How algorithms diffuse and amplify misinformation

Digital platforms are designed to maximize engagement. Recommender systems play a key role as they shape digital social networks, and the flow of information. Weaknesses in their underlying algorithmic systems are already today being exploited to try to influence public opinion on climate and sustainability issues. Automation through social bots also play a role, although their impacts are contested.

In 2010, a team of political scientists in collaboration with researchers at Facebook conducted a gigantic experiment on the platform. It included “all users of at least 18 years of age in the United States who accessed the Facebook website on 2 November 2010, the day of the US congressional elections”. This 61-million-person experiment tested whether a simple tweak in the newsfeed of users – adding an “I Voted” button and information about friends who had voted – could impact voting behaviour.

The results were clear: this small design change led to higher real-world voting turnout. Facebook users who saw a message about one of their close friends on Facebook having voted were 2.08% more likely to vote themselves (Bond et al., 2012).

This controversial experiment illustrates a number of issues that are key if we are to understand how digital platforms interplay with the creation and diffusion of mis- and disinformation, and their impacts.

First, digitalization and the expansion of social media have not only fundamentally expanded the scale, but also transformed the properties of social networks. 3,6 billion people use social media today. Through it, they access a digital media environment where language barriers slowly but surely are eroding as new digital tools make communication across borders increasingly frictionless. These new connections are transforming the structures of social networks in ways that allow for immediate communication across vast geographical space (Bak-Coleman et al., 2021).

Second, the way information moves in these vastly spanning digital social networks is not random. Its movement is fundamentally affected by the way digital media platforms such as recommender systems are designed to maximize engagement and induce other forms of behavioural responses amongst its users (Kramer et al., 2014; Coviello et al., 2014). The diffusion of such information can also be exploited by external attempts to manipulate public opinion online. Three common approaches include (1) specifically targeting the way algorithmic systems operate, (2) carefully crafting viral messages to sow confusion, and (3) through the use of automation tools like ‘social bots’.

The figure illustrates how social bots influenced the diffusion of different climate change frames via Tweets about the 2019/2020 Australia bushfires. The results indicate that different framings of climate change are associated with distinctive automation signatures. Figure: Azote.

Social bots mimic human behaviour online, and can be used in ways to amplify certain types of information – say, climate denialism –, or operate in ways that widen social divisions online (Gorwa and Guilbeault, 2018; Shao et al., 2018). Our own work (Daume et al., 2023) shows that social bots play an unignorable role in climate change conversation. They amplify information that supports and opposes climate action at the same time, especially information appealing to emotions, such as sympathy or humour.

Recommender systems play a key role for the dynamics of information diffusion and the evolution of digital social networks (Narayanan, 2023), and they have increasingly become infused with deep learning-based AI (Engström and Strimling, 2020). People-recommender systems for example (like “People You May Know” on Facebook or “Who to Follow” on Twitter) shape social network structures, thus influencing the information and the opinions a user is exposed to online (Cinus, 2021). Content-recommender systems (like “Trends for you”) can reinforce the human preference for content that aligns with a user’s ideology (Bakshy et al., 2015).

The figure below illustrates four highly simplified models of information propagation of one individual post. The expansion of digital social networks, the influence of recommender systems and social bots change the reach of mis- and disinformation. This diffusion has complex secondary effects on perceptions, on the formation of online communities, on collective action, and on identity formation.

The figure illustrates four models of information propagation of one individual post through: a) network, b) algorithm, c) affective, and d) automated amplification. In a) the post cascades through the network as long as other users choose to further propagate it by e.g., sharing or liking. In the algorithmic model b) propagation unfolds as users with similar interests (as determined by recommendation algorithms based on for example past engagement) are more likely to be recommended the post. In (c), users comment and reshare posts much more if they elicit strong emotions, and recommender systems pick up highly engaging posts and amplify them even further. In d) automated accounts (‘social bots’) purposefully share content that elicits strong emotions to further increase propagation. Figure based on (Narayanan, 2023), design by Azote.

Continue reading