Online behavioral data, such as digital traces from social media, have the potential to allow researchers an unprecedented new window into human behavior in ecologically valid everyday contexts. However, research using such data is often purely observational, limiting its ability to identify causal relationships. Here we review recent innovations in experimental approaches to studying online behavior, with a particular focus on research related to misinformation and political psychology. In hybrid lab-field studies, exposure to social media content can be randomized, and the impact on attitudes and beliefs measured using surveys; or exposure to treatments can be randomized within survey experiments, and their impact observed on subsequent online behavior. In field experiments conducted on social media, randomized treatments can be administered directly to users in the online environment - e.g. via social tie invitations, private messages, or public posts - without revealing that they are part of an experiment, and the impacts on subsequent online behavior observed. The strengths and weaknesses of each approach are discussed, along with practical advice and central ethical constraints on such studies.

Misinformation can come directly from public figures and organizations (referred to here as “elites”). Here, we develop a tool for measuring Twitter users’ exposure to misinformation from elites based on the public figures and organizations they choose to follow. Using a database of professional fact-checks by PolitiFact, we calculate falsity scores for 816 elites based on the veracity of their statements. We then assign users an elite misinformation-exposure score based on the falsity scores of the elites they follow on Twitter. Users’ misinformation-exposure scores are negatively correlated with the quality of news they share themselves, and positively correlated with estimated conservative ideology. Additionally, we analyze the co-follower, co-share, and co- retweet networks of 5,000 Twitter users and find an ideological asymmetry: estimated ideological extremity is associated with more misinformation exposure for users estimated to be conservative but not for users estimated to be liberal. Finally, we create an open-source R library and an Application Programming Interface (API) making our elite misinformation-exposure estimation tool openly available to the community.

Social corrections, wherein social media users correct one another, are an important mechanism for debunking online misinformation. But users who post misinformation only rarely engage with social corrections, instead typically choosing to ignore them. Here, we investigate how the social relationship between the corrector and corrected user affect the willingness to engage with corrective, debunking messages. We explore two key dimensions: (i) partisan agreement with, and (ii) social relationship between, the user and the corrector. We conducted a randomized field experiment with N=1,586 Twitter users and a conceptual replication survey experiment with N=812 Amazon Mechanical Turk workers in which posts containing false news were corrected. We varied whether the corrector identified as Democrat or Republican; and whether the corrector followed the user and liked three of their tweets the day before issuing the correction (creating a minimal social relationship). Surprisingly, we found that shared partisanship did not increase a user’s probability of engaging with the correction. Conversely, forming a minimal social connection significantly increased engagement rate. A second survey experiment (N = 1,621) found that minimal social relationships foster a general norm of responding, such that people feel more obligated to respond – and think others expect them to respond more – to people who follow them, even outside the context of misinformation correction. These results emphasize social media’s ability to foster engagement with corrections via minimal social relationships, and have implications for effective, engaging fact-check delivery online.

Misinformation can come directly from public figures and organizations (referred to here as "elites"). Here, we develop a tool for measuring Twitter users' exposure to misinformation from elites based on the public figures and organizations they choose to follow. Using a database of professional fact-checks by PolitiFact, we calculate falsity scores for 816 elites based on the veracity of their statements. We then assign users an elite misinformation-exposure score based on the falsity scores of the elites they follow on Twitter. Users' misinformation-exposure scores are negatively correlated with the quality of news they share themselves, and positively correlated with estimated conservative ideology. Additionally, we analyze the co-follower, co-share, and co-retweet networks of 5000 Twitter users and find an ideological asymmetry: estimated ideological extremity is associated with more misinformation exposure for users estimated to be conservative but not for users estimated to be liberal. Finally, we create an open-source R library and an Application Programming Interface (API) making our elite misinformation-exposure estimation tool openly available to the community.

We investigate the relationship between individual differences in cognitive reflection and behavior on the social media platform Twitter, using a convenience sample of N = 1,901 individuals from Prolific. We find that people who score higher on the Cognitive Reflection Test-a widely used measure of reflective thinking-were more discerning in their social media use, as evidenced by the types and number of accounts followed, and by the reliability of the news sources they shared. Furthermore, a network analysis indicates that the phenomenon of echo chambers, in which discourse is more likely with like-minded others, is not limited to politics: people who scored lower in cognitive reflection tended to follow a set of accounts which are avoided by people who scored higher in cognitive reflection. Our results help to illuminate the drivers of behavior on social media platforms and challenge intuitionist notions that reflective thinking is unimportant for everyday judgment and decision-making.

Online behavioral data, such as digital traces from social media, have the potential to allow researchers an unprecedented new window into human behavior in ecologically valid everyday contexts. However, research using such data is often purely observational, which limits its usefulness for identifying causal relationships. Here we review recent innovations in experimental approaches to studying online behavior, with a particular focus on research related to misinformation and political psychology. In hybrid lab-field studies, exposure to social-media content can be randomized, and the impact on attitudes and beliefs can be measured using surveys, or exposure to treatments can be randomized within survey experiments, and their impact on subsequent online behavior can be observed. In field experiments conducted on social media, randomized treatments can be administered directly to users in the online environment (e.g. via social-tie invitations, private messages, or public posts) without revealing that they are part of an experiment, and the effects on subsequent online behavior can then be observed. The strengths and weaknesses of each approach are discussed, along with practical advice and central ethical constraints on such studies.

Americans are much more likely to be socially connected to copartisans, both in daily life and on social media. However, this observation does not necessarily mean that shared partisanship per se drives social tie formation, because partisanship is confounded with many other factors. Here, we test the causal effect of shared partisanship on the formation of social ties in a field experiment on Twitter. We created bot accounts that self-identified as people who favored the Democratic or Republican party and that varied in the strength of that identification. We then randomly assigned 842 Twitter users to be followed by one of our accounts. Users were roughly three times more likely to reciprocally follow-back bots whose partisanship matched their own, and this was true regardless of the bot's strength of identification. Interestingly, there was no partisan asymmetry in this preferential follow-back behavior: Democrats and Republicans alike were much more likely to reciprocate follows from copartisans. These results demonstrate a strong causal effect of shared partisanship on the formation of social ties in an ecologically valid field setting and have important implications for political psychology, social media, and the politically polarized state of the American public.

How can online communication most effectively respond to misinformation posted on social media? Recent studies examining the content of corrective messages provide mixed results—several studies suggest that politer, hedged messages may increase engagement with corrections, while others favor direct messaging which does not shed doubt on the credibility of the corrective message. Furthermore, common debunking strategies often include keeping the message simple and clear, while others recommend including a detailed explanation of why the initial misinformation is incorrect. To shed more light on how correction style affects correction efficacy, we manipulated both correction strength (direct, hedged) and explanatory depth (simple explanation, detailed explanation) in response to participants from Lucid (N = 2,228) who indicated they would share a false story in a survey experiment. We found minimal evidence suggesting that correction strength or depth affects correction engagement, both in terms of likelihood of replying, and accepting or resisting corrective information. However, we do find that analytic thinking and actively open-minded thinking are associated with greater acceptance of information in response to corrective messages, regardless of correction style. Our results help elucidate the efficacy of user-generated corrections of misinformation on social media.