2024-06-12: Paper Summary: Exploring the Use of Personalized AI for Identifying Misinformation on Social Media

Figure 1 Jahanbakhsh et al. Assess tweets with AI assistance (UI used in Step 3). Participants assess each tweet in their feed while seeing AI model predictions. As shown in this screenshot, the user is in the middle of an experiment and has assessed the top tweet, while the rest of the tweets in this feed (for which the AI model predictions are shown) have not been assessed yet. The AI model predictions may change in response to user feedback. The tweets for which the AI predictions have recently changed are shown on the left pane. Newly changed predictions are differentiated visually with a border and a notification icon, similar to the bottom tweet in the image (Figure 3 in original paper).

Understanding Personalized AI

Personalized AIs refer to algorithms customized to individual users based on their preferences, behaviors, and past interactions. Social media platforms use personalized AI to curate users' newsfeeds and also show them content related to their interests.

The Role of Personalized AI in Misinformation Detection

By analyzing the user's past interactions, preferences and the credibility of the sources they typically engage with, personalized AI algorithms can asses the trustworthiness of the information they encounter on social media. For example, if a user typically engages with sources known for their accuracy and suddenly interacts with information from a source with a questionable reputation, the AI algorithm can flag it as potentially misinformation. Similarly, if a user frequently shares misinformation, the algorithm can adjust the content they see to include more credible sources.

Methodology

The researchers developed a platform enabling human-AI interaction. Participants assessed  feed of tweets and receive predictions from their personalized AI regarding how they would assess other tweets. As participants provided more assessments, their personalized AI would retrain and update its predictions. The task served two main purposes:

1. Exposing participants to interacting with a personalized AI to determine content accuracy on social media and gauge their perceptions.
2. Understanding whether a user's assessment of a tweet would be influenced by an AI predicting how the user might assess it.

To study the AI's influence on user decisions, they couldn't ask users to assess tweet accuracy before and after seeing AI predictions, as this might bias their assessments. Instead, they compared user agreement with the AI across tweets, some with AI predictions displayed and some without. To prevent confounding effects, they trained two separate AI models evolving independently. This allowed the researchers to display one model's predictions on some tweets and withhold the predictions of the other AI on different tweets. By comparing how often users agreed with the predictions, they ensured that any observed differences were attributed to users seeing or not seeing AI predictions, rather than differences in AI model performance.

Data Collection

The dataset includes user interactions and content shared on Twitter, a popular social media platform. It also has user profiles, posts, comments, likes, shares, and the credibility scores of the sources. Additional information on user engagement patterns, such as the types of content users typically interacted with and the frequency of their interactions. Table 1 shows sample topics from collected data. 

The tweets were collected based on the following criteria:

• Tweets must contain verifiable claims, excluding arguments, opinions, or life stories.
• Tweets should be in the form of statements or evident rhetorical questions, not questions, to ensure binary accuracy labeling.
• The topic of tweets should not relate to news outside the United States to maintain relevance for US-based users.
• Tweets must be self-contained, requiring no interpretation from other tweets to understand.
• The main topic of the tweet should be COVID-19, excluding topics like the pandemic's economic impact to limit subtopics.
• Tweets should not focus on COVID-19 statistics like death tolls, as these numbers may change before the user study begins.

Table 1 Jahanbakhsh et al. - The two sides of the arguments in COVID-19 related subtopics from pool of tweets. (Table 1 in original paper)

Preprocessing

Before training the personalized AI model, the authors preprocessed the collected data. This included cleaning the text data, removing stop words, tokenizing, and applying vectorization techniques such as TF-IDF (Term Frequency-Inverse Document Frequency) and word embeddings.

• Tweets for both AI processing and participant presentation were cleaned.
• Removed @username strings at the beginning of tweets, replacing other occurrences within the text with a special token.
• Replaced links with another special token.
• Removed hashtags but retained the word following the hashtag.
• Eliminated indicators of tweet threads at the end of the tweet text (e.g., 2/5).
• Only the tweet texts were used for training users' personalized AI models; other features such as tweet author or date were not considered due to their rarity in the dataset.

Training Data

The dataset divides into training, validation, and test sets. The training data included user interactions with both credible and non-credible sources of information. They labeled the data with user interactions as either reliable or unreliable, based on the credibility scores of the sources.

Personalized AI Model

They developed a personalized AI model using machine learning algorithms such as Support Vector Machines (SVM), Random Forest, and Neural Networks. The model trained to predict the credibility of information based on a user's past interactions and preferences. To train a personalized model for each user that evolved as the user provided more assessments, the researchers required an AI system that could be quickly retrained. They deployed an instance of an open-source system called Label Sleuth for this purpose. Label Sleuth is an interactive visual system that allows users to label text elements while the system automatically trains in the background. They also customized Label Sleuth to communicate with the experimental system. Continuously submit a user's assessments, check for updates to the user's model, and retrieve the most recent model's predictions using Label Sleuth's backend API. Label Sleuth trains a binary classifier that predicts which items from a document (tweet texts) likely belong to the positive category, which, in their case, was the category "inaccurate". The model training happens iteratively, starting a new model training iteration as more labels are provided.

To allow for fine-grained control over when a new iteration is invoked, Label Sleuth offers two customizable settings: positive threshold and changed element threshold. These settings denote after how many initial positive labels a model should start training for the user and the number of changes in user labels required to trigger the training of a new model, respectively. To ensure the user's model updated in reaction to their inputs but without causing wasted computation, the researchers set both thresholds to 4 based on empirical testing. Label Sleuth trains a SVM model for the user, allowing for fast retraining when new labels arrive. However, retraining still takes approximately a minute. After the training of a new model, the experimental platform determines which predictions by the new model are different from before, records them, and signals them to the user.

The predictions could change several times before the user finished assessing the tweets, they decided that each feed would consist of 26 tweets. With 26 tweets in each feed, each of the models Hidden and Visible would have seen 52 examples by the end of their life cycle. Additionally, providing free-text reasons for 3 tweets in each feed served the purpose of delaying users a little longer, allowing for the processing of model predictions.

Participants

The researchers recruited participants from Amazon Mechanical Turk, requiring them to have more than 500 HITs approved, an approval rate of over 98%, and the United States as their location. Additional eligibility criteria included being 18 years or older, occasionally reading news online, fluency in English, and being a US citizen or permanent resident. A total of 65 non-spammer workers participated. The researchers identified low-quality responses by examining participants' free-text responses to reasoning questions and the post-study survey. Four participants were paid but their submitted texts were unintelligible due to language errors. These cases were removed from the dataset as the participants did not fully understand the task.

Analysis Procedure

• Due to a bug in the open-source AI system, 11 participants' AI models (Hidden, Visible, or both) failed to retrain after initial iterations. Researchers removed these participants' data from their analyses comparing assessments across conditions, resulting in data from 50 users for these analyses. However, for analyzing post-study survey responses, they retained data from 54 users, excluding only those whose Visible model experienced training problems, regardless of Hidden model failures.
• For statistical tests predicting binary dependent variables, they fitted generalized linear models using the "glmer" function from the R package "lme4," with the family function "Binomial" and link "logit."
• For continuous outcomes, used the "lmer" function to fit linear models.
• In all models, included tweet and participant identifiers as random effects to account for variation attributed to unobserved characteristics of a particular tweet or participant.
• In regression estimate tables, presented exponentiated coefficients (odds ratios) as measures of effect size for binomial dependent variables (logistic regression), and partial χ2 as a measure of effect size for linear regressions predicting continuous outcomes.
• Additionally, they presented the marginal χ2 (variance in the outcome explained by fixed effects) and the conditional χ2 (variance explained by the entire model including both fixed and random effects) in all analyses.

Results

Figure 2 Jahanbakhsh et al.- The predicted values of user-AI agreement by the user’s confidence in their assessment. The higher the confidence of a user in their assessment, the more likely their AI is to correctly predict their accuracy rating. (Figure 6 in original paper)

Table 2 Jahanbakhsh et al.- Participants’ views in favor of and against adopting a personalized AI for identifying misinformation on social media. (Table 4 in original paper)

Figure 3 Jahanbakhsh et al. - The effect of the interaction between model and iteration on the predicted user-model agreement. Over time, the difference is user-model agreement across the two models increases (Figure 9 in original paper).

Figure 3 shows the effect of the interaction between model and iteration on the predicted user-model agreement. It demonstrates that over time (i.e., as the iteration increases), the difference in user-model agreement between models Hidden and Visible grows larger. This observation suggests that when users see AI predictions, they become more reliant on the AI over time.

Benefits and Challenges

The use of tailored AI for misinformation detection offers several benefits:

1. Real-time Detection: It can quickly identify and flag misinformation as it appears on social media, helping to prevent its spread.
2. Customized Solutions: By tailoring detection algorithms to individual users, it can provide more effective and targeted solutions.
3. Reduced Human Effort: Automated detection of misinformation reduces the burden on human fact-checkers, allowing them to focus on more challenging cases.

However, there are also challenges associated with the use of AI for misinformation detection:

1. Privacy Concerns: Tailored AI requires access to user data, raising concerns about privacy and data security.
2. Algorithmic Bias: There is a risk that AI algorithms may inadvertently reinforce existing biases, leading to unfair or inaccurate assessments of content.
3. Adversarial Actors: Those spreading misinformation are likely to adapt their tactics to evade detection by AI algorithms, posing an ongoing challenge.

The Future of Misinformation Detection

While tailored AI holds promise for preventing misinformation on social media, it is not a remedy. To be effective, it must be part of a broader strategy that includes human fact-checkers, community reporting, and efforts to improve media literacy. The future research and development would improve the accuracy and fairness of customized AI algorithms. By addressing these challenges, it can harness the power of AI to create a safer and more reliable online information environment.

Conclusion

References

-Nithiya