| The social media company that pioneered letting users fact-check each other’s posts is introducing a new twist: allowing artificial intelligence bots to fact-check users’ posts. Elon Musk’s X said this week it will begin allowing developers to create AI agents that can propose debunking as part of its Community Notes program. But don’t worry, said Keith Coleman, X’s vice president of product: Humans will still be in charge. And some misinformation experts say the idea isn’t as fantastical as it might sound — though they do have some concerns. X hopes the move will help it speed up and scale up a program that has become a model for the social media industry. Started as a pilot project called Birdwatch at Twitter before Musk bought the company in 2022, Community Notes allows volunteer users to propose “notes” adding context or corrections to popular posts on the platform. Other users then rate the proposed notes’ helpfulness and credibility. If a proposed note gets positive ratings from enough users — particularly those who have disagreed with each other over past ratings — it goes public on X for all to see, appearing right below the post in users’ feeds. Musk quickly latched onto the idea as an alternative to what he saw as the liberal bias of professional fact-checking organizations. The idea has since been copied by other leading social networks, including Facebook, YouTube and TikTok. For tech executives accustomed to being harangued by politicians for either allowing lies to spread unchecked or wrongly censoring hard truths, the idea of delegating fact-checking to their users held an obvious appeal. The system has its virtues and shortcomings. The notes that get shown on X are generally viewed as helpful by the people who see them. And research has found that they tend to slow the spread of posts that receive the notes. But they have hardly cured the site’s well-known misinformation problem. That’s partly because analyses by The Washington Post and others have found that relatively few notes clear the high bar for approval. And often by the time they do, the offending falsehood has already traveled far and wide. That’s where AI could come in handy, Coleman said. Coleman told The Tech Brief that he hopes allowing developers to program AI agents to propose notes will result in more notes that are written more quickly and applied to more posts across the site. That might sound like a recipe for chaos. AI models are known to fabricate falsehoods themselves — a problematic trait for a fact-checker. And X’s own AI model, Grok, has had some rather spectacular missteps. But Grok won’t be writing Community Notes, at least for now. Rather, human developers, including researchers at universities, will register their own purpose-built AI bots for the program. They’ll first have to pass a few tests to gain eligibility. Once they do, the notes they propose will face the same tough approval process as human-written notes before they can be shown on the site. They’ll also be labeled AI-generated. Like human contributors to the program, the AI bots will receive credibility scores based on how their notes are rated. “Very importantly, AIs do not rate notes,” Coleman said. Keeping humans in charge of the rating process is crucial, he added, because large language models “make mistakes, they hallucinate things, they can be biased. Everyone’s aware of these issues.” Still, Coleman believes a well-devised AI bot has the potential to apply established facts to at least some categories of obviously false or misleading posts more quickly than any human fact-checker could. “They’re good at reviewing lots of content, scrounging the web, looking for resources,” he said. “It’s very believable that they could be good at saying, ‘Hey, this video has been around since 2013’ or ‘There have been 10,000 notes shown on this [claim] in the past.’” X partnered with researchers at MIT, the University of Washington, Harvard and Stanford on a preprint academic paper explaining how the system could work. The paper also identifies several “risks and challenges,” including the potential for AI bots to write “persuasive but inaccurate notes” or to overwhelm human raters. Some experts say the idea has potential, but they worry it could go awry. “This is actually potentially interesting if they can find a way to avoid floods of spam,” said Renée DiResta, a professor at Georgetown University’s McCourt School of Public Policy, in a Threads post. She pointed to research that has found that specially trained AI models excel at producing neutral language on controversial issues, which can help people of different ideologies find common ground. Such neutral language is key for Community Notes to gain approval. Alexios Mantzarlis, director of Cornell Tech’s Security, Trust, and Safety Initiative, said AI could help with one of Community Notes’ biggest challenges: applying notes to all the variants of a particular false claim that may be circulating at a given time. He noted that the single most prolific contributor to Community Notes already uses automation to submit notes flagging crypto scams, as Mantzarlis reported for Indicator last month. That contributor, an antivirus company, has submitted 52,000 notes, with over 1,400 rated “helpful” and shown publicly on X. Still, Mantzarlis said he believes the introduction of AI is a bad sign for the project long-term. One problem, in his words: “Who is going to rate all those notes?” Using AI bots to propose notes might not speed things up much if X is relying on the same number of human volunteers to approve them. That’s why he believes X will eventually turn to AI to rate notes as well. And if that happens, Mantzarlis said, a system meant to democratize the fact-checking process could devolve into an AI feedback loop. “It is counter to the original mission and purpose of the project, which was to bring in real humans, everyday users into this moderation space,” he warned. | 
