Twitter bots promoting escorts and gambling drowned out protests in China

To protect the content from the reach of government censors, many people turned to Twitter to share what they were seeing.

But Twitter users who searched for that content ran into streams of spam, with thousands of accounts pushing escort services and gambling ads.

The spam posts were so overwhelming that they dominated results in Twitter searches for some of China’s largest cities.

Those posts were coming from bots. And despite claims from Twitter that it was cracking down on inauthentic accounts, Chinese-language Twitter feeds were inundated by them.

Twitter and its new owner, Elon Musk, have recently vowed to crack down on bots. But the flood of spam for Chinese users in recent weeks has underscored the challenges the company faces in policing fraudulent and inauthentic activity, especially in foreign languages that have traditionally been more loosely policed by large American social media platforms.

Discover the stories of your interest

For many Chinese who turned to the platform as demonstrations against COVID-19 restrictions had grown political and widespread, the experience of using the app appeared to capture a different reality.
Much of the spam is linked to commercial bot networks that have operated on Chinese-language Twitter since before the protests began Nov. 25, according to an analysis by The New York Times of thousands of tweets, interviews with people behind some of the bot accounts and a report by a researcher at Stanford University.

The Times searched on Twitter for 10 Chinese cities using Chinese characters, and reviewed the results for spam. Bots were active throughout, the analysis found, even for cities where protests were not held. When contacted, two businesses that appeared in spam tweets said that they had purchased the tweets through advertising services.

The Times also searched for six city names outside of China using English and simplified Chinese. Only the English searches were free from spam bots.

The Times analysis matched findings published last week by David Thiel, the chief technologist at the Stanford Internet Observatory. Thiel reviewed millions of tweets by searching for 30 Chinese cities and found that bots were active before the protests began and continued after they had ebbed. Twitter suspended many of the accounts, but the spam continued as other bots kept tweeting, he wrote in the report.

Musk, who took over Twitter a month before the protests began, has said that controlling bots is an “absolute top priority.” But in the weeks since acquiring the company, he cut Twitter’s workforce by about half. He also dissolved the company’s Trust and Safety Council, an advisory group formed in 2016 to address hate speech and other problems on Twitter.

In an internal email about the decision to end the Trust and Safety Council, the company said it would be “moving faster and more aggressively” to make Twitter safer, but provided no details on its plans. The company and Musk did not respond to a request for comment.

“I almost feel like it’s the Wild West,” Eirliani Rahman, a former member of the Trust and Safety Council, said in an interview. In a letter posted to Twitter announcing her resignation this month, she wrote that the “safety and well-being” of Twitter users was in decline, adding that “shooting from the hip is not how you do content moderation.”

In a recent live chat, Musk suggested that the company was being too aggressive in banning bots, suspending some legitimate accounts in the process. Yet among Chinese-language content on Twitter, the bots appear to maintain a significant presence.

Some internet watchdogs have suggested that the Chinese government was behind the spam bots. The Times and Thiel at Stanford found no evidence backing the idea.

In China, prostitution and pornographic content are illegal, and powerful censors wage an unending battle to cut back advertising from those services. So those businesses often flock to platforms like Twitter and Instagram, which have far fewer resources dedicated to monitoring and maintaining content shared in Chinese.

Estimates about the number of bots on Twitter vary. The company has said in public disclosures that bots account for fewer than 5% of its user base. In 2017, researchers at Indiana University and the University of Southern California estimated that number was closer to 15%.

Yoel Roth, the former head of safety and integrity at Twitter who resigned last month, has criticized efforts to scrutinize bot activity on the platform, writing in 2020 that automated accounts could be a “powerful tool” for customer service and other similar cases. He added that the company had instead focused its attention on limiting the “malicious use of automation” that aims to manipulate conversations.

As Twitter and other social media platforms have prioritized eliminating fraudulent activity, misinformation experts say the companies have largely failed in policing posts that are not in English. The Times analysis found that the number of bots appearing in searches about #Beijing was far higher for Chinese-language searches compared with English ones.

Thiel collected millions of tweets over several weeks to track the bot campaigns. He found hundreds of thousands of accounts with clear markers of bot activity, such as repeatedly tweeting the same content with a different combination of hashtags and photos. It often took Twitter days to suspend inauthentic accounts, which is typical for the social network, Thiel said. That allowed their content to overrun search results for days until the company acted – just in time for new accounts to be created and continue the drumbeat of spam.

Spam accounts can also be purchased and repurposed by governments from bot marketplaces, making it hard to tell whether a campaign is commercial or political, according to one former Twitter employee who worked in its safety and integrity group and has analyzed state-backed influence operations. The former employee spoke on the condition of anonymity because of the terms of a severance agreement with the company.

And although there was no evidence that the Chinese government was behind the spam bots that flooded Twitter last month, Thiel said he had concerns about what may lie ahead.

“Given how effective it was at drowning out other people, I wouldn’t be surprised if a government of some kind tries to use this technique in the future,” Thiel said.