Experiment Results: 47% of People Cannot Distinguish Between Bots and Humans on Social Media

The week-long experiment conducted by Surfshark and master's students at Malmö University involved more than 700 participants. The results of the experiment revealed that only 53% of participants were able to correctly identify bots more often than they mistakenly identified real humans as bots on a simulated social media platform. Meanwhile, approximately half, or 47%, were unable to correctly identify bots. Cybersecurity experts warn that the number of people unable to distinguish between bots and real humans on social media will increase even more rapidly in the future.

Justas Pukys, Senior Product Manager at Surfshark, stated the following:

"The game 'Bot or Not' and this experiment serve as important clues for understanding how malicious bots affect actual social media users. Earlier this year, it was revealed that major platforms remove over 6.3 billion fake accounts annually. This is approximately 47 times the number of newborns worldwide in a single year (about 135 million). Billions of bots are currently being generated, and the latest experiment showed that half of the participants can no longer distinguish them from real humans. Going forward, as technological advancements allow bots to blend more naturally into the profiles of real humans, this trend will only accelerate."

The more emotions are stirred, the harder it is to spot bots

The social media bot experiment revealed highly intriguing results. According to the data, when sensitive topics such as politics or social issues are discussed, people's ability to identify bots decreases, and the tendency to mistakenly suspect real humans of being bots increases.

The moment the discussion in the "Bot or Not" simulation shifted to emotional content, participants' bot detection capabilities dropped. Particularly in political discussions regarding immigration, the bot detection rate fell to 54%, and nearly half of the participants failed to spot bots on the social media platform. Furthermore, accuracy also dropped to 63%, increasing the instances where real humans were mistakenly identified as bots.

Moreover, topics related to women's rights yielded the most difficult results for bot identification. The bot detection rate plummeted to 49%, with participants missing more bots than they detected. In addition, accuracy dropped to 61%, marking the highest rate of participants misidentifying posts by real humans as being generated by bots.

Luis Costa, Head of Research at Surfshark, explained as follows:

"On the other hand, in discussions about data centers—a relatively technical topic for many people—participants recorded the highest bot detection rate at 71%, successfully identifying the majority of bots. Accuracy also maintained a high level at 76%. The results indicate that in situations where emotions are not strongly stimulated, people can not only detect more AI bots but are also less likely to mistakenly treat real humans as bots."

The "Bot or Not" game is currently available online for anyone to play.

Can we continue to distinguish between humans and bots in the social media age?

Costa further stated:

"The results of this experiment are very recent and contain important implications. It has become clear that simply 'deciphering' post content is not enough to cope with a social media environment flooded with bots. The ability to detect bots tends to be influenced by factors such as age, the social media platform used, and screen time. However, the most striking point was that 'emotion' is the biggest blind spot. When debates heat up, people's digital judgment is easily shaken. What is needed to counter automated disinformation and deception is not more advanced text analysis. It is maintaining composure and understanding one's own vulnerabilities."

Furthermore, Pukys explained practical countermeasures as follows:

"Please remember to always double-check information you see on social media. Also, do not take posts from unknown users at face value. You must be particularly cautious of messages claiming you have won a prize, links directing you to suspicious sites, or texts designed to incite anxiety, such as 'A family member has been in an accident!'"

Pukys also emphasized the importance of basic digital security measures, such as using anti-fraud tools on a daily basis. These tools analyze the contents of emails, SMS messages, and websites to determine whether they were generated by bots or attackers.