Major technology companies are falling short in addressing terrorist and violent extremist material on their platforms, according to a new transparency report released by Australia’s eSafety Commissioner.
The report, based on disclosures from Google, Meta, WhatsApp, X (formerly Twitter), Telegram, and Reddit, highlights inconsistencies in detecting and removing harmful content, as well as gaps in safeguards against misuse for radicalisation and recruitment.
eSafety Commissioner Julie Inman Grant said online services have repeatedly been exploited by violent extremists, contributing to real-world safety threats.
“Ever since the 2019 Christchurch attack, we have been particularly concerned about the role of livestreaming, recommender systems, and now AI, in producing, promoting, and spreading this harmful content,” Inman Grant said.
Reporting mechanisms insufficient
The report found that Meta’s Messenger, Telegram, and WhatsApp services did not employ measures to detect livestreamed terrorist and violent extremist material, despite Facebook Live—another Meta product—being used in the Christchurch attack.
Additionally, Facebook and YouTube lacked mechanisms for users who were not logged in with accounts to report livestreamed terrorist content.
Google reported receiving 258 user reports of suspected AI-generated deepfake terrorist or extremist material from its generative AI system, Gemini, between April 2023 and February 2024.
It also received 86 reports of suspected AI-generated child sexual exploitation and abuse material.
“This underscores how critical it is for companies developing AI products to build in and test the efficacy of safeguards,” Inman Grant said.
“Safety by Design must be deployed throughout every stage of the AI life cycle.”
Inconsistent enforcement and delayed response times
The report uncovered discrepancies within the same companies' services. For example, while Meta prohibits certain extremist groups from using its platforms, WhatsApp did not extend this ban to its private messaging service, potentially allowing these organisations to operate without action.
Hash-matching technology, used to identify harmful content, was also not consistently deployed.
Google only applied this technology to detect exact matches of extremist content on YouTube and Google Drive, despite technology existing to detect altered versions.
The measure is therefore insufficient, given that Meta reported over 800 different versions of the Christchurch attack video circulating in the aftermath.
The response time to user reports of terrorist content also varied significantly:
- WhatsApp took more than a day
- Reddit took 1.3 days
- Threads took 2.5 days
- Telegram took 18 hours for Chats and Secret Chats
Industry-wide challenges
eSafety issued a A$957,780 infringement notice to Telegram last week for failing to meet a transparency reporting deadline by more than five months. Telegram has since provided information, which is reflected in the report.
Meanwhile, X is challenging eSafety’s reporting notice in the Administrative Review Tribunal, and as a result, its responses are not included in the report.
The findings highlight broader concerns about the effectiveness of major tech companies in tackling violent extremism online.
While Google and Meta support 80 and 109 languages respectively, Telegram only covers two of the top five languages spoken in Australian homes, raising further concerns about the accessibility of moderation efforts.
eSafety’s report calls for stronger industry-wide enforcement, improved AI safeguards, and faster response times to prevent digital platforms from being used to promote extremism and harm.