Privacy-preserving Moderation of Illegal Online Content

Stockwell, Sam, Wake, Georgia, Dargahi, Tooska ORCID: https://orcid.org/0000-0002-0908-6483, Ajao, Oluwaseun ORCID: https://orcid.org/0000-0002-6606-6569, Latham, Annabel ORCID: https://orcid.org/0000-0002-8410-7950 and Danladi Abdullahi, Ahmed (2025) Privacy-preserving Moderation of Illegal Online Content. Research Report. The Alan Turing Institute.

Preview

Published Version Available under License Creative Commons Attribution. Download (4MB) | Preview

Abstract

This CETaS Research Report examines promising content moderation solutions that can help social media platforms and end-to-end encrypted (E2EE) services fulfil their new legal duties to remove illegal online content under the UK Online Safety Act (OSA). It also seeks to understand what metrics can be used to better assess the effectiveness of moderation methods, as well as measure their impact on user privacy when they involve E2EE protocols. As reflected in the real-world harm to users caused by rising volumes of illegal content disseminated across online domains, effective responses to this threat have been challenging to implement at scale. To further complicate these efforts, detecting such material on E2EE services – where only the sender and the recipient can view a message – involves a difficult balance between safeguarding users and minimising privacy intrusiveness. Based on an extensive analysis of existing literature and focus groups with experts from different sectors, this report explores current challenges in content moderation and makes a series of recommendations for improving the privacy-preserving nature of tools, frameworks and policies involved in illegal content detection and removal processes.