ML/AI Algorithms to Combat Online Gender-based Abuse

Researchers at the National Robotarium have been awarded £1 million in funding from UK Research and Innovation for the creation of AI tools to improve the detection, intervention, and prevention of online gender-based abuse.

The National Robotarium’s project is in response to a report published by Glitch, a UK charity that aims to end online abuse. The Glitch report highlights the current "epidemic of online abuse" experienced particularly by women and non-binary people. Forty-six percent of women and non-binary people surveyed said they experienced attacks online since the beginning of the COVID-19 pandemic. Gender was the most frequently cited reason for online abuse. Twenty-nine percent of people who had experienced online abuse prior to the pandemic reported that the abuse worsened during lockdown.

Principal investigator and professor of conversational AI at the National Robotarium, Verena Rieser, said, “Figures like those included in the Glitch report continue to highlight the discriminatory effect of gender-based online abuse and the disproportionate impact it has on non-binary people and women. One challenge is that the current means of identifying and defining abuse often relies on majority social viewpoints. From our research, we know these viewpoints don’t necessarily correlate with the experiences of victims.” 

Related:5 Tips for Inclusive Product Design

AI Chatbots To Provide Support to Victims

A key ambition of the National Robotarium project is to increase digital literacy concerning the safe use of social media from an early age. This will be achieved through the development of learning materials to help children identify what online gender-based abuse is and how to deal with it.

Dr. Fiona McNeill, coresearcher and reader in computer science education at the National Robotarium, noted that activities to mitigate online gender-based abuse tend to focus on adults. “[Online abuse] is a huge problem for children and young people, as well,” McNeill said. “As part of the project being driven by the National Robotarium, we will be working with young people to understand their experiences of online abuse, the language they use around this, and the way in which young victims need to be supported.”

The project plans to also produce data on the type and frequency of online abuse experienced by victims. New and updated datasets will be shared with the wider research community for the purposes of developing further strategies. “Our approach is based on the framework laid out by the Scottish Government's Equally Safe strategy, which aims to prevent violence against women, young people, and children,” Rieser noted. “In addition to automatically detecting online abuse, it will aim to rethink what it is we need to detect, how best to support victims, and the role education can play as a tool for prevention.”

National Robotarium researchers said they will engage with experts on gender-based abuse and online harassment to build mechanisms that automatically generate counter-narratives aimed at perpetrators. The institute will also develop a chatbot to provide proactive support to victims of online abuse.

The robotics and AI facility is part of the Data-Driven Innovation initiative and is supported by £21 million from the UK Government and £1.4 million from the Scottish Government through the £1.3 billion Edinburgh and South East Scotland City Region Deal -- a 15-year investment program jointly funded by both governments and regional partners.