Ensuring the Safety of Children in the Digital World: an article from the Business and Human Rights Review

Safe Online

A child uses her computer.

This article first appeared in the Winter 2020 edition of the Business and Human Rights Review. Download the full edition here.

In 1989, the UN Convention on the Rights of the Child, which sets out the human rights of every person under 18 years old, was adopted by the UN General Assembly. The Convention affirms that all children have the right to be treated with dignity and respect, develop to their full potential, participate in society and be protected. Since then, 196 States have become parties to the Convention and, as a result, now have an obligation to protect children from violence, including online child sexual exploitation and abuse (CSEA).

All businesses also have a responsibility to respect human rights, including children’s rights, in accordance with the UN Guiding Principles on Business and Human Rights.

All businesses also have a responsibility to respect human rights, including children’s rights, in accordance with the UN Guiding Principles on Business and Human Rights (UNGPs). The corporate responsibility to respect human rights exists independently of States’ willingness to fulfil their own human rights obligations to prohibit CSEA. This means that businesses have a responsibility to avoid contributing to impacts on children’s rights by, for example, allowing and failing to take adequate steps to prevent CSEA from appearing on their sites, even where domestic legal frameworks have yet to adequately address the problem. Businesses are key to preventing and tackling online CSEA, and a growing number have made public commitments to doing so. Yet evidence and End Violence’s experience and research suggest that much more can and should be done.

The pandemic of online child sexual exploitation and abuse

The COVID-19 pandemic has accelerated the adoption of digital technologies for work, online learning, socialising, games and keeping in touch with loved ones. In fact, digital connectivity has been labelled as ‘the silent hero’ of the pandemic. However, this increase in screen time and reliance on digital technologies has resulted in an explosion of online CSEA. With both children and sexual offenders confined at home and spending more time online, law enforcement authorities and reporting hotlines have seen a striking increase in the amount of child sexual abuse material (CSAM)* being shared online. In April 2020 alone, the National Center for Missing and Exploited Children registered four million reports of suspected CSAM online compared to one million for the same period in 2019.

That said, digital harms to children are by no means a product of the Covid-19 pandemic. The spread of CSAM and child sex trafficking increased exponentially with the rise of the internet, digital technologies and online markets. A decade ago, there were 3,000 reports of suspected online CSEA registered by authorities. In 2019, that number climbed to nearly 17 million. Another trend is also becoming apparent: the victims are getting younger, with 89% of victims aged between three and 13.

By itself, voluntary action is not enough

While voluntary commitments are important and laudable, more accountability and action are needed to address the horrendous crimes of online CSEA. In this regard, businesses should be aware of evolving legislative frameworks that require more from digital platforms when it comes to protecting children online, including: (i) a newly adopted Age Appropriate Design Code in the UK; (ii) potential amendments to Section 230 of the Communications Decency Act in the US, which provides immunity for websites vis-à-vis third party content; and (iii) the forthcoming European Union Digital Services Act which will revise the rules that affect how intermediaries regulate user activity on their platforms.

A number of businesses and governments have endorsed voluntary frameworks like the Child Online Safety Universal Declaration adopted by the UN Broadband Commission for Sustainable Development and the 11 Voluntary Principles to Counter Online CSEA, which have already been adopted by five governments and endorsed by Facebook, Google, Microsoft, Twitter, Snap, and Roblox, amongst other members of the Technology Coalition.

These frameworks are designed to guide industry players to: (a) review existing safety processes; (b) understand the level and nature of online CSEA threats and areas of high risk on their platform; (c) identify gaps, consider where existing measures can go further, and improve and invest in innovative tools and solutions; and (d) respond to the evolving threat and changing societal and offending behaviours to reduce foreseeable and unexpected risks for users.

What more specific actions can businesses take to avoid online harm to children?

Businesses that provide and promote digital technologies have a responsibility and fundamental role to play in addressing online CSEA. There is no single method through which industry players can help to tackle this issue. While businesses of varying sector focuses, sizes and resources will have different roles and responsibilities in the global fight to end online CSEA, every digital industry player will need to give consideration to the issues discussed in this article in order for there to be an effective global response to end online CSEA.

Technology companies and internet service providers (ISPs) have a particularly important position from which they can enact change, and this article will largely focus on the actions that they can take. However, businesses in other industries should also consider certain steps and recommendations as outlined below, as they can often implement change in how they use the services, products and platforms offered by technology companies and ISPs.

Put Safety-by-Design at the heart of your business

Following the guidance set out in the UNGPs, all companies today should undertake human rights due diligence on their own operations and those in their value chains. This means that all ISPs should undertake ‘Safety-by-Design (SbD)’ due diligence to put user safety, especially children’s safety, at the forefront of the design, development and release of online products and services. The responsible use of technology should be carefully considered throughout the design process of every new platform and service, not just as an afterthought.

There are a number of resources to guide companies, both large and small, during the SbD due diligence process, such as the Child Online Safety Self-Assessment Tool developed by UNICEF in consultation with over 50 companies, specific networks and resources for mobile operators via the GSMA, and the Australia eSafety Commissioner’s anticipated self-assessment tools for start-ups and more established companies.

There are a number of opportunities today for industry, in collaboration with government, academia and NGOs, to collaborate more closely to share best practices in addressing child online safety. Funded by the End Violence Fund, UNICEF’s Regional Office for East Asia and the Pacific, together with a consortium of technology companies, recently convened five multi-stakeholder roundtables to share promising industry practices. Leading private sector companies attended and contributed to the roundtables, including Google, Facebook, Microsoft, Telenor, Ericsson, MobiCom (Mongolia), Mobifone+ (Vietnam), Globe Telecom (the Philippines), True (Thailand), GSMA and civil society partners. Industry players are also creating networks, such as the joint industry think-tank that incubates innovative solutions for education content development and delivery platforms and monitors behaviour change.

Empower children and families

To address CSEA effectively, industry players also need to focus on empowering children themselves and families. Best practice for technology companies and ISPs would be to provide their users with access to cost-free child helplines, age-appropriate services and safe e-education platforms. They could also invest in digital literacy and skills for children, parents and caregivers, as well as use their platforms to raise public awareness of the risks for children and other vulnerable groups. End Violence, together with its partners, collected key resources that industry can use to make online platforms safe and accessible for children during the Covid-19 pandemic.

Lead the design and scaling-up of technology tools and increase investment

Addressing online CSEA will also require businesses to invest in tools required for swift detection, investigation, and intervention to address online CSEA. Businesses with strong engineering capacity, financial assets and global reach will need to support and work with key global actors to provide financial and technical resources in order to adapt existing tools to different contexts and systems around the world, and secure licensing, training and maintenance of such tools.

Generally, companies in all industries can help to make a substantial impact by providing resources to organisations that work to leverage existing and build new cutting‑edge technology tools to tackle online CSEA (such as machine learning, AI and data science). For example, End Violence invested in an AI‑based law enforcement tool which, so far, has helped to identify and rescue 1,792 child victims globally and arrest 1,151 offenders in less than two years; over 1,200 law enforcement officers in 55 countries also reported an estimated time saving of up to 60%. Companies that may not be able to provide expertise in relevant areas for online CSEA may equally find this to be an impactful step to take.

This is also an area in which the technology industry can have a particularly significant impact. There are impressive examples of involvement, including: (i) ‘PhotoDNA’, a collaboration between Microsoft and Dartmouth University, which set a global standard by creating a digital fingerprint of known CSAM that can be used for detection, reporting and removal at scale; (ii) Google’s AI-powered Content Safety API, which improves the ability of NGOs and other technology companies to review CSAM at scale; (iii) Facebook’s open-sourcing photo and video matching technology, which supports companies by keeping their services safe and allowing their systems to communicate with each other, making their safety features more effective; and (iv) Project Artemis, a text-based tool developed by Microsoft in collaboration with the Meet Group, Roblox, Kik, and Thorn, which detects and reports online predators attempting to groom and lure children for sexual purposes. As a next step, companies in the technology sector will need to create and collaborate on tools and resources to be used by government, NGOs and businesses in other sectors.

Collect and share data

Industry has a crucial role to play in collecting and sharing data on children’s online access, use and risks, and particularly on offending patterns, to create a stronger and broader evidence base in all countries and gain insight into the scale and nature of the problem. For example, ISPs can provide data for innovative and replicable research projects like Disrupting Harm to assess the scale, nature and context of online CSEA across countries. Companies can also fund data collection and research to inform product and policy design across sectors.

Without meaningful participation from private companies and industry, we cannot fully understand the scale and nature of online CSEA or identify solutions to it.

Without meaningful participation from private companies and industry, we cannot fully understand the scale and nature of online CSEA or identify solutions to it. In June 2020, End Violence launched a multi‑year collaboration with the Technology Coalition, a group of 18 of the biggest technology companies, including Google, Microsoft, Facebook, Apple and Twitter. Through this partnership, the Technology Coalition and End Violence have set up a Safe Online Research Fund to advance the understanding of patterns of online CSEA and learn from effective efforts (behavioural, technological and educational) in order to prevent, deter and eradicate it. The vision is to fund innovative research that produces actionable insights that can impact product and policy development, with a priority given to research that can inform the industry’s approach to combatting online CSEA.

Collaborate with public sector organisations

Over the past four years End Violence’s Safe Online team, industry players and other stakeholders have worked together to test and scale new solutions and technologies to keep children protected from online harms, and industry has played a critical role in this collaboration. Through our Safe Online investments, we tested new types of collaboration, with a number of successful examples, such as the following:

Customer access to tools and messages: The unique partnership between Capital Humano y Social Alternativo and Telefónica in Peru resulted in the availability of child online safety awareness raising messages and educational tools for over 14 million customers. Similarly, mobile operators in Namibia and Madagascar disseminated free messages on child online safety, and in Costa Rica seven ISPs committed to incorporating an e-mentoring programme into their child online safety offerings to customers. 

Stakeholder access to training: In the Philippines, Plan International delivered training on child online safety to 1,387 internet café and pisonet operators. Subsequently, 1,170 signed a code of conduct with the community to regulate their role and operations in preventing and responding to online CSEA.

Design of safe technology: In Vietnam, ChildFund Australia worked with the local government to design a safe and child-friendly internet café, which was piloted in an effort to build a safe and healthy environment for children, as well as raise revenue for them. It also delivered four workshops to 110 online game shop owners and managers.

Through continued collaboration between End Violence, technology companies and the ISP industry, as well as other businesses more generally, a variety of viewpoints, areas of expertise and ideas can be brought together to fight online CSEA.

A call to action for industry: Investment, innovation, and implementation

Through its Safe Online work, End Violence continues to invest to tackle this ever-growing problem; in 2020, End Violence’s Safe Online portfolio will reach USD45m with impact in over 70 countries. However, this investment cannot match the enormous scale of the fight against online CSEA. Industry players should continue to collaborate with the public sector and provide much-needed resources to match the fighting power against the reality of children’s risks online.

The UN Secretary-General, Antonio Guterres, recently noted that if there is one area in which the key global actors must come together and show the power of digital cooperation across countries and sectors, it is the area of tackling the scourge of online CSEA. It is one of the key goals of the Safe Online initiative to see more industry players come together, as enhanced cross-sector collaboration and engagement modalities remain paramount to ensure tangible results.

Industry players are vital stakeholders in the attempt to tackle this problem. Although it has made a start, the technology sector can do much more to address CSEA by engaging in meaningful SbD due diligence during product development and before providing the sort of services, products and platforms that are currently being misused. Businesses in all sectors that use digital services, products, and platforms should also be mindful of the impact that they can have and put in place measures to combat their misuse. Finally, it is fundamental that all businesses aim to use their power to drive technological policy and practice across sectors and countries to ensure that all children are safe online.

Online CSEA is an abhorrent and widespread violation of children’s rights. End Violence calls upon all stakeholders, particularly industry players, to work together to put an end to it. Time is short to meet the goal of ending all forms of violence against children by 2030. The time to act is now.

* Child sexual abuse material, as defined by the Optional Protocol to the Convention on the Rights of the Child on the sale of children, child prostitution and child pornography means “any representation, by whatever means, of a child engaged in real or simulated explicit sexual activities or any representation of the sexual parts of a child for primarily sexual purposes”. In the digital context, CSAM is captured in the form of digital images and videos, as well as live-streamed videos. The production, distribution and possession of this type of material is a criminal offence.


Marija Manojlovic, Trang Ho Morton and Serena Tommasino are staff members with the Global Partnership to End Violence Against Children (End Violence) and lead its Safe Online initiative. End Violence is a public-private partnership launched by the UN Secretary-General in 2016 to accelerate progress towards Sustainable Development Goal 16.2: ending all forms of violence against children by 2030. End Violence comprises over 500 partners, including governments, civil society organizations, UN agencies, the private sector and research institutions, and acts as a global platform for advocacy, evidencebased action, and investments to end all forms of violence against children. Through its Safe Online work, End Violence provides funding, policy and advocacy guidance, and coalition-building to significantly advance national, regional and global efforts to prevent and respond to online child sexual exploitation and abuse. In 2020, End Violence’s Safe Online investment portfolio reached USD45 million in grants to projects achieving tangible results in over 70 countries.