Isabella Shere

Dear Miss Hallett, Response to Regulation 28 report following inquest into the death of Isabella Shere We write in response to the Regulation 28 report to Prevent Future Deaths, which was issued to Ofcom following the death of Isabella Shere (‘the Report’). We received your report on 11 June 2024; thank you for agreeing a deadline for our response of 25 July. We would like to offer our deepest condolences to Isabella’s family and loved ones, and to assure them that ensuring online services tackle illegal and harmful suicide content on their platforms is a key priority for Ofcom as we implement the Online Safety Act 2023 (‘the Act’). We thank the coroner’s office for bringing to our attention the circumstances surrounding Isabella’s death. Intelligence about the real-world impacts of online harms and how these manifest on specific services will be crucial as we develop our approach to the online safety regime. Overview of our response to the Report In this response, we set out our proposed actions in relation to the issues raised by the Report, where these fall within the scope of the online safety regime, and the timetable for these actions. These actions are pursuant to Ofcom’s duties and powers under Act, as the UK’s regulator for online safety. We also highlight the steps we are taking to promote compliance with the regime across all relevant regulated services. Our response to this report includes:
• A brief overview of the scope of the Act, which sets out how the online safety regime protects all users from illegal suicide content and how the regime protects children from suicide content that is harmful to children.
• How Ofcom will implement the Act, which includes summaries of the three phases we have organised our work into: Illegal Harms, Protecting Children and Additional Duties on Categorised Services.
• A summary of Phase One: Illegal Harms.
• A summary of Phase Two: Protecting Children.
• A summary of Phase Three: Additional Duties on Categorised Services.
• Response conclusion.

In this response, we have taken a holistic approach to set out how the Act and our proposed implementation of it will address the risks of harm to adults and children online. We believe we have addressed all the individual areas/risk factors which you have identified in your Report. For ease of reference, we have signposted in footnotes throughout where the information we are providing is relevant to one of your specific concerns. The Online Safety Act 2023 The Act received Royal Assent on 26 October 2023 and makes companies that operate a wide range of online services legally responsible for keeping people safer online. The Act covers certain categories of internet services that have links with the UK. These include ‘user-to-user’ services and search services.1 The Act defines a user-to-user or search service as having links to the UK if it meets any one or more of the following criteria:
• Has a significant number of UK users; or
• Has UK users as one of its target markets; or
• Is capable of being used by UK users, and there are reasonable grounds to believe that there is a material risk of significant harm to UK users.

Any service which meets one or more of the above criteria, and which is not exempt2, will be expected to comply with the relevant duties under the Act. Among other things, the Act:
• Appoints Ofcom as the regulator for online safety and confers upon us a number of powers and duties (set out in detail below).
• Imposes a number of duties on those regulated services, which focus on improving the systems and processes online services operate to ensure the safety of their users, rather than on the presence of individual pieces of content.
• Requires regulated services to assess the risks their services pose to users in relation to illegal content and content that is harmful to children, and take steps to mitigate and manage those risks.
• Requires Ofcom to issue a number of regulatory publications to help regulated services understand how they can comply with their legal duties.
• Requires Ofcom to publish resources to help companies assess, understand and manage risk.
• Ofcom will also produce Codes of Practice, setting out recommended measures services can take to comply with the relevant duties under the Act in order to mitigate the risk of harm. As explained further below, there are three sets of duties contained in the Act which may relate to suicide content on user-to-user services. First, there are duties on all regulated user-to-user services relating to protecting their users from illegal harms that will require those services to understand and take steps to manage and mitigate the risks of users encountering illegal suicide content2, or of

1 Section 3 of the Act states that a “user-to-user service” means an internet service by means of which content that is generated directly on the service by a user of the service, or uploaded to or shared on the service by a user of the service, may be encountered by another user, or other users, of the service. “Search service” means an internet service that is, or includes, a search engine (see section 229 of the Act). 2 A number of exemptions also apply as set out in Schedule 1 to the Act. See: Vol 1, Section 3 of our Illegal Harms Consultation

their services being used for the commission or facilitation of this offence. The offence of encouraging or assisting suicide is a priority offence under the Act and user-to-user services will have to swiftly take down illegal suicide and illegal self-harm content when it is identified. Second, there are duties relating to content which is harmful to children. Content which “encourages, promotes or provides instructions for suicide” (‘suicide content’) and content which “encourages, promotes or provides instructions for an act of deliberate self-injury” (‘self-harm content’) have been designated as ‘Primary Priority Content’ under the Act.3 Where regulated services are likely to be accessed by children within the meaning of the Act, they will also have to understand the risks of, and take steps to prevent child users from encountering, suicide and self- harm content. Third, there are additional duties which will apply to certain user-to-user services which will be ‘categorised’ as either Category 1 or 2B, based on thresholds that will be decided by the Secretary of State and which include thresholds for user numbers and functionalities. These services will have duties which are designed to make these services more transparent and accountable to their users about the steps they take to protect them from harm. Category 1 services will have duties to enable adult users to have more control over the type of content they encounter, including by having access to tools to reduce their potential exposure to suicide and self-harm content. A set of separate duties apply to regulated search services. These focus on understanding the risks of harm and taking steps to minimise the risk of individuals encountering illegal content and content that is harmful to children in search results. Although the Act is now law, the duties are not yet in force. The Act requires us to draft Codes of Practice and consult on them before submitting them to the Secretary of State for approval. If approved, the Codes must then be laid before parliament for 40 days. Following approval by Parliament, the Codes will come into force 21 days after they have been issued. We explain our plans to implement the regime (consultation and enforcement timings) for Ofcom’s Illegal Harms Consultation and Protection of Children Consultation below. More information on our approach to implementing the Act can be found in our published Roadmap. In the meantime, we are already encouraging in-scope service providers to take meaningful steps to improve safety on their platforms. To this end, we are committed to driving industry improvements by engaging with the largest and riskiest services via continuous ‘regulatory supervision.’ We also have a dedicated team for identifying, prioritising and escalating emerging issues, particularly on services where we do not have an existing supervisory relationship, which we call the Triage team. The purpose of the Triage team is to ensure Ofcom responds effectively, promptly, and proportionately to new or growing harms and risks, focusing our reactive work on the most harmful issues. Ofcom’s implementation of the Online Safety Act The Act has identified three core areas for Ofcom to work on over the next three years, which are summarised below. The Act received Royal Assent on 26 October 2023, which means we have 18 months to submit our Codes of Practice on illegal harms (Phase one) and publish associated guidance to the Secretary of State by the end of 2025. Phase one: Illegal Harms Our first step in protecting users from illegal harms was the publication of our consultation Protecting people from illegal harms online (‘Illegal Harms Consultation’) on 9 November 2023. As part of this consultation, we published draft Codes of Practice which set out how

3 Relevant to concern (2)

services can comply with their illegal content duties in the Act.4 We also published draft Illegal Content Judgements Guidance on how services can identify illegal suicide and illegal self-harm content. Phase two: Protecting Children We published our consultation Guidance for service providers publishing pornographic content on 5 December 2023. This included draft guidance on age assurance and other duties for services providing pornographic content. Following this, we published our consultation Protecting children from harms online (‘Protection of Children Consultation’) on 8 May 2024. This consultation includes our draft Children’s Safety Codes which set out more than 40 practical steps that services can take to keep children safer online. Phase three: Additional Duties on Categorised Services On 25 March 2024, we published our Call for evidence: Third phase of online safety regulation. We will publish further detailed proposals in relation to the additional duties on categorised services in 2025. As part of our preparatory work for implementation, we have been actively engaging with a range of expert stakeholders including the government, law enforcement, and charities such as Samaritans to develop our understanding, expertise and evidence base in relation to suicide and self-harm, and to ensure that we are aware of developing areas of risk. We have also been concentrating on growing our internal expertise in relation to this complex and important harms area. We will continue our programme of engagement with relevant experts as we consult on our initial proposals on how services can comply with their duties. Phase one Ofcom’s Illegal Harms Consultation: assessing risks The Act requires Ofcom to produce a register of risks for illegal harms, and guidance to assist services in conducting their own risk assessment. Our draft guidance sets out a four-step risk assessment process which we propose as the best way to ensure that services meet their risk assessment obligations. We also consulted on ‘Risk Profiles’, which set out the risk factors that we consider are associated with an increased risk of harm on services, based on our Register of Risks. Services will be required to take account of our Risk Profiles when conducting their risk assessments. For illegal suicide content and self-harm content, we set out risk factors relating to service type, user base, functionalities of the service, and recommender systems.5 We expect to publish final Illegal Harms Risk Assessment Guidance by the end of this year. At that point regulated services such as Quora will need to complete an Illegal Harms Risk Assessment, including assessing the risk of illegal content, including illegal suicide content on its platform, within three months.

4 While services are not required to implement all measures in our Codes of Practice, in the event that they choose not to take the steps recommended, they will need to be able to explain how their chosen approach allows them to be compliant with their legal duties. 5 Recommender systems are defined as an algorithmic system which, by means of a machine learning model, determines the relative ranking of an identified pool of user-generated content on content feeds such as newsfeeds and reels. Content is recommended based on factors that it is programmed to account for, such as popularity of content, characteristics of a user, or predicted engagement.

Ofcom’s Illegal Harms Consultation: mitigating the risk of harms The Act requires Ofcom to produce Codes of Practice setting out the measures that in-scope services may take to comply with their duties under the Act.5 The Codes will recommend proportionate systems and processes across a number of areas, including: content moderation, governance, and user complaints. While services are not required to implement all measures in our Codes of Practice, in the event that they choose not to take the steps recommended, they will need to be able to explain how their chosen approach allows them to be compliant with their legal duties. We published our Illegal Content Codes of Practice in draft form alongside our Illegal Harms Consultation. The proposed measures in our Codes of Practice would require services to, among other things:
• have a named person, who is accountable to the most senior governance body, for compliance with illegal content safety duties, and reporting and complaints duties; 6
• have in place effective and easy-to-find content reporting and complaint mechanisms, so that users who encounter illegal content (including illegal suicide and self-harm content) can report it and see action taken;
• in the case of medium or high-risk7 services that use algorithms to recommend content to users, measure the risk that changes to algorithms increase the likelihood of users’ exposure to illegal content (including illegal suicide and self-harm content);8
• in the case of user-to-user services: have in place content moderation systems or processes that are designed to take down known illegal content (including illegal suicide and self-harm content) swiftly; and 9
• in the case of search services: have systems and processes in place that are designed so that search content that is illegal content is deprioritised or deindexed for UK users.10 In addition, our draft Codes of Practice for illegal harms include a proposal that search services should provide crisis prevention information in response to search requests that contain general queries regarding suicide and queries seeking specific, practical or instructive information regarding suicide methods. This information should include a helpline and links to freely available supportive information provided by a reputable mental health or suicide prevention organisation. It should also be prominently displayed to users in the search results. Once the final Illegal Harms Codes of Practice are approved by Parliament, regulated services such as Quora will need to take appropriate steps to keep their users safe from illegal harms, including using the measures contained in our final Codes to protect users from illegal suicide content, or other steps that ensure compliance with the duties. Ofcom’s Illegal Harms Consultation: Illegal Content Judgements Guidance Our Illegal Harms Consultation includes a draft version of Ofcom’s Illegal Content Judgements Guidance.7 This document provides guidance to in-scope services on how they may identify illegal content (content which may be reasonably inferred to amount to a relevant offence) including under Section 2 of the Suicide Act 1961.

6 Relevant to concern (7) 7 See draft Service Risk Assessment Guidance for definitions for medium and high risk services 8 Relevant to concern (4) 9 Relevant to concern (1), concern (4) and concern (7) 10 Relevant to concern (3)

In our guidance, we have proposed that, in certain contexts, the provision of practical information about how to take one’s life, and content inducing someone to enter into a ‘suicide pact’ are likely to be able to be inferred to be illegal content. Our guidance therefore suggests that content of this type should be removed from services in order for providers to be compliant with their illegal content safety duties. After Ofcom’s Illegal Harms Consultation and statement We expect to publish final versions of our illegal harms guidance, together with a statement setting out our response to issues raised by stakeholders, around the end of 2024. The Online Safety Act requires Ofcom to submit our Codes of Practice on illegal harms to the Secretary of State and to publish associated guidance within 18 months of Royal Assent. Once we issue our statement, services will have three months to undertake their illegal content risk assessments. At this point we will also submit the Codes of Practice to the Secretary of State, which, subject to their approval, are to be laid in Parliament for 40 days. Following approval by Parliament, the Codes will come into force 21 days after they have been issued. Assuming that Parliament immediately approves the Codes, these duties will become enforceable in early 2025. Phase two Ofcom’s Protection of Children Consultation: assessing access All user-to-user and search services are required to carry out children’s access assessments. A children’s access assessment is a process for establishing whether a service is ‘likely to be accessed by children’ within the meaning of the Act.11 As part of our Protection of Children Consultation we published draft Children’s Access Assessments Guidance which explains to services what their duties are in relation to children’s access assessments and how they can carry out this process. Services that conclude they are likely to be accessed by children will be in scope of the children’s safety duties in the Act. This means that they must:
• carry out children’s risk assessments (for more details, see draft Children’s Risk Assessment Guidance); and
• take steps to comply with the relevant safety duties protecting children (for more details, see Volume 5 of our consultation). We anticipate that most services not using highly effective age assurance12 are likely to be accessed by children within the meaning of the Act. We intend to publish our final Children’s Access Assessments Guidance in early 2025. Regulated services, such as Quora, will then have three months to complete children’s access assessments. If they conclude they are likely to be accessed by children, they will need to carry out a children’s risk assessment. If a service concludes they are not likely to be accessed by children, then they must carry out a new children’s access assessment within a year.13

11 Section 37 of the Act explains the meaning of “likely to be accessed by children”. 12 We explain what we mean by highly effective age assurance in our draft guidance for Part 5 services. 13 Services may also have to undertake a new children’s access assessment if they make a significant change to the service’s operation or design; if evidence shows a reduced effectiveness in a form of age assurance; or if evidence shows a significant increase in the number of children using the service.

Ofcom’s Protection of Children Consultation: assessing risks Services that conclude they are likely to be accessed by children are required to carry out children’s risk assessments. Our Protection of Children statement, including final Children’s Risk Assessment Guidance, will be published in spring 2025. By this point, services will have had three months to complete their first children’s access assessment and determine whether they need to comply with the children’s safety duties. The Act requires Ofcom to produce a Register of Risks for the Children’s Safety Codes, and guidance to assist services in conducting their own children’s risk assessment. Our draft Children’s Risk Assessment Guidance sets out a four-step risk assessment process which we propose as the best way to ensure that services meet their risk assessment obligations. This is the same process we propose for services’ illegal harms risk assessments. We are also consulting on Children’s Risk Profiles, which set out the risk factors that we consider are associated with an increased risk of harm on services, based on our children’s Register of Risk. Services will be required to take account of our Risk Profiles when conducting their risk assessments. This approach is also consistent with our illegal harms recommendations. For suicide and self-harm content, we set out risk factors relating to service type, user base, functionalities and recommender systems, and business model and commercial profile. We explain that certain functionalities, including notifications and alerts, are likely to encourage children to return to a service. This can increase the risk that children will encounter harmful content, including suicide and self-harm content. Services should consider notifications and alerts as risk factors when carrying out their risk assessment.14 The Act also requires Ofcom to provide guidance for providers of user-to-user and search services which gives examples of content, or kinds of content, that Ofcom considers to be, or considers not be, harmful to children (‘Guidance on Content Harmful to Children’). This is intended to support service providers that may need to make judgements about whether content on their service amounts to content that is harmful to children as defined in the Act. In our draft Guidance on Content Harmful to Children, we have proposed examples of kinds of content that Ofcom considers to be suicide content that is harmful to children. This includes content describing methods of suicide; content containing detailed instructions for methods of suicide; content, such as internet challenges or dares, instructing suicide; content featuring real-life suicides or suicide attempts. Once services have completed the process set out in the Children’s Risk Assessment Guidance, they should have an adequate understanding of the risks to children that arise on their service and implement measures to manage and mitigate those risks. If regulated services such as Quora conclude from their children’s access assessment that they are likely to be accessed by children, then they will need to carry out a children’s risk assessment. Following the outcome of the risk assessment, they will need to take steps to mitigate the risks of harm they have identified on their service. Ofcom’s Protection of Children Consultation: mitigating the risk of harm The Act requires Ofcom to produce Codes of Practice setting out the measures that in-scope services may take to comply with their duties under the Act. We have set out in our draft Children’s Safety Codes the measures that services can take to comply with the Children’s safety duties. The draft

14 Relevant to concern (5)

Children’s Safety Codes recommend proportionate systems and processes across a number of areas, including: age assurance; recommender systems; content moderation; governance and accountability; user tools and user reporting and complaints. We propose that services should adopt the measures recommended in the draft Children’s Safety Codes to mitigate the risks they have identified in their Children’s Risk Assessment – the combination of measures for each service would depend on the risks they have identified and on the size of the service. Services which are required to comply with the safety duties protecting children will be under a duty to operate a service using proportionate systems and processes designed to prevent children of any age from encountering Primary Priority Content. The proposed measures in our draft Children’s Safety Codes include:
• Implementing highly effective age-checks on user-to-user services to prevent children from seeing such content. This would apply to services whose principal purpose is the hosting or dissemination of content harmful to children (including suicide content); services that do not prohibit such content for all users; and services at higher risk of such harmful content being shared on the service.15
• Configuring algorithms to filter out the most harmful content (including suicide content) from children’s feeds. 16
• For user-to-user services, having content moderation systems and processes that ensure swift action is taken against content harmful to children (including suicide content) to either take down that content or take steps to prevent children from encountering it.17
• For search services, having appropriate moderation systems and, where large search services believe a user to be a child, a ‘safe search’ setting which children should not be able to turn off should filter out the most harmful content (including suicide content).18
• Having a named person as accountable for compliance with the safety duties protecting children; an annual senior-body review of all risk management activities relating to children’s online safety; and an employee Code of Conduct that sets standards for employees around protecting children online.19
• Having clear and accessible information for children and carers, with easy-to-use reporting and complaints processes, and giving children tools and support to help them stay safe.
• For search services, providing crisis prevention information in response to search requests regarding suicide and self-harm. This information should be prominently displayed so that it is the first information users encounter in search results. It should include links to freely available, supportive information and helplines, provided by reputable mental health or suicide charities that hold relevant and accessible materials that are comprehensible and suitable in tone to all users, including children, in the UK.
• For user-to-user services, signposting children who encounter relevant kinds of content harmful to children (including suicide and self-harm content) to appropriate support resources at key points in the user journey.20

15 Relevant to concern (1) and concern (3) 16 Relevant to concern (4) 17 Relevant to concern (1), concern (4) and concern (7) 18 Relevant to concern (3) 19 Relevant to concern (7) 20 Relevant to concern (6)

Once the final Children’s Safety Codes are passed by Parliament, regulated services such as Quora will need to take appropriate steps to keep children safe from content harmful to children, including using the measures contained in our final Codes to protect children from suicide content. If they decide not to follow the Children’s Safety Codes, they will need to ensure that their alternative approach is still compliant with their legal duties. After Ofcom’s Protection of Children Consultation and Statement The Act requires Ofcom to submit our Children’s Safety Codes to the Secretary of State and to publish associated guidance within 18 months of Royal Assent which was received on 26 October
2023. We intend to publish a final version of our Children’s Access Assessments guidance early in 2025. Services will then have three months to carry out children’s access assessments. Providers who have completed children’s access assessments and determined that their services are ‘likely to be accessed by children’ within the meaning of the Act will have three months from completion of a children’s access assessment to carry out children’s risk assessments. At this point we will also submit the Children’s Safety Codes to the Secretary of State, which, subject to their approval, are to be laid in Parliament for 40 days. By the end of April 2025, we will publish a statement, setting out our response to issues raised by stakeholders during the Protection of Children Consultation, our final policy decisions, and final versions of guidance to services and the Children’s Safety Codes. Like the illegal harms Codes of Practice, the Children’s Safety Codes will come into force 21 days after they have been issued following approval by Parliament.

Phase three As noted above, Phase three focuses on additional duties for categorised services, including transparency, and other duties such as user empowerment which will apply only to Category 1 services.21 Duties for Category 1 services include a duty to include, to the extent that it is proportionate to do so, features which adult users may use or apply if they wish to increase their control over certain kinds of content including suicide and self-harm content. Services will be categorised based on thresholds, including for user numbers and functionalities. The Secretary of State is currently considering where these thresholds should be and, following this, Ofcom will begin the process of establishing the register of categorised services. Once we publish this register, a certain number of user-to-user services will be formally categorised as Category 1 or 2B, and will be therefore be subject to a number of additional duties, depending on which category they fall within. Conclusion The death of Isabella highlights the impact that suicide content can have on children and their families. Ofcom is committed to holding services to account for addressing the risk of harm from

21 ‘Category 1’ refers to certain user-to-user services categorised based on user numbers and functionalities. Services in this category are subject to additional duties related to transparency, user empowerment and protection of democratic and journalistic content. ‘Category 1 threshold conditions’ are set the Secretary of State, with advice provided by Ofcom. Ofcom will then be responsible for designating services into categories according to these thresholds.

such content effectively and proportionately. As such, we will contact Quora to request their response to this report and will seek to engage with them further on this matter. One of the aims of the Act is to secure a higher level of protection for children, and we want to see wider deployment and improvements in services’ measures to address areas which pose the greatest risk to children in the UK. We are committed to working with industry to ensure compliance with these duties, and will work directly with services to promote compliance, including – where appropriate – through targeted supervision. Where we identify non-compliance, we can investigate and may take enforcement action where appropriate to protect all users, especially children from harm. We will have the powers to require non-compliant service providers to take steps to mitigate the risk of harm and to impose financial penalties, as well as powers to seek business disruption measures in the most serious cases.22 We will also continue to seek input and engagement not only with academic and industry experts but with children and parents, including those with lived experience of online harms. Evidence included in reports from coroners and other experts will play an important role as we implement the regime, and we will of course take the evidence in your report into account as we continue our policy development. We hope that this response provides helpful information about the significant steps Ofcom is taking as we continue to work through the implementation of the Act. If further information or clarification is required, we would be happy to provide this.

Dear Ms Hallet, Thank you for the opportunity to respond to this Report to Prevent Future Deaths, regarding the tragic death of Isabella Charlotte Shere. Following the change of government, I am considering the issues you have raised in your report and I am grateful to you for bringing them to my attention. I have taken advice from my officials in preparing this response and wanted to reply promptly to avoid further delay. First and foremost, I would like to extend my deepest condolences to the family and friends of Isabella. I have recently met with bereaved families who have had similar experiences and I will consider your report carefully when deciding how the government moves forward in ensuring greater online safety. I am responding in place of the Department for Culture, Media and Sport, given my department’s responsibility for the online safety policy. The Online Safey Act received Royal Assent on 26 October 2023 and will place duties on tech companies to protect their users online, especially children. The Online Safety Act regime will be overseen and enforced by Ofcom, the independent regulator. I will address the concerns you have listed (referred to as (1)-(7) in your report) in my response, as per my department’s remit, and having engaged with the Ministry of Justice on the relevant offences which fall within their remit. Under section 2(1) of the Suicide Act 1961 (as amended by section 59 of the Coroners and Justice Act 2009) it is an offence for a person to do an act capable of encouraging or assisting the suicide or attempted suicide of another person, with the intention that their act will encourage or assist suicide or an attempt at suicide. The person committing the offence need not know the other person or even be able to identify them. An offence may be committed whether or not a suicide or attempted suicide takes place. Simply making available, or facilitating the provision of, resources which provide information detailing methods of suicide, where there is no intention to encourage or assist suicide, is not an offence under the Suicide Act. The criminal offence has a high threshold to avoid criminalising people who are expressing suicidal feelings and those offering them support, by for example sharing their own experiences. I will now turn to the provisions in the Online Safety Act. The Act introduced a new communications offence of encouraging or assisting serious self-harm (akin to the offence under the Suicide Act but covering non-fatal self-harm) and a regulatory framework to address online

content harmful to children, such as the type of content that Isabella accessed shortly before her death. As the material accessed related to suicide, it would likely be captured under the Act as either 'primary priority content' (PPC) harmful to children or ‘priority illegal content’. PPC is defined under Section 61 of the Act, and includes content which encourages, promotes or provides instructions for suicide. Priority illegal content, defined under Section 59 of the Act, includes content which amounts to a priority offence, such as an offence under Section 2(1) of the Suicide Act. The Online Safety Act imposes duties relating to both types of content on in- scope ‘user-to-user’ services. These are services which allow users to post material online, or to interact with each other. The Act also imposes the similar duties on search services. Under the duties protecting children, all user-to-user services and search services that are likely to be accessed by children must assess the risk of any content harmful to children (including primary priority content) being encountered on their service. User-to-user services – such as Quora – then have a duty to put in place systems and processes to prevent children from encountering user-generated content which is PPC, while search services must minimise the risk of children encountering this content in search results or on the pages they land on when they click on them. The Act states that risk assessments must consider the functionalities of the service. Ofcom has proposed in its draft Children’s Safety Codes that functionalities such as alerts and notifications or those that allow users to react to content are specific risk factors that should be considered. Your report highlighted similar functionalities on the Quora website as matters of concern. Ofcom published the draft Children’s Safety Codes on 8 May. The Codes include proposed recommended measures that regulated services put in place to fulfil their safety duties. For user- to-user services, these measures include:
• the use of highly effective age assurance to prevent children seeing PPC which has been identified on the service, such as content encouraging, promoting or giving instructions for suicide
• complaints and reporting systems that are easy to access and use, enabling people to make user reports about content which might be harmful to children
• well-resourced content moderation systems or processes designed to swiftly take action against identified content harmful to children, for example by filtering identified PPC content so that it is only visible to confirmed adult users
• measures for recommender systems and algorithms that ensure content likely to be PPC is not recommended to children, preventing them from encountering PPC through notification systems or being recommended similar material in the manner described in your report. This ensures that content likely to the PPC is reduced in prominence on children’s recommender feeds and enables children to provide negative feedback on content that is recommended to them. Ofcom’s draft measures also include proposals for search services to protect children from encountering harmful content through moderation, in line with their duties under the Act, and mitigate the impact of such content. These measures include: downranking or blurring of PPC where a user is believed to be a child; filtering identified PPC out of their search results; and provision of crisis prevention information in response to known PPC-related search requests regarding suicide. Furthermore, the Act’s illegal content safety duties require user-to-user services to have systems and processes in place to proactively prevent users from encountering priority illegal content via their service and to minimise the length of time for which such content is present. Ofcom has

also produced a draft Illegal Content Codes of Practice for user-to-user services and search services. There will be sites and services that choose not to comply with the Act’s regulatory framework. In these instances, Ofcom has a suite of robust enforcement powers to support its regulatory functions. These extend to instances where companies are based overseas but have a significant number of UK users or the UK as a target market. Ofcom’s powers include the ability to apply to the court for business disruption measures. These are court orders that require third parties (including those who provide an access facility to such services, such as internet service providers, or ancillary services such as payment providers) to prevent, restrict or deter access to non-compliant services in extreme circumstances.

Part of our Government’s mission is to reduce the lives lost to suicide, and the Online Safety Act will help with this aim. My department is working closely with Ofcom to ensure that the Online Safety Act is implemented as quickly as possible so that children are protected from encountering harmful content online, especially at their most vulnerable moments, as happened with Miss Shere.