4th Report - Human Rights and the Regulation of AI

AI supply chains are complex and opaque, involving multiple actors. This makes it difficult to identify the source of human rights risks, and the actors responsible for identifying and addressing those risks. (Conclusion, Paragraph 20) Human rights issues raised by AI

The Council of Europe Framework Convention on AI provides a strong basis for both effective regulation of AI within the UK, and collective international action. We urge the government to seize the opportunity to be an early adopter of the Framework Convention. (Conclusion, Paragraph 26)

The UK should set out a timeline for ratification of the Council of Europe Framework Convention detailing what steps will be taken and when. This plan for ratification should be subject to public consultation. (Recommendation, Paragraph 27)

AI systems are prone to producing unfairly discriminatory outcomes, which may arise from bias in the datasets on which they are trained as well as from the way they are developed and deployed. This threatens the human rights of people in the UK and elsewhere. (Conclusion, Paragraph 38)

AI systems pose particular risks to privacy and personal data, because of the prevalence of web scraping to collect personal data to train large language models (LLMs), and the potential for AI systems to be used in highly intrusive ways. This threatens the right to privacy of people in the UK and elsewhere. (Conclusion, Paragraph 45)

AI systems pose particular risks to the right to an effective remedy in circumstances where there is insufficient transparency concerning the use of AI systems to support significant decisions about affected persons, and in 75 circumstances where it is difficult for those persons to access mechanisms to challenge those decisions. This threatens the human rights of people in the UK. (Conclusion, Paragraph 49)

Our inquiry has not focused on intellectual property rights, or the rights of performers. The issues which witnesses have raised clearly merit further investigation, and could require specific measures going beyond the scope of this inquiry. It is important for the UK to protect its rich creative industries and those who contribute to them, and to safeguard performers’ human rights as well as their livelihoods. (Conclusion, Paragraph 53)

Our inquiry did not focus on the potential effects of AI on the justice system, but witnesses raised some very concerning issues which merit detailed consideration, and might require specific measures going beyond the scope of this inquiry. The system is already under great strain. We note the Justice Select Committee has also recently discussed with ministers the use of AI in connection with the legal system, as part of its ongoing inquiry into Access to Justice. (Conclusion, Paragraph 55)

We welcome the government’s stated commitment to ensuring that in the UK, development and use of AI systems is grounded in human rights. AI has the potential to enhance the protection of human rights, but can also give rise to significant interferences with those rights. (Conclusion, Paragraph 60)

Human rights principles must inform all aspects of the government’s policies and practices relating to AI systems, including its approach to the regulation of AI. (Recommendation, Paragraph 61)

Government should encourage the development of AI use cases which can demonstrably improve human rights. (Recommendation, Paragraph 62) Legal and policy framework

The AI Action Plan commissioned by the previous administration, while it brings a welcome focus on innovation and economic opportunity, fails to pay sufficient attention to the need to secure the protection of human rights from AI-generated interference throughout the supply chain. (Conclusion, Paragraph 89)

The UK’s leading position as the third largest AI market in the world affords opportunities, not only for economic growth but also for setting strong standards on the responsible adoption and development of new technologies. (Conclusion, Paragraph 90) 76

The Committee is encouraged by the prominence given to AI in the new administration and urges the government to seize the opportunity to regulate AI in a way which puts human rights first. (Recommendation, Paragraph 91) Problems with existing protections

Taken together, existing UK laws that could apply to AI systems do so primarily at the point of deployment, so that deployers are the primary holders of responsibility for harms arising from AI systems. This means that existing laws do not take due account of the complexity of the AI lifecycle and supply chain and are not directed to the actor that may be best placed to identify and avert the source of risks. They are therefore likely to allow human rights harm that could have been prevented. (Conclusion, Paragraph 104)

Regulators lack the power to test and evaluate AI systems before they are publicly released, or to prevent their release if they are considered to pose unacceptable risks, except in very limited circumstances. (Conclusion, Paragraph 108)

Model developers engage with the AISI on a voluntary basis. The AISI has no statutory power. (Conclusion, Paragraph 109)

No cross-economy regulators, agencies or bodies have systematic powers to prevent AI models or systems being deployed even if they pose significant risks. (Conclusion, Paragraph 110)

Existing law is deficient by failing to impose legal requirements on AI system deployers to be transparent about their use of AI systems. (Conclusion, Paragraph 118)

The current legal framework does not provide for adequate transparency in relation to the development and deployment of AI systems. (Conclusion, Paragraph 130)

Existing law does not require the provision of notice to individuals that an AI system is in use. In the absence of such knowledge, individuals who have been directly affected by the use of AI systems will be in practice unable to challenge such use or seek an effective remedy for human rights violations. The evidence we heard strongly suggests that many people may be in this situation. (Conclusion, Paragraph 131) 77

There are no mandatory obligations on AI developers and providers to provide downstream actors, including deployers, with information concerning the potential risks arising from the deployment of AI systems. This can reduce transparency and traceability across the AI supply chain. (Conclusion, Paragraph 132)

The ATRS provides some transparency concerning the use of AI systems in the public sector, but is limited in its coverage. (Conclusion, Paragraph 133)

The recent OpenAI/Hugging Face security incident starkly highlights the societal risks posed by the growing power of AI systems. It illustrates how the newest AI models can engage in activities that would be unlawful if undertaken by a legal person, with the potential to seriously undermine security, including in critical infrastructure. It also suggests that at present society largely relies on private companies to take voluntary action to guard against such incidents. This is unsatisfactory. (Conclusion, Paragraph 143)

The general-purpose nature of the foundation models which allows them to be embedded into a wide and diverse range of AI systems and contexts means that any vulnerabilities and sources of risk within the model can propagate very rapidly and at scale once deployed. The power of the newest models, and their potential for mass harm, justifies a precautionary approach to regulation. (Conclusion, Paragraph 144)

Existing law is inadequate to enable those who suffer human rights harm generated by AI systems to secure an effective remedy. (Conclusion, Paragraph 145)

Even if some existing laws could address some human rights risks posed by AI systems and provide redress to individuals in specific circumstances, their application is often speculative and uncertain, and the cost of individual litigation is in practice prohibitive. (Conclusion, Paragraph 146)

The current legal framework that applies to AI systems is fragmented and difficult to navigate. It relies primarily on harm-specific and sector-specific legislation leaving gaps in protection which fails to address preventable human rights risks. (Conclusion, Paragraph 155)

Current product liability law provides strict liability for AI systems incorporated into physical products, but not in relation to stand-alone AI systems. This difference in legal liability is difficult to justify in principle. The government must carefully consider any recommendations from the ongoing Law Commission review, including on this specific point. (Conclusion, Paragraph 156)

Existing regulators cannot effectively address human rights harms arising from AI systems using their present powers and resources. (Conclusion, Paragraph 162) 78

Human rights risks produced by AI systems cannot be adequately addressed by relying on individual litigants to bring proceedings. (Conclusion, Paragraph 163)

To address the global nature of AI harms, and protect human rights, governments need to collaborate to identify risks and promote consistent standards. (Conclusion, Paragraph 169)

The UK can be justifiably proud of its record of leadership on AI. This has been maintained across administrations from different sides of the political debate. (Conclusion, Paragraph 170)

An appropriate regulatory regime would not only enhance human rights protections but also help to reinforce the UK’s leadership position in AI. (Conclusion, Paragraph 171)

The UK should continue to play a leading role in international AI summits and seek to be party to international declarations and agreements on AI safety. In particular, the government should use its G20 presidency and other available global leadership opportunities to make the case for enabling or building upon international mechanisms to address AI harms and intervene early where necessary in the development of foundational models. (Recommendation, Paragraph 172) How risks to human rights should be addressed

The UK needs an AI-specific legal framework that applies to all sectors of society and the economy, which comprehensively addresses AI-generated risks and harm across the AI lifecycle to address AI’s unique challenges. To ensure that the UK remains at the forefront of regulation, this framework must be proportionate. (Conclusion, Paragraph 177)

The government should introduce a new AI Bill, as promised in the 2024 King’s Speech, to place AI regulation on a sound basis and give full effect to the Framework Convention. (Recommendation, Paragraph 178)

A risk-based approach is needed to address AI harms. The Hugging Face incident illustrates the significant risks that can arise from the newest and most advanced AI models, and legislation should address those risks on a precautionary basis with a view to avoiding harm, by analogy with the regulation of other potentially harmful products such as pharmaceuticals. It should only impose legal obligations where they are necessary and proportionate to prevent unacceptable risks and ensure effective redress for harm. Low risk AI systems should be subject to fewer 79 and less demanding legal requirements. Legislation should not place unjustified burdens on businesses, especially SMEs, to enable UK companies to innovate and for the UK to benefit from the advantages AI can confer. (Conclusion, Paragraph 186)

To help ensure that regulation can keep pace with technological developments, it is preferable for legislation to take the form of broadly framed principles, with more detailed guidance provided by codes of practice. (Conclusion, Paragraph 187)

The AI Bill we recommend should provide for a regulatory regime that would classify different risk levels, and mandate proportionately more demanding obligations for higher-risk AI systems and AI models. (Recommendation, Paragraph 188)

AI supply chains are complex. Actors which have the greatest capacity to prevent harm are often upstream developers. However at present it is too easy for the risks to be passed down the chain to those who are not in a position to identify and address the source of those risks located upstream. (Conclusion, Paragraph 193)

Many AI systems developed for a specific purpose can be deployed or adapted for use for different purposes and contexts from those intended by the AI developer. Those deploying the system may lack understanding of the risks arising from the way in which the AI system has been designed and developed, and how its deployment could adversely impact human rights. (Conclusion, Paragraph 194)

The AI Bill we recommend should impose proportionate obligations on actors at all stages of the AI lifecycle and supply chain to ensure that risks are identified, evaluated and effectively addressed at the appropriate stage. (Recommendation, Paragraph 195)

Some uses of AI should be prohibited, because they are incompatible with human rights. (Conclusion, Paragraph 199)

The AI Bill we recommend should prohibit the use of AI in ways that are inconsistent with respect for human rights, such as subliminal techniques, emotional inference or inappropriate use of profiling or biometric data. Because of the very serious human rights risks, this would mean that the development and provision of very powerful AI systems (such as Artificial General Intelligence and Artificial Superintelligence) which risk causing widespread and very serious harm, including the capacity to evade effective human control, would be prohibited. (Recommendation, Paragraph 200)

The detail of such prohibitions should be established following public consultation. (Recommendation, Paragraph 201) 80

The AI Bill we recommend should include a requirement that AI systems posing a high risk of causing harm to human rights should need prior approval before they can be provided or deployed. (Recommendation, Paragraph 204)

The AI Bill we recommend should place requirements on actors at all stages of the AI supply chain to undertake due diligence obligations necessary to prevent unacceptable human rights risks when designing, developing or deploying high risk AI systems. (Recommendation, Paragraph 207)

We recommend that the due diligence requirements contained in the AI Bill should be differentiated, based on the role of the relevant actor along the AI supply chain, and the nature and seriousness of the risk posed by the relevant AI model or system. To identify the nature and scope of those due diligence obligations, the EU AI Act may provide a useful point of departure, but the provisions should be tailored to the requirements of the UK. (Recommendation, Paragraph 208)

The AI Bill we recommend should impose mandatory transparency requirements on all actors across the AI lifecycle. The deployment of AI systems which may have significant impacts on individuals, groups and communities should include an obligation to state when AI systems are being used and provide a full and comprehensible explanation of what it is being used for. There should also be an obligation to provide information about the source of data used by the AI system. (Recommendation, Paragraph 213)

Although data protection law provides for specific safeguards when automated decision-making is used, there is a need for greater clarity in the law to ensure that the safeguards operate effectively in practice, and that reliance on human involvement is not misplaced. (Conclusion, Paragraph 217)

The regulation-making powers in the UK GDPR should be used to ensure that there are robust protections against misuse of automated decision-making. The regulations should say that the mere presence of a ‘human in the loop’ is not enough to constitute meaningful human involvement or intervention. It should be necessary to show that the person is sufficiently informed and independent to be able to reach an objective view that has not been improperly influenced by the automated decision. The regulations should also say that safeguards include a requirement to provide information relating to the data subject’s individual circumstances rather than the general operation of the system, and that data subjects must be given enough information to allow them to mount an effective challenge against the decision. (Recommendation, Paragraph 218) 81

We agree with our witnesses that the present regulatory framework is fragmented and difficult to navigate. Urgent action is needed to close gaps in protection arising from the current framework. (Conclusion, Paragraph 224)

A single AI oversight body would be able to act as a single point of contact for raising concerns about use of AI, carry out oversight and monitoring of AI harms and risks, and maintain an adverse incident repository to help identify where more systemic issues may lie. We are confident this can be done without creating duplication. (Conclusion, Paragraph 225)

The government should establish an independent AI oversight body on a statutory footing, either as a new body or by expanding the powers of an existing body. The government should consult carefully with existing regulators to avoid unnecessary overlap and promote regulatory consistency and coherence. (Recommendation, Paragraph 226)

The new oversight body should: a. have the power to facilitate co-ordination with other national regulators in relation to monitoring, investigation and enforcement, knowledge sharing, and capacity building b. make regular reports on its activities to Parliament, identifying opportunities to fill emerging gaps in protection and highlighting trends and sectoral developments in best practice c. have a statutory duty to maintain a public repository of AI incidents and proactively identify patterns of reported harm and any emerging systemic threats, and to consider cases for further investigation d. have the power to publish mandatory codes of practice for the responsible use of AI in the private and public sector, and to sanction developers, deployers and other actors who do not fulfil their responsibilities under these codes of practice e. have the powers necessary to implement and enforce the provisions of the AI Bill we recommend, including: i. to carry out testing and evaluation of AI systems, including powers to implement a prior testing, auditing and evaluation regime for high-risk uses and applications of AI ii. to prohibit AI models or systems from being launched or deployed, and to order their withdrawal from the market if unacceptable risks to human rights are identified 82 iii. to issue codes of practice which would provide greater specificity concerning the due diligence obligations which must be complied with (for example, requiring high risk AI providers to establish and maintain a risk management system, to ensure that practical and acceptable complaints mechanisms are in place and so forth), and to receive reports if certain types of risk are identified iv. to conduct investigations of allegations concerning breaches of the statutory prohibitions and mandatory codes of conduct, including allegations that AI systems have caused harm to human rights v. to sanction actors who fail to comply with mandatory codes of practice vi. to order that remedies be provided as appropriate in individual cases without requiring the affected individuals to incur prohibitive costs vii. to prescribe transparency requirements through its codes of practice, identifying the information actors at each stage of the AI lifecycle must disclose and to whom, including other actors along the AI lifecycle and public sector AI deployers viii. where actors in the AI supply chain are located outside the UK, to be able to assess the risks of deploying their systems in the UK, having regard to how those actors are regulated in their home state. If the new AI oversight body considers that deployment in the UK would give rise to unacceptable risks which could not be addressed within the UK, the oversight body should be able to limit or prohibit such deployment. (Recommendation, Paragraph 227)

The government should place AISI on a statutory basis. AISI should have the power to review new and revised powerful AI models. It should also have a duty to assess, publish findings pre-release, and issue preliminary ‘warning’ statements associated with release of new AI foundation models. To avoid duplication, the government should consider combining the existing functions of the AISI with those of the new regulator. (Recommendation, Paragraph 231)

Developers of powerful AI models must be required to submit new models (and new versions of existing models) to AISI for review, evaluation, and testing. They must also provide technical specifications, including model properties, training data and process, intended use, security and safety testing and risk control measures. (Recommendation, Paragraph 232) 83

The government must give the AISI a statutory duty to inform the authorities, including the new AI oversight body, of potential risks posed by AI models and systems. This should include a duty to advise the oversight body of the need to prevent AI systems from being launched or deployed, or to order them to be withdrawn from the market, if AISI identifies significant risks to human rights. (Recommendation, Paragraph 233) 84