Source · Select Committees · Public Accounts Committee
Recommendation 37
37
Accepted
Public Law Project warns of bias and transparency risks in DWP's machine learning for fraud.
Recommendation
However, written evidence we received from the Public Law Project raised a series of risks around DWP’s use of machine learning to tackle fraud. In particular, it noted the risk of machine learning taking on human biases when it was trained on historical data and the potential for widescale detrimental impacts on claimants if there was a system error. It said that DWP needed to be more transparent to address the issue of public confidence in its machine learning techniques.73 67 Q 46 68 Qq 46, 57 69 C&AG’s Report (on accounts), paras 32, 34 70 Committee of Public Accounts, The Department for Work & Pensions Annual Report and Accounts 2022–23, Fourth Report of Session 2023–24, HC 290, 6 December 2023; Committee of Public Accounts, The Department for Work and Pensions’ Accounts 2021–22 – Fraud and error in the benefits system, Twenty-Sixth Report of Session 2022–23, HC 44, 9 November 2022 71 Committee of Public Accounts, The Department for Work & Pensions Annual Report and Accounts 2022–23, Fourth Report of Session 2023–24, HC 290, 6 December 2023 72 C&AG’s Report (on accounts), para 34 73 DCSA0003 17
Government Response Summary
The government accepts the recommendation, committing to developing a new publishable form of fairness analysis assessment for machine learning by Summer 2025, with improved governance and independent oversight to address transparency concerns.
Government Response
Accepted
HM Government
Accepted
7.1 The government agrees with the Committee’s recommendation. Target implementation date: Summer 2025 7.2 The department will support an in-confidence session with the Committee and departmental officials to set out the 2024 fairness analysis assessment. 7.3 Moving forward, the department has made a commitment at the Work and Pensions Select Committee on 29 January 2025 (Q18) to develop a new publishable form of fairness analysis assessment. 7.4 Across the public sector, this department is at the forefront of producing fairness analysis such as these. There is no set government standard for fairness analysis, nor any best practice examples that the department could identify, therefore it has had to adopt a test and learn approach to fairness analysis. The fairness analysis method has been endorsed by statistical experts. 7.5 At every stage of machine learning development, the department ensures checks and balances are in place and have safeguards to minimise the risk of unfair treatment or detrimental impact on legitimate claimants. 7.6 The department has reflected on how it can assure Parliament and the public of its processes and have committed to a new approach to fairness analysis of machine learning models designed to tackle fraud. To introduce additional independence and scrutiny into the process, the department will: • Improve upon existing governance around the assessment of the fairness analysis to determine whether each model is effective and remains reasonable and proportionate. • Assure both the statistical analysis and the assessment will be overseen by a team independent of those running the machine learning models, with reference back to the appropriate internal governance board when issues are discovered that require action. • Draft fairness analysis assessments in such a way that they can be published unredacted, setting out the rationale for why the department assesses the models to be reasonable and proportionate but without divulging the detail of its fraud and error controls that would put the security and integrity of the social security system at risk from fraud. 7.7 The aim of this new approach is to provide the Committee, Work and Pensions Select Committee and the wider public with assurance on the department’s fairness analysis.