Source · Select Committees · Public Accounts Committee
Recommendation 7
7
Accepted
Share fairness impact assessment results for machine learning to reassure against unfair claimant treatment.
Conclusion
We remain concerned about the potential negative impact on protected groups and vulnerable customers of DWP’s use of machine learning to identify potential fraud. The previous Public Accounts Committee repeatedly raised concerns about the impact of data analytics and machine learning on legitimate benefit claims being delayed or reduced, the number of people affected, and whether this is affecting specific groups of people. Written evidence from the Public Law Project raises a series of risks around DWP’s use of machine learning to tackle fraud. In particular, it notes the risk of machine learning taking on human biases when it is trained on historical data and the potential for wide–scale detrimental impacts on claimants if there is a system–error. DWP has undertaken a fairness impact assessment on its use of data analytics, which it says raised no concerns about the impact on customers. However, it has not published the results because it says it wants to avoid releasing information that could assist fraudsters. recommendation DWP should share with us – in confidence if necessary – the results of its 2024 fairness impact assessment in order to provide reassurance that its use of machine learning is not resulting in claimants being treated unfairly. 6 1 Customer service Introduction
Government Response Summary
The government agrees to share the 2024 fairness impact assessment and commits to developing a new, publishable form of fairness analysis assessment by January 2025. This new approach will involve improved governance, independent oversight, and assessments drafted for unredacted publication to enhance transparency.
Government Response
Accepted
HM Government
Accepted
The government agrees with the Committee’s recommendation. departmental officials to set out the 2024 fairness analysis assessment. Moving forward, the department has made a commitment at the Work and Pensions Select Committee on 29 January 2025 (Q18) to develop a new publishable form of fairness analysis assessment. Across the public sector, this department is at the forefront of producing fairness analysis such as these. There is no set government standard for fairness analysis, nor any best practice examples that the department could identify, therefore it has had to adopt a test and learn approach to fairness analysis. The fairness analysis method has been endorsed by statistical experts. At every stage of machine learning development, the department ensures checks and balances are in place and have safeguards to minimise the risk of unfair treatment or detrimental impact on legitimate claimants. The department has reflected on how it can assure Parliament and the public of its processes and have committed to a new approach to fairness analysis of machine learning models designed to tackle fraud. To introduce additional independence and scrutiny into the process, the department will: • Improve upon existing governance around the assessment of the fairness analysis to determine whether each model is effective and remains reasonable and proportionate. • Assure both the statistical analysis and the assessment will be overseen by a team independent of those running the machine learning models, with reference back to the appropriate internal governance board when issues are discovered that require action. • Draft fairness analysis assessments in such a way that they can be published unredacted, setting out the rationale for why the department assesses the models to be reasonable and proportionate but without divulging the detail of its fraud and error controls that would put the security and integrity of the social security system at risk from fraud. The aim of this new approach is to provide the Committee, Work and Pensions Select Committee and the wider public with assurance on the department’s fairness analysis.