Select committee: Public Administration and Constitutional Affairs Committee

Transforming the UK’s Evidence Base

Closed Opened 14 Nov 2023 Closed 24 May 2024 15 recommendations 21 conclusions
Inquiry scope (the committee's words)
This inquiry is exploring how officials produce statistics and analysis, how demands for data are changing, and whether the privacy of citizens is being adequately protected as new and innovative sources become available.

Where it stands

The inquiry is closed, opened 14 November 2023.

The committee has published 1 report, most recently Fifth Report - Transforming the UK’s Evidence Base on 24 May 2024.

The government's response

Published 6 March 2025. The deadline was 24 July 2024.

Summary written by Accountability Index from the government's response. It is not the government's own words.

The Government accepts some of the Committee's recommendations addressed to it, rejects others, and notes two on funding, while referring the remaining recommendations to the UK Statistics Authority.

  • On a new data-sharing programme (recommendation 1), the Government "rejects the Committee's recommendation", saying it is "developing a more comprehensive approach", including a National Data Library, with funding implications "considered in the round" at Spending Review 2025.
  • It rejects the recommendation to publish evidence and data underpinning major policy announcements (recommendation 14), stating that such decisions lie with departments, which weigh transparency against factors such as public safety and commercial sensitivity.
  • It accepts the recommendation to review health and social care data in the Digital Economy Act 2017, saying it will implement this "by the end of 2025", and states it has "now implemented" the recommendation on training communications staff in Intelligent Transparency.
  • On future funding of the analysis function and the Office for Statistics Regulation (recommendations 10 and 12), the Government "notes" them, and HM Treasury "will consider all UKSA funding arrangements as part of Spending Review 2025".

Read the government's response ↗ Report an error

Recommendations and conclusions 36 shown

1: UK public data landscape demonstrates strengths in independence, innovation, and regulatory framework.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
There is much to be proud of across the UK’s public data-landscape. The independence of its statisticians from the government of the day, the innovative work being undertaken by a skilled researcher community, and our unique regulatory framework all received praise from our witnesses.

Link to this item · Read this item

2: UK's public data landscape is fragmented, lacking effective collaboration between actors.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
As it stands, the UK’s public data landscape is highly fragmented. This need not be a problem (indeed the model offers several conceivable advantages) if the many actors involved in generating, analysing and communicating evidence work together effectively. Our inquiry suggests that in several areas, however, this is not yet the case.

Link to this item · Read this item

3: New data sources offer benefits but possess limitations, requiring integration with traditional data.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
It is clear that the volume and variety of data generated within the UK has exploded in scale over recent years. While new sources of data have much to offer - not least in improving the timeliness of evidence, and allowing us to delve into issues in greater detail - there are things they simply cannot do. They cannot tell us about citizen’s intentions, they cannot be easily tweaked to capture the information an analyst is seeking, and they are prone to bias. The UK’s public evidence base will be best served, therefore, by bringing together old sources and new.

Link to this item · Read this item

4: Government departments fail to share data due to lack of incentives and central coordination.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
Despite the passage of relevant legislation in 2017, the UK has failed to bring its disparate datasets together to enrich its public evidence base. Instead, data withers in silos across countless government bodies. Witnesses to our inquiry were clear; the problems here are not legislative, and they do not result from challenges around data protection. The issue is much simpler; departments and public bodies choose not to share data because they do not share an incentive to do so. And no central, coordinating leader has yet made a sufficiently strong case for the benefits of sharing data for statistics and research across government.

Link to this item · Read this item

5: Develop comprehensive programme with ONS to improve cross-government data-sharing for research and statistics.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
It is time for Government to do what it promised to do seven years ago, and to join up the UK’s evidence base. Given that the Cabinet Office’s existing initiatives for improving data sharing are self-evidently insufficient, it should in partnership with the Office for National Statistics develop a comprehensive new programme aimed at improving data-sharing for statistical and research purposes. The programme must clearly define deliverables and timelines, and must be owned by a senior responsible officer at an appropriately high level. In line with the recommendations of the Lievesley report, we also recommend that HM Treasury establish mechanisms so that the costs are not borne by individual Departments, but rather centrally. The Cabinet Office should prepare and publish an annual progress report on delivery against the programme.

Link to this item · Read this item

6: Publish annual ONS information on requested datasets, rationale, and request status online.

Recommendation · Fifth Report - Transforming the UK’s Evidence Base

Recommended (the committee's words)
Separately, the Office for National Statistics should publish information on the datasets it is seeking on an annual basis, setting out its rationale for seeking those data, and details on the status of the request - all of which should be made available on the ONS website.

Link to this item · Read this item

7: Review exclusion of health and social care data from Digital Economy Act 2017.

Recommendation · Fifth Report - Transforming the UK’s Evidence Base

Recommended (the committee's words)
While time in this Parliament now runs short, we recommend that the next Government review the exclusion of health and social care data from the Digital Economy Act 2017. There are understandable sensitivities around the sharing of health data, but it may be that the implementation of research provisions over the last seven years, and the work that has taken place in the context of the Caldicott review, offer a helpful new basis for discussion concerning their possible inclusion.

Link to this item · Read this item

8: ONS ambition to deliver high-quality population statistics and new data sources supported.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
We support the Office for National Statistics in its ambition to deliver high-quality and timely population statistics. It is right to be considering whether new data sources might offer opportunities to improve the UK’s evidence base, and it is also right to be engaging closely with users of that evidence base, as decisions are taken on the future of the decennial census.

Link to this item · Read this item

9: Undertake further ONS work on proposals for future migration and population statistics.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
This Committee’s view - particularly in light of challenges around data-access - is that officials have not yet demonstrated that they can deliver the evidence users need, without a decennial census. We therefore recommend that the Office for National Statistics undertake further work on proposals for the future of migration and population statistics.

Link to this item · Read this item

10: Absence of data prioritisation framework causes significant evidence gaps across government.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
Currently, the UK has no framework by which to identify, and then prioritise, demands for data and evidence. In the absence of such a framework, and with an ongoing need to have regard to budget constraints, Ministers and HM Treasury wield an inordinate amount of power in deciding what evidence the UK collects and communicates through the decisions they take on departmental budgets. In the context of these systemic shortcomings, significant data gaps have emerged; the UK lacks suitable evidence on the performance of its different health services, for example, and on key policy challenges like school absenteeism.

Link to this item · Read this item

11: Establish UK Statistics Authority framework for prioritising evidence demands and submit triennial findings to Parliament.

Recommendation · Fifth Report - Transforming the UK’s Evidence Base

Recommended (the committee's words)
It is time to democratise access to data and evidence. The UK Statistics Authority should establish a framework for identifying and prioritising demands for evidence. We recommend that it use a high-level Assembly (of the kind recently recommended by Professor Denise Lievesley) to draw together information from communities across the UK about their needs for evidence and the benefits new evidence would bring, alongside research on data gaps, and public understanding. We further recommend that the UK Statistics Authority submit its findings on the nation’s demands for evidence to Parliament on a triennial basis, for scrutiny by this Committee.

Link to this item · Read this item

14: Confirm Government support for the public good principle of statistics in response to report.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
We ask the Government to confirm, in its response to this report, that it supports the principle - enshrined in the Statistics and Registration Service Act 2007 - that statistics are for the public good; and that the public good includes not just assisting in the development and evaluation of policy, but also informing the public about social and economic matters.

Link to this item · Read this item

15: Inadequate data comparability across UK nations hinders public service evaluation.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
It is disappointing that - despite the ever-increasing amount of data available to policy-makers - there are many areas in which it is impossible to compare the experiences of those living in each of the four nations of the UK. This is detrimental to individual citizens, who are deprived of the ability to compare public services delivered in their part of the UK with those delivered in other parts. These issues were highlighted in a recent review of the UK Statistics Authority; but three months on from the report’s publication, although the issue appears to be a priority for Government, it has not yet made its full response on the issue known.

Link to this item · Read this item

17: Government analysis function vision for evidence-based policymaking is highly commendable.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
The vision of the government analysis function - to deliver better outcomes for the public by providing the best analysis to inform decision-making - is a commendable one. In an age in which Ministers are required to respond to complex policy challenges - whether climate change, cost-of-living challenges, or the performance of the National Health Service - it is surely right that we equip them to do so by making available the best possible evidence.

Link to this item · Read this item

18: Inadequate evaluation prevents robust conclusions on policymaker's access to evidence.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
It was not possible for us to form a robust conclusion about how well served our policy-makers are by evidence in 2024. Ironically, for a group of people dedicated to the cause of informed decision-making, analysts appear to have done little by way of evaluating the function’s success in delivering its vision. What we did hear anecdotally, however, suggests that there might be significant room for improvement.

Link to this item · Read this item

20: Reaffirm government commitment to analysis function and review its future funding options.

Recommendation · Fifth Report - Transforming the UK’s Evidence Base

Recommended (the committee's words)
We recommend that Government reaffirm its commitment to the analysis function, and that HM Treasury review options for its future funding. If Government truly wishes to improve its use of analysis and deliver better outcomes for the public, it clearly needs to fund that change.

Link to this item · Read this item

21: Mandate National Statistician to review analysis function scope and publish evaluation plans.

Recommendation · Fifth Report - Transforming the UK’s Evidence Base

Recommended (the committee's words)
In parallel, the National Statistician should review the analysis function’s scope and standard, with a view to defining an achievable set of next-steps, and clear plans for honest evaluations of the function’s success. This review and subsequent evaluations 42 Transforming the UK’s Evidence Base should be made publicly available, so that Parliament is in future better equipped to scrutinise both the Government’s use of evidence and the progress of the analysis function.

Link to this item · Read this item

22: Government communications frequently disregard evidence, undermining public trust in democracy.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
Too frequently, Government communications exhibit a disregard for evidence. This is helpful neither to the Government, in building trust in our democracy and support for policies of the day, nor to citizens who rightly expect to be able to scrutinise the work of Ministers and officials.

Link to this item · Read this item

24: Publish annual report card on departments' transparency compliance and agree sustainable funding model.

Recommendation · Fifth Report - Transforming the UK’s Evidence Base

Recommended (the committee's words)
We commend the OSR for its work on Intelligent Transparency and recommend that it publish an annual report card on departments’ compliance with its guidance, so that Parliament and external bodies might support it in holding departments to account, and making the case for well-informed policy. Recognising that this important work expands the remit of the OSR beyond official statistics, and into the larger world of government analysis, we also suggest that at the next Spending Review, it works with HM Treasury to agree a sustainable funding model for this work, given the vital role it plays.

Link to this item · Read this item

25: Train government communications professionals on OSR guidance and update Communication Functional Standard.

Recommendation · Fifth Report - Transforming the UK’s Evidence Base

Recommended (the committee's words)
We recommend that all government communications professionals are trained on the OSR’s Intelligent Transparency guidance, and that the Government Functional Standard for Communication be updated to make it clear that officials are expected to comply with that guidance.

Link to this item · Read this item

27: Routinely publish evidence for major policy announcements and develop options to deliver this.

Recommendation · Fifth Report - Transforming the UK’s Evidence Base

Recommended (the committee's words)
We recommend that, at a minimum, governments in future routinely publish the evidence and data underpinning their major policy announcements. Making this happen will not be a straightforward task, and we suggest that in the first instance leaders of the analysis and communications functions develop options to deliver this ambition, for the consideration of Ministers.

Link to this item · Read this item

28: Public sector evidence producers generally comply with data protection framework.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
Those responsible for collecting, analysing, and communicating evidence in the UK public sector are bound by a strict data protection framework, enforced by the Information Commissioner’s Office (ICO). We were pleased to learn that - on the whole - producers of public evidence comply with this framework, and work hard to protect the privacy of the public.

Link to this item · Read this item

29: Continued ICO monitoring of data protection compliance crucial for new data sources.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
The ICO’s evidence to this inquiry has been an invaluable resource to us and as officials continue to explore the potential of new data sources in the production of evidence, its continued monitoring of analysts’ compliance with the current data protection framework will be crucial.

Link to this item · Read this item

32: Public involvement in personal data use decisions crucial; engagement risks declining with new data.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
It is crucial that members of the public are involved in making decisions about how the UK chooses to use personal data in the development of its public evidence base. Traditionally, statisticians have engaged with members of the public about the use of their data in the conduct of surveys, but as officials embrace new sources of data there is a risk that this important dialogue falls away.

Link to this item · Read this item

34: Impressive early work by organisations on ethical data use in government.

Conclusion · Fifth Report - Transforming the UK’s Evidence Base

Concluded (the committee's words)
We have been impressed by the great range of early work conducted by organisations in and outside the public sector, to help policymakers understand what it means to use data ethically, and to establish mechanisms for applying those considerations to the every-day work of government.

Link to this item · Read this item

35: Jointly review varying data ethics frameworks for greater consistency and accountability mechanisms across government.

Recommendation · Fifth Report - Transforming the UK’s Evidence Base

Recommended (the committee's words)
It is now time to consolidate the excellent exploratory work that has been done on data ethics, and to embed it more formally into the collection, analysis, and communication of evidence in the UK. We recommend that the Cabinet Office’s Central Digital and Data Office and the Office for National Statistics jointly review the varying data ethics frameworks available to analysts across the UK; considering opportunities for greater consistency, and possible accountability mechanisms, to encourage a wider adoption of data ethics across government.

Link to this item · Read this item

Reports

1 report

Oral evidence sessions

5 sessions

On smaller screens, scroll horizontally to read every column. Keyboard users can focus the table region and use the arrow keys.

Date Session and witnesses Source
12 Mar 2024
Transforming the UK’s Statistical Evidence Base
Baroness Neville-Rolfe DBE CMG · Cabinet Office, Professor Denise Lievesley CBE · Oxford University, Steffan Jones · Cabinet Office
View ↗
6 Feb 2024
Transforming the UK’s Statistical Evidence Base
Ed Humpherson · Office for Statistics Regulation
View ↗
5 Dec 2023
Transforming the UK’s Evidence Base
Gavin Freeguard · Connected by Data, John Edwards · Information Commissioner's Office, Reema Patel · Ipsos UK
View ↗
9 Nov 2023
Transforming the UK’s Evidence Base
Chris Morris · Full Fact, Dr Gemma Tetlow · Institute for Government, Hetan Shah · British Academy
View ↗
5 Sep 2023
Transforming the UK’s Evidence Base
Professor Sir Ian Diamond · Office for National Statistics
View ↗

Who gave evidence

11 witnesses

On smaller screens, scroll horizontally to read every column. Keyboard users can focus the table region and use the arrow keys.

WitnessOrganisationSessions
Baroness Neville-Rolfe DBE CMG · Minister of State Cabinet Office 1
Chris Morris · Chief Executive Full Fact 1
Dr Gemma Tetlow · Chief Economist Institute for Government 1
Ed Humpherson · Director General for Regulation Office for Statistics Regulation 1
Gavin Freeguard · Policy associate Connected by Data 1
Hetan Shah · chief executive British Academy 1
John Edwards · Information Commissioner Information Commissioner's Office 1
Professor Denise Lievesley CBE · Lead on the Review of the UK Statistics Authority and Honorary Fellow and former Principal of Green Templeton College Oxford University 1
Professor Sir Ian Diamond · The National Statistician Office for National Statistics 1
Reema Patel · Head of Deliberative Engagement Ipsos UK 1
Steffan Jones · Director of Joint Data and Analysis Centre Cabinet Office 1