Select Committee · Public Administration and Constitutional Affairs Committee

Transforming the UK’s Evidence Base

Status: Closed Opened: 14 Nov 2023 Closed: 24 May 2024 15 recommendations 21 conclusions 1 report

This inquiry is exploring how officials produce statistics and analysis, how demands for data are changing, and whether the privacy of citizens is being adequately protected as new and innovative sources become available.

Reports

1 report
Title HC No. Published Items Response
Fifth Report - Transforming the UK’s Evidence Base HC 197 24 May 2024 36 Responded

Recommendations & Conclusions

36 items
1 Conclusion Fifth Report - Transforming the UK’s Ev… Rejected

UK public data landscape demonstrates strengths in independence, innovation, and regulatory framework.

There is much to be proud of across the UK’s public data-landscape. The independence of its statisticians from the government of the day, the innovative work being undertaken by a skilled researcher community, and our unique regulatory framework all received praise from our witnesses.

Government response. The government rejects a specific 'recommended programme structure' for data sharing, instead committing to its own comprehensive approach including reinvigorating DSIT's digital centre and creating a National Data Library.
2 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

UK's public data landscape is fragmented, lacking effective collaboration between actors.

As it stands, the UK’s public data landscape is highly fragmented. This need not be a problem (indeed the model offers several conceivable advantages) if the many actors involved in generating, analysing and communicating evidence work together effectively. Our inquiry suggests that in several areas, however, this is not yet …

Government response. The government commits to issuing further publications to enhance transparency on data acquisition and processing, and will add data in the process of being acquired to its transparency reporting, developing the optimal format with suppliers.
3 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

New data sources offer benefits but possess limitations, requiring integration with traditional data.

It is clear that the volume and variety of data generated within the UK has exploded in scale over recent years. While new sources of data have much to offer - not least in improving the timeliness of evidence, and allowing us to delve into issues in greater detail - …

Government response. The government accepts the recommendation, committing to keep health data arrangements under the Digital Economy Act 2017 under review and to consider amendments to data sharing powers, with implementation by the end of 2025.
4 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

Government departments fail to share data due to lack of incentives and central coordination.

Despite the passage of relevant legislation in 2017, the UK has failed to bring its disparate datasets together to enrich its public evidence base. Instead, data withers in silos across countless government bodies. Witnesses to our inquiry were clear; the problems here are not legislative, and they do not result …

Government response. The ONS is continuing work to develop and improve admin-based population estimates, aiming for them to become official mid-year population estimates in 2025. The Authority also expects to publish its recommendation on the future of population and migration statistics in …
5 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

Develop comprehensive programme with ONS to improve cross-government data-sharing for research and statistics.

It is time for Government to do what it promised to do seven years ago, and to join up the UK’s evidence base. Given that the Cabinet Office’s existing initiatives for improving data sharing are self-evidently insufficient, it should in partnership with the Office for National Statistics develop a comprehensive …

Government response. The UK Statistics Authority will establish a UK Statistics Assembly, which is set to meet for the first time on 22 January 2025, to bring together users and producers to discuss and give direction on evidence priorities. The ONS has …
6 Recommendation Fifth Report - Transforming the UK’s Ev… Accepted

Publish annual ONS information on requested datasets, rationale, and request status online.

Separately, the Office for National Statistics should publish information on the datasets it is seeking on an annual basis, setting out its rationale for seeking those data, and details on the status of the request - all of which should be made available on the ONS website.

Government response. The OSR accepts the recommendation in principle and will enhance its focus on data gaps in future editions of its annual 'State of the Statistical System' report. The report, last published in July 2024, is the OSR's key publication for …
7 Recommendation Fifth Report - Transforming the UK’s Ev… Deferred

Review exclusion of health and social care data from Digital Economy Act 2017.

While time in this Parliament now runs short, we recommend that the next Government review the exclusion of health and social care data from the Digital Economy Act 2017. There are understandable sensitivities around the sharing of health data, but it may be that the implementation of research provisions over …

Government response. The Government accepts the recommendation but states that HM Treasury would consider findings of any future reports from the UKSA (subject to responses to other recommendations) to inform funding decisions related to public evidence, rather than directly committing to review …
8 Conclusion Fifth Report - Transforming the UK’s Ev… Acknowledged

ONS ambition to deliver high-quality population statistics and new data sources supported.

We support the Office for National Statistics in its ambition to deliver high-quality and timely population statistics. It is right to be considering whether new data sources might offer opportunities to improve the UK’s evidence base, and it is also right to be engaging closely with users of that evidence …

Government response. The government accepts the recommendation and confirms it supports the principle that statistics are for the public good, as enshrined in existing legislation. No specific new actions are committed to regarding new data sources or user engagement for the census.
9 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

Undertake further ONS work on proposals for future migration and population statistics.

This Committee’s view - particularly in light of challenges around data-access - is that officials have not yet demonstrated that they can deliver the evidence users need, without a decennial census. We therefore recommend that the Office for National Statistics undertake further work on proposals for the future of migration …

Government response. The OSR accepts the Committee’s recommendation to review and publish a report on the adequacy of UK-wide comparable data, committing to publish an update in 2025 that will share more detailed views on UK comparability.
10 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

Absence of data prioritisation framework causes significant evidence gaps across government.

Currently, the UK has no framework by which to identify, and then prioritise, demands for data and evidence. In the absence of such a framework, and with an ongoing need to have regard to budget constraints, Ministers and HM Treasury wield an inordinate amount of power in deciding what evidence …

Government response. The Analytical Function Central Team (AFCT) will work with HM Treasury to assess the best option for future funding for a dedicated central AF team. Additionally, the Head of the AF will work with Chief Analysts across government to raise …
11 Recommendation Fifth Report - Transforming the UK’s Ev… Deferred

Establish UK Statistics Authority framework for prioritising evidence demands and submit triennial findings to Parliament.

It is time to democratise access to data and evidence. The UK Statistics Authority should establish a framework for identifying and prioritising demands for evidence. We recommend that it use a high-level Assembly (of the kind recently recommended by Professor Denise Lievesley) to draw together information from communities across the …

Government response. The government response does not address the committee's recommendation to establish a framework and high-level assembly for evidence demands, nor to report findings to Parliament. Instead, it discusses the ONS's plans to review the Analysis Function's scope and standard, evaluate …
12 Recommendation Fifth Report - Transforming the UK’s Ev… Acknowledged

Prepare regular public OSR reports detailing data gaps across the United Kingdom.

We recommend that the OSR support this activity by preparing regular and public reports on data gaps in the UK. (Paragraph 64) Transforming the UK’s Evidence Base 41

Government response. The OSR accepts the recommendation to prepare regular and public reports on data gaps but states that this would expand its remit and require appropriate funding. It is beginning to develop proposals for a monitoring and reporting approach for intelligent …
13 Recommendation Fifth Report - Transforming the UK’s Ev… Deferred

Use OSR data gap reports to inform Treasury funding decisions for public evidence.

We recommend that in its conduct of future Spending Reviews, HM Treasury uses the findings from these reports to inform the decisions it takes on the funding of activity relating to the collection, analysis and communication of public evidence.

Government response. The Office for Statistics Regulation (OSR) is exploring ways to provide Intelligent Transparency materials online, developing new social media guidance, and refreshing the Code to articulate IT standards more clearly. It states it already encourages transparency around policy announcements but …
14 Conclusion Fifth Report - Transforming the UK’s Ev… Deferred

Confirm Government support for the public good principle of statistics in response to report.

We ask the Government to confirm, in its response to this report, that it supports the principle - enshrined in the Statistics and Registration Service Act 2007 - that statistics are for the public good; and that the public good includes not just assisting in the development and evaluation of …

Government response. The government response does not confirm support for the principle that statistics are for the public good, as requested. Instead, it discusses the ONS's acceptance of a recommendation to develop options for Ministers on the routine publication of evidence and …
15 Conclusion Fifth Report - Transforming the UK’s Ev… Deferred

Inadequate data comparability across UK nations hinders public service evaluation.

It is disappointing that - despite the ever-increasing amount of data available to policy-makers - there are many areas in which it is impossible to compare the experiences of those living in each of the four nations of the UK. This is detrimental to individual citizens, who are deprived of …

Government response. The government response does not address the committee's conclusion regarding the lack of comparable data across the four nations of the UK. Instead, it states the ONS accepts a recommendation for the Analysis Function to explore options for improving transparency …
16 Recommendation Fifth Report - Transforming the UK’s Ev… Acknowledged

Require Office for Statistics Regulation to publish report on UK-wide data adequacy by April 2025.

We recommend that the Office for Statistics Regulation review and publish a report on the adequacy of UK-wide comparable data, by themes, before April 2025. (Paragraph 68) Evidence in policymaking

Government response. The Office for Statistics Regulation (OSR) states its response has focused on developing a framework for reporting data gaps across the UK and on improving intelligent transparency. It agrees harmonisation is desirable but does not explicitly commit to publishing a …
17 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

Government analysis function vision for evidence-based policymaking is highly commendable.

The vision of the government analysis function - to deliver better outcomes for the public by providing the best analysis to inform decision-making - is a commendable one. In an age in which Ministers are required to respond to complex policy challenges - whether climate change, cost-of-living challenges, or the …

Government response. The government accepts the recommendation, highlighting its ongoing annual survey by the Responsible Technology Adoption Unit, which tracks public opinions on data-driven technology to support policymakers. It also commits to keeping the committee updated on work by the UK Statistics …
18 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

Inadequate evaluation prevents robust conclusions on policymaker's access to evidence.

It was not possible for us to form a robust conclusion about how well served our policy-makers are by evidence in 2024. Ironically, for a group of people dedicated to the cause of informed decision-making, analysts appear to have done little by way of evaluating the function’s success in delivering …

Government response. The ONS accepts the implicit recommendation to review the AF's scope, standard, and evaluate its impact. The AF Central Team commits to undertaking a light-touch assessment and updating the Committee on findings by Q2 2025/26, subject to funding.
19 Conclusion Fifth Report - Transforming the UK’s Ev… Acknowledged

Significant mismatch exists between analysis function's ambitious vision and its limited funding.

We also identified a mismatch between the ambitious vision of the analysis function, and the very limited funding made available to deliver that vision.

Government response. The ONS accepts the principle of the conclusion regarding funding for the Analysis Function (AF), recognizing its importance. The AF Central Team will work with HM Treasury to assess the best option for future funding, but no specific funding commitment …
20 Recommendation Fifth Report - Transforming the UK’s Ev… Accepted

Reaffirm government commitment to analysis function and review its future funding options.

We recommend that Government reaffirm its commitment to the analysis function, and that HM Treasury review options for its future funding. If Government truly wishes to improve its use of analysis and deliver better outcomes for the public, it clearly needs to fund that change.

Government response. The government accepts the recommendation in principle, confirming the Analysis Function Central Team will work with HM Treasury to assess options for future funding.
21 Recommendation Fifth Report - Transforming the UK’s Ev… Accepted

Mandate National Statistician to review analysis function scope and publish evaluation plans.

In parallel, the National Statistician should review the analysis function’s scope and standard, with a view to defining an achievable set of next-steps, and clear plans for honest evaluations of the function’s success. This review and subsequent evaluations 42 Transforming the UK’s Evidence Base should be made publicly available, so …

Government response. The ONS accepts the recommendation for the AFCT to review the Analysis Function's scope and standard, reflecting findings in an updated strategy by Q2 2025/26. Subject to funding, the AFCT will also evaluate the impact of their work, providing updates …
22 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

Government communications frequently disregard evidence, undermining public trust in democracy.

Too frequently, Government communications exhibit a disregard for evidence. This is helpful neither to the Government, in building trust in our democracy and support for policies of the day, nor to citizens who rightly expect to be able to scrutinise the work of Ministers and officials.

Government response. The OSR accepts the recommendation to further its work on intelligent transparency, stating it has a working group to embed IT principles across government and is developing proposals for a monitoring and reporting approach.
23 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

OSR's Intelligent Transparency guidance improves evidence access but requires further development.

Since its launch in 2022, the Office for Statistics Regulation’s Intelligent Transparency guidance has helped to unlock important evidence for Parliament, business, researchers and citizens, but there remains more to do.

Government response. The OSR accepts the conclusion and is forming a working group to promote and embed Intelligent Transparency principles. They are developing proposals for a cross-departmental monitoring and reporting approach, considering incentives and automated tools, subject to a sustainable funding model.
24 Recommendation Fifth Report - Transforming the UK’s Ev… Accepted in Part

Publish annual report card on departments' transparency compliance and agree sustainable funding model.

We commend the OSR for its work on Intelligent Transparency and recommend that it publish an annual report card on departments’ compliance with its guidance, so that Parliament and external bodies might support it in holding departments to account, and making the case for well-informed policy. Recognising that this important …

Government response. The OSR accepts the recommendation and is beginning work to develop proposals for a monitoring and reporting approach for intelligent transparency. They will consider these proposals alongside discussions with HM Treasury on a sustainable funding model for this expanded work.
25 Recommendation Fifth Report - Transforming the UK’s Ev… Accepted

Train government communications professionals on OSR guidance and update Communication Functional Standard.

We recommend that all government communications professionals are trained on the OSR’s Intelligent Transparency guidance, and that the Government Functional Standard for Communication be updated to make it clear that officials are expected to comply with that guidance.

Government response. The government accepts the recommendation, stating the GCS has incorporated Intelligent Transparency guidance into its Modern Communications Operating Model and its online Data & Insight for Communicators course, which is available to all GCS professionals, thereby implementing the recommendation.
26 Recommendation Fifth Report - Transforming the UK’s Ev… Rejected

Require Government to publish evidence and data underpinning announced policy decisions.

We concur with Lord Maude’s recent recommendation that, when a policy decision is announced, the Government should publish the evidence and data underpinning that decision.

Government response. The Government rejects the recommendation to publish evidence and data underpinning policy decisions. They assert that departments determine data sharing based on considerations like public safety, commercial sensitivity, and international relations.
27 Recommendation Fifth Report - Transforming the UK’s Ev… Rejected

Routinely publish evidence for major policy announcements and develop options to deliver this.

We recommend that, at a minimum, governments in future routinely publish the evidence and data underpinning their major policy announcements. Making this happen will not be a straightforward task, and we suggest that in the first instance leaders of the analysis and communications functions develop options to deliver this ambition, …

Government response. The Government rejects the recommendation to routinely publish evidence and data underpinning major policy announcements. They state that such decisions lie with individual departments, who consider transparency alongside factors like public safety and commercial sensitivity.
28 Conclusion Fifth Report - Transforming the UK’s Ev…

Public sector evidence producers generally comply with data protection framework.

Those responsible for collecting, analysing, and communicating evidence in the UK public sector are bound by a strict data protection framework, enforced by the Information Commissioner’s Office (ICO). We were pleased to learn that - on the whole - producers of public evidence comply with this framework, and work hard …

29 Conclusion Fifth Report - Transforming the UK’s Ev…

Continued ICO monitoring of data protection compliance crucial for new data sources.

The ICO’s evidence to this inquiry has been an invaluable resource to us and as officials continue to explore the potential of new data sources in the production of evidence, its continued monitoring of analysts’ compliance with the current data protection framework will be crucial. (Paragraph 99) Transforming the UK’s …

30 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

Limited information provided on personal data usage in government analysis.

Although statisticians and researchers publish a wealth of information on which data sources they hold, and how they are used, very little information is made available about how personal data are being used for the purposes of government analysis.

Government response. The ONS accepts the recommendation and commits the AFCT to investigate options for improving transparency around the use of personal data in official analyses by Q4 2025/26, working with relevant bodies.
31 Recommendation Fifth Report - Transforming the UK’s Ev… Accepted

Explore options for improving personal data transparency in official analyses and publish findings.

We recommend that the analysis function explore options for improving transparency around the use of personal data in official analyses, and that this work be made publicly available.

Government response. The ONS accepts the recommendation for the Analytical Function to explore options for improving transparency around personal data in official analyses. The Analytical Function Central Team commits to completing this investigative work by Q4 2025/26, subject to sufficient funding.
32 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

Public involvement in personal data use decisions crucial; engagement risks declining with new data.

It is crucial that members of the public are involved in making decisions about how the UK chooses to use personal data in the development of its public evidence base. Traditionally, statisticians have engaged with members of the public about the use of their data in the conduct of surveys, …

Government response. The government accepts the importance of public involvement, stating that the Responsible Technology Adoption Unit already conducts an annual survey to track public opinions on data-driven technology, with the next wave due in December 2024.
33 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

Public attitudes indicate openness to data use with clear and equitable benefits.

The information that we do have about public attitudes suggest an openness to the use of data, where there are clear benefits to be gained, and where those benefits are seen to be shared equitably.

Government response. The Government accepts the conclusion, stating that the Responsible Technology Adoption Unit has been conducting an annual survey since 2021 to track public opinions on data-driven technology, which provides insights that support policymakers.
34 Conclusion Fifth Report - Transforming the UK’s Ev… Accepted

Impressive early work by organisations on ethical data use in government.

We have been impressed by the great range of early work conducted by organisations in and outside the public sector, to help policymakers understand what it means to use data ethically, and to establish mechanisms for applying those considerations to the every-day work of government.

Government response. The Government accepts the conclusion, explaining that the Responsible Technology Adoption Unit's annual survey, ongoing since 2021, helps policymakers understand and align data use with public values, demonstrating existing work in this area.
35 Recommendation Fifth Report - Transforming the UK’s Ev… Rejected

Jointly review varying data ethics frameworks for greater consistency and accountability mechanisms across government.

It is now time to consolidate the excellent exploratory work that has been done on data ethics, and to embed it more formally into the collection, analysis, and communication of evidence in the UK. We recommend that the Cabinet Office’s Central Digital and Data Office and the Office for National …

Government response. The government rejects the recommendation for a joint review of data ethics frameworks, citing existing close collaboration and the need for flexibility over consistency. They plan to update the CDDO data ethics framework by the end of 2025 and assess …
36 Recommendation Fifth Report - Transforming the UK’s Ev… Accepted

Continue monitoring public attitudes on the Government’s use of data by the CDEI.

In parallel, the Centre for Data Ethics and Innovation should continue its excellent work in monitoring public attitudes on the Government’s use of data. (Paragraph 114) 44 Transforming the UK’s Evidence Base

Government response. The government accepts the recommendation, stating that the Responsible Technology Adoption Unit (formerly CDEI) already conducts an annual survey to monitor public opinions on data use, with the fourth wave due in December 2024.

Oral evidence sessions

5 sessions
Date Witnesses
12 Mar 2024 Baroness Neville-Rolfe DBE CMG · Cabinet Office, Professor Denise Lievesley CBE · Oxford University, Steffan Jones · Cabinet Office View ↗
6 Feb 2024 Ed Humpherson · Office for Statistics Regulation View ↗
5 Dec 2023 Gavin Freeguard · Connected by Data, John Edwards · Information Commissioner's Office, Reema Patel · Ipsos UK View ↗
9 Nov 2023 Chris Morris · Full Fact, Dr Gemma Tetlow · Institute for Government, Hetan Shah · British Academy View ↗
5 Sep 2023 Professor Sir Ian Diamond · Office for National Statistics View ↗

Correspondence

4 letters
DateDirectionTitle
12 Mar 2024 To cttee Letter from Professor Sir Ian Diamond, National Statistician, UK Statistics Aut…
9 Jan 2024 To cttee Letter from Professor Sir Ian Diamond, National Statistician on supplementary w…
19 Dec 2023 To cttee Letter from John Edwards, UK Information Commissioner on follow-up after the or…
17 Oct 2023 To cttee Letter from Professor Sir Ian Diamond, National Statistician to on follow up wr…