Transforming the UK’s Evidence Base

There is much to be proud of across the UK’s public data-landscape. The independence of its statisticians from the government of the day, the innovative work being undertaken by a skilled researcher community, and our unique regulatory framework all received praise from our witnesses.

As it stands, the UK’s public data landscape is highly fragmented. This need not be a problem (indeed the model offers several conceivable advantages) if the many actors involved in generating, analysing and communicating evidence work together effectively. Our inquiry suggests that in several areas, however, this is not yet the case. (Paragraph 24) An explosion of data: Navigating new data sources and technologies

It is clear that the volume and variety of data generated within the UK has exploded in scale over recent years. While new sources of data have much to offer - not least in improving the timeliness of evidence, and allowing us to delve into issues in greater detail - there are things they simply cannot do. They cannot tell us about citizen’s intentions, they cannot be easily tweaked to capture the information an analyst is seeking, and they are prone to bias. The UK’s public evidence base will be best served, therefore, by bringing together old sources and new.

Despite the passage of relevant legislation in 2017, the UK has failed to bring its disparate datasets together to enrich its public evidence base. Instead, data withers in silos across countless government bodies. Witnesses to our inquiry were clear; the problems here are not legislative, and they do not result from challenges around data protection. The issue is much simpler; departments and public bodies choose not to share data because they do not share an incentive to do so. And no central, coordinating leader has yet made a sufficiently strong case for the benefits of sharing data for statistics and research across government.

It is time for Government to do what it promised to do seven years ago, and to join up the UK’s evidence base. Given that the Cabinet Office’s existing initiatives for improving data sharing are self-evidently insufficient, it should in partnership with the Office for National Statistics develop a comprehensive new programme aimed at improving data-sharing for statistical and research purposes. The programme must clearly define deliverables and timelines, and must be owned by a senior responsible officer at an appropriately high level. In line with the recommendations of the Lievesley report, we also recommend that HM Treasury establish mechanisms so that the costs are not borne by individual Departments, but rather centrally. The Cabinet Office should prepare and publish an annual progress report on delivery against the programme. (Paragraph 45) 40 Transforming the UK’s Evidence Base

Separately, the Office for National Statistics should publish information on the datasets it is seeking on an annual basis, setting out its rationale for seeking those data, and details on the status of the request - all of which should be made available on the ONS website.

While time in this Parliament now runs short, we recommend that the next Government review the exclusion of health and social care data from the Digital Economy Act 2017. There are understandable sensitivities around the sharing of health data, but it may be that the implementation of research provisions over the last seven years, and the work that has taken place in the context of the Caldicott review, offer a helpful new basis for discussion concerning their possible inclusion.

We support the Office for National Statistics in its ambition to deliver high-quality and timely population statistics. It is right to be considering whether new data sources might offer opportunities to improve the UK’s evidence base, and it is also right to be engaging closely with users of that evidence base, as decisions are taken on the future of the decennial census.

This Committee’s view - particularly in light of challenges around data-access - is that officials have not yet demonstrated that they can deliver the evidence users need, without a decennial census. We therefore recommend that the Office for National Statistics undertake further work on proposals for the future of migration and population statistics. (Paragraph 49) Who should evidence serve? Delivering evidence for the public good

Currently, the UK has no framework by which to identify, and then prioritise, demands for data and evidence. In the absence of such a framework, and with an ongoing need to have regard to budget constraints, Ministers and HM Treasury wield an inordinate amount of power in deciding what evidence the UK collects and communicates through the decisions they take on departmental budgets. In the context of these systemic shortcomings, significant data gaps have emerged; the UK lacks suitable evidence on the performance of its different health services, for example, and on key policy challenges like school absenteeism.

It is time to democratise access to data and evidence. The UK Statistics Authority should establish a framework for identifying and prioritising demands for evidence. We recommend that it use a high-level Assembly (of the kind recently recommended by Professor Denise Lievesley) to draw together information from communities across the UK about their needs for evidence and the benefits new evidence would bring, alongside research on data gaps, and public understanding. We further recommend that the UK Statistics Authority submit its findings on the nation’s demands for evidence to Parliament on a triennial basis, for scrutiny by this Committee.

We recommend that the OSR support this activity by preparing regular and public reports on data gaps in the UK. (Paragraph 64) Transforming the UK’s Evidence Base 41

We recommend that in its conduct of future Spending Reviews, HM Treasury uses the findings from these reports to inform the decisions it takes on the funding of activity relating to the collection, analysis and communication of public evidence.

We ask the Government to confirm, in its response to this report, that it supports the principle - enshrined in the Statistics and Registration Service Act 2007 - that statistics are for the public good; and that the public good includes not just assisting in the development and evaluation of policy, but also informing the public about social and economic matters.

It is disappointing that - despite the ever-increasing amount of data available to policy-makers - there are many areas in which it is impossible to compare the experiences of those living in each of the four nations of the UK. This is detrimental to individual citizens, who are deprived of the ability to compare public services delivered in their part of the UK with those delivered in other parts. These issues were highlighted in a recent review of the UK Statistics Authority; but three months on from the report’s publication, although the issue appears to be a priority for Government, it has not yet made its full response on the issue known.

We recommend that the Office for Statistics Regulation review and publish a report on the adequacy of UK-wide comparable data, by themes, before April 2025. (Paragraph 68) Evidence in policymaking

The vision of the government analysis function - to deliver better outcomes for the public by providing the best analysis to inform decision-making - is a commendable one. In an age in which Ministers are required to respond to complex policy challenges - whether climate change, cost-of-living challenges, or the performance of the National Health Service - it is surely right that we equip them to do so by making available the best possible evidence.

It was not possible for us to form a robust conclusion about how well served our policy-makers are by evidence in 2024. Ironically, for a group of people dedicated to the cause of informed decision-making, analysts appear to have done little by way of evaluating the function’s success in delivering its vision. What we did hear anecdotally, however, suggests that there might be significant room for improvement.

We also identified a mismatch between the ambitious vision of the analysis function, and the very limited funding made available to deliver that vision.

We recommend that Government reaffirm its commitment to the analysis function, and that HM Treasury review options for its future funding. If Government truly wishes to improve its use of analysis and deliver better outcomes for the public, it clearly needs to fund that change.

In parallel, the National Statistician should review the analysis function’s scope and standard, with a view to defining an achievable set of next-steps, and clear plans for honest evaluations of the function’s success. This review and subsequent evaluations 42 Transforming the UK’s Evidence Base should be made publicly available, so that Parliament is in future better equipped to scrutinise both the Government’s use of evidence and the progress of the analysis function.

Too frequently, Government communications exhibit a disregard for evidence. This is helpful neither to the Government, in building trust in our democracy and support for policies of the day, nor to citizens who rightly expect to be able to scrutinise the work of Ministers and officials.

Since its launch in 2022, the Office for Statistics Regulation’s Intelligent Transparency guidance has helped to unlock important evidence for Parliament, business, researchers and citizens, but there remains more to do.

We commend the OSR for its work on Intelligent Transparency and recommend that it publish an annual report card on departments’ compliance with its guidance, so that Parliament and external bodies might support it in holding departments to account, and making the case for well-informed policy. Recognising that this important work expands the remit of the OSR beyond official statistics, and into the larger world of government analysis, we also suggest that at the next Spending Review, it works with HM Treasury to agree a sustainable funding model for this work, given the vital role it plays.

We recommend that all government communications professionals are trained on the OSR’s Intelligent Transparency guidance, and that the Government Functional Standard for Communication be updated to make it clear that officials are expected to comply with that guidance.

We concur with Lord Maude’s recent recommendation that, when a policy decision is announced, the Government should publish the evidence and data underpinning that decision.

We recommend that, at a minimum, governments in future routinely publish the evidence and data underpinning their major policy announcements. Making this happen will not be a straightforward task, and we suggest that in the first instance leaders of the analysis and communications functions develop options to deliver this ambition, for the consideration of Ministers. (Paragraph 93) Privacy and ethics in an age of data

Those responsible for collecting, analysing, and communicating evidence in the UK public sector are bound by a strict data protection framework, enforced by the Information Commissioner’s Office (ICO). We were pleased to learn that - on the whole - producers of public evidence comply with this framework, and work hard to protect the privacy of the public.

The ICO’s evidence to this inquiry has been an invaluable resource to us and as officials continue to explore the potential of new data sources in the production of evidence, its continued monitoring of analysts’ compliance with the current data protection framework will be crucial. (Paragraph 99) Transforming the UK’s Evidence Base 43

Although statisticians and researchers publish a wealth of information on which data sources they hold, and how they are used, very little information is made available about how personal data are being used for the purposes of government analysis.

We recommend that the analysis function explore options for improving transparency around the use of personal data in official analyses, and that this work be made publicly available.

It is crucial that members of the public are involved in making decisions about how the UK chooses to use personal data in the development of its public evidence base. Traditionally, statisticians have engaged with members of the public about the use of their data in the conduct of surveys, but as officials embrace new sources of data there is a risk that this important dialogue falls away.

The information that we do have about public attitudes suggest an openness to the use of data, where there are clear benefits to be gained, and where those benefits are seen to be shared equitably.

We have been impressed by the great range of early work conducted by organisations in and outside the public sector, to help policymakers understand what it means to use data ethically, and to establish mechanisms for applying those considerations to the every-day work of government.

It is now time to consolidate the excellent exploratory work that has been done on data ethics, and to embed it more formally into the collection, analysis, and communication of evidence in the UK. We recommend that the Cabinet Office’s Central Digital and Data Office and the Office for National Statistics jointly review the varying data ethics frameworks available to analysts across the UK; considering opportunities for greater consistency, and possible accountability mechanisms, to encourage a wider adoption of data ethics across government.

In parallel, the Centre for Data Ethics and Innovation should continue its excellent work in monitoring public attitudes on the Government’s use of data. (Paragraph 114) 44 Transforming the UK’s Evidence Base