Source · Select Committees · Public Accounts Committee
Recommendation 16
16
Transparency of the models that departments use is poor: only 9 out of 17 departments...
Recommendation
Transparency of the models that departments use is poor: only 9 out of 17 departments have published their list of business-critical models since 2013, and only four of those have updated this since 2017.54 HM Treasury has not published its own list of business-critical 44 Qq 43,55 45 Q 43 46 Q 49 47 C&AG’s Report, Evaluating government spending, para 20 48 Q 43 49 Q 44 50 C&AG’s Report, Evaluating government spending, para 3.14 51 Qq 33, 44 52 Q 33 53 Q 7 54 C&AG’s Report, Financial modelling in government, para 12 Use of evaluation and modelling in government 13 models since 2013. It told us it intends to update its list within the next few months given the importance of leading by example.55 There can be a public interest in publishing a model in its entirety; for example, the Office of Budgetary Responsibility publishes the code and variables for its model of the economy so that others can use it.56 In other cases, publishing details of the model such as the outputs or assumptions gives parliament and the public assurance about how decisions are made. However, 45 models out of the NAO’s sample of 75 had no information published about them.57 HM Treasury told us it is discussing how to improve guidance to clarify what information departments should publish about models.58 Uncertainty
Government Response
Not Addressed
HM Government
Not Addressed
5.1 The government agrees with the Committee’s recommendation. Target implementation date: Summer 2023 5.2 The government agrees with the Committee’s recommendation to publish and maintain an evaluation registry. As there is currently no centralised place where government departments can publish evaluation evidence, the Evaluation Task Force is developing an online registry of UK government evaluations that will be publicly accessible. It will allow departments to publish evaluation plans/protocols and reports, in line with expectations outlined in the Government Social Research Publication Protocol and Concordat to Support Research Integrity. It will also allow departments to indicate estimated publication dates of evaluation reports. 5.3 The evaluation registry's functionality will enable users in departments to add or update evaluation information, and the government expects that departments will use this functionality to provide explanations where publications are delayed significantly or withheld. 5.4 The government agrees with the Committee’s recommendation. Target implementation date: Summer 2022 5.5 The Analysis Function is currently reviewing its Functional Standard which incorporates a variety of different quality dimensions with respect to analysis. Included within this standard are the various codes of practice and guidance books which support analytical best practice, of which the Aqua Book, which covers guidance on producing quality analysis for government, is one. As part of this standards piece – and in light of the Committee’s recommendation – there is a work programme over summer 2022 to review procedures around the quality assurance of models and to incorporate improved practices across departments by the end of the year.