Recommendations & Conclusions
19 items
2
Recommendation
Fourth Report - Use of evaluation and m…
Not Addressed
HM Treasury is not making full use of the spending levers it has at its disposal to deliver a step change in the use of evaluation across government. We welcome the creation of the Evaluation Task Force and the increased emphasis on evaluation in the 2020 and 2021 Spending Reviews, …
Government response. The response does not address the recommendation of setting up a formal process for tracking spending review settlement conditions relating to evaluation, and instead discusses internal performance measures and consultations with the sector.
HM Treasury
3
Recommendation
Fourth Report - Use of evaluation and m…
No single body is responsible for upholding modelling and evaluation standards and monitoring their implementation. Within departments, ultimate responsibility for upholding standards and securing improvements lies with that department’s accounting officer. Supporting the accounting officer, the departmental directors of analysis own the application of the Analysis Functional Standard in their …
Government response. 3.1 The government agrees with the Committee’s recommendation. Target implementation date: March 2023 3.2 The Government Analysis Function is currently in the process of piloting the assessment framework which will allow departments to monitor their implementation of the Analysis Functional …
HM Treasury
4
Recommendation
Fourth Report - Use of evaluation and m…
Good quality modelling and evaluation is hampered by challenges in sharing data and a lack of common data standards. Government does not always see data as a priority and data quality is often inadequate. We have drawn attention to poor data and a lack of joined up systems in numerous …
Government response. 4.1 The government agrees with the Committee’s recommendation. Target implementation date: April 2025 4.2 The strategy will be completely delivered by April 2025; however, an interim progress report will be produced by December 2023. 4.3 The Central Digital and Data …
HM Treasury
5
Conclusion
Fourth Report - Use of evaluation and m…
Departments are not meeting government requirements on publishing evaluation plans and findings, and on transparency of models and their outputs. Transparency is critical for the public to understand the evidence behind decisions and how money is being spent. It also allows departments to learn lessons from past projects and from …
Government response. 5.1 The government agrees with the Committee’s recommendation. Target implementation date: Summer 2023 5.2 The government agrees with the Committee’s recommendation to publish and maintain an evaluation registry. As there is currently no centralised place where government departments can publish …
HM Treasury
6
Conclusion
Fourth Report - Use of evaluation and m…
Model producers and users do not adequately assess the range of plausible outcomes and are overly reliant on central estimates that do not reflect inherent uncertainty. All modelled information is uncertain, and decision-makers need information on the possible range of outcomes to plan well, manage risks and make better decisions. …
Government response. 6.1 The government agrees with the Committee’s recommendation. Target implementation date: March 2023 6.2 The Finance Function will work with the Analysis Function to identify appropriate analysis and modelling considerations relating to uncertainty and incorporate these into the Finance Standard, …
HM Treasury
1
Conclusion
Fourth Report - Use of evaluation and m…
On the basis of two reports by the Comptroller and Auditor General, we took evidence from HM Treasury, the Finance Function, the Analysis Function and the Evaluation Task Force about the use of evaluation and modelling across government.1 Cabinet Office introduced cross-government ‘functions’ in 2013 to provide professional support to …
Government response. 1.1 The government agrees with the Committee’s recommendation. Target implementation date: August 2022 1.2 The Evaluation Task Force (ETF) was established in 2021 as a joint HM Treasury and Cabinet Office unit following a commitment made in the June 2021 …
HM Treasury
7
Conclusion
Fourth Report - Use of evaluation and m…
HM Treasury has used the last two spending reviews to increase the emphasis on evaluation in departments, but acknowledges it has a lot more to do to address the persistent problems which prevent the provision and use of evaluation.17 Working with the Evaluation Task Force, HM Treasury asked departments to …
Government response. The Evaluation Task Force is working with HM Treasury to ensure that departments respond to and deliver on their settlement conditions relating to evaluation. The ETF has developed a central tracking system that captures updates on departments’ delivery of evaluation …
HM Treasury
8
Conclusion
Fourth Report - Use of evaluation and m…
Outside of the spending reviews, HM Treasury has few levers and exercises little scrutiny to ensure that departments comply with evaluation requirements and expectations.20 6 C&AG’s Report, Evaluating government spending, para 3.33; C&AG’s Report, Financial modelling in government, para 13, Figure 4 7 Qq 1, 18–19 8 Q 25 9 …
Government response. The Evaluation Task Force is working with HM Treasury to ensure that departments respond to and deliver on their settlement conditions relating to evaluation. The ETF has developed a central tracking system that captures updates on departments’ delivery of evaluation …
HM Treasury
9
Conclusion
Fourth Report - Use of evaluation and m…
Within departments, ultimate responsibility for upholding standards and securing improvements lies with that department’s accounting officer.24 Managing Public Money states that financial modelling and evaluation are an important part of the accounting officer’s role, and that they should comply with the Functional Standards.25 However, there continues to be important weaknesses …
Government response. Within departments, ultimate responsibility for upholding standards and securing improvements lies with that department’s accounting officer. Managing Public Money states that financial modelling and evaluation are an important part of the accounting officer’s role, and that they should comply with …
HM Treasury
10
Conclusion
Fourth Report - Use of evaluation and m…
While HM Treasury is an important user of department’s models, it does not comprehensively scrutinise and challenge a department’s quality assurance arrangements for the models it uses. There is no requirement for the Aqua Book to be followed as part of the investment approval process nor as part of the …
Government response. 3.1 The government agrees with the Committee’s recommendation. Target implementation date: March 2023 3.2 The Government Analysis Function is currently in the process of piloting the assessment framework which will allow departments to monitor their implementation of the Analysis Functional …
HM Treasury
11
Conclusion
Fourth Report - Use of evaluation and m…
To address the lack of overall oversight for both evaluation and modelling standards, the Analysis Function intends to strengthen the governance framework for the Analysis Functional Standard, including introducing a self-assessment framework to assess performance against the standard.31 The Analysis Function told us there will be an expectation in the …
Government response. 3.1 The government agrees with the Committee’s recommendation. Target implementation date: March 2023 3.2 The Government Analysis Function is currently in the process of piloting the assessment framework which will allow departments to monitor their implementation of the Analysis Functional …
HM Treasury
12
Conclusion
Fourth Report - Use of evaluation and m…
Government does not always see data as a priority and data quality is often inadequate.36 The NAO reported in 2019 that the quality and sharing of data is a clear example of a neglected and poorly planned activity, and there is a culture of tolerating and working around poor-quality data.37 …
Government response. 4.1 The government agrees with the Committee’s recommendation. Target implementation date: April 2025 4.2 The strategy will be completely delivered by April 2025; however, an interim progress report will be produced by December 2023. 4.3 The Central Digital and Data …
HM Treasury
13
Conclusion
Fourth Report - Use of evaluation and m…
When producing and using models, poor data can lead to additional time and increased difficulty when quality assuring the model inputs. This difficulty is compounded by a lack of data standards across government which means there are inconsistent ways of recording the same information.40 When dealing with new situations, such …
Government response. 4.1 The government agrees with the Committee’s recommendation. Target implementation date: April 2025 4.2 The strategy will be completely delivered by April 2025; however, an interim progress report will be produced by December 2023. 4.3 The Central Digital and Data …
HM Treasury
14
Conclusion
Fourth Report - Use of evaluation and m…
Government’s view is that that transparency is critical for the public to understand the evidence behind decisions and how money is being spent, and that transparency should be the default position.44 The Analysis Function consider that transparency of evaluation findings allows departments to learn lessons from past projects and make …
Government response. 5.1 The government agrees with the Committee’s recommendation. Target implementation date: Summer 2023 5.2 The government agrees with the Committee’s recommendation to publish and maintain an evaluation registry. As there is currently no centralised place where government departments can publish …
HM Treasury
15
Conclusion
Fourth Report - Use of evaluation and m…
HMT told us it is only recently that it has started to ask departments to explain why they do not plan to publish some evaluations.48 The Evaluation Task Force raised the issue that there were evaluations that the centre of government did not know about because they had never been …
Government response. 5.1 The government agrees with the Committee’s recommendation. Target implementation date: Summer 2023 5.2 The government agrees with the Committee’s recommendation to publish and maintain an evaluation registry. As there is currently no centralised place where government departments can publish …
HM Treasury
16
Recommendation
Fourth Report - Use of evaluation and m…
Transparency of the models that departments use is poor: only 9 out of 17 departments have published their list of business-critical models since 2013, and only four of those have updated this since 2017.54 HM Treasury has not published its own list of business-critical 44 Qq 43,55 45 Q 43 …
Government response. 5.1 The government agrees with the Committee’s recommendation. Target implementation date: Summer 2023 5.2 The government agrees with the Committee’s recommendation to publish and maintain an evaluation registry. As there is currently no centralised place where government departments can publish …
HM Treasury
17
Conclusion
Fourth Report - Use of evaluation and m…
Uncertainty is inherent in modelled information and should be considered as part of all analysis. HM Treasury believes any model that does not explicitly address uncertainty is deficient.59 Decision-makers need information on the range of outcomes that may occur and their relative likelihoods to develop contingency plans and to avoid …
Government response. 6.1 The government agrees with the Committee’s recommendation. Target implementation date: March 2023 6.2 The Finance Function will work with the Analysis Function to identify appropriate analysis and modelling considerations relating to uncertainty and incorporate these into the Finance Standard, …
HM Treasury
18
Conclusion
Fourth Report - Use of evaluation and m…
Despite the importance of uncertainty analysis, model producers do not routinely assess or communicate to users the uncertainty in their models. There is limited evidence of departments using uncertainty analysis or developing contingency plans, and departments generally present outputs as single estimates.64 Analysis of uncertainty can give clarity on the …
Government response. 6.1 The government agrees with the Committee’s recommendation. Target implementation date: March 2023 6.2 The Finance Function will work with the Analysis Function to identify appropriate analysis and modelling considerations relating to uncertainty and incorporate these into the Finance Standard, …
HM Treasury
19
Conclusion
Fourth Report - Use of evaluation and m…
Departments present outputs from their models to HM Treasury as part of the spending review and budget process. HM Treasury uses these outputs for forecasting, budget planning and to monitor emerging risks. Departments usually provide HM 55 Qq 52–53 56 Q 54 57 C&AG’s Report, Financial modelling in government, paras …
Government response. 6.1 The government agrees with the Committee’s recommendation. Target implementation date: March 2023 6.2 The Finance Function will work with the Analysis Function to identify appropriate analysis and modelling considerations relating to uncertainty and incorporate these into the Finance Standard, …
HM Treasury