Select committee inquiry
Challenges in implementing digital change
Closed
What the inquiry looked at
“Government plans to transform public services are increasingly led by digitally-enabled business change, and it is essential that public bodies deliver high quality digital services in a time when our way of life is increasingly digital. But a recent NAO report found that in “25 years of government strategies and countless attempts to deliver digital business change successfully , there is a consistent pattern of underperformance. This underperformance is often the result of decision makers fixing on technology solutions before fundamental aspects of projects and programmes are sufficiently thought through.” The NAO found that “only a small proportion of senior officials in government have first-hand experience of digital business change”, and that “many of the problems that occur in large digital operational change programmes stem from senior decision-makers’ inability to understand the issues and make the decisions required to implement digital change in an effective way”. The NAO concluded that current pressures on public finances mean there is an urgent need for those designing and delivering digital business change programmes to learn from past mistakes. The Committee will question senior officials at the Cabinet Office, Government Digital Service and HMRC on the capacity and capability of senior decision-makers in the public sector to deliver digital business change. If you have evidence on these questions, please submit it here by 6pm on Monday 20 September 2021.”
The committee's words, from the inquiry page on parliament.uk ↗.
What it found
In the committee's words, from the summary of Thirtieth Report - Challenges in implementing digital change ↗
Digital change, on which government estimates it spends around £20 billion each year, offers a real opportunity for the government to transform its ways of working and how it provides services to citizens. However, we are concerned by the number of complex, large-scale digital programmes we continue to see fail and the impact this has on important government services and taxpayers’ money. For example, failures in NHS England’s efforts to transform primary care services potentially put patients at risk of serious harm. More recently, the Home Office’s programme to replace the police national computer has been delayed by at least five years with an associated cost overrun of more than £400 million.
It is clear to us that government faces significant long-term barriers to successful digital change that will take much time, effort, and money to overcome. For example, there is a lack of digital skills and capability among government’s senior non-specialist leadership, who need to understand the scope of these vast programmes, many of which have embedded ‘legacy’ systems. Such systems are widespread across government but can be unreliable, hard to support, and frustrate efforts to modernise services. This situation is not helped by the scarcity of specialist digital, data and technology skills across both the public and private sectors.
We have heard about progress made over the last decade and what government feels are some important recent success stories, such as the COVID-19 furlough schemes.
However, most of these are not large-scale transformational programmes and we remain sceptical of the ability to succeed in this area given the examples we have seen of recent programme failures. Therefore, we also welcome the renewed efforts from the centre of government towards tackling the long-term barriers that prevent real modernisation.
There is much to do at both the centre and within departments if effective digital change is to become business as usual for government. In addition, Ministers generally spend a relatively short time in any one post, while Permanent Secretaries typically only serve five-year terms. Neither is likely to remain in post for the entire duration of a major digital change programme. Digital change planning therefore needs to be a core activity for Whitehall to deliver as “business as usual”, as some programmes could take up to 20 years to deliver fully. The citizen and the taxpayer deserve a better service and we will continue to challenge departments in front of us about how they are delivering on these ambitions.
What the government said
The government responded on 1 March 2022. The deadline was 10 February 2022.
Summary written by Accountability Index from the government's response.
The government states that it agrees with the Committee's recommendation on each of the six conclusions set out in the response, with target implementation dates between summer 2022 and the end of December 2022.
- On digital education for leaders, it agrees with the recommendation (target: August 2022). It says training for Ministers and senior officials "has already begun and will be expanded upon in the 2022-23 financial year".
- On legacy systems, it agrees (target: end of December 2022). CDDO is working with departments on a common methodology for identifying and prioritising legacy risk, and "aims to implement this across ministerial departments by the end of 2022".
- On IT operations, it agrees (target: Summer 2022). CDDO will support departments through "the publication of an updated DDaT Functional Standard and an accompanying DDaT Assessment Framework".
- On the digital skills gap, it agrees (target: Summer 2022, and annually thereafter). A report detailing agreed metrics "will be shared with the Committee within 6 months".
Read the government's response ↗ Read the report ↗ Report an error
How the inquiry ran
Each dot is a dated item listed on this page, from the inquiry's opening on 6 September 2021 to its close on 1 March 2022. The dashed mark is the deadline Parliament records for the government's response.
- Oral evidence 23 Sep 2021
- Report 10 Dec 2021
- Government response 1 Mar 2022
What it recommended 10 recommendations · 10 conclusions
2There is no clear plan to replace or modernise legacy systems and data that are...
Recommended (the committee's words)
There is no clear plan to replace or modernise legacy systems and data that are critical to service provision but are often old, unsupportable, vulnerable and a constraint on transformation. Legacy systems, some of which date back to the 1970s, are widespread across government, which relies on them for important services such as managing the UK’s borders and paying the State Pension. Many of these systems are stable and performing acceptably at reasonable cost, but others are high-risk, unreliable, contain security vulnerabilities, or frustrate business transformation. Despite this, departments have a limited understanding of their legacy estate and the centre of government does not have an ongoing process for assessing and understanding what cross-government legacy risk looks like. The Central Digital and Data Office (CDDO) is now testing how it can gather information on departmental legacy systems. The CDDO hopes to prioritise where departments most need to take remedial action using a risk-based model and to roll this out more widely at the start of 2022. This is encouraging and we look forward to seeing the centre and departments take concrete action to begin to address high-risk legacy systems next year. Recommendation: At the start of 2022 the CDDO should work with departments to map legacy systems across government to document what is there, why it exists and how critical it is. By the end of 2022 the CDDO should use this to produce a pipeline of legacy systems they have prioritised with milestones for action. This pipeline should be shared with the Committee.
3Departments have failed to understand the difference between improving what currently exists and real digital...
Recommended (the committee's words)
Departments have failed to understand the difference between improving what currently exists and real digital transformation, meaning that they have missed opportunities to move to modern, efficient ways of working. The Cabinet Office concedes that too often operations and services fail to recognise the fundamental 6 Challenges in implementing digital change inefficiencies of legacy systems and their data; and instead build new systems on top of these using the old data. Programmes have not sufficiently considered the potential and longer-lasting benefits for the citizen of redesigning and transforming public services around new business processes, data, and emerging technologies. Government has developed several new systems in response to specific needs arising from EU exit, but these were all developed to meet a narrow purpose by fixed deadlines. We recognise the delivery of initiatives such as GOV.UK Notify and Pay, but we note these are relatively straightforward commodity services that are widely available commercially. We are not convinced these demonstrate government’s ability to secure the deeper levels of transformation needed to achieve a more fundamental redesign of business services for the digital age. Cultivating better systems analysis, data analysis and design skills would help government gain a greater understanding of where to focus its efforts. Recommendation: The Cabinet Office and departments should introduce a structured way of deciding whether the changes they are making represent incremental improvements to existing systems, or a more transformational redesign of business processes. The Cabinet Office and departments should reflect this in the depth and rigour of the initial scoping and design of programmes.
4Digital programmes often fail to have their own single programme office to support the programme...
Recommended (the committee's words)
Digital programmes often fail to have their own single programme office to support the programme director to align all aspects throughout the lifetime of the programme, including integration of legacy and future systems. Although the Cabinet Office has started to try to strengthen central oversight of digital programmes, government’s digital functions are relatively immature when compared with the long-standing control exercised by HM Treasury over financial management. The breadth and complexity of government presents a significant challenge to putting in place common approaches and ensuring programmes do what they are supposed to do in the right way. Departments find it difficult to manage integration and connections between different systems, especially legacy systems, both within and outside programmes. Departments need a strong, internal programme office for each major programme. These programme offices need a depth of digital expertise beyond what might be provided by a non-specialist programme board, backed up by a strong function at the departmental level and at centre of government (e.g., a centre of excellence) to ensure this happens. A recent report commissioned by the Cabinet Office found there has been too much reliance on contractors or external consultants for core design or architectural decisions, which need a long-term and holistic view across government. The Cabinet Office told us that it has already accepted and is implementing the report’s recommendations. These include a strong, expert central team with real ‘teeth’ to take ownership of digital, data and technology standards including technology choices, privacy, security, and interoperability. Recommendation: The Cabinet Office should develop guidance on how to approach legacy integration, and mandate rigorous and professional design, data and infrastructure controls and practices, with appropriate accountabilities.
5Departments have failed to develop a modern professional approach to IT operations needed to support...
Recommended (the committee's words)
Departments have failed to develop a modern professional approach to IT operations needed to support business change and transformation and have created an over-reliance on outsourcing. Departmental IT functions are often not structured or funded in a way that allows them to run and update their core systems Challenges in implementing digital change 7 and support business transformation. This is not helped by the current problem of alignment between Spending Review allocations and annual funding and the need to deliver outcomes over long periods in major multi-billion-pound programmes, which have led to under-investment in resolving legacy system issues and in maintaining systems’ ongoing fitness for purpose. Government accepts the less successful programmes were typically excessively outsourced by departments who had a thinly resourced internal client function. This led to a dependence on external consultants who may have considerable expertise in some areas but who have little or noknowledge of the existing systems. Government also concedes that in the past it selected technology too early in programmes and incorporated it into supplier contracts, before fully understanding the business problem to be solved. This has inhibited flexibility and the ability to change direction when needed. Government has been slow to pivot or stop programmes when it has become clear they are not working, as typified by GOV.UK Verify. Recommendation: The Central Digital and Data Office should set out what departments need to put in place to improve the maturity of departments’ approach to IT operations and change including: • what the Intelligent Client Function should do; • what influence Digital specialist leaders should have; • who should be accountable and responsible for contracting; and • the assurance mechanisms at the beginning and throughout the lifecycle.
6There is a large gap between the demand for and supply of the digital specialists...
Recommended (the committee's words)
There is a large gap between the demand for and supply of the digital specialists that government needs, and it is hard to get the right balance of in-house and outsourced skills. Government has been excessively reliant on outsourcing and has failed to retain sufficient in-house capability. The Central Digital and Data Office now has responsibility for government’s capability building strategy, but recognises government does not yet have the depth of experience needed to deliver across the full breadth of its digital portfolio. The demand for digital skills is increasing, not just in the public sector but also in the private sector and there are shortages in key areas such as architects and people who can lead delivery. We are concerned that despite its intent government is yet to make meaningful progress in overcoming long-standing challenges. It wishes to bring more digital activity in-house but faces a difficult external recruitment market. Specialists in scarce supply earn significantly more money in the private sector than the government is willing to pay, so it can be hard for the civil service to attract and retain them at appropriate levels. Recommendation: The Central Digital and Data Office should write to us, within 6 months, setting out how it intends to measure progress against its capability strategy, and annually thereafter to report what progress it has made against those metrics. 8 Challenges in implementing digital change 1 Leadership and capability
1On the basis of a report by the Comptroller and Auditor General, we took evidence...
Concluded (the committee's words)
On the basis of a report by the Comptroller and Auditor General, we took evidence from the Cabinet Office and HM Revenue & Customs (HMRC) on the challenges in implementing digital change in government.1
7Many of the systems government use are not really geared up to modern technology and...
Concluded (the committee's words)
Many of the systems government use are not really geared up to modern technology and some date back to the 1970s.14 The Cabinet Office told us that government is a difficult environment for digital projects, partly because these previous systems and old data, known as ‘legacy’, can stand in the way of progress.15 Government relies on legacy systems for many important services.16 For example legacy systems are used by the Home Office to manage the UK border and by police forces to access criminal records and share intelligence.17 This means government is often building on top of existing systems with complex operational dependencies.18 This makes it challenging to move off legacy systems to modern replacements.19
8The risks associated with legacy systems include that they can be difficult and expensive to...
Recommended (the committee's words)
The risks associated with legacy systems include that they can be difficult and expensive to support, lack operational resilience for key government services, and be vulnerable to cyber-attack. This exposes government to what is likely to be an uncertain but high level of financial risk from potential operational and cyber-related incidents. Legacy systems need a significant level of resource to make more regular, incremental updates over time.20 Given there are so many legacy systems in government, we asked how long it will be before it becomes necessary to start replacing them.21 The Cabinet Office responded that it wants to focus on those systems that are high-risk, or which present an impediment to business transformation.22 This requires departments to have a plan for how to manage their legacy estate, including sufficient funding for maintenance.23
9Departments typically do not have a good understanding of their IT estate, legacy systems are...
Concluded (the committee's words)
Departments typically do not have a good understanding of their IT estate, legacy systems are often poorly understood because of their age, and replacing these systems 10 Q 7 11 Dr Jerry Fishenden, Professor Mark Thompson and Assistant Professor Will Venters submission page 2 12 Qq 13–14 13 Qq 27, 7 14 Chair opening statement 15 Q 1 16 C&AG’s Report, para 2.15 17 Committee of Public Accounts, Digital Services at the Border, Forty-Eighth Report of Session 2019–21, HC 936, 12 March 2021; Committee of Public Accounts, e-Borders and successor programmes, Twenty-seventh Report of Session 2015–16, HC 643, 4 March 2016; Committee of Public Accounts, The National Law Enforcement Data Programme, Twenty-Ninth Report of Session 2021–22, HC 638, 8 December 2021 18 Q 1 19 C&AG’s Report, para 2.17 20 C&AG’s Report, paras 2.25–2.16 21 Q 39 22 Q 40 23 C&AG’s Report, para 8h 10 Challenges in implementing digital change carries risk.24 We asked the Cabinet Office if anyone has a clear picture as to what major legacy systems exist across government. The Cabinet Office told us that it has recently started to try and build-up this overall picture and that departments are getting more detailed in their understanding. However, it acknowledged there is not yet a clear on- going process for assessing and understanding cross-government legacy risk. The Cabinet Office is piloting a simple, risk-based model with three departments that it hopes will provide this dynamic view of legacy IT across government and enable conversations about funding with HM Treasury. This process is due be rolled out more widely in 2022.25 Missed opportunities for transformation
10The Cabinet Office concedes that too often operations and services fail to recognise the fundamental...
Concluded (the committee's words)
The Cabinet Office concedes that too often operations and services fail to recognise the fundamental inefficiencies of legacy systems and the data they contain.26 The National Audit Office reported that government has been trying to improve its digital performance for the past 25 years27 but has not been successful in achieving the intended results and departments have continued to build on top of legacy systems and data.28 Programmes have concentrated on automating existing way of working without focusing sufficiently on trying to redesign public services around new business processes and use of data and technology in new ways.29
11Government has developed several systems to meet new requirements, such as for checking and processing...
Recommended (the committee's words)
Government has developed several systems to meet new requirements, such as for checking and processing applications under the EU Settlement Scheme. However, the deadline for EU Exit and Government decisions on the timetable for withdrawing the rights previously exercised by EU nationals meant that these systems all had to be in place by a fixed deadline. Government designed and built these systems to meet own their specific, narrow purposes so they could deliver them on time.30 This represents another missed opportunity to think further ahead about how these systems and their associated data flows could be more widely joined-up across government.
12The Cabinet Office pointed to the role that central platforms such as GOV.UK Pay and...
Recommended (the committee's words)
The Cabinet Office pointed to the role that central platforms such as GOV.UK Pay and Notify have played, for example sending alerts and test results to citizens during the COVID-19 pandemic.31 But the functions they perform are relatively straightforward commodity services that are already easily available and used extensively in the private sector. We are not convinced these examples demonstrate government’s ability to secure the deeper levels of transformation and business change needed to achieve a more fundamental redesign of services for the digital age.32 Cultivating better systems analysis, data analysis and design skills would help government gain a greater understanding of where to focus its efforts.33 24 C&AG’s Report, para 2.18 25 Q 39 26 C&AG’s Report, paras 2.15–2.21 27 C&AG’s Report, Figure 3 28 Q 18; Cabinet Office, Organising for Digital Delivery, published 22 July 2021, p. 2. Available at https://www.gov. uk/government/publications/organising-for-digital-delivery 29 Qq 22, 67 30 Qq 7, 36–37 31 Q 68 32 Q 4 33 Dr Jerry Fishenden, Professor Mark Thompson and Assistant Professor Will Venters submission page 4 Challenges in implementing digital change 11 2 Co-ordination, skills and outsourcing Strong and accountable co-ordination across government
13Government has not properly structured and resourced technology departments against good standards.
Recommended (the committee's words)
Government has not properly structured and resourced technology departments against good standards. This makes it difficult to manage the integration and connections between different systems, especially legacy systems, both within and outside programmes.34 This lack of resource and structure means that digital programmes do not have a strong technical design function and programme office for each major programme. Departments need to staff these programme offices with people who have sufficiently deep digital expertise, and the centre should support them with a strong central team. This can help ensure departmental programmes deliver what is in the best interests of government as a whole.35
14Government’s central digital functions are relatively immature.
Concluded (the committee's words)
Government’s central digital functions are relatively immature. Government only created the Central Digital and Data Office this year and set up the other central functions within the last decade. The Cabinet Office told us that they have started to strengthen central oversight of digital programmes, but we do not yet see a similar degree of oversight and control as established institutions such as HM Treasury exercise over the way departments manage their finances.36 Government operations are broad and complex, and departments are responsible for their own digital programmes, operating in a federated manner. This presents a significant challenge to putting in place common approaches.37
15According to a recent report, government has had to rely on external contractors and consultants...
Recommended (the committee's words)
According to a recent report, government has had to rely on external contractors and consultants to make critical decisions on core design and architecture choices, which need a long-term and holistic view across government.38 The report recommends government creates a strong, expert central team with real ‘teeth’ to take ownership of digital, data and technology standards including technology choices, privacy, security, and interoperability. The Cabinet Office told us that it accepts the recommendations of this report and has already implemented four of them with the others being in progress.39 Approach to suppliers and technology
16Funding for departmental technology functions is often not allocated in a way that enables them...
Concluded (the committee's words)
Funding for departmental technology functions is often not allocated in a way that enables them to both run and update their core systems and support business transformation. This Committee has seen in other digital programmes that the current spending review processes, which set separate capital and revenue allocations in short, fixed cycles, have contributed to an under-investment in resolving legacy system issues and ensuring systems remain fit for purpose.40 34 Qq 52–53 35 Qq 2, 70–71 36 Qq 2–3, 11–12 37 Q 70 38 Cabinet Office, Organising for Digital Delivery, published 22 July 2021, p. 9. Available at https://www.gov.uk/ government/publications/organising-for-digital-delivery 39 Q 59; Cabinet Office, Organising for Digital Delivery, published 22 July 2021, pp. 5–6. Available at https://www. gov.uk/government/publications/organising-for-digital-delivery 40 Qq 39, 41, 43; Cabinet Office, Organising for Digital Delivery, published 22 July 2021, p. 7. Available at https:// www.gov.uk/government/publications/organising-for-digital-delivery 12 Challenges in implementing digital change
17Government accepts that it is important to have deep digital, data and technology expertise preserved...
Concluded (the committee's words)
Government accepts that it is important to have deep digital, data and technology expertise preserved within departments. Government has excessively outsourced programmes and has very thinly resourced the internal client function capability within departments. That has been a problem and has led to a dependence on external experts, without sufficient internal expertise.41
18Departments have made technology choices too early in the life of digital programmes before they...
Concluded (the committee's words)
Departments have made technology choices too early in the life of digital programmes before they have fully understood the business problem the programme is seeking to address.42 Departments have then over specified supplier contracts to a degree of detail which does not match the department’s knowledge and this has resulted in a loss of flexibility.43 Consequently, departments have struggled to change the direction of programmes in response to later events.44 Departments have also been slow to stop programmes when it has become clear they are not working. An example is GOV.UK Verify, which after years of development still does not work for around half of the people trying to use it.45 The Cabinet Office acknowledges government can learn from the private sector how to do this better. This will require a change in culture and a willingness not to regard such changes in direction as failures, when they are a response to additional information emerging over time.46 Skills and outsourcing
19Ten years ago, government had little in-house digital capability and outsourced most technology programmes.47 The...
Concluded (the committee's words)
Ten years ago, government had little in-house digital capability and outsourced most technology programmes.47 The Cabinet Office blamed the failure of many programmes in the past on too much outsourcing of poorly understood services, and insufficient resourcing of the intelligent client function within departments.48 For example, NHS England did not believe it had the skills in-house to transform primary care services, but also did not know enough about those services to be able to set the supplier achievable performance standards. The supplier underestimated the scale of the task, and patient safety was potentially put at risk.49 The National Law Enforcement Data Service programme needed to manage multiple supplier contracts but did not take enough ownership of technical development work, and had limited ability to integrate the work and the way the suppliers needed to work together.50 Working with large suppliers is an important way to access the experienced delivery expertise which the Cabinet Office accepts it still does not have enough of.51
20The Cabinet Office accepted that government still has a capability problem and is not as...
Concluded (the committee's words)
The Cabinet Office accepted that government still has a capability problem and is not as sophisticated as it needs to be.52 There are not yet enough people with the depth 41 Qq 12, 73 42 Q 4, 73 43 Qq 4, 46, 63 44 Q 4 45 Q 48 46 Qq 43–44, 62, 65–66, 73 47 Q 7 48 Q 12 49 Committee of Public Accounts, Supporting Primary Care Services: NHS England’s contract with Capita, Fifty- Seventh Report of Session 2017–19, HC 698, 25 July 2018; C&AG’s Report, figure 8 50 C&AG’s Report, The National Law Enforcement Data Programme, Session 2021–22, HC 663, 10 September 2021, Para 2.15 51 C&AG’s Report, The challenges in implementing digital change, Session 2021–22, HC 575, 21 July 2021, para 2.11 52 Qq 21, 73 Challenges in implementing digital change 13 of experience needed to deliver across the breadth of the governments digital portfolio.53 Government is trying to build its internal skills and capabilities at a time when the external recruitment market is challenging.54 The Cabinet Office agreed that the supply of digital skills is not enough to match the demand in the public and private sectors. This leads to shortages of key skills, such as digital architects and software developers.55 Government also faces challenges in building capability beyond those faced by the private sector. Government digital and technology leaders surveyed by the National Audit Office identified the amount departments can pay specialists and civil service recruitment processes as the biggest barriers to recruitment and retention.56 53 Q 73 54 C&AG’s Report, para 3.4 55 Q 7 56 C&AG’s Report, figure 13 14 Challenges in implementing digital change
Evidence
Oral evidence sessions 1
- 23 Sep 2021Challenges in implementing digital change4 witnesses, listed belowparliament.uk ↗
Who gave evidence 4 witnesses
- Alex ChisholmChief Operating Officer Civil Service and Permanent Secretary, Cabinet Office
- Angela MacDonaldDeputy Chief Executive and Second Permanent Secretary, HMRC
- Joanna Davinsoninterim Government Chief Digital Officer, Cabinet Office
- Tom ReadChief Executive Officer, Government Digital Service