Challenges in implementing digital change

There is no clear plan to replace or modernise legacy systems and data that are critical to service provision but are often old, unsupportable, vulnerable and a constraint on transformation. Legacy systems, some of which date back to the 1970s, are widespread across government, which relies on them for important services such as managing the UK’s borders and paying the State Pension. Many of these systems are stable and performing acceptably at reasonable cost, but others are high-risk, unreliable, contain security vulnerabilities, or frustrate business transformation. Despite this, departments have a limited understanding of their legacy estate and the centre of government does not have an ongoing process for assessing and understanding what cross-government legacy risk looks like. The Central Digital and Data Office (CDDO) is now testing how it can gather information on departmental legacy systems. The CDDO hopes to prioritise where departments most need to take remedial action using a risk-based model and to roll this out more widely at the start of 2022. This is encouraging and we look forward to seeing the centre and departments take concrete action to begin to address high-risk legacy systems next year. Recommendation: At the start of 2022 the CDDO should work with departments to map legacy systems across government to document what is there, why it exists and how critical it is. By the end of 2022 the CDDO should use this to produce a pipeline of legacy systems they have prioritised with milestones for action. This pipeline should be shared with the Committee.

Departments have failed to understand the difference between improving what currently exists and real digital transformation, meaning that they have missed opportunities to move to modern, efficient ways of working. The Cabinet Office concedes that too often operations and services fail to recognise the fundamental 6 Challenges in implementing digital change inefficiencies of legacy systems and their data; and instead build new systems on top of these using the old data. Programmes have not sufficiently considered the potential and longer-lasting benefits for the citizen of redesigning and transforming public services around new business processes, data, and emerging technologies. Government has developed several new systems in response to specific needs arising from EU exit, but these were all developed to meet a narrow purpose by fixed deadlines. We recognise the delivery of initiatives such as GOV.UK Notify and Pay, but we note these are relatively straightforward commodity services that are widely available commercially. We are not convinced these demonstrate government’s ability to secure the deeper levels of transformation needed to achieve a more fundamental redesign of business services for the digital age. Cultivating better systems analysis, data analysis and design skills would help government gain a greater understanding of where to focus its efforts. Recommendation: The Cabinet Office and departments should introduce a structured way of deciding whether the changes they are making represent incremental improvements to existing systems, or a more transformational redesign of business processes. The Cabinet Office and departments should reflect this in the depth and rigour of the initial scoping and design of programmes.

Digital programmes often fail to have their own single programme office to support the programme director to align all aspects throughout the lifetime of the programme, including integration of legacy and future systems. Although the Cabinet Office has started to try to strengthen central oversight of digital programmes, government’s digital functions are relatively immature when compared with the long-standing control exercised by HM Treasury over financial management. The breadth and complexity of government presents a significant challenge to putting in place common approaches and ensuring programmes do what they are supposed to do in the right way. Departments find it difficult to manage integration and connections between different systems, especially legacy systems, both within and outside programmes. Departments need a strong, internal programme office for each major programme. These programme offices need a depth of digital expertise beyond what might be provided by a non-specialist programme board, backed up by a strong function at the departmental level and at centre of government (e.g., a centre of excellence) to ensure this happens. A recent report commissioned by the Cabinet Office found there has been too much reliance on contractors or external consultants for core design or architectural decisions, which need a long-term and holistic view across government. The Cabinet Office told us that it has already accepted and is implementing the report’s recommendations. These include a strong, expert central team with real ‘teeth’ to take ownership of digital, data and technology standards including technology choices, privacy, security, and interoperability. Recommendation: The Cabinet Office should develop guidance on how to approach legacy integration, and mandate rigorous and professional design, data and infrastructure controls and practices, with appropriate accountabilities.

Departments have failed to develop a modern professional approach to IT operations needed to support business change and transformation and have created an over-reliance on outsourcing. Departmental IT functions are often not structured or funded in a way that allows them to run and update their core systems Challenges in implementing digital change 7 and support business transformation. This is not helped by the current problem of alignment between Spending Review allocations and annual funding and the need to deliver outcomes over long periods in major multi-billion-pound programmes, which have led to under-investment in resolving legacy system issues and in maintaining systems’ ongoing fitness for purpose. Government accepts the less successful programmes were typically excessively outsourced by departments who had a thinly resourced internal client function. This led to a dependence on external consultants who may have considerable expertise in some areas but who have little or noknowledge of the existing systems. Government also concedes that in the past it selected technology too early in programmes and incorporated it into supplier contracts, before fully understanding the business problem to be solved. This has inhibited flexibility and the ability to change direction when needed. Government has been slow to pivot or stop programmes when it has become clear they are not working, as typified by GOV.UK Verify. Recommendation: The Central Digital and Data Office should set out what departments need to put in place to improve the maturity of departments’ approach to IT operations and change including: • what the Intelligent Client Function should do; • what influence Digital specialist leaders should have; • who should be accountable and responsible for contracting; and • the assurance mechanisms at the beginning and throughout the lifecycle.

There is a large gap between the demand for and supply of the digital specialists that government needs, and it is hard to get the right balance of in-house and outsourced skills. Government has been excessively reliant on outsourcing and has failed to retain sufficient in-house capability. The Central Digital and Data Office now has responsibility for government’s capability building strategy, but recognises government does not yet have the depth of experience needed to deliver across the full breadth of its digital portfolio. The demand for digital skills is increasing, not just in the public sector but also in the private sector and there are shortages in key areas such as architects and people who can lead delivery. We are concerned that despite its intent government is yet to make meaningful progress in overcoming long-standing challenges. It wishes to bring more digital activity in-house but faces a difficult external recruitment market. Specialists in scarce supply earn significantly more money in the private sector than the government is willing to pay, so it can be hard for the civil service to attract and retain them at appropriate levels. Recommendation: The Central Digital and Data Office should write to us, within 6 months, setting out how it intends to measure progress against its capability strategy, and annually thereafter to report what progress it has made against those metrics. 8 Challenges in implementing digital change 1 Leadership and capability

On the basis of a report by the Comptroller and Auditor General, we took evidence from the Cabinet Office and HM Revenue & Customs (HMRC) on the challenges in implementing digital change in government.1

Many of the systems government use are not really geared up to modern technology and some date back to the 1970s.14 The Cabinet Office told us that government is a difficult environment for digital projects, partly because these previous systems and old data, known as ‘legacy’, can stand in the way of progress.15 Government relies on legacy systems for many important services.16 For example legacy systems are used by the Home Office to manage the UK border and by police forces to access criminal records and share intelligence.17 This means government is often building on top of existing systems with complex operational dependencies.18 This makes it challenging to move off legacy systems to modern replacements.19

The risks associated with legacy systems include that they can be difficult and expensive to support, lack operational resilience for key government services, and be vulnerable to cyber-attack. This exposes government to what is likely to be an uncertain but high level of financial risk from potential operational and cyber-related incidents. Legacy systems need a significant level of resource to make more regular, incremental updates over time.20 Given there are so many legacy systems in government, we asked how long it will be before it becomes necessary to start replacing them.21 The Cabinet Office responded that it wants to focus on those systems that are high-risk, or which present an impediment to business transformation.22 This requires departments to have a plan for how to manage their legacy estate, including sufficient funding for maintenance.23

Departments typically do not have a good understanding of their IT estate, legacy systems are often poorly understood because of their age, and replacing these systems 10 Q 7 11 Dr Jerry Fishenden, Professor Mark Thompson and Assistant Professor Will Venters submission page 2 12 Qq 13–14 13 Qq 27, 7 14 Chair opening statement 15 Q 1 16 C&AG’s Report, para 2.15 17 Committee of Public Accounts, Digital Services at the Border, Forty-Eighth Report of Session 2019–21, HC 936, 12 March 2021; Committee of Public Accounts, e-Borders and successor programmes, Twenty-seventh Report of Session 2015–16, HC 643, 4 March 2016; Committee of Public Accounts, The National Law Enforcement Data Programme, Twenty-Ninth Report of Session 2021–22, HC 638, 8 December 2021 18 Q 1 19 C&AG’s Report, para 2.17 20 C&AG’s Report, paras 2.25–2.16 21 Q 39 22 Q 40 23 C&AG’s Report, para 8h 10 Challenges in implementing digital change carries risk.24 We asked the Cabinet Office if anyone has a clear picture as to what major legacy systems exist across government. The Cabinet Office told us that it has recently started to try and build-up this overall picture and that departments are getting more detailed in their understanding. However, it acknowledged there is not yet a clear on- going process for assessing and understanding cross-government legacy risk. The Cabinet Office is piloting a simple, risk-based model with three departments that it hopes will provide this dynamic view of legacy IT across government and enable conversations about funding with HM Treasury. This process is due be rolled out more widely in 2022.25 Missed opportunities for transformation

The Cabinet Office concedes that too often operations and services fail to recognise the fundamental inefficiencies of legacy systems and the data they contain.26 The National Audit Office reported that government has been trying to improve its digital performance for the past 25 years27 but has not been successful in achieving the intended results and departments have continued to build on top of legacy systems and data.28 Programmes have concentrated on automating existing way of working without focusing sufficiently on trying to redesign public services around new business processes and use of data and technology in new ways.29

Government has developed several systems to meet new requirements, such as for checking and processing applications under the EU Settlement Scheme. However, the deadline for EU Exit and Government decisions on the timetable for withdrawing the rights previously exercised by EU nationals meant that these systems all had to be in place by a fixed deadline. Government designed and built these systems to meet own their specific, narrow purposes so they could deliver them on time.30 This represents another missed opportunity to think further ahead about how these systems and their associated data flows could be more widely joined-up across government.

The Cabinet Office pointed to the role that central platforms such as GOV.UK Pay and Notify have played, for example sending alerts and test results to citizens during the COVID-19 pandemic.31 But the functions they perform are relatively straightforward commodity services that are already easily available and used extensively in the private sector. We are not convinced these examples demonstrate government’s ability to secure the deeper levels of transformation and business change needed to achieve a more fundamental redesign of services for the digital age.32 Cultivating better systems analysis, data analysis and design skills would help government gain a greater understanding of where to focus its efforts.33 24 C&AG’s Report, para 2.18 25 Q 39 26 C&AG’s Report, paras 2.15–2.21 27 C&AG’s Report, Figure 3 28 Q 18; Cabinet Office, Organising for Digital Delivery, published 22 July 2021, p. 2. Available at https://www.gov. uk/government/publications/organising-for-digital-delivery 29 Qq 22, 67 30 Qq 7, 36–37 31 Q 68 32 Q 4 33 Dr Jerry Fishenden, Professor Mark Thompson and Assistant Professor Will Venters submission page 4 Challenges in implementing digital change 11 2 Co-ordination, skills and outsourcing Strong and accountable co-ordination across government

Government has not properly structured and resourced technology departments against good standards. This makes it difficult to manage the integration and connections between different systems, especially legacy systems, both within and outside programmes.34 This lack of resource and structure means that digital programmes do not have a strong technical design function and programme office for each major programme. Departments need to staff these programme offices with people who have sufficiently deep digital expertise, and the centre should support them with a strong central team. This can help ensure departmental programmes deliver what is in the best interests of government as a whole.35

Government’s central digital functions are relatively immature. Government only created the Central Digital and Data Office this year and set up the other central functions within the last decade. The Cabinet Office told us that they have started to strengthen central oversight of digital programmes, but we do not yet see a similar degree of oversight and control as established institutions such as HM Treasury exercise over the way departments manage their finances.36 Government operations are broad and complex, and departments are responsible for their own digital programmes, operating in a federated manner. This presents a significant challenge to putting in place common approaches.37

According to a recent report, government has had to rely on external contractors and consultants to make critical decisions on core design and architecture choices, which need a long-term and holistic view across government.38 The report recommends government creates a strong, expert central team with real ‘teeth’ to take ownership of digital, data and technology standards including technology choices, privacy, security, and interoperability. The Cabinet Office told us that it accepts the recommendations of this report and has already implemented four of them with the others being in progress.39 Approach to suppliers and technology

Funding for departmental technology functions is often not allocated in a way that enables them to both run and update their core systems and support business transformation. This Committee has seen in other digital programmes that the current spending review processes, which set separate capital and revenue allocations in short, fixed cycles, have contributed to an under-investment in resolving legacy system issues and ensuring systems remain fit for purpose.40 34 Qq 52–53 35 Qq 2, 70–71 36 Qq 2–3, 11–12 37 Q 70 38 Cabinet Office, Organising for Digital Delivery, published 22 July 2021, p. 9. Available at https://www.gov.uk/ government/publications/organising-for-digital-delivery 39 Q 59; Cabinet Office, Organising for Digital Delivery, published 22 July 2021, pp. 5–6. Available at https://www. gov.uk/government/publications/organising-for-digital-delivery 40 Qq 39, 41, 43; Cabinet Office, Organising for Digital Delivery, published 22 July 2021, p. 7. Available at https:// www.gov.uk/government/publications/organising-for-digital-delivery 12 Challenges in implementing digital change

Government accepts that it is important to have deep digital, data and technology expertise preserved within departments. Government has excessively outsourced programmes and has very thinly resourced the internal client function capability within departments. That has been a problem and has led to a dependence on external experts, without sufficient internal expertise.41

Departments have made technology choices too early in the life of digital programmes before they have fully understood the business problem the programme is seeking to address.42 Departments have then over specified supplier contracts to a degree of detail which does not match the department’s knowledge and this has resulted in a loss of flexibility.43 Consequently, departments have struggled to change the direction of programmes in response to later events.44 Departments have also been slow to stop programmes when it has become clear they are not working. An example is GOV.UK Verify, which after years of development still does not work for around half of the people trying to use it.45 The Cabinet Office acknowledges government can learn from the private sector how to do this better. This will require a change in culture and a willingness not to regard such changes in direction as failures, when they are a response to additional information emerging over time.46 Skills and outsourcing

Ten years ago, government had little in-house digital capability and outsourced most technology programmes.47 The Cabinet Office blamed the failure of many programmes in the past on too much outsourcing of poorly understood services, and insufficient resourcing of the intelligent client function within departments.48 For example, NHS England did not believe it had the skills in-house to transform primary care services, but also did not know enough about those services to be able to set the supplier achievable performance standards. The supplier underestimated the scale of the task, and patient safety was potentially put at risk.49 The National Law Enforcement Data Service programme needed to manage multiple supplier contracts but did not take enough ownership of technical development work, and had limited ability to integrate the work and the way the suppliers needed to work together.50 Working with large suppliers is an important way to access the experienced delivery expertise which the Cabinet Office accepts it still does not have enough of.51

The Cabinet Office accepted that government still has a capability problem and is not as sophisticated as it needs to be.52 There are not yet enough people with the depth 41 Qq 12, 73 42 Q 4, 73 43 Qq 4, 46, 63 44 Q 4 45 Q 48 46 Qq 43–44, 62, 65–66, 73 47 Q 7 48 Q 12 49 Committee of Public Accounts, Supporting Primary Care Services: NHS England’s contract with Capita, Fifty- Seventh Report of Session 2017–19, HC 698, 25 July 2018; C&AG’s Report, figure 8 50 C&AG’s Report, The National Law Enforcement Data Programme, Session 2021–22, HC 663, 10 September 2021, Para 2.15 51 C&AG’s Report, The challenges in implementing digital change, Session 2021–22, HC 575, 21 July 2021, para 2.11 52 Qq 21, 73 Challenges in implementing digital change 13 of experience needed to deliver across the breadth of the governments digital portfolio.53 Government is trying to build its internal skills and capabilities at a time when the external recruitment market is challenging.54 The Cabinet Office agreed that the supply of digital skills is not enough to match the demand in the public and private sectors. This leads to shortages of key skills, such as digital architects and software developers.55 Government also faces challenges in building capability beyond those faced by the private sector. Government digital and technology leaders surveyed by the National Audit Office identified the amount departments can pay specialists and civil service recruitment processes as the biggest barriers to recruitment and retention.56 53 Q 73 54 C&AG’s Report, para 3.4 55 Q 7 56 C&AG’s Report, figure 13 14 Challenges in implementing digital change