Evaluating innovation projects in children’s social care

We are not yet convinced the Department’s dissemination of learning from the programme is delivering widespread improvement. The Innovation Programme and its successor schemes have spread practice from six promising innovations across 57 further local authorities. There is some evidence that practice from the schemes is being taken up outside the scope of the funded programme. There is encouraging early data indicating the potential impacts of projects in these successor schemes. The Department does not yet have a complete picture of the impact of the programme on outcomes for children, however, and the evaluations from the successor schemes are only due between Autumn 2022 and 2027. The Department recognises the need to balance the tensions between the time needed for its formal evaluations to report, and the desire in the sector for early action. Understanding impact is also complicated by the lack of a coherent set of outcomes for children’s social care. The Independent Review of Children’s Social Care considers that the programme’s ‘scale and spread’ approach is limited in the absence of fundamental change and that there is sufficient evidence already for investment in new approaches, warning the costs of inaction are too high. Recommendation: The Department must set out a coherent set of outcomes it expects from the sector in its response to the Independent Review of Children’s Social Care, and further report on the impact of the innovation programme and successor schemes in supporting these outcomes. Recommendation: The Department should set out how it will secure a better understanding of the take-up of learning by local authorities across the country.

The Department does not yet have the data it needs to understand the impact of the innovation programme. Systems that provide high quality information to everyone involved can contribute to a culture of evaluation and an understanding of 6 Evaluating innovation projects in children’s social care what is working. Lack of data has proved a challenge for many Innovation Programme projects. Furthermore the small scale of projects within the Innovation Programme risks creating statistical challenges understanding the impacts of these schemes on particular minority groups. The Department needs to balance support for local authorities collecting the information they need locally, with considerations for what is required nationally to understand the effectiveness of these schemes across population groups. More widely, the Department concedes there is not currently a shared view on what data needs to be collected across the children’s social care sector. The Department reports it has work underway in response to recommendations around data raised by the Independent Review of Children’s Social Care. Recommendation: The Department should set out the standards it expects for local data collection, and make clear the benefits for local authorities of collecting good quality data. The Department should also use its new outcomes framework to shape its own data collection strategy.

Potential innovation risks being hampered by inflexibility in the wider system of children’s social care. The Department intended the Innovation Programme to both improve outcomes for children in the social care system and produce savings. Residential care in children’s homes is especially expensive, while outcomes for children appear better for those supported in kinship settings. The Department intends its response to the Independent Review of Children’s Social Care to promote support for keeping children in kinship groups or foster care as preferred alternatives. However, we have seen many examples of local barriers to supporting children in these settings, including inflexibility around the costs required to accommodate children taken into kinship care or foster homes, and help around the work and lifestyle changes required to effectively support them. The local government funding system is not always good at adapting to these needs, which risks resulting in more expensive residential provision being required. Recommendation: The Department should work with the Department for Levelling Up, Housing & Communities and HM Treasury to develop plans for addressing the local funding boundaries and barriers that stop children getting the help they require.

A challenging funding environment requires that government maintains its commitment to evaluation, and applies its learning to secure better outcomes. As the Innovation Programme and its successor schemes move towards the end of 14 years of funding by 2027, the Department wants to see innovation and learning activity move into mainstream practice. The phasing out of dedicated funding comes at a time the Department accepts is challenging for local authority budgets. The Innovation Programme has demonstrated that quality evaluation is expensive, but we share the Department’s view that it will usually represent what it called a ‘rounding error’ when set alongside the value of mainstream services. The need for the best evidence to support the most impactful practice remains, and it is vital that the opportunities to secure better outcomes created by the Innovation Programme are not lost. Evaluating innovation projects in children’s social care 7 Recommendation: The Department should set out how it will demonstrate the benefits of its spending on innovation and evaluation for local authorities and other Departments to secure the full benefits of this spending. 8 Evaluating innovation projects in children’s social care 1 Delivering impact through innovation and evaluation Introduction

On the basis of a report by the Comptroller and Auditor General, we took evidence from the Department for Education (the Department), including its Chief Social Worker for Children and Families, about the Children’s Social Care Innovation Programme (the Innovation Programme). The Department has overall policy responsibility for children’s social care and is accountable to Parliament for ensuring that the social care services local authorities provide is of adequate quality to protect and support children.1 In 2019–20 local authorities spent some £9.2 billion on children’s social care.2

The Department believes the impact of the Innovation Programme on the sector has been ‘transformational’. It describes the demand stimulated by the programme, having received over 200 applications for the first funding round of a programme where evaluation was a condition of funding.15 The Department also described issues caused by the limited capacity and expertise in the market to provide the evaluations the programme required, and the decision to set up a What Works Centre for Children’s Social Care in 2017 to take this agenda forward.16 The Department told us there is more of that capacity as a legacy of the Innovation Programme.17 What Works for Children’s Social Care also emphasised the work it now does through its Evidence Hubs, sharing research findings with front-line practitioners and aiming to ensure the importance of evidence is understood by senior leaders in the sector.18.

The Department concedes there is further to go to embed the culture of evaluation it wants to see in children’s social care. The Department and the sector have learned that quality evaluation is expensive and can take a long time to do.19 However one evaluation provider told us that robust and rigorous evaluations are not yet routine in children’s social care, with many practitioners opposing the more robust techniques such as randomisation on principle.20 Following the recommendations of the Independent Review of Children’s 9 C&AG’s Report para 3.1 10 C&AG’s Report paras 18, 20 11 C&AG’s Report Figure 2 12 C&AG’s Report para 2.4 13 Qq 15, 53 14 Q 15 15 Q 42 16 Q 31 17 Q 42 18 EPC0004 p.3 19 Qq 31, 43 20 EPC0001 p.2 10 Evaluating innovation projects in children’s social care Social Care (the Care Review), What Works for Children’s Social Care and the Early Intervention Foundation have announced they will merge. The Department sees this as an opportunity to strengthen methodologies and evidence standards, and its ability to embed evaluation more effectively across children’s social care.21 Understanding impact

The Department intended the Innovation Programme to achieve replication of successful new approaches, better life chances for children receiving help from the social care system and better value for money across children’s social care. The Department funded 94 projects through the Innovation Programme, before selecting six it considered to be most promising from initial evidence for further testing across 57 further locations through its two key successor programmes.22 In all, the Department reported 58 of the original 94 projects continuing after initial funding ended, 38 of which had spread beyond their initial locations.23 Hertfordshire County Council, one of the innovator authorities funded by the Innovation Programme, described extending the reach of its Family Safeguarding project to 27 other authorities, only 10 of which were directly funded by the Innovation Programme and key successor funds.24

We agree with successor programme evaluator Coram, that the key measure of success for the Department’s Innovation Programme must be what has changed for children as a result.25 The Department told us that it cannot yet report any ‘perfectly causal line’ between the programme and improved outcomes for children. The Department highlighted encouraging data from one of its successor projects including a 38% reduction in arrests and a 57% reduction in A&E visits for the young people referred to the scheme, with similar good evidence from other projects.26 Understanding the impact of the scheme is made harder by the lack of clarity over the outcomes government wants for children’s social care. The Department has undertaken to provide a national framework of outcomes in response to recommendations of the Care Review.27

The Care Review, while recognising the role the Innovation Programme has played in creating stronger practice, asserts that the impact of its scale and spread approach is limited in the absence of more fundamental change. The Care Review considers the evidence backing its own recommendations is already ‘compelling and comprehensive’, while the evaluations from the Department’s successor schemes are only due between Autumn 2022 and 2027.28 The Department is having to balance the tensions between the time taken to learn about outcomes from its longer-term evaluations and the desire in the sector for early action.29 The Department considers that it is moving towards the point of transitioning from scale and spread into changing mainstream approaches of social care practice in order to influence impacts on a greater scale.30 21 Q 52 22 C&AG’s report, paras 9, 18, 2.2 and 3.1 23 Q 56 24 EPC0003 p.2 25 EPC0001 p.1 26 Qq 54, 57 27 Qq 44, 53 28 Josh MacAlister, The Independent Review of Children’s Social Care Final Report, May 2022 p.46, and C&AG’s Report para 20 29 Q 14 30 Qq 72–73 Evaluating innovation projects in children’s social care 11 2 Supporting the system for improving children’s social care Improving data in children’s social care

The lack of high-quality data was one of the significant challenges and common limitations identified in the Department for Education’s (the Department’s) independent assessments of evaluations of Innovation Programme projects.31 We share the Department’s view that systems that give better information to everyone involved can contribute to a culture of evidence and evaluation “so that people can see much more readily what is working and then put things into practice.”32

There are particular data challenges where evaluations of small-scale innovations make it statistically difficult to understand the impact of schemes on particular population groups. We received evidence from the Traveller Movement, a population disproportionately represented in children’s social care, highlighting risks that lack of precision in monitoring impact on the distinct Traveller, Gypsy and Roma populations masks the different experiences of these groups.33 This mirrors other localised challenges we have seen around monitoring groups which are large in a specific area, but small nationally. The Department agrees that there are issues around what should be done locally monitoring specific communities and what needs to be done centrally, and accepts the need to be alert to that challenge.34

The Department could point to examples in other work where it has used technology to reduce the burdens of data collection, and shared data back with front line organisations in ways that incentivise local use.35 The Independent Review of Children’s Social Care (the Care Review) also reported on encouraging good practice, and recommended a National Data and Technology Taskforce to support improvement in data collection, sharing and use to inform decisions.36 The Department accepts that there is more to do to get agreement on what is important data in children’s social care. The Department told is it is working with the Association of Directors of Children’s Social Care in shaping its response to data issues raised in the Review.37 Tackling barriers to innovation

The Innovation Programme was intended to both improve outcomes for children in the social care system and produce savings. The Department accepts that residential care in children’s homes can be expensive and it is often a last resort where they do not ideally want children to be.38 The Care Review reports the cost of places in children’s homes running as high as £5000–7000 per week.39 The Care Review also sets out the improved health, attainment, employment and sibling connection outcomes for children supported 31 C&AG’s Report, para 16 32 Q 34 33 EPC0005 p.2 34 Qq 40–41 35 Qq 32–34 36 The independent review of children’s social care, Final Report, p.12 37 Qq 32, 35 38 Q 78 39 The independent review of children’s social care, Final Report, p.121 12 Evaluating innovation projects in children’s social care through kinship care compared to other residential settings.40 Our witnesses described the ‘huge priority’ around supporting children to live in families, whether kinship care or fostering in line with the recommendations of the Care Review.41

We have seen many cases of the potential for better outcomes and potential savings being blocked by inflexibility in the care system. These typically involve the costs of the space required to provide kinship or foster care, or support with the changes to working patterns that providing such care might require. The system can be complex, particularly where multiple organisations and different local authority areas are involved.42 Further, the Care Review highlights the financial challenges faced by kinship carers, many of whom ‘live in more deprived areas and are generally poorer than foster carers. Some studies report that financial distress is reported by 70% of kinship carers, and in one study 39% of kinship carers reported being in debt.’43

The Department told us it wants the care system to be more flexible, and accepts the system is currently not very good at adapting to innovative or usual requests. The result is that the child will often end up in care settings that cost more than the adaptation might have done. The Department explained it wants the system to start from an understanding of “What will help this child get the right outcome, and how can we deploy this amount of money that we are highly likely to spend on the child anyway?” The Care Review estimated that the average cost of the provision of public services for those who enter care was £70,900, and above £200,000 for the annualised costs of an independent sector residential care placement.44 The Department stressed however that the funding system is not within its individual control, and can be hard for it to influence.45 Securing the benefits of evaluation

Having started out in 2014–15, the delivery phases of the Innovation Programme and its successor schemes are approaching their end. Programme funding is expected to finish in March 2024, with the remaining evaluations reporting by 2027.46 The Department stresses that ring-fenced funding streams for innovation and learning are not indefinite, and that they want to see this activity shifted into mainstream funding and practice.47 The Department is confident that enough progress has been made, with sufficient positive gains in practice that the argument for continued investment in evidence-based policy should be easier to make.48 Hertfordshire County Council for instance highlighted experience it has already had supporting local authorities making invest to save bids for further work.49

The phasing out of dedicated funding for evaluation comes at a time the Department accepts is challenging for local authority budgets.50 The Department described how the Innovation Programme has helped both the Department and the wider sector understand that quality evaluation is expensive, but we share the Department’s view that the costs 40 Care Review p.94 41 Q 78 42 Q 81 43 The independent review of children’s social care, Final Report, p. 104 44 Q81–82; The independent review of children’s social care, Final Report, p. 47, The Case for Change, p.59 45 Q81 46 C&AG’s Report, paras 9, 20 47 Q 14 48 Q 46 49 EPC0003 p.1 50 Q 51 Evaluating innovation projects in children’s social care 13 often represent a ‘rounding error’ when contrasted with the £9 billion local authorities spend on children’s social care each year.51 A sense of potential savings is indicated by a participant in the successor schemes citing early evidence of 40% reductions in children being taken into care, with estimated cost avoidance of £117 million over 5 years.52

We would not want to see financial pressures making it harder for local authorities to make the case for building evaluation and learning into their future ways of working.53 The Local Government Association stresses the need to build on the success of the Innovation Programme as councils deal both with financial pressures and children requiring support for more complex needs, citing the need for the best evidence to be used to support ongoing improvement.54 It is vital that the Department tracks the use and impact of evidence-based policy in children’s social care to ensure the opportunities created by the Innovation Programme are not lost. 51 Qq 31, 43 52 EPC0003 p.2 53 Q 46 54 EPC0002 p.3 14 Evaluating innovation projects in children’s social care