What do crime rates, NHS waiting times and the attainment gap in schools have in common? The answer is simple but tragic: all three measures have got worse in England over the last decade. In fact, almost across the board public services are failing to deliver for citizens. This is despite the UK having highest tax revenue since the 1940s (though still lower than comparable countries).

It appears that the basic social contract – by which voters pay in tax to a collective pot and government spends this effectively to provide a safety net and enable people to thrive – is now broken. It is little wonder then that trust in politics is in freefall and that measures of public satisfaction with public services like the NHS have been falling as well.

Why are public services failing? Has government really become less effective at teaching our children, keeping us safe and providing us with care? The answer to these questions is of course complex – and varies significantly between different public services – but two general trends can be outlined:

  • First, there has been an increase in demand for public services. This is the result of growing populations, more complex needs (eg an ageing population) and higher public expectations as both social norms and the frontier of what is scientifically and technologically possible shifts.
  • Second, the state has been constrained in its ability to deliver more and better services. The most widely discussed shift here is the choice to impose austerity in 2010, followed more recently by spending constraints imposed on the back of Covid-19 and the inflation crisis.

But, public spending constraints are not the only factor limiting the state’s ability to deliver. The other major change has been the decline of public service reform in recent decades, with much less emphasis put on it by politicians and public debate. This is a lesser told story but one that is just as vital in explaining how we got here – and how we can chart a course out.

The rise and fall of public service reform

The rise and fall of public service reform is most clearly illustrated by comparing the 2001 and 2019 election winning manifestos. In 2001, the Labour Party, made public service reform the centrepiece of their offer to the country. The word ‘reform’ was mentioned 78 times in their manifesto. This compares to just 10 mentions by the Conservatives in 2019.

This is not just a reflection of the shift in governments (both the Labour and Conservative party have championed public service reform), changing political discourse or shifts in how manifestos are written. It is because parties and leaders have invested much less political capital in public service reform.

Notably, whilst recent Conservative (and coalition) governments have driven through some specific experiments in reform – the evidence revolution in schools or the devolution of power to Greater Manchester and West Midlands – this has failed to add up to a coherent cross-government approach reform.

Why has this happened? It can be argued that it is the result of the last major attempt at public service reform, New Public Management (NPM), running out of road in the late noughties. And despite multiple attempts to renew this agenda, there has been no coherent and practical alternative available for politicians to draw on to replace it.

This is why IPPR has kicked off a new research programme of work on Great Government. In the context of failing public services and angry citizens we are seeking to establish what works in public service reform and how this can be built into an ambitious new public service reform agenda for the 2020s. This first output seeks to set the scene for this programme.

New public management: running out of road

New public management was developed by a group of academics, think-tankers and journalists in the 1980s and 90s. The main argument put forward by this group was that in the absence of market forces public services suffered from weak or misaligned incentives. They argued that you simply cannot rely on the altruism of public servants alone to drive improvement.

The answer to this, they claimed, was reform attempts to bring stronger and better incentives into to the public sector. This was to be achieved through two main mechanisms:

  • the importation of private sector practices to the workings of public administrations including performance indicators (eg targets) and performance management (eg regulators and delivery units); and
  • the introduction of quasi-markets such as choice for ‘consumers’ (usually on quality rather than price) and competition between providers, often including private and third sector organisations.

This policy agenda has filtered into the British state (in particular in England) since the 1970s. But the most ambitious and coherent application of NPM occurred under New Labour between 1997 and 2010 alongside significant increases in public spending.

The policies undertaken at the time undoubtedly led to some significant improvements in public service outcomes - waiting times in the NHS fell, attainment in schools improved and crime was reduced. However, there is a still a big debate about what drove these improvements, and in particular, the relative impact of increased funding and the reform agenda.

Since 2010, the Coalition and subsequent Conservative governments have in many ways sought to deepen the NPM agenda. This can be seen most clearly in the Lansley reforms to the NHS, the academisation and free school agenda in schools and the contracting out of services like probation. This time, these reforms were attempted at the same time as significant public spending constraint overall, and cuts in many services. This has, if anything, raised further questions over the efficacy of NPM approaches.

An overview of this literature based on a literature review by Dr Eleanor Woodhouse commissioned as part of this research can be seen in figure 1.

Figure 1: Summary of effectiveness of NPM levers

LeverEffectiveness Explanation
TargetsCan drive improvement in the measured metrics but often at the expense of other outcomes. Can lead to gaming. When combined with top-down control can demotivate staff. Not conducive to complex problems which need local knowledge and flexibility.Focussing resource and attention on specific outcomes drives improvement, but at the expense of wider system performance.
Choice and competition Limited evidence of improvement of outcomes (access, quality) but evidence of increased inequality as a result of choice through worse outcomes for those from lower socioeconomic backgrounds. Outsourcing can reduce costs but sometimes at the expense of quality. Evidence that ability of the state to write complete contracts and actively commission is a key determinant of success. Limited-service user take up of choice and unwillingness of the state to allow providers to fail means incentives created by choice are weak. Providers compete by reducing their costs but cut quality enhancing inputs (eg staff).
Regulation Some evidence that it can drive providers from poor to good - but limited evidence it can drive excellence. When combined with top-down control can demotivate staff or drive perverse incentives. Can help identify service failures and target accountability and support on these providers. But often reinforces top-down control and disempowers providers and staff.

To simplify greatly, this finds that:

  • There is evidence that some levers, such as targets, can drive improvements, for tame rather than complex problems (eg increasing activity in hospitals vs support people with chronic conditions), though they often come with undesirable side effects such as gaming or falls in performance in non-measured outcomes.
  • Meanwhile, other components of the NPM agenda seem to have had more limited positive impact. For example, competition and outsourcing in public services, can reduce costs, but usually to the detriment of quality and fairness (eg by provider reducing input costs such as staffing levels or pay).

Explaining the limitations of NPM

Why did NPM approaches fail to transform public services? There are a range of reasons that explain its shortcomings including the design of these policies and at times the capability of the state to implement them effectively. However, there is a far deeper, more fundamental failing at the heart of the NPM revolution in public services: it fails to understand what motivates people and drives change in many cases.

NPM is based on the idea that people, staff and service users in public services, require extrinsic motivations - reward and punishment - to drive behaviour change. However, scientists, psychologists and behavioural economists increasingly find that unlocking intrinsic motivation is a stronger driver of behaviour change in many circumstances. Moreover, they also find that attempts to use extrinsic motivators in these cases can “crowd out” intrinsic motivation leading to worse outcomes.

These insights are based on a growing school of thought called Self Determination Theory (SDT) pioneered by academics Edward Deci and Richard Ryan. This theory suggests that humans rather than being simple economic man or woman are in fact what the economist Bruno Frey calls mature economic man or woman. That is, as well as having both basic biological drives and responding to incentives, they also have what is considered the ‘third motivation’, by which they mean intrinsic motivation.

These thinkers argue that this third motivation is underpinned by three fundamental needs. These are the pursuit of: competence (mastery), autonomy (choice) and relationships (connection). The environment people are in - for example, the workplace - can feed these needs, unlocking intrinsic motivation, to drive effort and performance, or can starve them and thus “crowd it out”. Extrinsic motivators - such as “contingent rewards” - often result in the latter as they depend on control and therefore impinge on autonomy.

For example, Deci, Koestner and Ryan (1999) replicated 128 experiments that sought to test the impact of extrinsic motivators on intrinsic motivation and performance. Their conclusion is clear: “tangible rewards tend to have a substantially negative effect on intrinsic motivation…(they) undermine people taking responsibility for motivating or regulating themselves. When institutions…focus on the short term and opt for controlling people’s behaviour, they may be having a substantially negative long-term effect”.

This does not mean targets and incentives can never work. Where tasks are simple (eg solvable by following reliable procedures) and routine (do not inspire intrinsic motivation) they are demonstrably effective. However, policy makers implementing NPM approaches have drastically overestimated the number of activities that this applies to across the public sector. Many more activities are complex, creative and relational. And, in turn, they have underestimated the long-term cost - in the form of crushed intrinsic motivation - of over-relying on these mechanisms.

A new agenda for public service reform

Overall, this suggests that while there are some elements of the NPM reform agenda which can be utilised going forward (with some adaptation), we are - beyond a small number of more ‘simple’ social problems conducive to more transactional approaches - reaching the limits of NPM to drive improvements in public services. For the increasing number of policy issues which are complex and require organisations and people to be driven by intrinsic motivations, we need a renewed approach to public service reform.

Fortunately, there is an emerging cohort of ‘green shoots’ experiments and innovations in public service reform to draw on across the UK and world. These experiments start to show how we can redesign services, stripping back extrinsic motivators and seeking to create the conditions that unlock the intrinsic motivations of staff and citizens to drive performance, innovation and behaviour change.

Internationally, these include countries that have taken fundamentally different paths to the UK such as the schools’ system in Finland or the care system in Holland. Whilst, in the UK local leaders in England, Wales and Scotland have used devolution to push through the constraints of Westminster’s centralised straight-jacket to deliver more innovative public services, with the most innovative examples including the Wigan Deal and the Preston Model.

In contrast to the low trust, skill and autonomy NPM playbook of top-down targets, regulation, financial incentives, choice and competition, these systems rely more on a high trust, skill, autonomy alternative. They draw on a different set of levers including shared missions, devolution of power to the frontline, investment in skills/capabilities, peer-to-peer learning and empowerment of citizens in services.

The lessons of these models can be brought together to start to set out a reformed ‘public service reform playbook’ for delivering personal, preventative, and productive public services (see figure 2).

Figure 2: A new public service reform playbook

This new playbook sets out six key drivers of improvement that can be utilised within public services such as education, health, care and policing. These include:

  1. Setting bold missions - There is a growing interest in the use of long term, outcome focussed missions for driving coordinated action across government, private and third sectors.
  2. Making smart investment – Additional funding for public services can drive improvement in outcomes, and is needed, but is most effective when it is used to drive reform in how services operate (eg a shift to prevention).
  3. Freeing up the frontlineDecentralisation can help join up services and ensure they are responsive to local needs. Similarly, creating great autonomy with public services can help unlock the intrinsic motivations of staff.
  4. Cultivating the talent of the team – Investing in recruitment, training and development of staff can help drive better public service outcomes. There is also strong evidence behind investment in developing excellent leadership.
  5. Building a learning ecosystem – Creating robust evidence on ‘what works’, a culture of improvement and learning, and mechanisms to spread best practice can drive improved outcomes.
  6. Unleashing technology and data – The application of new technologies, including AI, and the use of big-data can help make public services more efficient and effective.

In addition, we set out two further drivers which sit outside public services:

  1. Empowering citizens and communities – Mechanisms to enable citizens to more effectively self-manage or prevent ill-health, self-direct learning, or drive positive change in their communities.
  2. A new partnership with business Businesses use their role as a producer and employer to drive better social outcomes, with the state using procurement, investment, regulation, and tax to support this to happen.

This is still an early version - a ‘beta version’ - of our playbook. This project will now seek to refine and test it with academics, policy makers and public service practitioners - and then to establish how we can embed this in our public services through policy change. But we cannot achieve this alone. So, we would love to hear from you, and work with you, on our search for better public services, greater social outcomes, and more satisfied citizens.

The authors would like to thank KPMG and Reed in Partnership for supporting this work.