What the auditors found - The Online Court: will IT work?

The National Audit Office (NAO) scrutinises public spending on behalf of parliament. It is independent of the government and its head, known as the comptroller and auditor general, is an officer of the House of Commons. The NAO’s report Early Progress in transforming courts and tribunals, published in May 2018, was the first objective overview of the HMCTS reform programme.¹ Although its findings were played down on publication day by David Gauke, the lord chancellor, and Susan Acland-Hood, chief executive of HMCTS, they cannot have found it comfortable reading.

“HMCTS faces a daunting challenge in delivering the scale of technological and cultural change necessary to modernise the administration of justice and achieve the savings required,” the NAO report concluded. “It has responded to early concerns by extending the timetable and improving its governance and programme management. But there is a long way to go to achieve the planned transformation and overall HMCTS is behind where it expected to be at this stage.”

This is how the NAO summarised its key findings:²

HMCTS’s change portfolio presents a very significant challenge. In 2016, HMCTS commissioned a review which found that the changes it is proposing are far broader than those in comparable programmes in other countries. The transformation programme includes introducing new technology, rationalising estates, restructuring the workforce and operations, and managing critical dependencies. It also involves multiple stakeholders, some of which are constitutionally independent of government.
HMCTS has changed the timescale and scope of the portfolio significantly since 2016. Recognising the breadth and ambition of its original plans, HMCTS extended the timetable from four to six years in 2016, though it did not change the budget. It has since reduced the scope of the Common Platform Programme and brought others such as the Transforming Compliance and Enforcement Programme and other smaller change programmes into a single portfolio. The large number of individual projects within the change portfolio means HMCTS has flexibility to prioritise aspects of planned transformation.
Delivering the reforms successfully remains extremely challenging, despite HMCTS’s work to reduce risk. The revised six-year timescale for the reforms is still shorter than the time taken to complete smaller programmes in other countries. The Infrastructure and Projects Authority’s most recent assurance review concluded that successful delivery of the programme was in doubt, and that there were major risks or issues in a number of key areas. It noted, however, that those leading the programme were aware of these issues and were taking action to resolve them.
HMCTS has made less progress overall than it had expected to at this stage. HMCTS completed the first of four “interim states” at the end of September 2017, including rolling out early versions of several technical components such as online applications for divorce. At this point, it reported that it had fully completed 62% of planned outcomes and partially completed 25%, with 11% significantly incomplete and the remaining 2% adversely affecting the delivery of the next state. The estates reform project has generated more income than expected. The programme at greatest risk of not achieving its outcome is the Common Platform Programme. It has suffered significant delays in development and delivery.
Expected costs have increased and planned benefits have decreased. Since 2015, HMCTS has revised its business cases for the Reform Programme and the Common Platform Programme twice. The 10-year economic case has weakened in each successive iteration. This is in part due to the longer timescale for rolling out the programmes. Annual “steady-state” benefits have also fallen as HMCTS has reduced the scope of the portfolio. The business cases only quantify benefits in terms of savings to HMCTS and the Crown Prosecution Service. It currently excludes estimates of wider benefits to other organisations and court users.
There are gaps in the funding for reforms in later years. HMCTS plans to pay for changes using funding from HM Treasury (£810 million) and retained savings and receipts from property sales (£282 million). The business case projects a funding shortfall of £61 million, assuming that HM Treasury will agree that all previous years’ underspends can be carried forward. Without this agreement, the funding gap could be £177 million. Underspends have arisen because of delays to projects or the two-year extension in the timetable.
HMCTS has taken steps to improve governance and the way it manages the portfolio. HMCTS has simplified programme structures and governance in response to recommendations from assurance reviews. This includes integrating the Common Platform Programme more closely with the Reform Programme and redesigning its governance structure to improve decision-making. It also brought in a new delivery partner, PwC, in the autumn of 2017 to provide support across the portfolio.
HMCTS still needs to develop how the new services will work in practice. HMCTS has developed high-level “target operating models”. But assurance reviews and our own interviews highlighted that stakeholders do not fully understand how the reformed services will work in detail. The large number of interconnected projects in the portfolio makes it difficult to establish and communicate how all the areas of activity fit together. In other programmes, such a lack of clarity has contributed to delays and programme failings.
Delays in introducing primary legislation create a significant degree of uncertainty. The 2017 general election changed the planned legislative timetable, and the timing of the Courts Bill is currently unclear. Some elements of reform, such as the planned extension of virtual hearings, will depend on primary legislation. Without this, HMCTS may have to re-scope elements of the portfolio which is likely to cause delays, increase costs and reduce benefits. It could potentially signal a lack of commitment to the changes which could weaken support and also increase the dependence on the judiciary as certain changes will need to be enacted through an online procedure rules committee. HMCTS may need to re-prioritise its ambition should this risk or others materialise. In doing this it will need to be clear about which elements of reform it considers essential to achieving its vision.
Failure to sustain commitment from all delivery organisations will significantly reduce the likelihood of success and the benefits achieved. HMCTS relies heavily on other organisations to invest in new technology and change their working practices but has limited influence over these groups. It has established relationships with the senior judiciary, the police, the Crown Prosecution Service and representatives of legal professionals, but the tight timetable creates challenges in maintaining meaningful engagement with these organisations and ensuring alignment across all parties. Engagement is also affected by limited transparency. HMCTS does not yet have effective arrangements to measure and report on progress and communicate this clearly to its stakeholders.
Delivering change on this scale at pace means that HMCTS risks making decisions before it understands the system-wide consequences. There are ongoing challenges in understanding and managing dependencies across the portfolio and the possible consequences of changes for other parts of the justice system. Planned changes are far-reaching and could increase costs for other organisations. Any resulting unintended consequences could displace costs, create new inefficiencies and undermine confidence in the justice system.
The benefits claimed so far by HMCTS exceed expectations but risk putting pressure on its ability to maintain services. In other major programmes we have found that financial pressures can lead departments to seek savings before programmes are completed or users adopt new services. Around 65% of the benefits from the reform programme so far have come from not replacing staff who have left rather than from fully implementing new ways of working or moving services online. Our experience reviewing other transformation programmes suggests that expected savings tend to be over-ambitious. They are often uncertain and highly dependent on how change is implemented. For example, some future benefits rely on assumptions that at least 70% of users will move to online services within five years. Recognising such uncertainty, HMCTS, applied an optimism bias of 15% to its savings estimates – although one project has already reduced its original benefits estimates by 30% following live testing.

The NAO made a number of recommendations:

HMCTS should allow enough time to engage with affected parties within the justice system. As the reforms pick up speed, there is pressure to make changes quickly and to expect partners to work at the same pace. HMCTS should build in sufficient time to consult widely in a meaningful way and act on the results of that consultation. HMCTS must provide more detail on how the modernised services will work in practice, what has already happened and what else needs to be done. HMCTS should ensure it builds and maintains relationships as key staff in organisations change.
HMCTS should resist pressure to claim savings until planned changes are fully embedded. Banking savings before new processes and systems have fully embedded can compromise the quality of front-line services, which are then less able to respond to changes in demand. HMCTS should test whether the new processes can cope with fluctuations in demand without reducing the quality of service or effective operation of the justice system.
HMCTS should provide greater transparency of its objectives and progress and be clear how it is adapting plans in response to risks. It needs to be clear what it considers essential to achieving its vision and those areas where it has a degree of flexibility to manage around defined tolerances. Being open in this way will help ensure taxpayers and stakeholders have a clearer picture of what is happening and can hold HMCTS to account for its performance.
HMCTS should work with the Ministry of Justice and HM Treasury to address the system-wide consequences of planned changes. It needs to actively anticipate and mitigate possible adverse consequences resulting from the operational and financial impact of changes on organisations in the justice system. HMCTS has established forums to improve its understanding of the distribution of costs and benefits across organisations. It should use these to agree how costs will be shared across the reformed system.

The House of Commons Public Accounts Committee endorsed the NAO findings. After taking evidence³ from HMCTS and its critics in June 2018, it published its own report six weeks later.

This is how the committee summarised its conclusions and recommendations:⁴

We have little confidence that HMCTS can successfully deliver this hugely ambitious programme to bring the court system into the modern age. While other countries have attempted elements of what HMCTS is doing, no other country has attempted to change its whole system at this scale or pace. Such sweeping changes will be extremely challenging to deliver and the performance of HMCTS to date shows that it has much to learn if it intends to do everything it plans. Despite extending its timetable from four to six years, HMCTS has already fallen behind, delivering only two-thirds of what it expected. The Common Platform Programme, a key part of the reforms, is at significant risk of not delivering. The reforms have suffered from poor progress measures and weak governance, which HMCTS has taken steps to improve. The Infrastructure Projects Authority has also cast doubt on the ability of HMCTS to deliver the programme successfully. There are a number of external factors which could derail progress, such as the delay in primary legislation and uncertainty over long-term funding.

Recommendation 1: HMCTS should write to the Committee, by January 2019, to provide assurance about its updated timetable for delivery. It should update the Committee every six months thereafter in the same format so we can monitor progress.

HMCTS has failed to articulate clearly what the transformed justice system would look like, which limits stakeholders’ ability to plan for, and influence the changes. HMCTS was unable to explain what the transformed justice system would look like and how it would measure whether the changes had been delivered successfully. Organisations representing users of the justice system told us that although the reforms presented many opportunities they had a poor understanding of what HMCTS is aiming for and how the new system will work in practice. Resolution, the Magistrates Association and the Bar Council told us that they were frustrated with the quality of HMCTS’s engagement with them so far and said they felt informed rather than always consulted or meaningfully engaged. This lack of transparency means that interested parties feel that changes are being imposed rather than co-created.

Recommendation 2: By January 2019, HMCTS should provide the Committee with a clear and detailed articulation of what the changes will mean in practice for all the users of the justice system, and when users can expect these changes to be in place.

Despite the revised timescale, HMCTS’s imperative to deliver at such a fast pace risks not allowing time for meaningful consultation or evaluation and could lead to unintended consequences. With so much to deliver in the remaining four years of the programme, HMCTS is operating at a rapid pace. This limits the time available to stand back and consider the wider impacts of the changes on the justice system as a whole and on those that use it. Representatives of the legal profession told us that they are concerned that HMCTS is ploughing ahead without evaluating the impact of changes on user access to justice or consulting properly with stakeholders. They told us that HMCTS was paying “lip service” to engagement, rather than listening to and acting on concerns. Resolution told us that the lack of transparency over the closure of the court in Chichester led to a protracted debate about the court’s use and the impact on users. We share the concerns raised with us that rushing through court closures and fundamental changes to how cases are processed could impede access to fair justice and increase costs elsewhere in the system.

Recommendation 3: By November 2018, HMCTS should publish plans on how and when it will engage with stakeholders and be clear about how it will act on the feedback received and adjust plans if necessary.

HMCTS has not adequately considered how the reforms will impact access to, and the fairness of, the justice system for the people using it, many of whom are vulnerable. We are concerned that the reforms are being pursued at the possible expense of people’s access to fair justice. HMCTS has already closed 258 courts between 2010–11 and December 2017. These courts have been closed before moving services online, meaning that many people are having to travel further to access justice. This can cause many people severe difficulties, particularly for those who rely on public transport or have caring responsibilities. More closures are underway, yet HMCTS has undertaken limited work to review the impact of the closures on users or how demand for court time has been affected. HMCTS has not properly tested the use of new technology in accessing justice. Although HMCTS assured us that it is testing digital services, like online forms, with users, this does not amount to a proper evaluation of the wider impacts of the changes in the real world. We are concerned that HMCTS told us a great deal about processes and products and not enough about how the changes might affect people. Moving services online without assessing the impact could have serious implications for users of the justice system. We share concerns raised by legal professionals and in written submissions that, without sufficient access to legal advice, people could make uninformed and inappropriate decisions about how to plead, and that the roll-out of virtual hearings could introduce bias and lead to unfair outcomes.

Recommendation 4: HMCTS should write to the Committee by January 2019, setting out how it will identify and evaluate the impact of changes on people’s access to, and the fairness of, the justice system, particularly in relation to those who are vulnerable.

One third of the way through the programme, the Ministry of Justice still does not understand the financial implications of its planned changes on the wider justice system. HMCTS expects to save £265 million a year through its planned changes to the courts and tribunal system. These changes will have financial implications across the justice system. Some changes, such as the increased use of video hearings, will create additional costs for other organisations such as the prison service and the police because they will have to buy the new technology needed and make staff available to supervise defendants. However, there may also be indirect costs. These could include costs related to increases in witnesses not attending trials due to the need to travel greater distances to courts or delays processing divorce applications due to staff shortages meaning some people may have to access benefits as they wait for their financial situation to be resolved. Such cost-shunting across the justice system needs to be well understood given the stress that all parts of the system are already under. HMCTS told us that it has established a cross-system group that is modelling the costs and benefits of the changes to the justice system. Although we recognise work is underway to better understand and address this, we are surprised and disappointed it is not more advanced.

Recommendation 5: The Ministry should work with HM Treasury to quantify the likely financial implications of the reforms on the wider justice system. They should involve affected parties to address the implications of any cost-shunting and ensure future funding settlements reflect the cost of delivering services in the transformed system.

We remain concerned that the Ministry of Justice is taking on significant amount of change, without a clear sense of its priorities, at a time when it is facing severe financial and demand pressures. The Ministry faces competing demands for limited resources and increasing demand for its services. The changes to the courts and tribunal system is only one of a number of significant change programmes within the Ministry’s portfolio and the prison, probation and legal aid systems are all under significant strain. Brexit also creates uncertainty about the future demand on the courts and tribunals system. The Ministry is under severe financial pressure, which is only likely to increase, and it faces some hard decisions about what it can continue to fund. It is not clear to us how the Ministry aims to flex its priorities and balance its budget whilst maintaining critical public services.

Recommendation 6: The Ministry should write to the Committee in advance of the next spending review to explain how it plans to ensure its portfolio of change is well-balanced and appropriately prioritised to enable it respond to financial pressures. This should include setting out those elements of reform that are essential and those which could be put on hold.

* * *

Meanwhile, there were concerns elsewhere in Whitehall. The government’s Infrastructure and Projects Authority gives major projects a delivery confidence rating based on “the project’s likelihood of achieving its aims and objectives, and doing so on time and on budget”. In July 2018, the HMCTS reform project received an “amber/red” rating in its annual report for the fifth year in a row.⁵

An amber/red rating means that “successful delivery of the project is in doubt, with major risks or issues apparent in a number of key areas”. It also indicates that “urgent action is needed to address these problems and/or assess whether resolution is feasible”.

It’s probably fair to add that HMCTS is by no means alone in this category.

* * *

In February 2019, the Ministry of Justice and HMCTS published detailed responses to the Public Accounts Committee report.⁶My summaries of these responses are preceded by reminders of each recommendation.

1. HMCTS should write to the Committee, by January 2019, to provide assurance about its updated timetable for delivery.

As of January 2019, we have fully delivered 20 of the 23 indicators we planned to meet at this stage, and which we provided to the committee in June 2018. This includes the national rollout of four new digital services and two new private beta services. We have also opened our first Courts and Tribunals Service Centres in Stoke and Birmingham and started to support new national services through these centres. Considerable steps have also been made in our enabling services, where the Video Hearings project has launched further pilots and our bulk scanning and printing services are now available. Crime has also made good progress in taking more cases through the digital Single Justice Service, and the early version of Common Platform has been developed.⁷

2. By January 2019, HMCTS should provide the committee with a clear and detailed articulation of what the changes will mean in practice for all the users of the justice system, and when users can expect these changes to be in place.

Reform to our services means catering better to the needs of the people who rely on it. This boils down to designing our processes around them wherever we can, instead of asking the public to conform to what we think the process ought to be. In practice, it means more cases started, handled and prepared online or by video; better information flows and communication between the people involved, and to and from the court; better tools to understand what is happening, when it is happening, and why, including better online and telephone access and a better-functioning backoffice to support users; and more varied tools to resolve cases, drawing on mediation, online settlement and judicial led online resolution. It does not mean the end of oral hearings; the courtroom; or even – for the public, at least – paper…

Reform does not in any way challenge or compromise the independence of the judiciary. This essential tenet of the justice system remains steadfast. Access to the justice system will not be diminished; it will be enhanced through the addition of new routes that complement, but don’t necessarily replace, the old. Our justice system will remain as open and transparent as it is now. The public and media will continue to be able to see, hear and read about hearings and judgments, whether they are made in, or away from, the hearing room.

The physical hearing room, and face-to-face interaction, will still play a critical role in the administration of justice. The sincerity, gravitas, integrity, along with the many traditions and ceremonial aspects, of the courtroom will not change. And many familiar roles – ushers, clerks, advisors, security officers – will continue with the important work of supporting proceedings and making sure they run smoothly.

The changes that reform will bring to users of our services can be grouped into three main themes:

There will be a better overall experience – whether in a building, on the telephone or online. Instead of having to understand complex language and navigate their way around paper-based processes, there will be straightforward questions to answer. Our processes will be consistent, predictable and easy to understand. People will get help when and how they need it, via text, phone, email or face-to-face. Legal professionals will also experience simpler processes and good quality digital systems.
Our system will be more accessible, in the broadest sense – with less reliance on physical access to a building and greater use of technology to provide a range of ways of engaging with us (including paper, online or by telephone to a dedicated support team in a Courts and Tribunals service centre). We will be a user-first organisation – this means that we will provide multichannel services and users can choose the right channel for them at any given time. Paper forms, in person hearings and telephone support will remain part of the service offered. These things will be improved as part of reform alongside improvements to the physical environment of our estate to support people to present their case well.
There will be clear proportionality – complex and serious cases will continue to be handled with the full majesty of the courtroom. But all cases will not be handled alike, using modern technology to enable people to pay and submit applications online and access the right information when they need it, enable parties to consider or give evidence digitally, and to have a greater range of options to resolve their dispute and to enable the judiciary to dispense justice and manage cases in new ways.⁸

3. By November 2018, HMCTS should publish plans on how and when it will engage with stakeholders.

We provide regular information and updates about the Reform Programme, some of it generic, some of it targeted at particular stakeholder groups. This work also seeks to encourage and invite feedback and opportunities to engage further with HMCTS.

We share ideas, plans and proposals and exchange views and knowledge, so that we can understand each other’s perspectives on key issues.

We collaborate with real users, giving them the opportunity to test and feedback on new services very early in their design. We also invite stakeholders with particular expertise and experience to work with our project teams as part of the “agile” (step-by-step) development process adopted by the Reform Programme to design new and improved services.⁹

4. HMCTS should write to the committee by January 2019, setting out how it will identify and evaluate the impact of changes on people’s access to, and the fairness of the justice system, particularly in relation to those who are vulnerable.

We will be establishing an advisory panel to support analysts as well as ministers on evaluating, assessing and reviewing the reform programme. Our intention is to draw on a wide range of external expertise, including academics and legal practitioners, as well as those who have practical experience in the delivery of significant reform programmes. The panel’s role will be to ensure that analysts and ministers benefit from a wide range of external, expert advice on how the evaluation can best be undertaken. We will also be seeking the views of the senior judiciary to explore how we can make the best use of the expertise they can bring to the evaluation.

We will aim:

to have completed the scoping stage of the evaluation work, and have the advisory panel in place, by the end of spring 2019;
to have commissioned any further research for the evaluation, and publish a call for relevant evidence from academics by the end of the summer 2019, if required;
on that basis, we would be aiming to complete an interim report by summer 2021. There will also be regular updates throughout the evaluation, as findings become available.
The intention would then be to continue to evaluate the programme at regular intervals as the reform programme is concluded.

The HMCTS Portfolio Finance Team tracks the actual costs incurred and benefits realised by reform compared with those set out in the business case. By the end of 2018-19 we will have spent £546m, £83m less than the spending envelope in the fourth business case on reform, and our benefits to date of £158m have exceeded planned levels for this point in the programme.¹⁰

5. The Ministry should work with the Treasury to quantify the likely financial implications of the reforms on the wider justice system.

We have been working collaboratively with partners, agencies and other government departments that are likely to be affected by the programme to prevent cost-shunting by, wherever possible, co-designing services.¹¹