How AI Solutions And Strong Leadership Could Have Avoided The UK’s Worst-Ever Miscarriage Of Justice

The 14-year-long prosecution of hundreds of employees in the United Kingdom, dubbed the Post Office Horizon scandal, shook a global IT and AI community of which I am a part. This was because this miscarriage of justice was founded on information from an IT computer system baptized Horizon. I have spent years developing what I coin “intelligent anomaly detection systems” both in the United States and in France where my team and I develop AI tools to propose solutions and safeguards to real-world challenges in business. I admit to being baffled at the Post Office’s belligerence in wrongly accusing 736 employees of misreporting and theft instead of questioning the possibility that Horizon, designed by Fujitsu, may be malfunctioning. This scandal made me reflect more about not only the design and deployment of IT systems and AI solutions, but also possible ways to manage the risk of using those technological solutions and how to build accountability into those solutions.

With hindsight, many managerial and leadership actions could have been implemented to prevent a scandal which has destroyed hundreds of lives. To begin with, we should look at it through the risk management perspective. The Post Office should have set a clear tone at the top regarding the transparency and integrity of the IT systems put into place. It could have conducted a thorough investigation of any potential defects in the Horizon system before signing the contract with Fujitsu. Furthermore, it would have been well advised to make a robust risk management and monitoring plan of the system after its implementation.

Moreover, the Post Office should have taken the queries by the whistleblowing sub-postmaster, Alan Bates, more seriously when he asked for access to the data to try to understand the discrepancies in his branch. In 2000, Bates reported the problems linked to Horizon to the Post Office management team. Unfortunately, his reports were dismissed and his contract with the Post Office was terminated three years later. In its Bates & Others vs Post Office Limited judgment, the High Court of Justice stated that the Post Office’s approach was “a simple institutional obstinacy or refusal to consider any possible alternatives to their view of Horizon”. It went on to conclude: “It amounts to the 21^st century equivalent of maintaining that the earth is flat.”

Detecting And Flagging Anomalies With AI

We believe the AI solutions we have developed could avoid these obsolescent attitudes. For years, we have been working on an anomaly detection system designed for internal audit, risk management, and compliance purposes. When you put accounting data into the detection system, it can assign anomaly scores to each financial transaction. It tells you why certain transactions or patterns of transactions are anomalous-based on the metadata and the accounting structure of transactions. In this case, our detection system should be able to detect the changes in transaction patterns after implementing the Horizon system and flag many of the incorrect records generated by Horizon as highly anomalous. Furthermore, our algorithm can also generate explanations concerning how the anomaly scores were assigned based on the algorithm’s logic. Incidentally, our graph-based machine learning algorithms can also be used to counter anti-money laundering, as well as email communication monitoring, and fraud detection purposes.

But still, we would need the Post Office management to take the red flags of the Horizon system seriously and investigate accordingly. After all, this miscarriage of justice is not only about a flawed IT system, it is also about how the Post Office dealt with it.

Fundamental Changes For More Robust Cloud-based POS Systems

We are also convinced there is a valuable leadership lesson for Fujitsu. We believe the multinational company should not only be more effective in reducing product defects but also look at how its clients are using the output of its systems. Horizon is a locally based point-of-sale (POS) system that records transactions, but the Post Office also used the data output to prosecute its employees. More attention should have been paid to the data output at that point. Perhaps, Fujitsu should not have handed over data packs to the UK Post Office as court evidence.

Instead, there appears to be a travesty of justice surrounding Fujitsu’s involvement in the Horizon scandal. After a decade of investigation, award-winning reporter Nick Wallis claims in his book “The Great Post Office Scandal” that the Japanese technology corporation was no bit-part player, but “at the heart of the scandal” due to the faulty retail and accounting software it created and installed. Yet, Fujitsu has not only refused to pay any compensation money to the victims, it has just signed two huge UK government IT contracts worth over $520 million. This has generated strong criticism of the British government – even from within its own ranks where one Conservative peer said: “The government wants to act as if it’s ‘business as usual’ with Fujitsu. What is the incentive on Fujitsu to contribute to the massive compensation cost about to fall on the taxpayer?”

Eyes On The Cloud

More generally, fundamental digital changes are upon us to integrate intelligent solutions in business. More and more major companies in retail are adopting cloud-based POS systems integrated with robust cloud-based Enterprise Resource Planning (ERP) systems. They are more user-friendly and businesses can embed powerful AI into their ERP systems. These, for example, can embed algorithms into ERP structures to identify problems in the financial flows. Thus, it should replace traditional cash register systems like Horizon. Many businesses are also considering integrating blockchains into their ERP systems to improve the integrity of transaction records. However, adding blockchain technology requires more work in building the infrastructure and collaborating with business partners.

All these technological explorations are at the heart of our research mission at the HI! Paris Center on Data Analytics and Artificial Intelligence. This involves developing machine learning-based tools to improve our understanding of accounting data. It is research that seeks intelligent solutions to real-life challenges like the ones we saw in the UK Post Office affair. In this way, we hope that, never again, will a lack of rigorous monitoring of the IT system lead to such travesties of justice in which hundreds of innocent workers are wrongfully prosecuted.

Aluna Wang is HEC Assistant Professor of Accounting and Management Control. She is also chairholder at HI! Paris Center on Data Analytics and Artificial Intelligence.

Daniel Brown is Chief Editor at HEC Paris.