As the government considers its ambitious agenda to drive wider roll out of AI across the public sector in areas that directly affect people’s lives, we need to find different approaches to innovation that avoid failures like the Horizon Post Office scandal.
For almost a decade, public dialogues have been highlighting what people want from AI: technologies that tackle challenges affecting our shared health and wellbeing; tools that strengthen our communities and personal interactions; and systems that support democratic governance. As these conversations continue, they reveal a growing public scepticism about AI’s ability to deliver on these promises.
This scepticism is warranted. Despite impressive technical advances and intense policy activity over the last ten years, a significant gap has emerged between AI’s capabilities in the lab and its ability to deliver meaningful benefits in the real world. This disconnect stems in part from a lack of understanding of real-world challenges.
We’ve seen the impact of this lack of understanding in previous attempts to drive technology adoption. In the UK, both the Horizon Post Office and Lorenzo NHS IT scandals demonstrated how IT projects can fail catastrophically.
These failures share common patterns that we must avoid repeating. Insufficient understanding of local needs led to systems being designed without considering how they would integrate into existing workflows. Lack of effective feedback mechanisms prevented early identification of problems and blocked adaptation to user experiences. Rigid implementation approaches imposed technology without allowing for local variation or iteration based on real-world testing. Together, these factors created systems that burdened rather than benefited their intended users.
