I watched the film Mercy and was struck by its focus on an AI that administers justice. Although the film takes creative liberties to keep things interesting, it offers glimpses of how advanced AI might influence the delivery of justice in the real world. This connects to themes I explore in my book, where I examine how technology can amplify or challenge historical power structures, reminding us that the tools of the past, like the Athenian assemblies, also sought to balance authority and fairness.
The film's premise is intriguing: an AI that determines who lives or dies through justice algorithms. However, from the very first minutes, technical and conceptual inconsistencies emerge. The AI in Mercy acts as an omniscient entity with immediate access to all information, something our current architectures simply don't allow. In my experience, I've seen similar systems struggle with data and processing limitations, which makes this portrayal more an exercise in imagination than reality.
What troubles me most is the total omission of algorithmic bias. Any AI trained on historical data will absorb the prejudices inherent in those records. If it's fed police, judicial, or social data, it will replicate systemic injustices on a massive scale, disguised as neutrality. Consider how criminal risk algorithms in U.S. courts already discriminate against minorities, as research like ProPublica's has shown. The film ignores this entirely, presenting the AI as if it could magically overcome such flaws.
This calls to mind historical experiments in automated justice, like ostracism in ancient Athens. There, popular voting was meant to be democratic, but it often served to exile political rivals. A modern AI would operate at speeds that would bypass any equivalent human oversight. Isn't it curious how ancient patterns persist in the digital realm? As a brief aside, I recall excavations at Greek sites where ostraka — pottery shards inscribed with exile votes — reveal not just collective decisions, but also subtle manipulations, a reminder that human 'automation' already had its biases.
Current systems repeat this pattern. Risk assessment algorithms are used in U.S. courts, and studies document their systematic discrimination. Mercy takes this to a logical extreme: what happens if such systems are given power over life and death? What's fascinating is that the film brushes up against a central dilemma of our era: power concentrated in opaque technologies. But it treats this superficially, as a magical black box, without acknowledging that these systems arise from concrete human choices.
Here a promising alternative emerges: decentralized decision-making systems. Instead of a central AI dictating justice, imagine distributed frameworks where independent entities check one another. This isn't science fiction; prototypes in decentralized autonomous organizations already explore this. Transparency would be key: training data, algorithms, and processes auditable by multiple parties. This way, we avoid the tyranny of unappealable decisions, like those the film depicts.
Human systems, with their appeals and reviews, offer valuable lessons. An automated system without corrections would be inherently oppressive. But perhaps Mercy's greatest flaw is assuming that justice is purely algorithmic. It requires context, empathy, and the ability to question unjust rules. Optimizing existing rules doesn't produce true fairness; this is more complicated than it seems, and I'm still exploring how to integrate it into technological design.
The film illustrates why we need alternative paths. If we persist with centralized, opaque systems, scenarios like Mercy's could become reality. The question isn't whether technology can achieve perfect justice, but how we distribute power so that no single entity decides on lives alone, with constant oversight and verification. This calls for experimenting with technological governance: mandatory transparency, mutually verifiable algorithms, and ever-present human oversight.
Mercy paints a dystopian future, but it urges us to act today in designing systems that expand human justice, not replace it. There's still room to choose better, fostering technologies that serve collective fairness.
Stones don't lie, but historians sometimes do.
