Beyond the Red Flag: Building ConsistencyCheck to Strengthen Legal Judgement at Hack the Law 2026
Written by ConsistencyCheck (Dequn Teng, Arnau Salat, Bernard Liu and Mehmet Murat Cobanoglu)
When our team began the White & Case challenge at Hack the Law, the task seemed straightforward: build a tool that could scan a legal document, identify case citations, and classify them as verified, mischaracterised, or fabricated.
That was the brief. But one of the most valuable lessons from the experience was that the brief was only the starting point. What mattered more was the deeper question behind it: how can AI support legal work without weakening the lawyer’s own judgement?
That question shaped our thinking and, ultimately, the direction of ConsistencyCheck.
Looking beyond the brief
Like many teams, our first instinct was to focus on functionality: how to extract citations, check them, and present the output clearly. Those things mattered, but they were only part of the challenge.
What stood out during the event was the discussion around automation and professional judgement, including the comparison with aviation. In aviation, over-reliance on automated systems can gradually erode the very skills a pilot needs when the system fails. The comparison to legal practice was striking. If lawyers rely too heavily on AI outputs without maintaining active judgement, the risk is not only error, but the weakening of careful legal reasoning itself.
That helped us see the challenge differently. The goal was not simply to create a tool that “does the checking.” It was to design something that could strengthen, rather than replace, legal judgement.
The harder problem was the more interesting one.
A fabricated citation is serious, but in some ways it is the easier issue to identify. If a case cannot be found in the sources checked, that raises a clear flag.
The more difficult and realistic problem is when a citation is real, but the way it is used is misleading. A case may exist, and even be well known, but still be cited for a proposition it does not actually support. That is a subtler kind of error, and often a more dangerous one in practice.
This pushed us to think more carefully about how legal reasoning works. It is one thing to verify whether a citation exists; it is another to assess whether it has been used faithfully and consistently. That distinction became central to ConsistencyCheck.
Being careful about claims
Another important lesson was the value of precision and restraint. Early on, it was tempting to label missing citations as “fabricated.” But on reflection, that felt too absolute. Not finding a case in the sources checked is not the same as proving it does not exist.
So instead, we used language closer to “not found in the sources checked” or “suspected inconsistency requiring review.” That may sound less bold, but it is more honest. In a legal setting, that matters.
This reflected the broader principle behind the project: the tool should support the lawyer’s judgement, not claim to replace it. Where uncertainty exists, it should be surfaced clearly, not hidden behind overconfident language.
Designing for real workflow
One practical insight from the hackathon was that usefulness depends not only on what a tool does, but also on how it fits into the user’s existing workflow.
At one stage, we considered a more elaborate dashboard-style interface. But the more we thought about it, the more it seemed disconnected from how lawyers actually work. Most lawyers do not want to interrupt their drafting process to navigate a separate verification platform. They want support within the document, or as close to it as possible.
That led us towards a simpler concept: a system that highlights potential issues inline, much like a spelling or grammar check, while allowing the user to inspect the reasoning behind each flag when needed. The aim was not to overwhelm the user with analytics, but to provide targeted support at the point where legal judgement is being exercised.
A broader takeaway
We were fortunate that our work on the day was well received, but the biggest value of the hackathon was not the outcome itself. It was the reminder that good legal technology is not only about technical capability. It is also about humility, workflow, and respect for professional judgement.
For us, ConsistencyCheck is not about claiming that AI can “solve” legal reasoning. Rather, it is about asking how technology can help lawyers work more carefully, more consistently, and with greater confidence in what they are reviewing.
The through-line is simple: a hackathon rewards the team that understands the problem deeply, not necessarily the one that ships the most features. Read past the brief. Find the harder version of the problem. Say only what is true. Meet the user where they already work.
That, more than anything, was the lesson we took from Hack the Law 2026.
Dequn Teng is a PhD researcher at Cambridge Institute of Manufacturing studying human–algorithm interaction and analytical creativity. He is the founder of LLM+ and has experience across academia, finance, and industry.
Arnau Salat is a Cambridge Master of Laws (LLM) graduate with a profile at the intersection of law, finance, and technology. His work focuses on international financial law, capital markets, and the regulation of digital platforms and emerging technologies.
Bernard Liu is a PhD researcher at Cambridge Institute of Manufacturing studying spin-out commercialisation, IP, and technology governance. He has an interest in human-in-the-loop judgement in AI-enabled decision-making.
Mehmet Murat Cobanoglu is a visiting PhD researcher at Cambridge working at the intersection of systemic risk, contracts, institutional design, and AI-enabled decision-making. He has nearly 20 years’ experience in public debt management and macro-financial policy.