What will govern AI, whether it seems to be regulated or not? Due process.

Explainability might seem to some people like a luxury of design in AI tools. But explainability (and its cousin, transparency) is the beating heart of due process, one of the greatest inventions of humankind.

Courts of appeal in the U.S. provide a compelling analogue: appellate judges review decisions from lower courts not merely for outcome, but for how the lower-court judge got there. That is, for their reasoning, aka their explanation. And how the lower-court judge reached his or her decision is contained IN WRITING. 

Under standards like “abuse of discretion,” the appellate court asks whether the judge considered all relevant factors—often prescribed by statute or precedent—and articulated a reasoned basis. Under the standard of “plain error,” the court scrutinizes whether a legal mistake was obvious and affected the outcome. The standard of de novo (Latin for from the start) demands even more: the appellate court may substitute its own judgment, but it remains grounded in what the lower court wrote.

These standards are guardrails for accountability. Appellate panels typically include *three* judges precisely because multiple minds reduce arbitrariness. Explainability is what allows oversight to be effective. Without access to a judge’s reasoning, there’s no way to tell if bias or blind spots crept in. AI is no different. A system that cannot show its work cannot be governed.

So yes, human oversight and explainability serve distinct purposes. But without explainability, oversight becomes ornamental—watching a black box from the outside, unable to intervene where it matters most: the logic within.

In short, expect that jurisdictions worldwide will require increasing levels of explainability and transparency in AI, driven by the primary language judges and lawyers speak: that of due process. They simply won’t be able to function without speaking the language they know. That will be how AI gets shadow-regulated.