Because when you look closely, AI governance does something far more interesting. It acts like a mirror. It shows you how your organisation actually works beneath the surface: the habits, the shortcuts, the blind spots, the strengths. It reveals governance maturity long before it reveals model risk.

In that sense, AI governance is less about managing technology and more about diagnosing institutional maturity.

When governance lives on paper instead of in practice

A lot of AI governance today feels familiar: templates, frameworks, long documents that signal control. They look impressive, but they rarely touch the messy reality of how models are built, trained, or deployed.

The truth is that AI risk doesn’t behave like traditional compliance risk. It shifts with context, data, and behaviour. A checklist can show that you’re aware of the risk — but it cannot show that you’re in control of it. That gap between what’s written and what’s real is where most organisations quietly struggle.

Real governance forces uncomfortable questions about ownership, data quality, and accountability. Those answers don’t live in a policy binder. They live in culture.

Tools don’t fix weak foundations

Explainability dashboards, bias scanners, model inventories — they’re useful, but only if the foundations are solid. Without clean data, explainability becomes noise. Without clear accountability, bias reports gather dust. Without ownership, model inventories turn into abandoned spreadsheets.

These aren’t AI problems. They’re governance problems that AI simply makes impossible to ignore.

Strong AI governance doesn’t start with new tools. It starts with old disciplines: good data stewardship, clear decision rights, and leaders who take responsibility. AI governance only works when it’s woven into the organisation, not layered on top of it.

AI governance as an organisational health check

If you treat AI governance as a diagnostic, the patterns become obvious. Technical issues often point to structural ones:

• Missing data lineage usually means unclear process ownership.
• Poor training sets often reflect weak ethical review.
• Model drift exposes gaps in operational accountability.

These are symptoms of the organisation’s governance DNA — its culture, its decision hierarchies, its systems of trust.

This is where governance becomes valuable. It tells you how well your organisation governs itself. And when findings are treated as insights rather than failures, governance becomes a learning mechanism instead of a burden.

The real frontier: judgment and accountability

The most mature organisations don’t rely on compliance checks alone. They connect technical assurance with human judgment. They ask: Are we ready to stand behind this decision?

Some of the best‑governed companies voluntarily slow down, test longer, or limit automation — not because the law requires it, but because they know they aren’t ready to defend the outcome ethically or publicly. That restraint isn’t a lack of innovation. It’s a sign of strength.

Where that discipline exists, AI becomes an extension of sound judgment, not a compliance experiment.

Reframing the purpose of AI governance

If AI governance stays trapped in regulatory checklists, its impact will stay small. But when used as a mirror, it becomes transformative. It connects compliance to culture, technical risk to ethical responsibility, and process control to leadership integrity.

The systems may be powered by algorithms, but the real work is human.

The value of AI governance isn’t in the frameworks we build — it’s in what those frameworks reveal about us. Done thoughtfully, governance becomes a path to clarity, accountability, and the confidence to innovate without losing trust.

The EU AI Act establishes the legal baseline. The NIST AI RMF shapes how organisations operationalise risk. And ISO 42001 provides the certifiable management system that makes AI governance visible to the outside world.

Individually, each framework matters. In practice, none operates in isolation.

Most organisations are not choosing between them — they are navigating all three at once. Which leads to the real question practitioners are asking behind closed doors: not “Which framework should we adopt?” but “How do we integrate them without creating three parallel governance programmes?”

That is the right question. And the answer is more simple than most people expect.

Where they complement each other

The three frameworks were designed for different purposes, which is precisely why they work together rather than against each other.

The EU AI Act is external and mandatory — it defines the legal obligations a deployer or provider must meet and the penalties for failing to meet them. It tells you what the destination is, but it does not give you a detailed map for getting there. And for organisations outside the EU operating AI systems that affect EU individuals, it sets the jurisdictional floor regardless of where the organisation is registered.

The NIST AI RMF is internal and methodological — it gives your organisation the structured thinking tools to identify, assess, and manage AI risk across the full system lifecycle. Its four (4) functions — Govern, Map, Measure, Manage — create the internal architecture that EU AI Act deployer obligations presuppose but do not specify. So, an organisation working through the NIST AI RMF is building exactly the kind of internal governance capability the EU AI Act will eventually ask it to demonstrate.

ISO 42001 is the management system layer that sits across both — it embeds governance into organisational operations, makes it auditable, and produces the certification that clients, procurement teams, and regulators are beginning to ask for by name. Where NIST AI RMF is a methodology and the EU AI Act is a regulation, ISO 42001 is infrastructure.

Together, the three (3) frameworks create a complete governance stack: legal obligation, internal methodology, and certifiable management system.

Where they overlap

All three (3) frameworks ask a version of the same question: have you assessed the risks your AI system introduces, and do you have a documented, accountable process for managing them?

The EU AI Act calls this conformity assessment. The NIST AI RMF calls it the Map and Measure functions. ISO 42001 calls it risk and impact assessment. The language differs, the level of prescription differs, and the legal weight differs — but the underlying governance requirement is identical.

This overlap is actually useful. It means that an organisation doing genuine risk assessment work for one framework is simultaneously building evidence of compliance for the others. A well-documented NIST AI RMF risk assessment strengthens your ISO 42001 audit and supports your EU AI Act conformity documentation. The three are not competing demands — they are reinforcing ones.

Where the gaps remain

None of the three (3) frameworks fully resolves the question of what happens before deployment — specifically, the configuration decisions that shape how an AI system behaves before it goes live and before any compliance framework can reach it.

The EU AI Act places obligations on deployers after a system is classified and before it is placed in operation. The NIST AI RMF’s Map function asks organisations to identify context and risk. ISO 42001’s operational controls govern how systems are deployed and monitored. But none of them fully govern the moment when an organisation configures a third-party AI system — deciding which data sources to connect, which populations to include, which thresholds to set — because that moment sits in a space the frameworks assume has already been addressed.

It has not. And in most organisations, it is not governed at all.

What a practical governance stack looks like

For an organisation operating across jurisdictions — a European company expanding into Asia, an Asian company entering Europe, or a multinational managing AI systems across both — the practical stack is this: use the EU AI Act to define your legal obligations and risk classification, use the NIST AI RMF to build your internal governance methodology, and use ISO 42001 to certify that your management system is real and auditable. The three frameworks are not alternatives. They are layers.

And as China, Singapore, Japan, and other jurisdictions continue developing their own AI regulatory frameworks, the organisations that have built a coherent internal governance architecture, rather than reacting to each new regulation separately, will adapt far more efficiently than those that have not.

This article goes one level deeper. Because SEE does not happen in one way. It happens in two. And understanding the difference between them is essential to understanding why the problem is so difficult to see, so difficult to measure, and so difficult to govern.

The construct that captures this distinction is one I call the Death of Traits.

What the Death of Traits means

Every AI system is built on a training corpus — a body of data that determines what the system knows, what it can reason about, and what it will never be able to see. That corpus was assembled through decisions: what to include, what to scrape, what to license, what to prioritise. And those decisions, made predominantly by Western technology companies operating within a specific commercial and academic logic, left out an enormous proportion of humanity’s accumulated knowledge.

The Death of Traits is the name I give to that disappearance. It describes the structural loss of cultural practices, legal customs, governance frameworks, epistemic styles, and ways of knowing that never meaningfully enter AI training ecosystems — and that therefore never exist, for the AI system, at all.

But not all knowledge disappeared the same way. And that distinction is the sharpest analytical edge of this theory.

The first mode: Passive Death of Traits

The first mode is passive. It describes knowledge that was never digitised — knowledge that exists in living communities, in oral practice, in embodied tradition, in forms that the internet never captured and that no scraping algorithm could ever find.

Consider the Mizo customary law system of Mizoram, Northeast India — the community I come from. For generations, the Mizo people developed frameworks for land governance, community dispute resolution, and collective resource management that were finely calibrated to the specific ecological, social, and political conditions of the Northeast Indian hills. These frameworks worked. They produced stable, functional governance outcomes in conditions where written, bureaucratic legal systems could not easily reach.

None of that knowledge is in any AI training corpus. Not because it was deliberately excluded. Because it was never digitised. It exists in practice, in memory, in community, in the living transmission of knowledge from one generation to the next. And an AI system trained on the internet has no way of knowing it exists — let alone reasoning from it, applying it, or recognising its relevance when a problem arises that it was specifically designed to solve.

This is happening everywhere. Yoruba legal traditions in West Africa. Andean governance frameworks in South America. Pacific Islander navigation and ecological knowledge. Aboriginal Australian land management systems. The oral legal traditions of hundreds of communities across Asia, Africa, and the Pacific. Knowledge systems that are sophisticated, functional, and in many cases uniquely equipped to address exactly the kinds of complex, multi-variable problems that AI is now being deployed to help solve.

None of it made it into the ark.

The second mode: Active Death of Traits

The second mode is active. And in some ways it is more troubling than the first — because the knowledge exists. It is digitised. It is online. And it still did not make it in.

Active Death of Traits describes knowledge that was systematically filtered out by the curation decisions, scraping choices, licensing frameworks, and moderation norms of organisations that did not recognise its value — or did not prioritise it commercially.

A traditional medicine knowledge base documented in a regional South Asian language. A community archive of customary legal decisions published on a low-traffic website that no major scraping operation reached. A body of governance literature written in Swahili, Amharic, or Tagalog that exists digitally but was ranked too low in relevance signals to be included in training data. A collection of oral histories transcribed and uploaded by a community organisation, sitting in a corner of the internet that commercial AI development never looked at.

This knowledge was not lost before the digital age. It was lost during it — through the specific, consequential decisions about what counts as high-quality data, what languages are worth prioritising, what sources are authoritative, and what communities are worth engaging. Those decisions were not made with malicious intent. They were made with commercial logic. And the outcome was the same: active, systematic exclusion of knowledge that existed but that the dominant AI development infrastructure did not know how to value.

Why the distinction matters for governance

The difference between passive and active Death of Traits is not just analytical. It has direct implications for who bears responsibility and what governance mechanisms can actually reach the problem.

Passive Death of Traits — knowledge that was never digitised — requires a different kind of response. It requires investment in digitisation partnerships, community archiving programmes, and language documentation initiatives that bring knowledge into a form AI systems can eventually engage with. It requires time, resources, and genuine community partnership. And it requires doing this work before the governance window closes — before the current generation of foundational models cements its epistemic architecture so completely that catching up becomes structurally impossible.

Active Death of Traits — knowledge that exists but was filtered out — requires a different intervention. It requires accountability for the curation, scraping, and licensing decisions that produced the exclusion. It requires asking, of the organisations that built the foundational models: what did you choose not to include, why, and what were the consequences of that choice? These are governance questions. They are questions about decision-making, accountability, and the allocation of responsibility for outcomes that were foreseeable even if they were not intended.

This is why in my working paper I propose the Epistemic Provenance Document — a mandatory filing by the most powerful AI providers that documents what knowledge communities were included in training data, what was excluded, and what was done about it. Not as a bureaucratic exercise, but as the mechanism that makes epistemic accountability visible, auditable, and enforceable for the first time.

The irreversibility problem

There is one aspect of the Death of Traits that distinguishes it from almost every other AI governance challenge, and that makes it uniquely urgent.

Most AI governance failures are, in principle, correctable. A biased output can be retrained away. A privacy violation can be remediated. A deployment decision can be reversed. The damage is real, but the path to correction exists.

The Death of Traits is different. Knowledge traditions that are not digitised before the communities that carry them disperse, or before the living transmission of that knowledge breaks, are gone. Not inaccessible. Gone. No amount of governance reform, no Epistemic Provenance Document, no Ethical AI Data Fund can recover knowledge that no longer exists anywhere in a recoverable form.

And for knowledge that exists digitally but was filtered out — the active mode — every model generation that trains on the previous one’s outputs compounds the exclusion. The signal, already weak, becomes weaker relative to the dominant. Recovery becomes progressively harder with each iteration.

So the urgency is not rhetorical. The governance window to address the Death of Traits — both modes — is genuinely time-limited in a way that most regulatory problems are not.

What this means beyond the Global South

It would be a mistake to read the Death of Traits as a problem that belongs only to the communities most visibly affected. It is a problem for everyone who depends on AI systems to make good decisions — which, increasingly, means everyone.

When an AI system deployed in a European financial institution cannot read the economic behaviour of a Nigerian entrepreneur, that is a risk management failure for the institution. When an AI educational assessment tool cannot recognise the intellectual tradition of a South Asian student, that is a measurement failure that affects the student’s outcomes and the institution’s accuracy. When a medical AI system trained predominantly on Western clinical data makes recommendations for patients whose physiological and social contexts it was never trained to understand, that is a patient safety failure.

The Death of Traits is not a values question dressed up as a governance problem. It is a governance problem with measurable, institutional consequences — for the organisations deploying AI, for the regulators overseeing them, and for the people whose lives those systems increasingly shape.

The Theoretical Framework

The full theoretical framework for SEE and the Death of Traits is developed in my working paper “The Death of Traits: Epistemic Exclusion in AI and a Framework for Decolonial AI Governance” (SSRN 6283019). Open access and freely available.

About the Author

Helen Lalthanpari is a Partner at Morgan & Colney, an AI governance and cross-border advisory practice based in Île-de-France, France. She is also an Advocate enrolled at the High Court of Delhi, India and holds an LLB, MBA in International Finance from EDHEC Business School, an AI Governance Professional (AIGP) certification from IAPP and a Post Graduate Diploma in International Human Rights. She is from the Mizo tribe of Mizoram, Northeast India.

She has over seventeen (17) years working experience across multiple roles from AI governance consulting, investment analyst at Silicon Valley based Venture Capital firm to General Counsel/Head of Legal at multinational corporations across Asia and Europe, including Bank of America, Coca-Cola, WeWork, OYO, Miniso, and Denso. Her research sits at the intersection of law, technology, and governance, with a particular focus on upstream AI governance, epistemic justice, and cross-border AI governance across the Asia-Europe corridor. She writes and advises at the intersection of where AI law is written and where it fails to reach.

She can be reached at: helen@morganandcolney.com

Targeted Federal Preemption: Narrowing the Patchwork

The Framework calls for Congress to preempt state AI laws that create conflicting or burdensome requirements, particularly around AI development, which it frames as an interstate and national‑security matter. This would significantly narrow the current patchwork of state proposals and create a more uniform national baseline.

However, the Framework explicitly preserves state authority over children’s safety, state procurement, and general consumer‑protection laws. Preemption is targeted, not absolute.

EU AI Act interaction: Federal preemption does nothing to blunt the EU AI Act’s extraterritorial reach. Article 2 still applies to US developers placing systems on the EU market.

NIST AI RMF / ISO 42001: A more unified US baseline makes it easier to operationalise a single governance system nationwide—an inference, but a reasonable one.

No New Federal AI Regulator: A Structurally Different Model

The Framework rejects creating a standalone AI regulator. Instead, it relies on existing agencies (FTC, FDA, DOT) and industry‑led standards.

This is structurally different from the EU AI Act, which centralises oversight through the AI Office, notified bodies, and market surveillance authorities. US firms will therefore face voluntary, sectoral governance at home and mandatory, centralised compliance in Europe.

NIST AI RMF / ISO 42001: NIST AI RMF is elevated as the de facto US reference model. ISO 42001 is well‑placed to serve as the international bridge for organisations seeking a single governance system across jurisdictions.

Regulatory Sandboxes: Divergent Risk Philosophies

The Framework supports regulatory sandboxes designed to minimise ex ante regulatory friction. While lighter than the EU model, it does not eliminate documentation or supervision, especially in safety‑critical or children‑focused contexts.

EU AI Act interaction: Articles 57–60 of the EU AI Act impose structured oversight, documentation, and safeguards. The US model is more flexible, creating a foreseeable regulatory arbitrage risk: high‑risk development may occur in looser US environments while outputs are marketed into the EU, shifting Article 26 due‑diligence burdens onto EU deployers.

ISO 42001: In such flexible environments, ISO 42001 is likely to become the primary structured assurance mechanism.

Training Data and Copyright: Legal Uncertainty as a Governance Variable

The Framework signals support for treating model training on publicly available copyrighted material as generally permissible, but acknowledges that Congress and the courts must ultimately resolve the issue.

EU AI Act interaction: Article 10 requires documentation of data provenance, rights status, and governance controls. The absence of a clear US statutory standard deepens the upstream governance gap for EU deployers of US‑trained models.

NIST AI RMF / ISO 42001: Both frameworks require data lineage and provenance controls that exceed current US legal obligations.

Children’s Safety: A Point of Convergence

The Framework calls for privacy‑protective age assurance and limits on data collection for minors. This aligns with the EU AI Act’s treatment of minors as a vulnerable group triggering heightened safeguards.

Across NIST AI RMF and ISO 42001, vulnerable‑population protections are already embedded, so this makes a rare convergence point of all such regulations today.

Conclusion

The US Framework and the EU AI Act now represent two distinct governance philosophies. For organisations operating across jurisdictions, the challenge is not choosing one model but designing governance systems, often anchored in NIST AI RMF and ISO 42001, that can withstand the structural asymmetry now embedded in global AI regulation.

The model goes back into production. The problem goes back into the gap between the committees.

Three mandates, three versions of the same model

When a bank governs a high-risk AI system for creditworthiness assessment, it does so through a structure that looks coherent on paper. Risk owns model performance. Technology owns infrastructure. Compliance owns regulatory mapping. Three committees, three mandates, one shared objective — or so the governance framework says.

What the framework does not say is that each of these functions is governing a different abstraction of the model, using a different definition of risk, and measuring safety against a different standard.

For Risk, the model is a statistical instrument. It ranks borrowers, maintains discriminatory power, and holds within tolerance thresholds. If the Gini coefficient is stable and the backtesting is clean, the model is performing. If the validation pack is green, the governance obligation is discharged. This is a rigorous standard. It is also an incomplete one, because it was designed to answer a specific question, “Does the model predict accurately?”. It says nothing about whether the model is doing fairly, or whether it can be explained to the borrower it just declined.

For Technology, the model is a system. It must run, integrate with infrastructure that was built before anyone used the word AI, and fail gracefully when something upstream breaks. If the monitoring dashboards are clean and the deployment pipeline is stable, Technology considers the system sound. That is also rigorous, and also incomplete, because a model can be technically flawless and legally indefensible at the same time. Technology’s definition of risk ends at the infrastructure boundary.

For Compliance, the model is a regulated entity. It must satisfy documentation requirements, transparency obligations, human oversight provisions, and under the EU AI Act, a Fundamental Rights Impact Assessment before it touches a single live application. If the controls are mapped and the documentation is complete, Compliance considers the obligation met. But completeness of documentation is not the same as correctness of the model, and Compliance is rarely in a position to challenge the statistical or operational assumptions that sit underneath the paperwork it is asked to sign off on.

The gap is not between the people. It is between the definitions.

None of this is a failure of individual competence. The people in these committees are good at what they do. The failure is institutional. Each function was built to answer its own question, and nobody was given a mandate to hold the question that spans across all three.

So when a fairness metric drifts, or a segment begins underperforming in ways the model did not price, or a regulator asks for documentation that does not exist in the form they expected, the response is fragmented by design. Risk recalibrates. Technology checks the logs. Compliance updates the framework. And the underlying governance gap — the fact that these three functions never agreed on what they were jointly responsible for — remains exactly where it was.

This is the institutional reality that the EU AI Act is about to make visible, and expensive. The Act does not care about internal committee structures. It cares about outcomes. It expects a bank to demonstrate unified governance, consistent definitions, traceable decisions, and meaningful human oversight across the full lifecycle of a high-risk system. A bank where Risk, Technology, and Compliance are governing three different versions of the same model cannot demonstrate that. Not because the will is absent, but because the architecture was never built for it.

What actually needs to change

The solution is not to collapse the committees. It is to collapse the definitions. Before the next credit model goes to deployment, the three functions need a shared answer to a set of questions that none of them can currently answer alone, “ What does this model assume, who does it affect, what would it mean for it to fail, and who is responsible when it does?”

High-risk AI does not require banks to reorganise. It requires them to have a conversation they have been structurally avoiding.

This article is part of the series,"Operationalising the EU AI Act in Banking: A Field Guide to High-Risk AI”. The upstream governance dimension is examined in “Invisible Borrowers, Unpriced Risk: The Upstream AI Governance Gap under Article 26 of the EU AI Act” (SSRN 6313443), open access.

Helen Lalthanpari is Partner at Morgan & Colney, France with an AIGP, LLB, MBA and a PG Diploma International Human Rights. She has 17+ years work experience spanning AI governance consulting, Silicon Valley VC analyst, and GC/Head of Legal at various multinational companies across diverse markets. She can be contacted at: helen@morganandcolney.com

This second article goes deeper. SEE does not happen in one way. It happens in two. And unless we understand the difference, we cannot understand why the problem is so hard to see, so hard to measure, and so hard to govern.

I call this distinction the Death of Traits.

What the Death of Traits means

Every AI system is shaped by the data it is trained on. That data determines what the system can recognise, what it can reason about, and what it will never be able to understand. The training data did not assemble itself. It was built through choices — what to scrape, what to license, what to prioritise, what to ignore. These choices were made mostly by Western technology companies working within a particular commercial and academic logic.

The result is that a vast amount of human knowledge never made it in.

The Death of Traits is the name I give to this disappearance. It describes the loss of cultural practices, legal traditions, governance systems, ways of reasoning, and ways of knowing that never enter AI training data — and therefore do not exist for the AI at all.

But this loss happens in two very different ways, and that difference matters.

The first mode: Passive Death of Traits

The first mode is passive. It describes knowledge that was never digitised. It lives in communities, in oral practice, in embodied tradition. It is transmitted through memory and participation, not through text on a website.

The Mizo customary law system of Mizoram — the community I come from — is one example. For generations, the Mizo people developed ways of managing land, resolving disputes, and organising community life that were finely tuned to the realities of the hills. These systems worked. They produced stability in places where formal legal systems struggled.

None of this knowledge appears in any AI training dataset. Not because someone removed it, but because it was never digitised. An AI system trained on the internet has no way of knowing it exists, let alone recognising its relevance when facing a problem it was designed to solve.

This is not unique to the Mizo people. Yoruba legal traditions, Andean governance systems, Pacific Islander ecological knowledge, Aboriginal Australian land management, and hundreds of oral legal and governance traditions across Asia, Africa, and the Pacific share the same fate. These systems are sophisticated and often better suited to solving complex, real‑world problems than the frameworks AI currently draws from. But because they were never digitised, they never entered the AI universe.

They simply vanished.

The second mode: Active Death of Traits

The second mode is active. It describes knowledge that was digitised — and still never made it into AI training data.

This happens through curation choices, scraping decisions, licensing rules, and moderation norms that quietly filter out knowledge that does not fit the priorities of commercial AI development. A traditional medicine archive written in a regional South Asian language. A community website documenting customary legal decisions that never reached major scrapers. Governance literature in Swahili, Amharic, or Tagalog that was ranked too low to be included. Oral histories transcribed and uploaded by community groups but ignored by commercial datasets.

This knowledge was not lost before the digital age. It was lost during it — through decisions about what counts as “high‑quality,” what languages matter, and which communities are worth engaging. These decisions were not malicious. They were commercial. But the effect is the same: active exclusion of knowledge that existed but was not valued.

Why the distinction matters for governance

The two modes of the Death of Traits require different responses.

Passive Death of Traits needs investment: digitisation, community partnerships, language documentation, and archiving. It requires time and resources, and it must happen before the window closes — before communities disperse or oral traditions break.

Active Death of Traits needs accountability: transparency about what was included, what was excluded, and why. This is why I propose the Epistemic Provenance Document — a mandatory disclosure from major AI providers that makes these decisions visible and auditable. Without this, we cannot govern what we cannot see.

The irreversibility problem

Most AI governance failures can be corrected. Bias can be reduced. Privacy breaches can be addressed. Deployment decisions can be reversed.

The Death of Traits is different. Knowledge that disappears before it is digitised is gone forever. And knowledge that is digitally weak becomes weaker with every model generation that trains on the outputs of the last. Exclusion compounds.

This is why the governance window is genuinely time‑limited.not.

What this matters everywhere

This is not only a Global South problem. It affects anyone who relies on AI systems to make decisions. When an AI system cannot read the behaviour of a Nigerian entrepreneur, a South Asian student, or a patient whose clinical profile was never represented in the data, the failure is not cultural. It is institutional. It affects accuracy, safety, and risk management.

The Death of Traits is not a moral argument. It is a governance problem with real‑world consequences for the organisations deploying AI, for the regulators overseeing them, and for the people whose lives those systems increasingly shape.

What comes next

The next article in this series examines the governance paradox at the heart of SEE: why existing frameworks — the EU AI Act, NIST AI RMF, ISO 42001 — cannot detect what is missing, and why a system produces confident, authoritative outputs from structural ignorance without any visible signal that something is wrong.

The full theoretical framework for SEE and the Death of Traits is developed in my working paper “The Death of Traits: Epistemic Exclusion in AI and a Framework for Decolonial AI Governance” (SSRN 6283019). Open access and freely available.

Someone raises it in a meeting. And then someone else says the thing that ends the conversation, “But the model was validated and is objective. If the pattern is there, it must be in the risk.”

And with that, the conversation ends. The possibility that the model might be mis‑specifying certain borrowers, or reproducing a historical bias embedded in the training data, evaporates under the weight of institutional certainty.

That is the moment the bias wins.

Automation froze the judgment. It did not remove it.

Automated credit scoring promised neutrality, that if the inconsistent human is taken out of the decision, the inconsistency disappears. As much as that logic is seductive, it is also wrong.

Credit models are trained on historical decisions. Those decisions were made by loan officers working under time pressure, institutional incentives, and risk appetites that shifted with every economic cycle. They approved some borrowers and declined others. They made overrides, undocumented exceptions, and judgment calls that reflected not just the risk in front of them, but the cultural assumptions of the institution around them. And then all of that — the approvals, the declines, the patterns — became training data.

So, the model did not replace the loan officer’s judgment. Instead, it learned from it faithfully, at scale. And then it began reproducing it, thousands of decisions a day, without the one thing the loan officer had that the AI model does not: the ability to pause, feel if something is off, and look again.

The second look was doing more governance work than anyone realised.

Inside a bank, the second look was never a formal process. It was the credit officer who overrode a decline because the applicant’s income structure was unusual but the underlying business was sound. It was the underwriter who escalated a borderline case because something in the file did not fit the pattern cleanly. It was the supervisor who asked a difficult question in a credit committee because the numbers felt right but the story felt incomplete.

None of this was documented. None of it was in the governance framework. And almost all of it disappeared when the model arrived. Because when a model declines an application, the credit officer does not override it — not because they cannot, but because the model is perceived as objective, and overriding an objective system requires a justification that most institutions have not built the language to support.

The result is that the model’s decision becomes the decision. And the institutional structures that occasionally, imperfectly, corrected for what the model would have gotten wrong are gone.

The EU AI Act sees the output. It does not yet see the origin.

The EU AI Act sees the output. It does not yet see the origin.

The classification of creditworthiness assessment as high-risk AI under the Act is analytically correct. The harm is structural and it operates at scale. But the conformity assessment framework under Article 9 is still largely designed to test whether the model performs correctly on the data it was given. Article 27, which requires a Fundamental Rights Impact Assessment before deployment, comes closer to the right question — it asks who is affected and how. But even that instrument, applied honestly, can only reach as far back as the deployment decision.

It is not designed to ask how the data got the way it is — what institutional choices, over what period of time, shaped the approval and decline patterns that are now the model’s entire understanding of credit risk. A bank can complete every required assessment, satisfy every audit, and demonstrate full regulatory compliance — and still operate a credit model that prices risk incorrectly for borrowers whose economic lives do not match the historical patterns the model was trained to recognise. Not because the bank did anything wrong at deployment. Because the data arrived already shaped by decades of decisions that nobody is now required to interrogate.

The real governance question

The question banks need to be asking before the next model goes to validation is not whether the model performs. It is what the model learned, and from whom. Whose repayment behaviour is in the training data, and whose is absent or underrepresented? Which institutional assumptions about what financial stability looks like are now encoded as risk parameters?

The loan officer made mistakes. Some of them were visible, challengeable, reversible. The model makes the same mistakes. Invisibly. Consistently. Across every application, every day.

Removing the human did not remove the bias. It removed the ability to notice it.

This article is part of the 5 article series “Operationalising the EU AI Act in Banking”. The upstream AI governance dimension is examined in detail in “Invisible Borrowers, Unpriced Risk: The Upstream AI Governance Gap under Article 26 of the EU AI Act” (SSRN 6313443), open access.

You would not call that consultant biased. You would call them structurally unprepared. And you would want to know, before they gave you a single recommendation, what they do not know — and whether the gaps in their knowledge happen to be the gaps that matter most to your business.

This is not a hypothetical. This is the situation every organisation deploying AI is in right now. And almost nobody is asking the right question.

What AI is actually built on

More than 90% of AI training data comes from Europe and North America — a region representing just 15% of humanity. The AI systems now making decisions in banks, hospitals, schools, and courtrooms around the world learned what they know from one in seven people on earth. The other six were largely not in the room. And here is the part nobody is talking about: the system does not know they are missing.

This is not a diversity problem in the conventional sense. It is not fixed by adding more languages to a chatbot or hiring a more diverse engineering team. It is a structural governance failure — one that operates before any AI system is deployed, before any regulation applies, and before any audit can reach it. And it is the most consequential AI governance failure of this generation.

I call it Systematic Epistemological Exclusion, or SEE.

What SEE actually means

Epistemology is the study of knowledge — how we know what we know, what counts as valid knowledge, and whose ways of knowing are recognised as legitimate. Systematic Epistemological Exclusion is the pre-deployment structural omission of entire knowledge systems from AI training data, resulting in AI that embodies the knowledge and values of dominant groups while rendering alternative ways of knowing invisible.

The word systematic is important. SEE is not the result of individual careless decisions. It is the predictable outcome of building on what was already digitised, already licensed, already legible to the organisations making training decisions. Those organisations were predominantly Western, English-language, and operating within a specific set of commercial and academic priorities. They built on what they had. And what they had reflected where they were.

The result is an AI system that does not know it has a gap. It produces confident, fluent outputs from the knowledge it possesses — without signalling, without flagging, without any visible indication of what it does not know. And the gap is not random. It follows a pattern: the knowledge most absent from AI systems is disproportionately the knowledge of the Global South, of non-Western legal and governance traditions, of communities whose intellectual heritage was never commercially valuable enough to digitise at scale.

Why this is different from bias

Most AI governance conversations focus on bias — the tendency of AI systems to produce unfair or discriminatory outputs. Bias testing, fairness metrics, and explainability requirements are all designed to catch and correct these output-level problems. And they matter.

But here is the critical distinction that SEE introduces: bias is a deviation from a neutral baseline. SEE describes a situation where the baseline itself was never neutral. You cannot audit your way to a neutral baseline that does not exist. You cannot test an output for something that was never in the input. And you cannot correct, at the deployment stage, for a structural absence that was encoded before the model existed.

So, for example, a bank deploying an AI credit scoring system can pass every EU AI Act conformity assessment, satisfy every audit requirement, and demonstrate full regulatory compliance — and still operate a system that structurally cannot read the financial behaviour of borrowers whose economic lives do not match the patterns the model was trained to recognise. Not because the bank did anything wrong at deployment. But because the foundational model it deployed inherited a worldview that never included those borrowers in the first place.

This is what makes SEE a governance problem of a different order. It is upstream of every instrument currently designed to address it.

Why it matters now

AI is not a passive tool. Unlike a spreadsheet or a database, AI generates — it produces new outputs from the knowledge encoded in its training data, outputs that become inputs to human reasoning and institutional decision-making. And increasingly, those outputs become training data for the next generation of models.

So, the exclusion encoded today does not remain static. It compounds. The knowledge traditions absent from this generation of AI systems become even less represented in the next, as models train on the outputs of models that were already incomplete. The epistemic universe narrows — systematically, invisibly, and at a speed no previous technology has approached.

This is why I describe SEE as having civilisational stakes. We are not talking about a system that occasionally gets things wrong. We are talking about the structural narrowing of what the world’s most powerful knowledge infrastructure can know — at precisely the moment when that infrastructure is being embedded into the decisions that shape lives, allocate resources, and distribute opportunity across the globe.

The governance window to address this is open now. It will not stay open indefinitely.

What this series will cover

This is the first article in the SEE Series, which develops the theory of Systematic Epistemological Exclusion and its companion construct, the Death of Traits, drawing on original research I have published on SSRN.

The full theoretical framework is developed in “The Death of Traits: Epistemic Exclusion in AI and a Framework for Decolonial AI Governance” (SSRN 6283019). A second working paper, “Invisible Borrowers, Unpriced Risk: The Upstream AI Governance Gap under Article 26 of the EU AI Act” (SSRN 6313443), examines how SEE travels into financial services specifically. Both papers are open access.

The next article in this series examines the Death of Traits in detail — the two specific mechanisms through which knowledge disappears from AI, and why one of them is invisible even to the communities experiencing it.