Two AIs disagree about the reason someone can’t borrow money

Imagine a small business owner's loan application has just been rejected by an algorithm. Under international standards, she is entitled to know why.

To explain the reasoning, lenders use a coefficient-based explanation tool, often known as explainable artificial intelligence (AI), to 'look inside' and decipher the algorithm's rationale. This process should always give more or less the same account – at least in theory.

These AI tools are marketed to lenders as tools for transparency and sources of truth.

Our benchmark study, soon to be published in the post-conference IntelliSys 2026 papers, tested these AI explanation methods against each other for 1200 real small-to-medium enterprise loan applications in Bangladesh.

In one example, the first AI answer centres on an applicant's repayment record as the reason for loan rejection. A history of late payments, the size of the loan requested, monthly cash flow and so on.

The second tool, known as LIME (Local Interpretable Model-agnostic Explanations), answers entirely differently. It focuses on how long her business has operated, how many people she employs and how often she has borrowed before.

Compare the two lists of reasons and not one common feature appears on both.

Sciences & Technology

Developing countries are writing AI laws they cannot enforce

Our study evaluated three explanation approaches used to interpret AI-driven credit decisions: Shapley Additive Explanations (SHAP), LIME and direct coefficient analysis, the plainest method of all.

Each explanation method was asked the same question, which features mattered most to the decision made by the credit-scoring algorithm and how much did each feature contribute to the decision.

Coefficient analysis and SHAP answered consistently, their full rankings correlated in close to perfect agreement.

LIME broke that pattern entirely.

It overlapped with none of the top five features determined by coefficient analysis. And the overlap was barely better when measured against SHAP.

However LIME was compared, it told a different story about the same underlying decision.

Not all explanations are equal

Researchers who study explainable AI have a name for this pattern: the disagreement problem.

It has mostly been documented as a technical curiosity, a reason for caution among machine learning specialists.

Our study gives it a specific, regulatory shape, because a lending institution does not typically run three explanation systems at the same time.

In reality, it uses one, embeds it in its loan-processing software and applies it uniformly.

Whichever method it chooses becomes the official answer for every rejected applicant, not because that method is more accurate than the alternatives, but because of a decision made long before an application was ever filed.

Sciences & Technology

What happens when AI runs out of pictures?

The disagreement is also not a case of three equally valid perspectives on the same truth.

When we retrained the credit models using only the top five features of each explainable AI tool, the coefficient analysis and SHAP models retained close to their full predictive accuracy. This means their five key features were enough to capture what the original model relied on for its decision.

LIME's chosen features didn't hold up as well by comparison.

Its stated reasons were far further from what the original model was actually doing, not merely phrased differently.

A borrower whose rejection letter is generated by LIME is not receiving an alternative but equally legitimate account of the decision. She is receiving an account that is simply less true.

What the law assumes about AI explanations

This distinction matters because two major legal frameworks already assume explanations of this kind can be trusted.

The European Union's General Data Protection Regulation (GDPR) entitles individuals to meaningful information about the logic behind an automated decision that affects them.

The United States' Equal Credit Opportunity Act goes further, requiring lenders to state specific reasons for an adverse credit decision, not a general disclaimer that a model was involved.

Both frameworks treat an explanation as a stable fact about a decision, something a lender can retrieve and report. And something a borrower can use to inform their future.

That assumption carries a real cost for the borrower, not just the regulator.

Health & Medicine

We need to regulate the ‘Wild West’ of medical AI scribes

If the reason she is given is the wrong one, she may spend the next year fixing a problem that was never the cause of her rejection, paying down the wrong debt or chasing the wrong metric, while the actual barrier to approval goes untouched.

If the explanation depends on which method generated it, the reason communicated to a borrower is not a fact drawn from the model. It is a choice made out of sight of both borrower and lender.

We have the answer already

The same evidence also points to a remedy that does not require new technology.

Coefficient analysis, the simplest and oldest of the three methods, matched or exceeded the others on faithfulness, producing stable explanations and running at a fraction of the computational cost that LIME requires for the same task.

It performs this well specifically because it suits the linear credit models that many microfinance institutions already use.

Better, fairer lending explanations do not require an unproven tool. It requires choosing the one the evidence already favours, and being honest about the one it does not.

Regulators overseeing AI-driven lending, in Bangladesh and in every market using automated credit decisions around the world, should stop treating explainable AI as a single box to be ticked.

Current disclosure law does not ask which AI model is closer to the truth.

Lenders should be required to disclose which explanation method produced an adverse-action notice and to then justify that choice against a fair standard, not merely certify that some explanation was produced.

Until that requirement exists, a rejected borrower is not being told why an algorithm decided against her. She is being told what one particular tool, out of several that disagree with each other, happened to produce.