Why ‘Glass Box’ Models Win: Credit Risk Analysis Software

As of the third quarter of 2024, the Consumer Financial Protection Bureau (CFPB) reported more than $1.65 trillion in outstanding loan balances, yet lenders continue to observe a gap between projected risk at loan origination vs. what reality looks like twelve months into the loan maturity. The core problem is that legacy prediction models that are the foundation of credit risk software for lenders and credit unions rely on lagging indicators – snapshots of credit data points reported by credit bureaus that are combined to create prediction scores. These credit risk models, combined with traditional multi-point scorecards that use (among others) FICO scores, suffer from the same challenge. Increasingly sophisticated fraud, economic instability, and rapid fluctuations in collateral depreciation have weakened the correlation between FICO bands and “safe” loans.

As more consumers fall behind on their auto payments, high-risk sectors such as subprime auto loans are especially vulnerable, with delinquencies 60+ days past due reaching record highs and exceeding 6.6% in recent quarters. The financial distress, however, is not limited to the subprime sector. Even the prime and super-prime segments have experienced rising delinquencies as inflationary pressures have reduced affordability across all FICO score bands. With average new-car payments exceeding $750 per month and used-car payments exceeding $540 per month, inflation and wage pressures have shifted consumers’ priorities from paying for car loans to acquiring necessities such as housing, food, and energy, which remain at elevated prices. With stimulus checks, forbearance programs, and paused student loan repayments ending, consumers are now facing the double burden of reduced affordability and higher vehicle purchase costs.

The challenge is that, to navigate this volatility, lenders and credit unions must move from black-box scores that produce a number with little, if any, explanation to transparent ‘glass box’ approaches that identify the specific behavioral signals driving customer credit risk exposure.

Why Legacy and Black Box Models Break in Credit Analysis

The auto lending industry is built on scorecards for financial risk management, and scorecards are largely based on logistic regression models. The challenge with this approach of credit risk assessment is that it assumes a linear relationship between higher income and (for example) lower risk. The scorecard, however, fails to account for non-linear stressors like what is affectionately referred to as the “HENRY” segment – High Earner, Not Rich Yet – who, despite high incomes, have low investable assets, they carry significant long-term debt, spend significant portions of their incomes on experiences or luxury items, and often need tailored loans for high-end purchases like homes or vehicles. Legacy models and scorecards are based on data that is a snapshot in time, a static row of data that does not account for the velocity, such as the rate of change in a borrower’s bank balance over the previous ten weeks, without weeks of manual coding.

To address the challenges of static models and scorecards in credit management, there is a temptation to use Neural Networks (Deep Learning). While the CFPB has been clear about its stance on “complex algorithms,” the use of “Post-hoc Explainability” tools such as SHAP has been growing as a means of approximating lending decisions. The problems with SHAP and other similar techniques in credit risk decisioning are that they do not provide the exact causal logic used by the model, creating a regulatory liability for lenders who are not able to list “Credit History” as a reason for loan denials when the model used an interaction of over 50 variables to make the decision. In fact, the CFPB’s notice explicitly states that the Equal Credit Opportunity Act (ECOA) and Regulation B.

“…require creditors to provide applicants a written statement of specific, principal reasons for any adverse action, regardless of the technology used to make the decision.”

Lenders must provide applicants with a written notice explaining the main reasons for any adverse action, regardless of how the decision was made. The reasons must be specific and reflect the lender’s “actual factors” as considered and scored, without using generic or vague explanations. Going one step further, even when using methods such as SHAP, lenders must validate that SHAP explanations are reliable and accurate reflections of the actual factors driving the credit decision, or risk non-compliance.

The “Glass Box” Architecture for Credit Risk Management

While Legacy and Black Box models are widely adopted, another technology approach can provide greater transparency and a “glass box” view. The idea behind this new credit risk management approach is to shift the focus from algorithms to the discovery of signals in data that financial institutions already have access to. By using advanced artificial intelligence technology, lenders can identify highly predictive signals (known as ‘features’) and incorporate them into transparent, interpretable models, such as EBMs or constrained boosting, that comply credit policies.

At the heart of the approach are three fundamental technological capabilities that make this glass box approach possible: 

• Raw Data: First is the ability for a system to ingest raw relational data inputs, such as transaction logs, payment histories, etc., to discover patterns such as “5+ Non-Sufficient Fund (NSF) events in last 90 days.” The specificity of such signals provides the type of transparency that regulators seek but has, until recently, been difficult to uncover due to the complexity of the internal and external data and the limited resources available to most lenders.
• SQL Transparency: The second core principle is SQL transparency. Systems such as dotData can provide the logic behind any discovered signals as standard SQL code, enabling the Risk team to audit exactly how a risk factor is calculated before it enters production.
• Multi-Table Capabilities: Finally, the ability to handle multi-table relationships is critical, as it enables systems such as dotData to traverse entity relationships and identify links between distinct tables, thereby enabling the exploration of deeper signals than a flat file could provide.

The most attractive element of this approach is that it replicates the human model of data exploration, in which a risk analyst or data scientist might formulate a hypothesis, test it by writing SQL code to validate it, and then implement it in production. The difference, of course, is that even the most skilled analyst or data scientist can generate only a finite number of hypotheses and will accumulate some bias over years or decades of work experience. Credit analysis systems such as dotData can scale this approach by exploring millions of hypotheses in an unbiased data-driven approach, identifying early warning signals based on their actual value and impact. As a result, lenders are able to identify high-risk applicants and improve portfolio performance.

Operationalizing Transparency: Agile Risk Mitigation

It’s critical to understand that we are not arguing that scorecards and legacy models are “useless,” but rather that they are no longer sufficient for auto lenders that are struggling to keep up with changing market conditions, emerging trends of fraud, and vehicle prices. By deploying glass-box technologies such as dotData, lenders can benefit from Post-Model Adjustments (PMAs) to have a competitive edge in the lending landscape. What are PMAs? A good way to think about it is this: if scorecards and legacy models can give a comprehensive view, PMAs can provide quarterly, monthly, or even weekly guidance on vital details that the models you use and update regularly miss. 

For example, while your scorecards might indicate that a score of 700 indicates a low-risk borrower profile, using PMAs to flag that the borrower’s social security number is less than 2 years old and that the same borrower is an Authorized User Tradeline on another profile might create a knockout rule for anyone who meets that specific segment. With synthetic fraud exposure surpassing $9.2 billion, the ability to spot potential fraudulent activity that a scorecard or model might have missed can be critical.

A second aspect of PMA techniques is the ability to leverage the discovered signals to build high-value, low-impact micro-segments of data. At the core of the signals that systems like dotData discover are magic thresholds – precise statistical “tipping points” of risk. For example, instead of an arbitrary 49% DTI cutoff, the system might identify that risk increases by 22 percentage points relative to the average at 56.5% DTI, allowing the lender to approve low-risk borrowers in the 50%-56% band that would otherwise have been rejected. Armed with these powerful signals, business leaders and analysts can combine them as “business drivers” to build multi-faceted micro-segments of data. 

For example, a lender might combine two signals to identify that in 1.3% of the applications, when “Job Tenure < 3 years” AND “DTI > 45%,” the historical likelihood of default increases by 33%. While this specific micro-segment is small, it nevertheless provides the lender with powerful, easy-to-explain and validate rules to augment knockout rules and reject applicants that meet or exceed these criteria. This type of “driver stacking” allows users to combine multiple drivers to identify niche yet statistically powerful customer segments that outperform the average portfolio but are rejected by competitors.

Advanced Analytics of Financial Impact: Basis Points and P&L

Using PMAs can mitigate the Loss Given Default (LGD) – the percentage of the total exposure (loan amount) that a lender fails to recover after a borrower defaults and the vehicle is repossessed and sold. With negative equity impacting a significant portion of vehicle trade-ins, static Loan-to-Value (LTV) calculations are dangerous at best. Glass box models that leverage PMAs can incorporate vehicle-specific depreciation curves to predict “future” LTV at the time of potential default, reducing loss severity that has remained high across both prime and subprime sectors.

In addition, the cost of “false positives” – rejecting a good borrower – is equal to the lifetime value of that customer. Assuming, for example, the roughly 5.72% average yield credit unions see on a used-car loan, capturing just 5% more “gray area” applicants who might typically be rejected adds millions to the bottom line without increasing the net charge-off ratio.

From Black Box to Glass Box

Moving from a traditional approach based on scorecards and black-box models to an innovative solution, the glass-box approach powered by PMAs entails two critical yet interrelated strategies: auditing the underlying signals that drive current models and expanding the “Green lane” of loan approvals by automating more decisions. In practical terms, it means two key changes:

• The Audit Step: Start by auditing all signals (aka “features”) in current risk models. Can your current model explain the difference between a 620 FICO borrower who pays on time and a 620 FICO borrower who defaults? If the answer is no, the model lacks behavioral nuance.
• Expand the Green Lane: Start by setting a strategic goal – like automating approvals of 60% of decisions. Glass box transparency gives your Risk Committee the confidence to widen the “Auto-Approve” lane because the guardrails that have been put in place are clear, specific, and easy to understand.

In a market marked by growing auto loan delinquencies and persistent affordability challenges, the challenge for traditional credit process and black-box models is keeping pace with rapidly changing conditions. Lenders can adopt ‘glass box’ architectures of advanced credit risk management software, avoid the confusing, opaque mathematics of complex models, and rely on granular, real-time behavioral signals that are easy to audit credit applications and ensure regulatory compliance. The shift to glass-box architectures enables lenders to implement Post-Model Adjustments, supporting agile risk mitigation and the confident expansion of their “green lane” of auto-approvals. Ultimately, given the current market landscape, winning lenders will not merely be those managing risk with the most sophisticated algorithms, but those with the clearest view of their business drivers.