All case studies

01 · fintech · LLM agents

AI underwriting analysis, including decisioning, fraud detection, cash flow scoring and paystub screening

Every loan application gets an AI second opinion across dozens of checks, with every rule outcome and tool call logged.

  • Azure OpenAI
  • OpenAI API
  • LangChain.js
  • LangGraph
  • Vision models
  • Node.js
  • Laravel
  • Playwright
  • Plaid
  • TransUnion
  • Clarity
  • IPQS
Client
Stepping Stone Loans, a US consumer lender
Role
Senior full-stack & AI engineer
Period
Dec 2024 – present
Engagement
Long-term contract through Upwork

Problem

Stepping Stone Loans wanted an independent second assessment of each loan application next to its in-house underwriting engine, without handing approval decisions to an AI.

The assessment had to work from the same evidence underwriters use: bank data and cash flow, credit and bank-behavior reports, paystubs and fraud signals.

Paystubs were the weak link: borrowers upload documents that are stale, incomplete or not payroll documents at all, and each upload needed a manual check before Plaid verification.

How it works

diagram · How an application is reviewed
  1. 01ApplicationA new loan application enters review.
  2. 02Screen paystubsA vision model fills a strict schema (employer, pay period, gross and net pay, deductions, garnishments), and deterministic checks catch stale, incomplete or non-payroll documents.
  3. 03Verify incomeNet pay is compared with bank deposits, and clean documents are uploaded to Plaid automatically.
  4. 04Score cash flowThe applicant’s bank data is analyzed into a cash flow score.
  5. 05Detect fraudDocument checks and bank-side forensic signals roll into one fraud flag, next to TransUnion, Clarity and IPQS credit and fraud checks.
  6. 06Walk the policyAgents check the lending policy rule by rule, and every rule outcome and tool call is logged.
  7. 07Second opinionThe assessment sits next to the in-house decision.

When a paystub fails a check

  • Re-upload requestThe borrower is told exactly what to re-upload instead of getting a silent rejection.
  • Flagged for reviewSuspicious documents are flagged before they reach a human underwriter.

What I built

  • Built multi-agent underwriting on Azure OpenAI with LangChain.js and LangGraph: tool-calling agents gather the evidence, cover onboarding verification and KYC, and walk the lending policy rule by rule.
  • Built fraud detection: vision checks on documents and bank-side forensic signals from Plaid roll into one fraud flag through a pluggable signal aggregator, so new fraud checks plug in without touching the core.
  • Built a cash flow score analyzer that turns the applicant’s bank data into a cash flow score for underwriting.
  • Built rules-based decisioning on a credit and fraud risk pipeline (TransUnion, Clarity, IPQS) with a visual rules editor, so underwriters author, version and audit policy without engineers.
  • Extracted paystub fields with a vision model into a strict schema, with an automatic fallback from Azure OpenAI to OpenAI on the Laravel side.
  • Wrote deterministic document checks that tell the borrower exactly what to re-upload instead of silently rejecting the file.
  • Built a Node.js service that uploads clean documents to Plaid by driving a headless browser with Playwright.
  • Logged every rule outcome and tool call, with durable agent state and retries when a provider rate-limits, so each AI assessment can be audited next to the recorded decision.
  • Ran champion vs. challenger prompt experiments on live applications and built a dashboard that compares AI and recorded decisions loan by loan.

Key decisions

A shadow reviewer, not an approver
The AI assessment sits next to the in-house decision instead of replacing it, so the lender gets an independent second opinion without handing lending decisions to a model.
Schema first, then checks
Paystub extraction is forced into a strict schema before any deterministic check runs.
Two model providers with automatic fallback
Azure OpenAI or OpenAI is selected by configuration, and paystub extraction falls back to OpenAI when the Azure call fails.
Benchmark models on real loan files
GPT-5.x and Grok models were compared on historical loan files to pick the best cost and quality balance.
Browser automation for the Plaid upload
Plaid has no API for uploading paystubs, only its UI SDK, so a Node.js service drives that flow in a headless browser with Playwright.

Guardrails

  • The AI never approves or denies on its own; decisions stay with the in-house rules engine and the underwriters.
  • New prompts run champion vs. challenger on live applications before they replace the current one.
  • Personal data is redacted before any model call, including the benchmark runs.

Testing and evaluation

  • Evaluation baselines run in CI, so a prompt or model change can’t silently degrade decisions.

Result

Handles hundreds of applications a day.

Manual paystub checks are gone from the operations team, and suspicious documents are flagged before they reach a human underwriter.

Underwriters get an independent, fully logged assessment for every application and author their own risk rules without engineers.

media note

The lender’s internal systems are private, so this case study uses words and a diagram instead of screenshots.