← Back to Projects
Machine Learning · Fraud & Risk Detection

FraudLens: an imbalance-aware fraud detection workflow

An end-to-end pipeline built around the defining challenge of fraud detection: the outcome you care about most is the one standard accuracy metrics ignore. Model comparison, cost-based threshold tuning, and an AI explainability layer, all grounded in real, verifiable numbers.

284,807Transactions analysed
0.17%True fraud rate
0.860Average precision
87%Fraud recall (tuned)
The Problem

Fraud is a needle in a 284,807-transaction haystack

In this dataset (the Kaggle/ULB Credit Card Fraud benchmark), fraud accounts for just 0.17% of all transactions. A model that predicts "not fraud" every single time scores 99.8% accuracy and catches zero fraud. That single fact shapes every decision in this project: which models to compare, how to evaluate them, and where to set the line between a flag and a pass.

Methodology

Three approaches to the imbalance problem, compared honestly

Rather than picking one technique and hoping it works, all three standard approaches were trained on the same engineered features (transaction velocity, rolling amount z-score, hour-of-day risk windows) and evaluated on the same held-out test set, then ranked by Average Precision, since accuracy would be meaningless here.

Approach 1

Class-Weighted Random Forest

Penalises misclassifying the minority class more heavily during training, no resampling needed.

0.804Average Precision
Approach 2

SMOTE-Resampled Random Forest

Synthetically oversamples fraud cases in the training set to ease the imbalance before fitting.

0.802Average Precision
Approach 3 · Winner

XGBoost, scale_pos_weight

Gradient boosting with a built-in imbalance correction term, no resampling or class weighting needed.

0.860Average Precision
Results

The threshold matters as much as the model

XGBoost won on Average Precision, but the more interview-relevant result is what happened next: tuning the decision threshold against an explicit business cost (a missed fraud costs far more than a false alarm) rather than leaving it at the default 0.5 cutoff.

Precision-recall comparison across the three imbalance-handling strategies
Precision-recall comparison across all three strategies. XGBoost holds precision longest as recall increases.
ThresholdPrecisionRecallF1
Default (0.5)0.8930.8130.851
Cost-tuned (0.01)0.4520.8700.594
Total business cost across decision thresholds, with the chosen threshold marked
Total illustrative business cost by threshold. The chosen threshold (0.01) minimises total cost given a missed fraud is far costlier than a false alarm.

At the cost-tuned threshold, recall rises from 81% to 87% of all fraud caught, at the price of precision falling from 89% to 45%. That trade is not a flaw in the model, it is a deliberate business decision: given this illustrative cost ratio, the system should accept more false alarms in exchange for catching nearly nine in ten fraud cases. A different cost ratio would call for a different threshold.

Explainability Layer

A risk score is not something an analyst can act on

The model outputs a probability. An analyst needs a reason. This layer calls the Claude API to convert a flagged transaction's feature values into a short, grounded explanation, turning the model from "a score" into a tool someone would actually use.

Before → After
risk_score: 0.94  |  amount_zscore_5min: 4.8  |  is_night: 1  |  txn_count_60s: 6
"Flagged due to an amount far above this 5-minute window's typical transaction, occurring during a low-activity overnight window with unusually frequent transactions in the preceding minute."
Honest Limitations

Worth raising before anyone else does

See It Yourself

Full code, dashboard, and write-up on GitHub

The repository includes the feature engineering pipeline, all three trained models, the cost-tuning logic, the explainability script, and an interactive Streamlit dashboard.

View on GitHub Back to Projects