Case study · Fintech · AI security“Can a bank’s chatbot be tricked into leaking customer data”
The product
FinBot is a support assistant for a fictional bank. The same attacks — prompt injection, data leakage, excessive agency, hijacking and hallucination — were sent to the original bot and to a hardened version. Every message, guardrail decision and score is traced and shown in a read-only monitoring view.
The challenge
Chatbots with access to account data fail quietly. The audit had to find the real leaks, fix them with input and output guardrails, and show the before and after so that someone who isn’t an engineer can see what changed.
How we built it
- A red-team suite of 20 tests across six attack categories
- Input and output guardrails — the pass rate went from 40% to 100%
- Side-by-side chat that sends one message to both versions
- Langfuse tracing for sessions, traces, scores and latency
What it proves
- AI with guardrails you can measure, not promise.
- Every answer is traced, so failures can be found and fixed.
- Results a security reviewer and a product owner can both read.
