Veil

Employees paste customer records, contracts and API keys into chat sites, and LLM-backed apps send whole tickets to models in other countries. India's DPDP Rules carry penalties up to ₹250 crore, and banning AI doesn't work. Companies need a way to say yes to AI while keeping the sensitive parts at home.

Personal project, end to end: problem research, the guard proxy and detection engine, control plane, audit pipeline, Angular console, browser extension, chaos and load tests, and Kubernetes manifests.

1. D/01

   No database on the hot path

   Policies, dictionaries and routes reach every proxy through a compacted Kafka topic, each proxy in its own consumer group, and live in memory. The proxy is stateless and scales out; a revoked key is enforced everywhere within seconds.

2. D/02

   Checksums, not just regexes

   Aadhaar uses Verhoeff, cards use Luhn, IBANs mod-97, GSTIN its check character, with context words raising or lowering confidence. Company codenames go through one Aho–Corasick pass, so 10,000 terms cost the same per character as ten.

3. D/03

   AI that runs inside the proxy

   Confidential topics and paraphrased prompt injections have no pattern, so an in-process all-MiniLM-L6-v2 model on ONNX votes over labelled examples. No API cost and nothing leaves the network. Dynamic batching lets one inference serve every waiting text.

4. D/04

   Reversible redaction, even while streaming

   The same value always gets the same placeholder, so the model can still reason. A placeholder split across SSE chunks ([EMA + IL_1]) is held back only as long as needed, tested at every split point.

5. D/05

   Semantic cache on the redacted prompt

   Redis 8 vector search caches answers by redacted prompt, so two customers' similar requests share one answer and each gets their own names back. Sensitive values never enter the cache.

6. D/06

   Audit without the data

   Audit events carry types, offsets and salted HMAC fingerprints, never values. Inserts are idempotent on the request id over at-least-once Kafka, into a day-partitioned table where retention is a DROP of an old partition.

• Under load the first version ran one embedding per text on every core; at 140 req/s the median embedding took 230 ms. Dynamic batching and load shedding by the org's fail mode (503 with Retry-After, or pattern-only checks) replaced a timeout convoy.
• The chaos test found that a control-plane outage took every app down. A stale-if-error key cache now keeps known keys working from memory while revocations still evict.
• A Kafka outage during a burst of 20 requests: all were served, and 20 of 20 audit events arrived after recovery. The dashboard's 'waiting' gauge now counts in-flight sends too.
• Detection accuracy measured on 510 labelled prompts with decoys: precision and recall 1.00 on all 15 span types (synthetic set).

• Java 21
• Spring Boot 4
• WebFlux
• Kafka
• Redis 8
• PostgreSQL
• ONNX
• LangChain4j
• Angular
• Chrome MV3
• Kubernetes