AI-assisted litigation preparation for Kenyan law: turns case text or a PDF into a structured legal brief, streamed step-by-step to the browser in real time. A research and productivity aid, not a substitute for qualified legal counsel.
Full request lifecycle from browser to Pinecone/Postgres/OpenAI, the data-layer split, and a decision table of every infrastructure trade-off with the problem it solves.
The 7-stage retrieval pipeline in full: query expansion, statute-scoped search, thresholds, dedup, LLM reranking, and compression, each with the failure mode it was built to fix.
The browser posts a case file to FastAPI on Cloud Run with a Clerk JWT, verified against Clerk's JWKS with a short in-memory cache. An orchestrator then runs the case through extraction and RAG retrieval concurrently (they're independent of each other), followed by RAG-grounded strategy generation, drafting, and a final QA pass: each stage streamed back to the browser as a `markdown_section` Server-Sent Event as soon as it finishes, then a closing `complete` event, so the user sees the brief build up live rather than waiting on one long request.
The statute corpus (Kenyan legal texts) is chunked, embedded with `text-embedding-3-small`, and upserted into a Pinecone serverless index (1536d, cosine) once, ahead of request time. Retrieval itself does query expansion, metadata filtering, score thresholds, deduplication, and an LLM-assisted rerank/compression pass before handing chunks to the Strategy agent. If Pinecone isn't configured, retrieval simply returns no chunks and the pipeline still runs; QA is similarly non-fatal, so a run always completes with whatever sections it managed to produce.
Neon Postgres stores case history and application data. Structured LLM outputs are enforced with the `instructor` library over Pydantic schemas rather than parsed from free-form text, so extraction and drafting stages return validated, typed data; LLM calls go through OpenAI or OpenRouter (gpt-4o by default, though OpenRouter can't be used for the embeddings step, so ingestion always needs a direct OpenAI key), and Langfuse provides observability into the pipeline's actual calls and costs.
This is the pipeline actually running behind the live demo: 17 Kenyan statutes (including the Work Injury Benefits Act and Occupational Safety and Health Act, added specifically to strengthen workplace-injury retrieval), sentence-aware chunked and embedded once into Pinecone, then retrieved through a multi-stage pipeline built specifically to fix cases where plain top-k vector search returned the wrong provision. It replaced an earlier, much simpler baseline (present in the original team capstone repo) that embedded the raw case text directly with no expansion, filtering, or reranking at all.
Builds the Pinecone index ahead of request time; nothing here runs while a user is waiting.
17 Kenyan statutes as plain-text files (Constitution, Contract Act, Land Act, Employment Act, WIBA, OSHA, and others).
A curated, named corpus beats crawling at query time; every retrieved chunk can be cited back to a real Act the drafting agent and the user can both verify.
Splits on sentence boundaries and accumulates to roughly 800 characters with a 100-character overlap, rolling back whole sentences rather than cutting mid-word.
An earlier character-only chunker was cutting legal sentences in half mid-word. Embedding a truncated sentence produces a misleading vector that retrieves against unrelated text.
Drops boilerplate ("Arrangement of Sections", publisher headers), chunks under 25 words, and table-of-contents-like blocks before they're ever embedded.
Index and TOC pages embed deceptively well, since they contain real legal terms like "Land Act" or "Section 38", but have no legal substance; left in, Pinecone would still return them and dilute the Strategy agent's context.
OpenAI `text-embedding-3-small` (1536d), batched 256 chunks per call; existing vectors for a source file are deleted before upserting new ones, batched 100 at a time.
Batching cuts API round-trips. Deleting by source before re-upserting makes re-ingestion idempotent, so re-running the script after editing a statute file doesn't leave duplicate vectors behind.
Fired as a background task alongside the Extraction stage rather than awaited sequentially, so its ~2s runtime hides inside Extraction's ~15s window.
One GPT-4o-mini call turns the full case text into legal issues, applicable statutes, and 7 focused search queries, with hard-coded rules (always search WIBA on workplace injury; one query per cited constitutional article).
Embedding a 2,000-word case as a single vector produces a diluted signal. Article 33(2), for example, embeds close to 33(1) and was consistently retrieved instead of the correct clause without a query written specifically for it.
A keyword map routes each sub-query to a Pinecone metadata filter naming the right Act; certain statutes also fire a fixed "guarantee query" for provisions that embed poorly, like WIBA's tort-bar clause.
An unfiltered query about "employer liability" pulls chunks from three unrelated Acts at once. The guarantee query exists because that specific tort-bar section can rank outside the normal top-k results on its own, semantically.
Filtered, single-statute searches use a lower cosine threshold (0.45) than unfiltered, full-index searches (0.60).
A filtered search competes against far fewer vectors, so its best match can legitimately score lower without being any less relevant. One shared threshold would wrongly discard it.
Exact-match dedup first, then fuzzy dedup on containment, 90% word overlap, and Jaccard similarity of 0.85 or higher.
The same provision surfaces from multiple sub-queries. The fuzzy threshold is set deliberately high because legal text reuses common words like "court" or "section" across unrelated provisions, so a looser threshold would collapse genuinely distinct clauses.
GPT-4o-mini scores every deduplicated candidate 0-10 against the case and keeps the ones scoring 5 or above, targeting about 8 chunks.
Cosine similarity alone can't tell "on point" from "same keywords, wrong branch of law." The judge is given explicit failure modes to screen for: boilerplate, wrong statute, surface-keyword-only matches.
One call per surviving chunk extracts only the exact sentence(s) relevant to the case, verbatim, never paraphrased; if nothing is relevant, the original chunk is kept rather than dropped.
An 800-character chunk might hold 8 sentences and only 2 relevant ones, especially for Constitution articles with several unrelated subsections in one chunk. Denser signal per token means a cleaner prompt for the Strategy agent.
Within the original team capstone, my assigned scope was the FastAPI backend, pipeline orchestration, and LLM integration, and I additionally took on and implemented the RAG system (Pinecone-grounded retrieval) end to end.
The bootcamp then required each teammate to deploy their own individual version. In mine (`github.com/mrithwik/legal-assitant`, running on AWS), I further built out the RAG system beyond the shared team baseline. The deployment shown here is a later step: a free-tier re-platform of that individual AWS version onto Vercel, Google Cloud Run, Neon, and Pinecone. That specific port changed only the infrastructure layer, not the application logic: the RAG improvements themselves were already in place before this port, not part of it.
This deployment is a deliberate free-tier re-platform of the original AWS version (App Runner + Aurora + Terraform), documented as a set of numbered decisions in the repo's `docs/decisions.md` with alternatives considered for each. The stated porting strategy: port the application logic verbatim and rebuild only the infrastructure-facing layer, since "the multi-agent RAG pipeline is the actual engineering work worth preserving."