LawyerUp.
Multi-agent legal AI · LangGraph · hybrid RAG · streaming
“Most legal chatbots hallucinate. The only thing worse than a wrong answer is a wrong answer with a confident citation.”
Most legal AI products either prompt-engineer a single model and hope, or do a thin RAG retrieval over a generic corpus. Neither holds up for a jurisdiction like India where the answer to 'what does Section 420 IPC say' has to be sourced from an Act, jurisdictionally correct, and not contaminated by case law from other countries. A single LLM call can't carry that responsibility — the routing, retrieval, and synthesis each need to be inspectable.
I architected LawyerUp around a graph runtime, not a chain. The router agent classifies intent and jurisdiction, then dispatches to one or more specialist agents (criminal, family, contract, employment, compliance, drafting, review). Each specialist runs RAG over a domain-scoped slice of the corpus and returns a structured response. A synthesizer composes the final answer; a finalizer attaches the safety envelope. Every numerical claim and every citation traces back to a retrieved chunk — nothing is hallucinated in the final response.
The choices that mattered, with the reasoning at the time.
- Hybrid retrieval, not pure vector
sqlite-vec for semantic + FTS5 for keyword, fused via Reciprocal Rank Fusion. Legal questions often hinge on a specific section number or statute name that vector search misses but BM25 nails. RRF + MMR diversity gives the best of both with explainable scoring.
decision · 01 - Custom graph runtime, not LangGraph (initially)
I wrote a small graph framework with retries, timeouts, middleware, and checkpoint snapshots — gives me visibility into per-node state I couldn't get from a black-box library. The node API mirrors LangGraph so a migration is one weekend if I need their tooling later.
decision · 02 - Server-Sent Events, not WebSockets
Chat streaming is one-directional. SSE survives proxies, is HTTP-native, and the per-step event types (routing → agent_start → agent_complete → delta → done) let me build a UI that shows the agent thinking — not just typing.
decision · 03
One code surface that captures the structural decision.
from .specialist_agents import RouterAgent, LegalResearchAgent, CriminalLawAgent
from .framework import Graph, Node, edge
def build_orchestrator(llm_router, retrieval) -> Graph:
g = Graph(name="lawyerup")
g.add(Node("prepare", fn=lambda s: prepare_state(s)))
g.add(Node("route", fn=RouterAgent(llm_router)))
g.add(Node("refine_jurisdiction", fn=JurisdictionDetector()))
g.add(Node("run_primary", fn=lambda s: dispatch(s.routing.primary, s)))
g.add(Node("run_secondary", fn=lambda s: parallel(s.routing.secondaries, s)))
g.add(Node("fallback", fn=lambda s: LegalResearchAgent(retrieval)(s)))
g.add(Node("synthesize", fn=synthesize_answers))
g.add(Node("finalize", fn=attach_safety_envelope))
edge("prepare", "route")
edge("route", "refine_jurisdiction")
edge("refine_jurisdiction", "run_primary", when=lambda s: s.routing.confident)
edge("refine_jurisdiction", "fallback", when=lambda s: not s.routing.confident)
edge("run_primary", "run_secondary")
edge("run_secondary", "synthesize")
edge("fallback", "synthesize")
edge("synthesize", "finalize")
return g.compile(retries=2, timeout_s=30, checkpoint=True)
The case study is more useful when you can poke it.
responses pre-canned for demo. live deployment in the repo — see github.
Backend is feature-complete: chat + structured drafting (NDA / offer-letter / demand-notice templates with LLM polish) + contract review (extract → review → structured findings with risk labels). Next.js 15 frontend ships streaming chat with the full per-step trace visible. API-key auth, sliding-window rate limiting, JSON access logs, pytest coverage on the orchestrator, retrieval, framework primitives, and LLM router.
- specialist agents
- 8
- rate limit
- 20/60s
- retrieval
- hybrid
I'd rebuild the retrieval scoring as a single learned reranker instead of RRF + MMR — the hand-tuned weights work but a fine-tuned cross-encoder would carry the jurisdictional priors better. And I'd ship streaming over Server-Sent-Events from day one; refactoring to add it after the synchronous endpoints existed cost me an afternoon I won't get back.
- Workflow graph: prepare → route → refine_jurisdiction → run_primary → run_secondary → fallback → synthesize → finalize
- 8 specialist agents: Router, Jurisdiction Detector, Intent Classifier, Legal Research, Criminal/Family/Contract/Employment/Compliance Law, Document Drafting, Contract Review
- Retrieval: sqlite-vec (384-dim MiniLM) + FTS5 → Reciprocal Rank Fusion → MMR diversity → citation envelopes attached
- LLM Router abstracts Gemini (default), OpenAI, Anthropic — per-role overrides for routing / research / synthesis
- Server-Sent Events streaming emits: request · start · routing · agent_start · agent_complete · sources · delta · done · error