# LightRAG retrieval-quality recovery — MC #900176 # LightRAG retrieval-quality recovery — MC #900176 **Status:** LIVE PATCHED / guarded rollout; agent routing and backlog remain disabled **Date:** 2026-08-23 **Worktree:** `/Users/makinja/system/.claude/worktrees/lightrag-recovery-900176` ## Verified starting point The Cloudflare route, authentication, health endpoint, text upload, background processing, and vector write path work. The failure is higher in the stack: 1. The MC #900165 source is present as a processed full document, two text chunks, and two vector entries. 2. For the natural query “What does SCR mean in MC 900165 IndyDevDan smart prompting plan?”, the correct chunk ranks third at cosine `0.5319866538`. 3. Retrieval-only output contains the correct definition `SCR = simplify/compress/repeat`. 4. Server-side synthesis nevertheless answers `Simplify, Clarify, Refine` and fabricates `/path/to/indidevdan_smart_prompting_plan_section_scr.md`. 5. A generated-answer request poisons a later retrieval-only request for the same query/mode: LightRAG core `1.3.4` computes its response-cache key from only `(mode, query)`, checks that cache before `only_need_context`, and can therefore return the cached hallucinated answer instead of raw context. 6. Deployed LightRAG core `1.3.4` accepts only `text` in `POST /documents/text`; client-supplied `metadata` is silently ignored, so current text uploads become `unknown_source`. 7. The old ingest queue contains 14,006 stale rows. Its adapters and drain worker remain disabled and must not be enabled or drained as part of this recovery. Root-cause evidence: `/Users/makinja/system/evidence/lightrag-test-2026-08-23/root-cause-after-client-fix.json`. ## Decision Repair retrieval and provenance without trusting LightRAG’s answer generator and without mutating the existing corpus destructively. Build and validate against an isolated blue/green canary corpus first. Agent routing remains disabled until the quality gate passes. ## Build scope ### W1 — Provenance contract - Add a deterministic provenance envelope inside ingested text because the deployed API ignores external metadata. - Apply it consistently to manual CLI upload, queue drain upload, and bulk upload. - Preserve source type, source ID/path/URL, content hash, and ingestion timestamp without including credentials or private environment data. - Ensure already-enveloped content is not double-wrapped. ### W2 — Agent-safe retrieval - Add a retrieval-only path that never invokes server-side answer synthesis. - Isolate retrieval-only requests from generated-answer cache entries; validate the server cache correction in the blue/green canary before any live-stack change. - Parse returned chunks, preserve source attribution, apply deterministic lexical/exact-ID reranking, and return bounded advisory context. - Reject or clearly mark `unknown_source`; never fabricate citations. - Keep ordinary LightRAG failures fail-open for agents, but expose diagnostics. ### W3 — Evaluation - Add deterministic unit tests for provenance, parsing, reranking, source filtering, bounded output, and no-generated-answer behavior. - Add an exact-source live canary including MC #900165. - Record retrieval, provenance, latency, and hallucination-safe outcomes separately. ### W4 — Blue/green canary corpus - Define and run an isolated LightRAG canary with separate ports and volumes; do not touch live volumes. - Ingest a small curated authoritative corpus only: current BookStack/runbook or equivalent local sources, active rules/specs, and selected completed MC outcomes. - Exclude session dumps, runtime evidence noise, credentials, and the 14,006-row stale queue. - Tear down or keep stopped after evidence capture unless explicitly approved for cutover. ## Non-goals - No blind backlog replay. - No automatic per-prompt injector. - No destructive edit of the current graph or Docker volumes. - No embedding-model switch before measurement. - No claim that LightRAG is safe merely because HTTP/status checks pass. ## Acceptance criteria 1. All changed JavaScript parses and relevant tests pass. 2. Every new canary document has deterministic source provenance in retrieved context. 3. The MC #900165 exact query returns the correct source and `SCR = simplify/compress/repeat` without server synthesis or invented citations. 4. Controlled-corpus Recall@5 is at least 0.80, provenance coverage is 100%, and exact canaries have zero fabricated references. 5. Context is bounded and retrieval failures remain non-blocking for agents. 6. Live LightRAG volumes and the 14,006-row stale queue are unchanged by the canary run. 7. Pi/Claude global LightRAG routing remains disabled until independent verification passes. 8. Builder and independent validator evidence are recorded before MC #900176 can move to review. ## Rollback Code rollback is one commit revert. Canary rollback is `docker compose down` against the dedicated canary compose project; its separate named volumes can be preserved for evidence or deleted only with explicit approval. For the live server cache patch, rollback requires reverting the patch commits and **force-recreating only the `lightrag` container** from the image; a plain restart retains the already-patched container filesystem. Live data volumes must never be removed. ## AFTER — implementation and guarded live deployment ### Shipped - Deterministic, allowlisted in-text provenance: `lib/lightrag-provenance.js`. - Retrieval-only parsing, exact ID/acronym reranking, token-bound matching, definition-centered excerpts, unknown-source handling, and bounded context: `lib/lightrag-safe-context.js`. - Provenance wired into manual upload, queue drain, and bulk upload clients. - `safe-query` never invokes server synthesis and rejects generated-answer-shaped cache collisions. - `discover.js` now uses raw naive context, retains enough chunks for attribution, reranks locally, and centers its summary on the answer. - Runtime LightRAG `operate.py` patch bypasses generated-answer cache reads for exactly four context/prompt query paths; it is idempotent and fails closed on upstream drift. - Isolated canary compose uses loopback `127.0.0.1:19621` and run-scoped `lr900176-canary-*` volumes. Live commits: `c55a98599`, `a92d8868c`, `5010a8ccc`, `b6d3ad7d6`, `79778b950`, `2dbebc585`, `82eb4641d`, `61d23a9df`. ### Verification - Unit tests: 13/13 Node PASS plus 3/3 Python PASS. - Isolated five-document canary: Recall@5 `1.00`, provenance coverage `1.00`, exact-term pass `1.00`, synthesis calls `0`, p95 `1064ms`. - Cache-poisoning probe: a generated query completed first; the same retrieval-only query still returned raw attributed context — PASS. - Live `lightrag` and `lightrag-neo4j`: running and healthy; runtime cache-isolation markers: `4`. - Controlled live upload increased processed documents `28,748 → 28,749`. - Exact live query: authoritative MC #900165 report ranked first, 100% selected-source provenance, `SCR = simplify/compress/repeat`, server synthesis disabled. - Live `discover.js`: shows that definition with `[type=manual] [company=alai] [file=~/system/reports/indydevdan-agent-improvements-2026-08-23.md]`. - Queue remains exactly 14,006 rows / 37,466,092 content bytes; all five ingest/drain LaunchAgents remain `NOT_LOADED`. - Pi and Claude Code now have **on-demand** `safe-query` routing in their global prompts and `lightrag-query` skills. Automatic per-prompt injection remains absent. Fresh Pi and fresh Claude Code sessions both resolved MC #900165 via the exact safe command, attributed the report, returned `SCR = simplify/compress/repeat`, and did not invoke generated query modes or `--allow-unknown`. - Independent implementation review: PASS, P0=0, P1=0. Independent post-deploy review: core/live verdict PASS; its bookkeeping/routing questions are resolved by resuming MC #900176 and this explicit guarded-rollout decision. Primary evidence: - `/Users/makinja/system/evidence/900176/canary-pass1/verdict.json` - `/Users/makinja/system/evidence/900176/canary-pass1/cache-isolation-probe.json` - `/Users/makinja/system/evidence/900176/live-deploy/live-safe-query-attributed-final.json` - `/Users/makinja/system/evidence/900176/live-deploy/live-discover-lightrag-final.txt` - `/Users/makinja/system/evidence/900176/live-deploy/postdeploy-queue.json` - `/Users/makinja/system/evidence/900176/live-deploy/postdeploy-ingest-services.txt` - `/Users/makinja/system/evidence/900176/pi-safe-routing-canary-v4-summary.json` - `/Users/makinja/system/evidence/900176/cc-safe-routing-canary-v2-summary.json` - `/Users/makinja/system/evidence/900176/claude-independent-review.txt` - `/Users/makinja/system/evidence/900176/claude-independent-postdeploy-review-final.txt` - `/Users/makinja/system/evidence/900176/native-peer-routing-verify.txt` - `/Users/makinja/system/evidence/900176/live-safe-query-bounded-final.json` - `/Users/makinja/system/evidence/900176/native-peer-bounded-output-verify.txt` ### Deliberately not enabled - No automatic per-prompt Pi/CC injection. - No drain or replay of the 14,006-row stale queue. - No activation of BookStack/MC/filesystem ingest daemons. - The legacy `lightrag-ingest-mc-directory.js` path is not loaded; it remains a separate cleanup candidate before any broad ingest reactivation. The safe on-demand command is: ```bash node ~/system/tools/lightrag.js safe-query "question" --json ``` Only this retrieval-only path is approved for advisory use. Generated `lightrag.js query` answers remain untrusted.