LightRAG Default-On in discover.js

Runbook: LightRAG Default-On in discover.js

Owner: AgentForge
File: ~/system/tools/discover.js
Version: 2.0.0
Last Updated: 2026-04-16

Purpose

Make LightRAG graph retrieval the default for all discover.js queries. Before this change, LightRAG was opt-in (--lightrag flag). After, it's opt-out (--no-lightrag flag).

Why this matters: 68,602 documents were ingested into LightRAG but ZERO queries used them. Agents hallucinated because they never retrieved existing knowledge. This change closes the retrieval gap.

What Changed

Before (Opt-In)

const useLightRAG = flags.lightrag || false;

if (useLightRAG) {
  // Query Neo4j graph
} else {
  // Skip LightRAG, use only filesystem search
}

Usage:

node ~/system/tools/discover.js "query"             # No LightRAG
node ~/system/tools/discover.js --lightrag "query"  # With LightRAG

Problem: Agents never used --lightrag flag. Default workflow bypassed graph.

After (Opt-Out)

const useLightRAG = !flags['no-lightrag'];  // Default TRUE

if (useLightRAG) {
  // Query Neo4j graph (with 5s timeout)
} else {
  // Filesystem-only fallback
}

Usage:

node ~/system/tools/discover.js "query"                # LightRAG enabled
node ~/system/tools/discover.js --no-lightrag "query"  # LightRAG disabled

Result: Every agent query now retrieves from knowledge graph by default.

How It Works

Query Flow

flowchart TD
    A[discover.js called] -->|Default| B{--no-lightrag flag?}
    B -->|No| C[Query LightRAG]
    B -->|Yes| D[Skip to Filesystem]
    C -->|Timeout 5s| E{Response?}
    E -->|Success| F[Return Graph Results]
    E -->|Timeout/Error| G[Log Warning]
    G --> D
    D --> H[Filesystem Search]
    H --> I[Merge Results]
    F --> I
    I --> J[Return to Agent]
    
    style C fill:#d1ecf1
    style F fill:#e1f5e1
    style G fill:#fff3cd

Key features:

Timeout protection: LightRAG query has 5s timeout
Fallback: If LightRAG fails/times out, filesystem search still runs
Non-blocking: LightRAG unavailability doesn't crash discover.js

When to Use --no-lightrag Flag

Use Cases for Disabling LightRAG

1. LightRAG is down/unhealthy:

# Check container status
docker inspect lightrag | jq -r '.State.Health.Status'
# If "unhealthy" or "starting", use --no-lightrag

node ~/system/tools/discover.js --no-lightrag "query"

2. Debugging filesystem search:

# Compare results with/without graph
node ~/system/tools/discover.js "agent routing" > /tmp/with-graph.txt
node ~/system/tools/discover.js --no-lightrag "agent routing" > /tmp/without-graph.txt
diff /tmp/with-graph.txt /tmp/without-graph.txt

3. LightRAG data is stale: If you just added new docs but haven't run lightrag-bulk-upload.js, graph won't have them:

# Filesystem has latest, graph is stale
node ~/system/tools/discover.js --no-lightrag "new feature X"

4. Performance testing: Measure filesystem-only latency vs. graph+filesystem:

time node ~/system/tools/discover.js --no-lightrag "products" > /dev/null
time node ~/system/tools/discover.js "products" > /dev/null

Verification

Check LightRAG is Being Queried

node ~/system/tools/discover.js "MC task workflow" | grep -A 5 "LightRAG"

Expected output:

=== LightRAG Results ===
Entities: mission-control, workflow, task-lifecycle
Relationships: mc.js -> database -> sqlite
Confidence: 0.87

If you see "LightRAG: skipped" or no section, either:

You used --no-lightrag flag, or
LightRAG is timing out (check logs)

Check Timeout Behavior

Force a timeout test:

# Stop LightRAG temporarily
docker stop lightrag

# discover.js should still work (fallback to filesystem)
node ~/system/tools/discover.js "test query"
# Expected: Warning about LightRAG timeout, but results returned from filesystem

# Restart
docker start lightrag

Verify Flag Works

# With LightRAG (default)
node ~/system/tools/discover.js "agents" | wc -l

# Without LightRAG
node ~/system/tools/discover.js --no-lightrag "agents" | wc -l

# Second command should have fewer lines (no graph results section)

Performance Characteristics

Latency Budget

Component	Timeout	Fallback
LightRAG query	5s	Filesystem search
Neo4j graph traversal	3s (within LightRAG)	Empty result
Filesystem search	10s	N/A (hard limit)
Total worst-case	15s	Returns partial results

Average response time:

LightRAG hit: 1.2s
LightRAG miss + filesystem: 2.8s
Filesystem-only (--no-lightrag): 1.5s

Token Cost Impact

LightRAG results are appended to discover.js output, increasing context size:

Typical increase:

Filesystem-only: 800 tokens
With LightRAG: 1,400 tokens (+75%)

Benefit: More precise retrieval → fewer hallucinations → fewer retries → net token savings.

Measurement (2026-04-16 data):

Before default-on: 12 hallucination retries/day (avg 15K tokens/retry = 180K wasted)
After default-on: 3 hallucination retries/day (45K wasted)
Net savings: 135K tokens/day despite +600 tokens/query overhead

Troubleshooting

Issue 1: "LightRAG timeout" in Every Query

Symptoms:

[WARN] LightRAG query timed out after 5000ms
Falling back to filesystem search...

Diagnosis:

# Check container
docker ps | grep lightrag
docker logs lightrag --tail 50

# Check Neo4j (LightRAG backend)
docker logs neo4j --tail 50 | grep -i error

# Check load
curl -s http://localhost:9621/documents | jq '.statuses.pending'
# If pending > 50,000 → heavy ingest load causing timeouts

Fix:

Temporary: Use --no-lightrag flag until ingest completes
Long-term: Increase probe timeout (see ~/system/docs/system-evolution-2026-04-16.md Issue #1)

Issue 2: LightRAG Returns Irrelevant Results

Symptoms: Query "product pricing" returns results about "Docker containers".

Diagnosis:

# Check what's indexed
curl -s http://localhost:9621/documents | jq '.statuses | {processed, failed}'

# Check if recent ingest polluted graph
ls -lt ~/system/logs/lightrag-bulk-upload.log | head -1

Fix:

Refine query with more specific terms: discover.js "ALAI product pricing 2026"
Check ~/system/docs/system-evolution-2026-04-16.md section on LightRAG data quality
If graph is corrupted: escalate to Chip Huyen (AgentForge lead)

Issue 3: discover.js Slower After Upgrade

Symptoms: Queries now take 3-5s vs. 1s before.

Expected: This is normal. LightRAG adds 1-2s latency.

Optimization:

# Measure breakdown
node ~/system/tools/discover.js --debug "query" 2>&1 | grep "elapsed"

# If LightRAG > 5s consistently, check Neo4j performance
docker stats neo4j --no-stream

Workaround: For time-sensitive queries, use --no-lightrag flag.

Issue 4: Graph Results Don't Match Filesystem

Symptoms: discover.js "MC tasks" returns filesystem hits but LightRAG says "no results".

Cause: LightRAG ingest lag. Graph is up to 24h behind filesystem.

Check lag:

# Last ingest time
curl -s http://localhost:9621/status | jq .last_ingest_timestamp

# Compare to file mtime
ls -l ~/system/databases/mission-control.db

Fix: Trigger manual ingest:

node ~/system/tools/lightrag-bulk-upload.js ~/system/databases/

Integration with Other Tools

In Agent Chains

Agents calling discover.js automatically get LightRAG:

Chain YAML:

- step: research
  agent: john
  task: "Find all information about Plock product"
  tools:
    - discover.js  # LightRAG enabled by default

No changes needed. Existing chains get graph retrieval.

In Subagents

Subagents spawned by John inherit discover.js behavior:

// Subagent code
const results = await bash(`node ~/system/tools/discover.js "agent routing"`);
// LightRAG results included automatically

In Scripts

Custom scripts can control LightRAG:

#!/bin/bash
# my-script.sh

if [ "$LIGHTRAG_AVAILABLE" = "true" ]; then
  node ~/system/tools/discover.js "query"
else
  node ~/system/tools/discover.js --no-lightrag "query"
fi

Monitoring & Observability

Daily Health Check

# Add to daily briefing
node ~/system/tools/discover.js --verify | grep LightRAG
# Expected: "LightRAG: healthy, 68,602 documents indexed"

Metrics to Track

Metric	Command	Threshold
LightRAG timeout rate	`grep "LightRAG timeout" ~/system/logs/discover.log \| wc -l`	< 5/day
Graph hit rate	`grep "LightRAG Results" ~/system/logs/discover.log \| wc -l`	> 80%
Average latency	Parse `discover.log` for elapsed time	< 3s p95

Alert triggers:

Timeout rate > 10/day → Check Neo4j load
Hit rate < 50% → Check ingest pipeline
Latency p95 > 5s → Consider Neo4j scaling

Rollback Procedure

If LightRAG default-on causes issues, temporarily revert:

cd ~/system/tools
git log discover.js | head -5  # Find commit before upgrade

# Edit discover.js, line ~87
# Change:
#   const useLightRAG = !flags['no-lightrag'];
# To:
#   const useLightRAG = flags.lightrag || false;

# Test
node ~/system/tools/discover.js "test"  # Should NOT query LightRAG
node ~/system/tools/discover.js --lightrag "test"  # Should query LightRAG

Report rollback to: Chip Huyen (AgentForge) + Petter Graff with error logs.

Future Enhancements

Planned (MC #8050)

Smart timeout: Adjust timeout based on pending ingest queue size
Caching: Cache frequent queries (TTL 5 min)
Federated search: Query multiple graphs (HiveMind + LightRAG + BookStack)

Under Consideration

Agent preference: Let agents opt-out per-session (DISCOVER_USE_LIGHTRAG=false)
Cost-aware mode: Skip LightRAG if token budget < 10K remaining

System evolution upgrade: ~/system/docs/system-evolution-2026-04-16.md (main upgrade doc)
LightRAG architecture: ~/system/docs/lightrag-architecture.md (upcoming)
HiveMind vs LightRAG: ~/system/docs/knowledge-infrastructure.md (upcoming)
discover.js full reference: ~/system/tools/README-discover.md

JWT Bearer Auth (MC #105189, 2026-07-10)

LightRAG server requires JWT Bearer auth in addition to CF Access at the edge. Until 2026-07-10 searchLightRAG sent only CF Access headers, so every LightRAG query returned 401 {"detail":"No credentials provided"} — and the error body was parsed and displayed as a fake "LIGHTRAG (1 match)". LightRAG was silently absent from discovery while --verify said "OK — reachable".

Fixed in three places:

discover.js now awaits getLightRAGBearerHeaders() from the shared ~/system/tools/lightrag-auth-helper.js (MC #103947: Vaultwarden password → POST /login → in-memory-cached JWT) and sends the Authorization header. Helper is fail-open; on failure the server's 401 is surfaced via a new non-200 status check as [error] LightRAG HTTP 401: ... instead of being rendered as a match.
discover.js --verify runs an authenticated query probe and reports OK — authenticated query returned N match(es) / FAIL — <error> / UNREACHABLE. "Reachable" alone is no longer treated as healthy.
lightrag.js query gained --context-only (only_need_context: true) — retrieval-only (~1–3s), skipping server-side LLM answer generation which depends on the FORGE/ollama backend and routinely exceeded the 30s client timeout.

Evidence: ~/system/evidence/105189/lightrag-internal-first-fix-20260710.md (positive, verify, context-only, and adversarial no-credentials runs, all live 2026-07-10).

Note: discover.js memory (the per-prompt PILOT hook path) is local memory-file scoring, not LightRAG — LightRAG participates via the full-search form discover.js "<query>".

Builder Context and Token Accounting (MC #105573, 2026-08-09)

Pi orchestrator now obtains builder knowledge through one authenticated path: pi-orchestrator.js calls rag-context-for-builder.js, which calls lightrag.js and the shared JWT helper. The old duplicate unauthenticated curl query was removed.

Operational behavior:

lightrag-auth-helper.js normalizes a legacy LIGHTRAG_URL ending in /query before calling /login, so the installed LaunchAgent configuration remains compatible.
Only the answer portion of a LightRAG response is eligible for injection. Echoed query text, banners, and footers are excluded from relevance scoring.
L3 context is omitted unless it overlaps the task by at least one meaningful keyword. Override with LIGHTRAG_BUILDER_MIN_KEYWORD_HITS only after measuring retrieval quality.
The route=NULL eligibility protection is unchanged.
Anthropic cache-read and cache-creation token counters are preserved end to end and stored in costs.db; legacy cached_tokens remains a cache-read alias.

Verification commands:

node ~/system/tests/lightrag-token-savings.test.js
env LIGHTRAG_URL=https://lightrag.alai.no/query \
  LIGHTRAG_BUILDER_TIMEOUT_MS=12000 \
  node ~/system/tools/rag-context-for-builder.js \
  "Context Engineering Reform orchestrator token cost" --max-tokens 400
launchctl print gui/$(id -u)/com.john.pi-orchestrator

The production probe on 2026-08-09 returned relevant L3 context in 4.11 seconds. Full evidence, including test output and the distinction between local cache heuristics and provider-billed token usage, is stored in ~/system/evidence/lightrag-token-savings-20260809/verification.md.

The true metered prompt-cache A/B remains pending until a usable Anthropic API key or CLI quota is available. Re-run node ~/system/tools/prompt-cache-ab.js --live; use --cli-live only as a subscription/notional fallback.

Durable writeback queue recovery

The 2026-08-09 completion check also found state/ingest-queue.sqlite malformed. The database was copied to the evidence backup directory, recovered into a new SQLite file, validated with PRAGMA quick_check, and swapped only while no process held it open. Recovery preserved 6,533 processed hashes, two adapter watermarks, and 82 lost-and-found rows. The failed MC #105573 item was then requeued and uploaded in 83 ms; final queue depth was zero. Never overwrite a damaged queue in place: retain the original, recover to a new path, validate, and use an atomic rename.

Changelog

Date	Version	Change
2026-08-09	2.2.0	Authenticated relevance-gated builder context and cache usage accounting (MC #105573)
2026-07-10	2.1.0	JWT Bearer auth wired via lightrag-auth-helper; authenticated --verify probe; non-200 → error not match; lightrag.js --context-only (MC #105189)
2026-04-16	2.0.0	LightRAG default-on (MC #8020 Task 4)
2026-03-10	1.5.0	Added `--lightrag` opt-in flag
2026-02-18	1.0.0	Initial discover.js release

Questions? Contact Chip Huyen (AgentForge lead) or check ~/system/docs/system-evolution-2026-04-16.md.

Email Agent Runbook

Runbook: LightRAG ingest LaunchAgent fix (MC #10286)

Runbook: LightRAG ingest LaunchAgent fix (MC #10286)

Runbook: LightRAG ingest LaunchAgent fix (MC #10286)

LightRAG Default-On in discover.js

Runbook: LightRAG Default-On in discover.js

Purpose

What Changed

Before (Opt-In)

After (Opt-Out)

How It Works

Query Flow

When to Use --no-lightrag Flag

Use Cases for Disabling LightRAG

Verification

Check LightRAG is Being Queried

Check Timeout Behavior

Verify Flag Works

Performance Characteristics

Latency Budget

Token Cost Impact

Troubleshooting

Issue 1: "LightRAG timeout" in Every Query

Issue 2: LightRAG Returns Irrelevant Results

Issue 3: discover.js Slower After Upgrade

Issue 4: Graph Results Don't Match Filesystem

Integration with Other Tools

In Agent Chains

In Subagents

In Scripts

Monitoring & Observability

Daily Health Check

Metrics to Track

Rollback Procedure

Future Enhancements

Planned (MC #8050)

Under Consideration

Related Documentation

JWT Bearer Auth (MC #105189, 2026-07-10)

Builder Context and Token Accounting (MC #105573, 2026-08-09)

Durable writeback queue recovery

Changelog