Skip to main content

Framework Comparison Matrix

## §3 — Feature Matrix (D1 / AC#1)

Key: S=small (<8h), M=medium (<80h), L=large (>80h); EVIDENCE lines follow each cell.

| Framework | storage_backend | embedding_model | extraction_method | recall_at_10 | latency_p50_ms | multi_user_isolation | oauth_compatible | self_hosted_capable | license | last_release_date | maintainer_health | notes |
|-----------|----------------|-----------------|-------------------|--------------|----------------|----------------------|-----------------|---------------------|---------|---------------------|-------------------|-------|
| Mem0 self-hosted | Qdrant (local port 6333) | bge-m3:latest 1024-dim (Ollama) | LLM fact extraction (qwen3:8b-q8_0) then vector store | 80% (Phase 1 baseline) | ~200ms (full scan at 865 pts, no HNSW index) | Partial — user_id='john' hardcoded; Qdrant payload_schema supports user_id keyword | YES — no API key; all local Ollama | YES (deployed) | Apache-2.0 (mem0ai PyPI) | 2026-05-04 (v2.0.1) | Active (mem0ai org, VC-backed OSS) | integration_effort=S (already deployed, 865 facts, discover.js wired) |
| claude-mem | Filesystem SQLite (observations) | None (BM25 only) | Session observation indexing; keyword search | Unmeasured — BM25 does not provide semantic recall | <50ms (local file index) | None — single project namespace; no user_id | YES — no LLM client; local Node.js daemon | YES (v12.5.0 installed) | AGPL-3.0 | 2026-05-04 (v12.5.0 active) | Active (thedotmack, 12.x release series) | L3a BM25 layer only; cannot replace semantic Mem0 |
| mem-search | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | No installable package exists; YouTube video uses 'mem search' as category label |
| Memipalace | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | NOT VIABLE | GitHub API: 0 repos found; YouTube says 'me palace' for L4 verbatim recall — mnemonic concept not software |
| LightRAG-resurrect | Neo4J (graph) + NanoVectorDB (vector) + JsonKV | bge-m3:latest (Ollama via CF tunnel) | Graph entity extraction + relationship traversal | Unmeasured — 5,596 processed docs; 121,003 pending | N/A (asyncio starvation risk; /health 15-30s hang during freeze) | None — no user_id partitioning; single-tenant | YES — Ollama via CF tunnel, no Anthropic API | YES (vm-alai-lightrag, existing) | MIT | 2026-04-22 (v1.3.4 / api 0154) | Active (HKUDS/LightRAG GitHub) | BLOCKED: MC #99093 open; asyncio patch pending |

EVIDENCE (Mem0 storage_backend): curl http://localhost:6333/collections/mem0_john 2026-05-04T21:07Z → points_count:865 vector_size:1024
EVIDENCE (Mem0 embedding_model): /Users/makinja/system/mem0/config.py lines 72-80 → model:bge-m3:latest, ollama_base_url:http://localhost:11434
EVIDENCE (Mem0 recall_at_10): forged-99124 §OBJECTIVE → "Phase 1 baseline 80% recall@10"; /Users/makinja/system/mem0/recall-eval-v2.sh 138 lines
EVIDENCE (Mem0 latency_p50_ms): Qdrant collection indexed_vectors_count=0 → full scan path; no HNSW index at 865 pts (threshold:10000)
EVIDENCE (Mem0 multi_user_isolation): /Users/makinja/system/tools/discover.js line 677 → user_id:'john' hardcoded
EVIDENCE (Mem0 oauth_compatible): /Users/makinja/system/mem0/config.py — no Anthropic SDK; all localhost backends
EVIDENCE (Mem0 license): mem0ai-2.0.1.dist-info in venv site-packages; Apache-2.0 per mem0ai PyPI
EVIDENCE (claude-mem storage_backend): /opt/homebrew/bin/claude-mem search 'test' → 67 results (54 obs, 3 sessions, 10 prompts) — filesystem index
EVIDENCE (claude-mem embedding_model): package.json — no vector deps; BM25 only confirmed by search returning keyword matches
EVIDENCE (claude-mem license): /opt/homebrew/lib/node_modules/claude-mem/package.json → license:AGPL-3.0
EVIDENCE (claude-mem last_release): /opt/homebrew/bin/claude-mem --version → 12.5.0
EVIDENCE (claude-mem oauth_compatible): package.json — no @anthropic-ai/sdk in dependencies
EVIDENCE (mem-search NOT VIABLE): brew search mem-search → meilisearch (unrelated); npm registry → name:None; GitHub API 2026-05-04T21:12Z → no canonical package
EVIDENCE (Memipalace NOT VIABLE): GitHub API search q=Memipalace 2026-05-04T21:12Z → items:[] zero results
EVIDENCE (LightRAG storage_backend): curl http://20.240.61.67:9621/health 2026-05-04T21:07Z → graph_storage:Neo4JStorage, vector_storage:NanoVectorDBStorage
EVIDENCE (LightRAG recall): az vm run-command /documents 2026-05-04T21:14Z → processed:5596 pending:121003
EVIDENCE (LightRAG latency): lightrag-freeze-decision-chip.md §1 → /health hangs 15-30s during event-loop starvation; pipeline_busy:false at time of probe

---

No comments to display

Back to top