AI Support Agent — bilko.cloud (HR)

AI Support Agent — bilko.cloud (HR)

Status: Phase 1 operational (internal copilot); Phase 2 customer-facing widget HELD for Securion review
MC Task: #104429
Owner: John → CodeCraft (Petter Graff lead)
Built: 2026-06-28

1. Purpose & Scope

The Bilko Cloud AI Support Agent is a dedicated support system for the bilko.cloud (Croatia) jurisdiction of the Bilko SaaS platform. It handles both technical problems (login/Entra SSO, e-invoice/OIB failures, RLS, deploy errors) and user/accounting questions (PDV rates, HR-FISK/eRačun, invoices, bookkeeping, general ledger) — grounded in the Bilko knowledge base, validated by HR accounting domain experts, and gated by anti-hallucination verification.

Phased deployment:

Jurisdiction: Croatia only. The agent reasons under HR tax/accounting rules and answers in Croatian (hr-HR).

Cost target: $0 per question (runs on local tier-2 models: Qwen 2.5 Coder 32B / MLX).

2. The Team — 5 Agents

RoleAgentRegistryJob
Lead architectPetter Graff~/.claude/agents/petter-graff.mdOwns architecture, gates phases, integration design
Research (Phase 0)Datavera / Exploreephemeral dispatchesHR support taxonomy, competitor patterns, KB extraction (completed)
Support personabilko-support~/.claude/agents/bilko-support.mdOrchestrates answer pipeline, enforces anti-hallucination, escalation
HR accounting expert #1bilko-racunovodstvo-hr~/.claude/agents/bilko-racunovodstvo-hr.mdDvojno knjigovodstvo, RRiF chart of accounts, GL, expenses, financial statements (RDG/bilanca); KB author + validation gate
HR tax/fiscalization expert #2bilko-porez-fiskalizacija-hr~/.claude/agents/bilko-porez-fiskalizacija-hr.mdPDV (25/13/5/0%), HR-FISK 2.0/eRačun (UBL 2.1/EN 16931), OIB validation, FINA, filing deadlines; KB author + validation gate

All three personas (bilko-support + 2 experts) registered in ~/system/agents/specialist-mapping.json:

3. Architecture — Answer Pipeline

Orchestrator: ~/system/tools/bilko-support-answer.js (305 lines, CLI + module.exports).

Pipeline (8 steps, deterministic):

question (+ contextBundle: errorCode, appRoute, country=HR, planTier, orgId…)
 ↓
 [1] classify + route         tier-router.js (local $0 model)
 ↓
 [2] cache check              rag-router.js (flywheel.db hit? early return)
      ⮑ if cached + confidence >= 0.70 (or not acct/tax/e-invoice) → DONE
      ⮑ if cached BUT classification ∈ {accounting, tax, e-invoice}
          AND confidence < 0.70 → escalate:true, return
 ↓
 [3] retrieve context         KB JSONL search (40 Q&A pairs)
                              + LightRAG hybrid (knowledge.db)
                              + HiveMind semantic (hivemind.db, fire-and-forget)
 ↓
 [4] draft answer (Croatian)  tier-2 local model (Qwen/MLX)
 ↓
 [5] anti-hallucination gate  mini-verifier.js
      ⮑ HALLUCINATION (cited file missing/empty) → escalate
      ⮑ DRIFT (file present but ambiguous/language mismatch) → reduce confidence
      ⮑ CONFIRMED (grounded) → boost confidence
      ⮑ SKIP (verifier offline) → neutral
 ↓
 [6] accounting validation    if classification ∈ {accounting, tax, e-invoice}:
      ⮑ re-verify vs domain-specific KB (hr-tax-fiscalization, hr-accounting sources)
      ⮑ if no domain KB → escalate
      ⮑ if HALLUCINATION or CONFIRMED < 0.70 → escalate
 ↓
 [7] confidence floor gate    belowAccountingFloor(cls, confidence)
      ⮑ if acct/tax/e-invoice AND confidence < 0.70 → escalate:true
      (same helper used on BOTH cache path [L139] and pipeline path [L258])
 ↓
 [8] cost log + return        {answer, confidence, evidence_paths,
                               classification, severity, escalate, source}

Output schema:

{
  "answer": "",
  "confidence": 0.0–1.0,
  "evidence_paths": [""],
  "classification": "technical | accounting | tax | e-invoice | billing | onboarding | other",
  "severity": "P1 | P2 | P3 | P4",
  "escalate": boolean,
  "source": "cached | local | escalated"
}

4. Escalation Policy

Hard rule (enforced in code): ACCOUNTING_CONF_FLOOR = 0.70

Any answer with classification ∈ {accounting, tax, e-invoice} AND confidence < 0.70 MUST escalate to a human specialist (or to bilko-racunovodstvo-hr / bilko-porez-fiskalizacija-hr personas in async workflow).

Escalation triggers:

Single source of truth: The belowAccountingFloor(cls, confidence) function (line 62) is called on BOTH the cache early-return path (line 139) and the post-step-6 gate (line 258) — guarantees no drift in threshold logic.

5. Knowledge Base — 40 Source-Cited Q&A Pairs

Storage (current): JSONL files read directly by orchestrator; ingestion into lightrag/knowledge.db is a pending enhancement.

Sources cited:

Each entry has: {q, a, source, confidence: "HIGH" | "MEDIUM", domain: "hr-tax-fiscalization" | "hr-accounting"}.

Confidence levels:

6. Integration with Bilko Backend

Existing infrastructure (reused, not built for this):

Phase 1 workflow:

  1. Staff create/triage ticket via ~/system/tools/support-ticket.js (SLA P1–P4)
  2. Call node ~/system/tools/bilko-support-answer.js "" ''
  3. Agent returns JSON with answer + evidence + classification + escalate flag
  4. Human reviews, approves, sends to customer (via email / admin panel)

Phase 2 workflow (planned, HELD):

  1. Customer opens in-app chat widget on bilko.cloud
  2. Widget calls POST /support/tickets with context_bundle
  3. Backend triggers bilko-support-answer.js (via internal job queue or sync call)
  4. Agent output written to triageJson field: {suggestedAnswer, classification, severity, escalate, confidence, evidencePaths}
  5. Render in UI: if escalate:false AND confidence >= 0.70 → show AI answer with citation; else → "Eskaliran na podršku, očekujte odgovor u "
  6. Guardrails: PII allowlist (already enforced), prompt-injection guard (add Securion layer), scope validation (no cross-org queries)

Security note: Phase 2 requires Securion/Parisa Tabriz security review before live deployment (MC follow-up task).

7. Product Bugs Found by Experts (During KB Build)

The 2 HR accounting experts found 8 Bilko product/documentation defects while authoring the knowledge base:

MCSeverityIssue
#104441P3Bilko MVP GL: only 1 D/P per transaction → PDV not split to konto 450 Obveze za PDV (tracking via reports only, not GL)
#104446P3OIB validation: CROATIA-ERACUN.md missing ISO 7064 MOD 11,10 detail (in code, not doc)
#104447P2CoA setup friction: not available in wizard, only post-onboarding in Settings (USER-ONBOARDING.md gap)
#104442P3Storno logic: cannot cancel paid invoices (needs credit note, not auto-generated in MVP)
#104448P4BUSINESS-LOGIC.md: HR PDV table lists only 25%/13%, omits 5%/0% (available via free input, just not in doc table)
3 additional minor doc inconsistencies logged

These are now tracked separately and do not block the support agent (KB answers work around them with "Bilko MVP limitation" disclaimers where needed).

8. Runbook — How to Invoke

CLI:

node ~/system/tools/bilko-support-answer.js "<question>" '<contextBundleJSON>'

Example:

node ~/system/tools/bilko-support-answer.js \
  "Koja je stopa PDV-a na knjige u Hrvatskoj?" \
  '{"country":"HR","planTier":"PRO","locale":"hr-HR"}'

Output (JSON to stdout):

{
  "answer": "Stopa PDV-a od 5% primjenjuje se na knjige...",
  "confidence": 0.78,
  "evidence_paths": ["/Users/makinja/.../CROATIA-ERACUN.md"],
  "classification": "tax",
  "severity": "P3",
  "escalate": false,
  "source": "local"
}

Module API:

const { answer } = require('~/system/tools/bilko-support-answer');
const result = await answer(question, contextBundle);

Cost tracking (automatic): Each call logs to ~/system/tools/cost-tracker.js with {source: 'bilko-support-agent', backend: 'ollama', model: '...', cost_usd: 0, duration_ms, metadata}.

9. Open Follow-Ups

MCTaskOwner
#104432Proveo UAT — end-to-end verification: empty-KB degradation, offline-model fallback, severity classification coverage, schema validation, real ticket replayProveo / Angie Jones
#104430Phase 1 deployment — staff onboarding, support-ticket.js workflow integration, SLA monitoringFlowForge / Kelsey Hightower
#104431Phase 2 customer-facing widget — frontend component, POST hook, guardrails (HELD pending Securion)Vizu / Brad Frost + Securion / Parisa Tabriz
(backlog)KB ingestion into lightrag/knowledge.db (enhancement: replace JSONL direct-read with Neo4j-backed hybrid retrieval)AgentForge / Chip Huyen
(backlog)Securion review: confidence thresholds, cache-of-escalated-answers audit, PII leak surface, prompt-injection testsSecurion / Parisa Tabriz

10. Verification Evidence

MC #104429 verdict: PASS (2026-06-28, john verified)
Method: Company-Mesh (CodeCraft built, John verified via real CLI runs)
Evidence: ~/system/evidence/104429/verdict.json

Verified behaviors:

2 under-escalation defects caught & fixed: Sub-0.70 tax answers on both pipeline AND cache paths were initially missing the confidence floor check; John's independent runs caught these, and CodeCraft fixed them via the shared belowAccountingFloor() helper.

This page is the deliverable for MC #104433 (ZAKON PLAN mandatory documentation task). Last updated: 2026-06-28.

Revision #1
Created 2026-06-29 06:44:37 UTC by John
Updated 2026-06-29 06:44:37 UTC by John