Skip to main content

Infrastructure Overview

Runbook: Local Infrastructure

Platform: Mac Studio M3 Ultra, 96GB RAM, macOS Services: Docker containers, LaunchAgents, Cloudflare tunnels

Docker Services

Status Check

docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'

Services

Container Image Port Health
mattermost mattermost/mattermost-enterprise 8065 healthcheck
mattermost-db postgres:13 5432 (internal) —
planka ghcr.io/plankanban/planka 3100→1337 healthcheck
planka-db postgres:15-alpine 5433 (internal) healthcheck
documenso documenso/documenso 3003 —
documenso-db postgres 5434 (internal) healthcheck
bookstack lscr.io/linuxserver/bookstack 6875→80 —
bookstack_db lscr.io/linuxserver/mariadb 3306 (internal) —

Restart a container

docker restart <container_name>
# Example: docker restart mattermost

Restart all

# Mattermost stack
cd ~/system/services/mattermost && docker compose down && docker compose up -d

# Planka stack
cd ~/system/services/planka && docker compose down && docker compose up -d

# Documenso
cd ~/system/services/documenso && docker compose down && docker compose up -d

# BookStack
cd ~/system/services/bookstack && docker compose down && docker compose up -d

View logs

docker logs <container_name> --tail 50
docker logs <container_name> -f  # follow

Disk cleanup (if disk >90%)

docker system prune -f            # Remove unused images, containers, networks
docker volume prune -f             # Remove unused volumes (CAREFUL: data loss)

Cloudflare Tunnels

Config

cat ~/.cloudflared/config.yml

Routes — local Mac tunnel (3315a609-7934-45c5-ad0c-56d86d16374d)

Verified live 2026-07-28 by reading ~/.cloudflared/config.yml directly (not from memory).

Hostname Target Service
sign.basicconsulting.nolocalhost:3003Documenso — only basicconsulting.no ingress rule exists in this tunnel's config; sign.alai.no still resolves live (HTTP 302, verified 2026-07-28) but has no matching ingress line here — routed some other way (separate DNS/Access rule not yet located). Needs FlowForge follow-up before assuming sign.alai.no is safe to rely on for Documenso incident response.
mm.basicconsulting.no DECOMMISSIONED 2026-05-18http://192.168.68.61:8065 Mattermost — stale entry: service was decommissioned 2026-05-18 (retired)see mattermost.md) but this ingress line was never removed from config.yml. Dead weight, not currently routable to anything since the LAN host is retired; safe to delete.
boards.alai.no not present as its own ingress linePlanka — service-registry.md claims boards.alai.no → localhost:3100 via this tunnel, but no boards.* hostname appears in ~/.cloudflared/config.yml as read 2026-07-28. Either Planka is routed via a different tunnel/mechanism, or the registry entry is stale. Flagged for FlowForge verification, not asserted as fact either way.
ollama.alai.no / ollama.basicconsulting.nolocalhost:11434 / http://10.0.0.2:11435Both hostnames present — this one IS a clean dual-domain migration example (see inline comments in config.yml, 2026-06-30).
lumiscare-.alai.no / lumiscare-.basicconsulting.nolocalhost 8090/4001/4002/4003/4567Both hostnames present for every lumiscare subdomain — another clean dual-domain pair.
lobby.basicconsulting.no, lobby-api.basicconsulting.no, api.basicconsulting.no, drop-api.basicconsulting.no, mc.basicconsulting.no, auth.basicconsulting.no, track.basicconsulting.no, ssh.basicconsulting.no, vnc.basicconsulting.no, kenan-hot.basicconsulting.no, bilko-demo.basicconsulting.no, bilko-demo-api.basicconsulting.novarious localhost portsbasicconsulting.no-only — no alai.no equivalent ingress line exists for any of these in this tunnel's config as of 2026-07-28. bilko-demo.basicconsulting.no is additionally known-dead (orphaned GCP Cloud Run origin, MC #10440). The rest are unverified live/dead here — this is an inventory of what the migration (MC #9392, paused) has NOT yet touched, not a claim that they're broken.

Azure VM tunnel — BookStack, Vaultwarden, Grafana (separate tunnel, not in the file above)

BookStack, Vaultwarden, Documenso-adjacent docs/vault/grafana hosts live on the Azure VM (4.223.110.181), not this Mac. Live-curl comparison run 2026-07-28 (curl -o /dev/null -w '%{http_code}'):

Servicealai.no hostStatusbasicconsulting.no hostStatus
BookStackdocs.alai.no200docs.basicconsulting.no200
Vaultwardenvault.alai.no200vault.basicconsulting.no200
Grafanagrafana.alai.no302grafana.basicconsulting.no302
Planka boards.alai.no200boards.basicconsulting.no200
Documenso sign.alai.no localhost:3003302 Documensosign.basicconsulting.no302

Reality check on MC #9392 ("Migracija basicconsulting.no → alai.no za sve tunnel hostnames + Azure VM servisi"): all 5 services still answer identically on BOTH domains. The migration is not a completed cutover — both DNS names are live in parallel, matching MC #9392's actual status (paused, priority L, owner pi-orchestrator, as of 2026-07-28). Treat *.alai.no as the canonical/primary name to use going forward (per ~/system/CLAUDE.md and service-registry.md), but do NOT assume *.basicconsulting.no is dead for these 5 services during an incident — it currently is not. See also the BookStack page "ALAI Domain Migration — basicconsulting.no → alai.no" (id 2666, tagged staleness: needs-review) and ~/system/docs/published/hosting-migration-log-basicconsulting-bilko-io-2026-07-28.md for the corporate-site (non-tunnel) side of this same basicconsulting.no/alai.no duality — do not conflate the two: that log covers the Cloudflare-Pages-hosted apex sites, this section covers the Azure-VM Cloudflare-Tunnel-hosted internal tools.

Status

cloudflared tunnel info mattermost3315a609-7934-45c5-ad0c-56d86d16374d

Restart tunnel

# Tunnel runs as LaunchAgent
launchctl unload ~/Library/LaunchAgents/com.cloudflare.tunnel.plist
launchctl load ~/Library/LaunchAgents/com.cloudflare.tunnel.plist

LaunchAgents (Daemons)

List all custom daemons

launchctl list | grep -E "com\.(john|edita|cloudflare)"

Expected daemons

Daemon Interval Location
com.john.ops-agent 5 min ~/Library/LaunchAgents/
com.edita.autowork 30 min ~/Library/LaunchAgents/
com.john.mc-dashboard always ~/Library/LaunchAgents/
com.john.mc-session-worker on events ~/Library/LaunchAgents/

Load/unload

launchctl load ~/Library/LaunchAgents/<plist-name>.plist
launchctl unload ~/Library/LaunchAgents/<plist-name>.plist

Ollama (Local AI)

Status

curl -s http://localhost:11434/api/tags | python3 -c "import sys,json; [print(m['name']) for m in json.load(sys.stdin)['models']]"

Models

Model Size Use
llama3.1:8b 5GB Fast classification (ops-agent)
qwen2.5-coder:32b 19GB Code generation, contextual responses
llama3.1:70b 40GB Research, writing

Restart Ollama

# Ollama runs as macOS app
killall ollama 2>/dev/null
open -a Ollama

Mission Control Dashboard

Status

curl -s http://localhost:3030 | head -1

Restart

launchctl unload ~/Library/LaunchAgents/com.john.mc-dashboard.plist
launchctl load ~/Library/LaunchAgents/com.john.mc-dashboard.plist

Full Health Check

# Human-readable
node ~/system/tools/health-check.js

# JSON (programmatic)
node ~/system/tools/health-check.js --json

# Quick (HTTP only)
node ~/system/tools/health-check.js --quick

After System Reboot

All LaunchAgents with RunAtLoad: true start automatically. Verify:

# 1. Check Docker is running
docker ps

# 2. Check all daemons
launchctl list | grep -E "com\.(john|edita|cloudflare)"

# 3. Run health check
node ~/system/tools/health-check.js

# 4. If anything missing, load it
launchctl load ~/Library/LaunchAgents/<missing>.plist

Incident Procedure — Azure VM tunnel service unreachable (BookStack/Vault/Grafana/Planka/Documenso)

Because the basicconsulting.no → alai.no tunnel-hostname migration (MC #9392) is paused, not complete, both domain variants currently resolve for these 5 services. Do not skip a step just because one domain "looks fine."

  1. Confirm which hostname actually failed. Check both variants before escalating: 
    for h in docs vault grafana boards sign; do
  curl -s -o /dev/null -w "$h.alai.no=%{http_code}  " -m 5 "https://$h.alai.no"
  curl -s -o /dev/null -w "$h.basicconsulting.no=%{http_code}\n" -m 5 "https://$h.basicconsulting.no"
done
    If only ONE domain variant fails, this is a DNS/Cloudflare-zone issue, not the origin (Azure VM) — the origin serves both from the same backend.
  2. If BOTH variants fail: origin is down. SSH to the Azure VM (ssh -i ~/.ssh/azure_alai [email protected]) and check the relevant container per ~/system/context/docs/runbooks/infrastructure.md Docker Services section / service-specific runbook.
  3. Do not "fix" a basicconsulting.no failure by declaring it decommissioned — unlike mm.basicconsulting.no (actually retired), these 5 hosts are still live and serving real traffic; treat a basicconsulting.no-only failure as a real incident, not expected drift.
  4. Report which domain(s) failed in the incident record — don't collapse alai.no/basicconsulting.no into one line item, since MC #9392 tracks them as distinct DNS surfaces still requiring cutover.

Changelog

2026-07-28 — basicconsulting.no → alai.no tunnel migration audit (MC #106446, source #9392)

Live-verified (curl, both this Mac's ~/.cloudflared/config.yml and the Azure VM's 5 tunnel-fronted services) that the MC #9392 migration is incomplete: BookStack/Vault/Grafana/Planka/Documenso all still answer on both *.alai.no and *.basicconsulting.no. Updated the Cloudflare Tunnels section above with the verified route inventory (including a stale mm.basicconsulting.no ingress line that should be deleted, and an unresolved discrepancy where sign.alai.no/boards.alai.no have no matching ingress rule in this tunnel's config yet still resolve). Added the incident procedure above for handling single-domain-only failures during the migration window. No production changes made — this is a documentation-only pass per MC #106446 scope; the actual tunnel-config cutover remains MC #9392 (paused).

2026-07-10 — LIGHTRAG_URL stale host fix (MC #105191)

com.john.pi-orchestrator.plist had LIGHTRAG_URL hardcoded to the dead host https://lightrag.basicconsulting.no/query (times out), breaking lightrag.js calls in sessions spawned under pi-orchestrator. Fixed to the live host https://lightrag.alai.no/query (verified reachable, HTTP 405 on GET / — expected for a POST-only API) in both live copies:

  • ~/Library/LaunchAgents/com.john.pi-orchestrator.plist
  • ~/system/config/launchagents/com.john.pi-orchestrator.plist

Also fixed the matching hardcoded fallback default in ~/system/kernel/pi-orchestrator.js:2101 (process.env.LIGHTRAG_URL || '...'), which pointed to the same dead host.

Action required: the already-running daemon (verify current pid via ps aux | grep pi-orchestrator) was started before this fix and still holds the stale value in its process environment (confirmed via ps eww -p <pid> | grep LIGHTRAG_URL). It will only pick up the new value on next restart (launchctl unload/load ~/Library/LaunchAgents/com.john.pi-orchestrator.plist or a reboot). Restart was NOT performed as part of this fix — CEO should confirm before bouncing a live daemon.

Created: 2026-02-10 Last Updated: 2026-07-1028

Back to top