# Bilko Prod Topology — app.bilko.cloud Cutover (reuse-demo-as-prod)

# Bilko Prod Topology — app.bilko.cloud Cutover (reuse-demo-as-prod)

**MC:** #103300 | **Completed:** 2026-06-10 | **Proveo verdict:** CUTOVER PASS | **Evidence root:** `/tmp/evidence-103300/`

> **CRITICAL REFERENCE — do not make assumptions about which DB is prod or what data is on it. Read this page first.**

---

## 1. Topology Overview

CEO decision 2026-06-09: reuse the existing "demo" Cloud Run services and Cloud SQL instance as production. No new infrastructure was provisioned. The prior bilko-demo-\* services are now the live prod environment.

<table id="bkmrk-domaincloud-run-serv"><thead><tr><th>Domain</th><th>Cloud Run Service</th><th>Status</th><th>Notes</th></tr></thead><tbody><tr><td>`app.bilko.cloud`</td><td>`bilko-web-demo`</td><td>Ready/True, cert Ready/True</td><td>Primary prod frontend</td></tr><tr><td>`app-api.bilko.cloud`</td><td>`bilko-api-demo`</td><td>Ready/True, cert Ready/True</td><td>Primary prod API</td></tr><tr><td>`api.bilko.cloud`</td><td>—</td><td>HTTP 000 (CF Worker SNI issue)</td><td>Pre-existing issue, separate follow-up</td></tr></tbody></table>

- **GCP Project:** `tribal-sign-487920-k0`
- **Region:** `europe-north1`
- **Prod DB:** Cloud SQL instance `bilko-demo-db` (PostgreSQL 16, RUNNABLE) — this is the REUSED demo instance, now serving production
- **Live health probes (C1 evidence):** `https://app.bilko.cloud/` → HTTP 200 | `https://app-api.bilko.cloud/api/v1/health` → HTTP 200 `{"status":"ok"}`

---

## 2. Hostname Recognition — Dual-Mode (C2, PR #334)

Three files on `main` now recognize BOTH `app.bilko.cloud` (prod) AND `bilko-demo.alai.no` (demo/legacy) simultaneously.

<table id="bkmrk-fileprod-hostnamedem"><thead><tr><th>File</th><th>Prod hostname</th><th>Demo hostname</th><th>Verification</th></tr></thead><tbody><tr><td>`apps/web/lib/api-base.ts`</td><td>`app.bilko.cloud` → `https://app-api.bilko.cloud/api/v1`</td><td>`bilko-demo.alai.no` preserved (dual-mode)</td><td>Vitest 4/4 PASS</td></tr><tr><td>`apps/web/middleware.ts`</td><td>`app.bilko.cloud` + `.bilko.cloud`</td><td>`bilko-demo.alai.no` preserved</td><td>Machine-verified on main</td></tr><tr><td>`apps/web/i18n/request.ts`</td><td>`app.bilko.cloud` + `.bilko.cloud`</td><td>`bilko-demo.alai.no` preserved</td><td>Machine-verified on main</td></tr></tbody></table>

**PR #334** (`feat/prod-hostname-c2-103300`, HEAD `c70dab73`) adds the test file `apps/web/test/api-base-hostname-103300.test.ts`. The source changes to the three files above were already on `main` from prior commits. Proveo confirmed no merge conflict; safe to merge.

**Landing CTAs:** `landing-hr` CTAs already point to `app.bilko.cloud`. `landing-io` and `landing-ba` use `#demo-form`/`mailto:` anchors — no repoint required.

---

## 3. Backend Features Live on Prod (MC #103323)

Deployed to `bilko-api-demo` (now prod):

- **Sentry error tracking** — wired to prod service
- **Flyway V72** — `audit_log.request_id` column (renamed from V71 to avoid collision with `V71__seed_e2e_test_user`)
- **Flyway V73** — `support_tickets` table (renamed from V72 for same reason)
- **SupportTicketRoutes live:**
    - `POST /api/v1/support/tickets` — auth-gated (returns 401 if no token)
    - `GET /api/v1/admin/support/tickets` — auth-gated (returns 401 if no token)
    - `PATCH /api/v1/admin/support/tickets/{id}` — auth-gated (returns 401 if no token)

**Route verification (Proveo C6-lite):** Both `POST /api/v1/support/tickets` and `GET /api/v1/admin/support/tickets` return 401 (not 404) — routes are live and correctly auth-gated.

---

## 4. Prod DB State After Cleanup (C3)

**Instance:** `bilko-demo-db` (Cloud SQL, PostgreSQL 16, project `tribal-sign-487920-k0`, region `europe-north1`)  
**Backup anchor taken before cleanup:** ID `1781094321949`, status SUCCESSFUL, 2026-06-10T12:25:21Z  
**Cleanup executed:** 2026-06-10T12:50Z by FlowForge, authorized by CEO

### Organizations on prod DB — post-cleanup (1 row only)

<table id="bkmrk-org-idnameclassifica"><thead><tr><th>Org ID</th><th>Name</th><th>Classification</th><th>Status</th></tr></thead><tbody><tr><td>`d9e364ca-e7fc-48ed-a836-821bcaf79c99`</td><td>Bilko E2E Test Organisation</td><td>SEED/TEST — V71 migration seeded 2026-06-10</td><td>KEPT — CI/Playwright infrastructure (see caveat (a) below)</td></tr></tbody></table>

Seed orgs from V13/V14/V29 migrations (sentinel UUIDs) were already absent before this cleanup — removed in a prior operation. Two real trial orgs (ALAI Holding AS `53349d6a` and "unknown's Organization" `4e96b6ff` / alembasic@gmail.com) were deleted per CEO authorization: *"all data on bilko-demo-db is test data, no real customers."*

### Rows deleted in cleanup transaction

<table id="bkmrk-tablerows-deletedmet"><thead><tr><th>Table</th><th>Rows deleted</th><th>Method</th></tr></thead><tbody><tr><td>organizations</td><td>2</td><td>Direct DELETE</td></tr><tr><td>users</td><td>3</td><td>CASCADE</td></tr><tr><td>entra\_external\_identities</td><td>3</td><td>Explicit DELETE (bilko\_admin role required)</td></tr><tr><td>refresh\_tokens</td><td>26</td><td>Explicit DELETE</td></tr><tr><td>chat\_conversations</td><td>1</td><td>Explicit DELETE</td></tr><tr><td>logged\_actions</td><td>1</td><td>Explicit DELETE (RLS had hidden this row from initial NO ACTION FK pre-check)</td></tr><tr><td>invoices</td><td>1</td><td>CASCADE</td></tr><tr><td>expenses</td><td>1</td><td>CASCADE</td></tr><tr><td>contacts</td><td>1</td><td>CASCADE</td></tr><tr><td>expense\_documents</td><td>1</td><td>CASCADE</td></tr><tr><td>**Total**</td><td>**~40**</td><td></td></tr></tbody></table>

**Pre-commit guards 1-6: ALL PASSED.** Post-commit SELECT confirmed `organizations = 1 row` (E2E org only).

### System/catalog tables — do not touch

- `role_permissions` — 158 rows (system catalog)
- `permissions` — 54 rows (system catalog)
- `account_types` — 5 rows (system catalog)
- `flyway_schema_history` — 73 rows (migration history)
- `schema_version` — 10 rows (migration registry)
- `adapter_config` — 3 rows (system config)

### Restore anchor

```
gcloud sql backups restore 1781094321949 --restore-instance=bilko-demo-db
```

Backup taken 2026-06-10T12:25Z, 10 minutes before cleanup. Contains full pre-cleanup state including both deleted orgs.

---

## 5. Known Caveats and Open Follow-ups

<table id="bkmrk-refissuemcseverity-%28"><thead><tr><th>Ref</th><th>Issue</th><th>MC</th><th>Severity</th></tr></thead><tbody><tr><td>(a)</td><td>E2E test org `d9e364ca` (Bilko E2E Test Organisation) still on prod DB. Must be moved to staging before first external CIAM customer is onboarded.</td><td>\#103374</td><td>M — no external customers yet</td></tr><tr><td>(b)</td><td>Stage → prod promotion gate (C4) not yet formalized. Current `gcp-deploy.yml` deploys on every main push with no UAT approval step. TODO comment exists in the workflow file.</td><td>\#103375</td><td>M — mitigated by semver tag requirement for prod deploy</td></tr><tr><td>(c)</td><td>`api.bilko.cloud` returns HTTP 000 — pre-existing Cloudflare Worker Host/SNI rewrite issue, unrelated to this cutover. Direct Cloud Run URL and `bilko-demo-api.alai.no` both return 200. App is healthy.</td><td>open</td><td>L — brand URL, not a functional path</td></tr><tr><td>(d)</td><td>**SECURITY REVIEW ITEM:** `POST /api/v1/auth/test/session` test-auth endpoint is present on the prod API surface. Flagged as Securion F7 (MC #103371). Must be removed or hard-gated before any external users are onboarded.</td><td>\#103371</td><td>H — Securion review required</td></tr></tbody></table>

---

## 6. C5 — AI Integration

`GROQ_API_KEY` bound to `bilko-api-demo` (rev00165). AI route is live. Evidence: `/tmp/alai/7d24e9bf/evidence-bilko-ai-fix/verification.json`. Full functional E2E (Entra login path) is pending.

---

## 7. Verification Summary (Proveo)

**Agent:** Angie Jones (Proveo) | **Timestamp:** 2026-06-10T15:03Z | **Verdict:** CUTOVER PASS

<table id="bkmrk-checkresultdetail-ap"><thead><tr><th>Check</th><th>Result</th><th>Detail</th></tr></thead><tbody><tr><td>app.bilko.cloud HTTP 200</td><td>PASS</td><td>curl confirmed</td></tr><tr><td>bilko-demo-api.alai.no /api/v1/health HTTP 200</td><td>PASS</td><td>curl confirmed</td></tr><tr><td>PR #334 — hostname recognition, all 3 files</td><td>PASS</td><td>dual-mode confirmed on main</td></tr><tr><td>Vitest api-base-hostname-103300 (4 tests)</td><td>PASS</td><td>4/4, 831ms</td></tr><tr><td>tsc --noEmit (apps/web)</td><td>PASS</td><td>0 errors</td></tr><tr><td>MC #103323 routes auth-gated (401 not 404)</td><td>PASS</td><td>support\_tickets POST + GET routes live</td></tr><tr><td>DB clean state</td><td>PASS (evidence-reviewed)</td><td>SQL SELECT output in cleanup.md is authoritative; Cloud SQL proxy not available in Proveo context</td></tr><tr><td>PR #334 merge conflict check</td><td>PASS — no conflict</td><td>5 CI-only commits on main since branch point; zero file overlap with PR files</td></tr></tbody></table>

**Full Proveo report:** `/tmp/alai/p2p-pairing-evidence/proveo-103300-c2c6-verdict.md`

---

## 8. Cutover Status Table (C1–C7)

<table id="bkmrk-itemstatusownernotes"><thead><tr><th>Item</th><th>Status</th><th>Owner</th><th>Notes</th></tr></thead><tbody><tr><td>C1 — domain mapping (app.bilko.cloud + app-api.bilko.cloud → bilko-web-demo / bilko-api-demo)</td><td>DONE</td><td>FlowForge</td><td>Both Ready/True, cert Ready/True, HTTP 200</td></tr><tr><td>C2 — hostname recognition (middleware / api-base / i18n)</td><td>DONE</td><td>CodeCraft</td><td>Dual-mode on main; PR #334 adds test coverage</td></tr><tr><td>C3 — prod DB cleaned of test/seed orgs</td><td>DONE</td><td>FlowForge/DB</td><td>1 org remains (E2E test org d9e364ca); ~40 rows deleted; backup 1781094321949</td></tr><tr><td>C4 — stage → prod promotion gate formalized</td><td>PENDING</td><td>FlowForge</td><td>MC #103375</td></tr><tr><td>C5 — AI fix (GROQ\_API\_KEY on bilko-api-demo)</td><td>DONE</td><td>—</td><td>Rev00165, route live</td></tr><tr><td>C6 — Proveo end-to-end validation</td><td>DONE (C6-lite)</td><td>Proveo</td><td>Full E2E blocked on Entra login flow; C6-lite PASS</td></tr><tr><td>C7 — Skillforge BookStack documentation</td><td>DONE</td><td>Skillforge</td><td>This page</td></tr></tbody></table>

---

## 9. Evidence Index

<table id="bkmrk-artifactpath-c1%E2%80%93c7-d"><thead><tr><th>Artifact</th><th>Path</th></tr></thead><tbody><tr><td>C1–C7 delta state</td><td>`/tmp/evidence-103300/cutover-state.md`</td></tr><tr><td>DB inventory (read-only probe, pre-cleanup)</td><td>`/tmp/evidence-103300-c3/inventory.md`</td></tr><tr><td>DB cleanup execution record + post-commit proof</td><td>`/tmp/evidence-103300-c3/cleanup.md`</td></tr><tr><td>Proveo C2 + C6-lite verdict</td><td>`/tmp/alai/p2p-pairing-evidence/proveo-103300-c2c6-verdict.md`</td></tr><tr><td>C5 AI fix verification</td><td>`/tmp/alai/7d24e9bf/evidence-bilko-ai-fix/verification.json`</td></tr></tbody></table>