Bilko Prod Topology — app.bilko.cloud Cutover (reuse-demo-as-prod)
Bilko Prod Topology — app.bilko.cloud Cutover (reuse-demo-as-prod)
MC: #103300 | Completed: 2026-06-10 | Proveo verdict: CUTOVER PASS | Evidence root: /tmp/evidence-103300/
CRITICAL REFERENCE — do not make assumptions about which DB is prod or what data is on it. Read this page first.
1. Topology Overview
CEO decision 2026-06-09: reuse the existing "demo" Cloud Run services and Cloud SQL instance as production. No new infrastructure was provisioned. The prior bilko-demo-* services are now the live prod environment.
| Domain | Cloud Run Service | Status | Notes |
|---|---|---|---|
app.bilko.cloud | bilko-web-demo | Ready/True, cert Ready/True | Primary prod frontend |
app-api.bilko.cloud | bilko-api-demo | Ready/True, cert Ready/True | Primary prod API |
api.bilko.cloud | — | HTTP 000 (CF Worker SNI issue) | Pre-existing issue, separate follow-up |
- GCP Project:
tribal-sign-487920-k0 - Region:
europe-north1 - Prod DB: Cloud SQL instance
bilko-demo-db(PostgreSQL 16, RUNNABLE) — this is the REUSED demo instance, now serving production - Live health probes (C1 evidence):
https://app.bilko.cloud/→ HTTP 200 |https://app-api.bilko.cloud/api/v1/health→ HTTP 200{"status":"ok"}
2. Hostname Recognition — Dual-Mode (C2, PR #334)
Three files on main now recognize BOTH app.bilko.cloud (prod) AND bilko-demo.alai.no (demo/legacy) simultaneously.
| File | Prod hostname | Demo hostname | Verification |
|---|---|---|---|
apps/web/lib/api-base.ts | app.bilko.cloud → https://app-api.bilko.cloud/api/v1 | bilko-demo.alai.no preserved (dual-mode) | Vitest 4/4 PASS |
apps/web/middleware.ts | app.bilko.cloud + .bilko.cloud | bilko-demo.alai.no preserved | Machine-verified on main |
apps/web/i18n/request.ts | app.bilko.cloud + .bilko.cloud | bilko-demo.alai.no preserved | Machine-verified on main |
PR #334 (feat/prod-hostname-c2-103300, HEAD c70dab73) adds the test file apps/web/test/api-base-hostname-103300.test.ts. The source changes to the three files above were already on main from prior commits. Proveo confirmed no merge conflict; safe to merge.
Landing CTAs: landing-hr CTAs already point to app.bilko.cloud. landing-io and landing-ba use #demo-form/mailto: anchors — no repoint required.
3. Backend Features Live on Prod (MC #103323)
Deployed to bilko-api-demo (now prod):
- Sentry error tracking — wired to prod service
- Flyway V72 —
audit_log.request_idcolumn (renamed from V71 to avoid collision withV71__seed_e2e_test_user) - Flyway V73 —
support_ticketstable (renamed from V72 for same reason) - SupportTicketRoutes live:
POST /api/v1/support/tickets— auth-gated (returns 401 if no token)GET /api/v1/admin/support/tickets— auth-gated (returns 401 if no token)PATCH /api/v1/admin/support/tickets/{id}— auth-gated (returns 401 if no token)
Route verification (Proveo C6-lite): Both POST /api/v1/support/tickets and GET /api/v1/admin/support/tickets return 401 (not 404) — routes are live and correctly auth-gated.
4. Prod DB State After Cleanup (C3)
Instance: bilko-demo-db (Cloud SQL, PostgreSQL 16, project tribal-sign-487920-k0, region europe-north1)
Backup anchor taken before cleanup: ID 1781094321949, status SUCCESSFUL, 2026-06-10T12:25:21Z
Cleanup executed: 2026-06-10T12:50Z by FlowForge, authorized by CEO
Organizations on prod DB — post-cleanup (1 row only)
| Org ID | Name | Classification | Status |
|---|---|---|---|
d9e364ca-e7fc-48ed-a836-821bcaf79c99 | Bilko E2E Test Organisation | SEED/TEST — V71 migration seeded 2026-06-10 | KEPT — CI/Playwright infrastructure (see caveat (a) below) |
Seed orgs from V13/V14/V29 migrations (sentinel UUIDs) were already absent before this cleanup — removed in a prior operation. Two real trial orgs (ALAI Holding AS 53349d6a and "unknown's Organization" 4e96b6ff / alembasic@gmail.com) were deleted per CEO authorization: "all data on bilko-demo-db is test data, no real customers."
Rows deleted in cleanup transaction
| Table | Rows deleted | Method |
|---|---|---|
| organizations | 2 | Direct DELETE |
| users | 3 | CASCADE |
| entra_external_identities | 3 | Explicit DELETE (bilko_admin role required) |
| refresh_tokens | 26 | Explicit DELETE |
| chat_conversations | 1 | Explicit DELETE |
| logged_actions | 1 | Explicit DELETE (RLS had hidden this row from initial NO ACTION FK pre-check) |
| invoices | 1 | CASCADE |
| expenses | 1 | CASCADE |
| contacts | 1 | CASCADE |
| expense_documents | 1 | CASCADE |
| Total | ~40 |
Pre-commit guards 1-6: ALL PASSED. Post-commit SELECT confirmed organizations = 1 row (E2E org only).
System/catalog tables — do not touch
role_permissions— 158 rows (system catalog)permissions— 54 rows (system catalog)account_types— 5 rows (system catalog)flyway_schema_history— 73 rows (migration history)schema_version— 10 rows (migration registry)adapter_config— 3 rows (system config)
Restore anchor
gcloud sql backups restore 1781094321949 --restore-instance=bilko-demo-dbBackup taken 2026-06-10T12:25Z, 10 minutes before cleanup. Contains full pre-cleanup state including both deleted orgs.
5. Known Caveats and Open Follow-ups
| Ref | Issue | MC | Severity |
|---|---|---|---|
| (a) | E2E test org d9e364ca (Bilko E2E Test Organisation) still on prod DB. Must be moved to staging before first external CIAM customer is onboarded. | #103374 | M — no external customers yet |
| (b) | Stage → prod promotion gate (C4) not yet formalized. Current gcp-deploy.yml deploys on every main push with no UAT approval step. TODO comment exists in the workflow file. | #103375 | M — mitigated by semver tag requirement for prod deploy |
| (c) | api.bilko.cloud returns HTTP 000 — pre-existing Cloudflare Worker Host/SNI rewrite issue, unrelated to this cutover. Direct Cloud Run URL and bilko-demo-api.alai.no both return 200. App is healthy. | open | L — brand URL, not a functional path |
| (d) | SECURITY REVIEW ITEM: POST /api/v1/auth/test/session test-auth endpoint is present on the prod API surface. Flagged as Securion F7 (MC #103371). Must be removed or hard-gated before any external users are onboarded. | #103371 | H — Securion review required |
6. C5 — AI Integration
GROQ_API_KEY bound to bilko-api-demo (rev00165). AI route is live. Evidence: /tmp/alai/7d24e9bf/evidence-bilko-ai-fix/verification.json. Full functional E2E (Entra login path) is pending.
7. Verification Summary (Proveo)
Agent: Angie Jones (Proveo) | Timestamp: 2026-06-10T15:03Z | Verdict: CUTOVER PASS
| Check | Result | Detail |
|---|---|---|
| app.bilko.cloud HTTP 200 | PASS | curl confirmed |
| bilko-demo-api.alai.no /api/v1/health HTTP 200 | PASS | curl confirmed |
| PR #334 — hostname recognition, all 3 files | PASS | dual-mode confirmed on main |
| Vitest api-base-hostname-103300 (4 tests) | PASS | 4/4, 831ms |
| tsc --noEmit (apps/web) | PASS | 0 errors |
| MC #103323 routes auth-gated (401 not 404) | PASS | support_tickets POST + GET routes live |
| DB clean state | PASS (evidence-reviewed) | SQL SELECT output in cleanup.md is authoritative; Cloud SQL proxy not available in Proveo context |
| PR #334 merge conflict check | PASS — no conflict | 5 CI-only commits on main since branch point; zero file overlap with PR files |
Full Proveo report: /tmp/alai/p2p-pairing-evidence/proveo-103300-c2c6-verdict.md
8. Cutover Status Table (C1–C7)
| Item | Status | Owner | Notes |
|---|---|---|---|
| C1 — domain mapping (app.bilko.cloud + app-api.bilko.cloud → bilko-web-demo / bilko-api-demo) | DONE | FlowForge | Both Ready/True, cert Ready/True, HTTP 200 |
| C2 — hostname recognition (middleware / api-base / i18n) | DONE | CodeCraft | Dual-mode on main; PR #334 adds test coverage |
| C3 — prod DB cleaned of test/seed orgs | DONE | FlowForge/DB | 1 org remains (E2E test org d9e364ca); ~40 rows deleted; backup 1781094321949 |
| C4 — stage → prod promotion gate formalized | PENDING | FlowForge | MC #103375 |
| C5 — AI fix (GROQ_API_KEY on bilko-api-demo) | DONE | — | Rev00165, route live |
| C6 — Proveo end-to-end validation | DONE (C6-lite) | Proveo | Full E2E blocked on Entra login flow; C6-lite PASS |
| C7 — Skillforge BookStack documentation | DONE | Skillforge | This page |
9. Evidence Index
| Artifact | Path |
|---|---|
| C1–C7 delta state | /tmp/evidence-103300/cutover-state.md |
| DB inventory (read-only probe, pre-cleanup) | /tmp/evidence-103300-c3/inventory.md |
| DB cleanup execution record + post-commit proof | /tmp/evidence-103300-c3/cleanup.md |
| Proveo C2 + C6-lite verdict | /tmp/alai/p2p-pairing-evidence/proveo-103300-c2c6-verdict.md |
| C5 AI fix verification | /tmp/alai/7d24e9bf/evidence-bilko-ai-fix/verification.json |