# Bilko Backoffice — Ops Infra (Logging Views + support@ Forwarding + Preflight)

# Bilko Backoffice — Ops Infra (MC #103325)

**Branch:** feat/103325-backoffice-infra | **PR:** #317 | **Proveo verdict:** PASS (2026-06-10) | **Sibling:** [Backoffice Backend MVP (page 3100)](/books/bilko/page/3100)

---

## 1. Cloud Logging Saved Views

**GCP project:** `tribal-sign-487920-k0`  
**Bucket:** `_Default` (global)  
**Verified via:** `gcloud logging views list --bucket=_Default --location=global --project=tribal-sign-487920-k0`

<table id="bkmrk-view-idscope-filteri"><thead><tr><th>View ID</th><th>Scope filter</th><th>Intended use / Log Explorer query to add</th></tr></thead><tbody><tr><td>`bilko-error-by-org`</td><td>`resource.type="cloud_run_revision" AND resource.labels.service_name=~"bilko-api-(demo|stage)"`</td><td>Add query `severity>=ERROR`. Group results by `orgId` (parse via `JSON_EXTRACT(textPayload, "$.orgId")` — orgId lives in textPayload JSON, not jsonPayload).</td></tr><tr><td>`bilko-request-trace`</td><td>`resource.type="cloud_run_revision" AND resource.labels.service_name=~"bilko-(api|web)-(demo|stage)"`</td><td>Add query `logName=~"stdout" OR logName=~"requests"`. Correlate requests end-to-end by `requestId` field in textPayload.</td></tr><tr><td>`bilko-5xx-demo`</td><td>`resource.type="cloud_run_revision" AND resource.labels.service_name=~"bilko-(api|web)-demo"`</td><td>Add query `httpRequest.status>=500`. Scoped to demo environment only.</td></tr></tbody></table>

### GCP constraint — view filter expressiveness

GCP `gcloud logging views create --log-filter` only accepts log source, resource type, appHub fields, user labels, and log ID conditions. Severity comparisons (`severity>=ERROR`) and field comparisons (`httpRequest.status>=500`) are **not valid in view filters** — they must be added as Log Explorer query refinements on top of the saved view scope. This is a documented GCP platform limitation. Each view description in GCP documents this explicitly.

**Log schema note:** Bilko API logs structured data as JSON inside `textPayload` (not `jsonPayload`). The textPayload schema is: `{"requestId":"...","method":"...","path":"...","status":N,"durationMs":N,"userId":"...","orgId":"...","ip":"..."}`. ERROR logs are stack traces in textPayload; `orgId` is present on request-completion log lines, not on exception lines.

---

## 2. support@bilko.cloud Email Forwarding

### MX provider — IMPORTANT

**bilko.cloud MX = Migadu** (`aspmx1.migadu.com` + `aspmx2.migadu.com`, confirmed via `dig MX bilko.cloud`). The **CF Email Routing section in DEPLOY-MAP.md is STALE** and must be corrected — Cloudflare does not handle bilko.cloud email.

### Implemented forwarding

Mail flow: `support@bilko.cloud` (Migadu mailbox, `may_receive=true`, `may_send=false`) → Migadu forwarding → `alem@alai.no`

**Key Migadu design constraint:** Alias `destinations` only accept same-domain addresses — external addresses are silently rewritten to `<localpart>@<same-domain>`. The correct mechanism for external cross-domain delivery is a **forwarding on a mailbox object** (not an alias).

**Implementation steps taken:**

1. Confirmed `GET /v1/domains/bilko.cloud/mailboxes/support/forwardings` — support@ was alias-only (no mailbox).
2. Created `support@bilko.cloud` mailbox: `may_receive=true`, `may_send=false`, IMAP/POP3 disabled (receive-only).
3. Added forwarding via `POST /v1/domains/bilko.cloud/mailboxes/support/forwardings {"address":"alem@alai.no"}` — response: `is_active: true`, `confirmed_at: 2026-06-10T08:17:01Z`, no confirmation email required.
4. Deleted the old support@ alias (superseded by mailbox).
5. Removed investigation-only forwarding from `sales@bilko.cloud` — **sales@ is left untouched** (`forwardings: []`).

**Verified state (Proveo independent GET):**

```
GET /v1/domains/bilko.cloud/mailboxes/support/forwardings
{"forwardings":[{"address":"alem@alai.no","confirmed_at":"2026-06-10T08:17:01Z","blocked_at":null,"is_active":true}]}

GET /v1/domains/bilko.cloud/mailboxes/sales/forwardings
{"forwardings":[]}
```

### Migadu admin path (for future changes)

To modify forwarding: [admin.migadu.com](https://admin.migadu.com) → bilko.cloud → Mailboxes → support → Forwardings. Do **not** use the Aliases section for external cross-domain targets.

---

## 3. Preflight Rollback Script

**File:** `scripts/ops/bilko-support-fix-preflight.sh` (committed at `67ed0ce5`, PR #317, mode `100755`)

### What it does

1. **STEP 1 — Cloud SQL backup** (write, skipped in dry-run): Takes an on-demand Cloud SQL backup of the Bilko DB before any deploy action. Provides a restore point.
2. **STEP 2 — Capture current Cloud Run revision** (read-only always): Records the live revision name and image SHA for both `bilko-api-demo` and `bilko-web-demo`.
3. **STEP 3 — Print rollback commands** (print only, never executes): Outputs the exact `gcloud run services update-traffic` commands needed to roll back to the captured revisions. These are `echo`-wrapped — they are never executed by the script.

### How to run

```
# Dry-run (safe, no writes — use to confirm rollback targets before deploy)
bash scripts/ops/bilko-support-fix-preflight.sh --dry-run

# Live run (takes SQL backup, captures revisions, prints rollback cmds)
bash scripts/ops/bilko-support-fix-preflight.sh
```

### Deploy-fragility rule enforced

All example re-deploy commands in the printed output use `--update-secrets`. The script documents and enforces: **NEVER use `--set-env-vars` for Bilko Cloud Run deploys** — it overwrites the Secret Manager binding and exposes secrets as plaintext environment variables.

---

## 4. Known Follow-up

<table id="bkmrk-itemstatusowner-depl"><thead><tr><th>Item</th><th>Status</th><th>Owner</th></tr></thead><tbody><tr><td>DEPLOY-MAP.md CF Email Routing section for bilko.cloud is stale (lists Cloudflare; MX is Migadu)</td><td>Open</td><td>John / next infra PR</td></tr><tr><td>PR #317 bundles MC #103323 application code (Sentry, SupportTickets, DB migrations V71+V72) — confirm separate QA validation for that scope</td><td>Open (Proveo advisory)</td><td>John</td></tr><tr><td>Merge PR #318 (smoke-test fix) before PR #317, or close #318 as superseded if #317 merges first</td><td>Open</td><td>John</td></tr></tbody></table>

---

*Created by Skillforge for MC #103325. Evidence: /tmp/evidence-103325/verification.md + /tmp/alai/p2p-pairing-evidence/proveo-multi-317-318-verdict.md. All facts machine-verified.*