# Incident — 2026-04-21 alai.no Contact Form Failure # 2026-04-21 — alai.no Contact Form Silent Failure ## Incident Classification **Severity:** HIGH — Silent data loss (potential lead loss) **Duration:** 2026-04-19 19:00 → 2026-04-21 11:30 (40.5 hours) **Detection:** Manual inspection via Himalaya IMAP client **Status:** RESOLVED (form handler redeployed to CF Pages Functions) ## Timeline - **2026-04-19 19:00** — alai.no migrated from Vercel to Cloudflare Pages (MC #8576) - **2026-04-19 19:00 → 2026-04-21 11:30** — Contact form submissions received HTTP 200 OK but no emails delivered - **2026-04-21 11:30** — CEO (Alem) noticed no inquiry emails received in days, requested investigation - **2026-04-21 11:35** — John inspected `info@alai.no` IMAP (via `himalaya search --folder INBOX from:noreply`) — zero messages from contact form - **2026-04-21 11:45** — Root cause identified: CF Tunnel routing hijack + documenso-webhook false-positive response - **2026-04-21 12:15** — CodeCraft dispatched to deploy dedicated contact handler as CF Pages Function (MC #8587) - **2026-04-21 14:00** — Fix deployed and verified (E2E browser test + inbox check) ## Impact Assessment - **Lost inquiries:** Unknown (no form submission logging). Estimated 0-5 potential leads during 40-hour window. - **User experience:** Users received "success" feedback but no confirmation email. No error notification. - **Business risk:** Medium — alai.no is not yet primary sales channel; minimal active marketing campaigns during incident window. ## Root Cause Analysis ### Technical Chain of Failure 1. alai.no contact form POSTs to `https://api.basicconsulting.no/contact` (hardcoded Vercel pattern from pre-migration code) 2. Cloudflare Tunnel ingress rule matches `api.basicconsulting.no/*` → routes ALL POST requests to `localhost:3001` 3. `documenso-webhook.js` listens on port 3001, designed for Documenso signature events 4. Webhook handler has catch-all route: `app.post('/*', (req, res) => res.json({ok: true}))` 5. Contact form receives HTTP 200 + `{ok: true}` → assumes success, displays "Message sent" 6. No email handler ever invoked → no SMTP call → no delivery ### Root Cause Categories - **Architectural:** Assumed serverless runtime (Vercel Functions) but deployed to static hosting (CF Pages) without serverless equivalent - **Migration process:** No pre-deployment checklist for "dynamic endpoints" (forms, APIs, webhooks) - **Testing gap:** No E2E validation of email delivery — only HTTP response validated (curl 200 != email delivered) - **Monitoring gap:** No alerting on zero-message rate for `info@alai.no` INBOX (expected rate: ~1-3/week) ## Detection Method Manual IMAP inspection using Himalaya CLI: ``` himalaya search --account info@alai.no --folder INBOX "from:noreply" "since:2026-04-19" # Result: No messages found ``` **Lesson:** HTTP 200 is NOT proof of delivery. Always verify end-to-end (inbox check, log inspection, user confirmation email). ## Fix Summary 1. CodeCraft deployed `/functions/contact.js` as CF Pages Function 2. Handler uses Resend API (`RESEND_API_KEY` in Bitwarden → CF Pages env vars) 3. Form target updated to `https://alai.no/api/contact` (CF Pages Functions route: `/functions/` → `/api/`) 4. Proveo validated: submit test form → received at `info@alai.no` within 5 seconds **MC Task:** [\#8587](https://docs.basicconsulting.no/books/operations/page/mission-control-dashboard) ## Lessons Learned ### What Went Well - CEO noticed absence of expected emails (operational intuition) - Himalaya CLI provided rapid IMAP audit without browser login - Root cause identified within 15 minutes of investigation start ### What Went Wrong - Migration checklist did NOT include "verify all POST endpoints have backend handlers" - No E2E test protocol for forms (HTTP 200 assumed sufficient) - No monitoring/alerting on email delivery rates (silent failure undetected for 40 hours) - Cloudflare Tunnel routing too broad (`/*` catch-all dangerous for multi-service proxy) ### Prevention Actions
ActionOwnerMC TaskStatus
Update site migration checklist: "Verify form handlers migrated"Skillforge\#8587DONE (this doc)
Create Forms E2E Testing Protocol (HTTP + inbox check required)Skillforge\#8587DONE (BookStack QA section)
Add Grafana alert: `info@alai.no` message rate < 1/week → notify #opsFlowForge\#8588OPEN
Audit all CF Tunnel ingress rules for overly-broad `/*` patternsSecurion\#8589OPEN
Migrate snowit.ba contact form (same silent failure risk)CodeCraft\#8591OPEN
Add form submission logging to all contact handlers (track volume even if email fails)CodeCraft\#8592OPEN
## Related Incidents - **snowit.ba contact form:** Same root cause (Vercel pattern, no CF Pages handler). Bouncing to `info@snowit.ba` (LumisCare side, not ALAI). MC #8591 tracks. - **getdrop.no waitlist:** Already migrated correctly (CF Pages Function + D1 storage). No issue. ## References - [Email Pipeline Runbook](https://docs.basicconsulting.no/books/operations/page/email-pipeline-edita-pa-runbook) - [Forms E2E Testing Protocol](https://docs.basicconsulting.no/books/testing-qa/page/forms-e2e-testing-protocol) (new) - [Static Hosting Migration — Progress Log](https://docs.basicconsulting.no/books/operations/page/static-hosting-migration-progress-log) - Himalaya setup: `~/.config/himalaya/config.toml` (`info@alai.no` IMAP credentials in Bitwarden) --- *Authored: 2026-04-21 | Owner: Skillforge | Reviewed: John*