Incident — 2026-04-21 alai.no Contact Form Failure
2026-04-21 — alai.no Contact Form Silent Failure
Incident Classification
Severity: HIGH — Silent data loss (potential lead loss)
Duration: 2026-04-19 19:00 → 2026-04-21 11:30 (40.5 hours)
Detection: Manual inspection via Himalaya IMAP client
Status: RESOLVED (form handler redeployed to CF Pages Functions)
Timeline
- 2026-04-19 19:00 — alai.no migrated from Vercel to Cloudflare Pages (MC #8576)
- 2026-04-19 19:00 → 2026-04-21 11:30 — Contact form submissions received HTTP 200 OK but no emails delivered
- 2026-04-21 11:30 — CEO (Alem) noticed no inquiry emails received in days, requested investigation
- 2026-04-21 11:35 — John inspected
info@alai.noIMAP (viahimalaya search --folder INBOX from:noreply) — zero messages from contact form - 2026-04-21 11:45 — Root cause identified: CF Tunnel routing hijack + documenso-webhook false-positive response
- 2026-04-21 12:15 — CodeCraft dispatched to deploy dedicated contact handler as CF Pages Function (MC #8587)
- 2026-04-21 14:00 — Fix deployed and verified (E2E browser test + inbox check)
Impact Assessment
- Lost inquiries: Unknown (no form submission logging). Estimated 0-5 potential leads during 40-hour window.
- User experience: Users received "success" feedback but no confirmation email. No error notification.
- Business risk: Medium — alai.no is not yet primary sales channel; minimal active marketing campaigns during incident window.
Root Cause Analysis
Technical Chain of Failure
- alai.no contact form POSTs to
https://api.basicconsulting.no/contact(hardcoded Vercel pattern from pre-migration code) - Cloudflare Tunnel ingress rule matches
api.basicconsulting.no/*→ routes ALL POST requests tolocalhost:3001 documenso-webhook.jslistens on port 3001, designed for Documenso signature events- Webhook handler has catch-all route:
app.post('/*', (req, res) => res.json({ok: true})) - Contact form receives HTTP 200 +
{ok: true}→ assumes success, displays "Message sent" - No email handler ever invoked → no SMTP call → no delivery
Root Cause Categories
- Architectural: Assumed serverless runtime (Vercel Functions) but deployed to static hosting (CF Pages) without serverless equivalent
- Migration process: No pre-deployment checklist for "dynamic endpoints" (forms, APIs, webhooks)
- Testing gap: No E2E validation of email delivery — only HTTP response validated (curl 200 != email delivered)
- Monitoring gap: No alerting on zero-message rate for
info@alai.noINBOX (expected rate: ~1-3/week)
Detection Method
Manual IMAP inspection using Himalaya CLI:
himalaya search --account info@alai.no --folder INBOX "from:noreply" "since:2026-04-19"
# Result: No messages found
Lesson: HTTP 200 is NOT proof of delivery. Always verify end-to-end (inbox check, log inspection, user confirmation email).
Fix Summary
- CodeCraft deployed
/functions/contact.jsas CF Pages Function - Handler uses Resend API (
RESEND_API_KEYin Bitwarden → CF Pages env vars) - Form target updated to
https://alai.no/api/contact(CF Pages Functions route:/functions/→/api/) - Proveo validated: submit test form → received at
info@alai.nowithin 5 seconds
MC Task: #8587
Lessons Learned
What Went Well
- CEO noticed absence of expected emails (operational intuition)
- Himalaya CLI provided rapid IMAP audit without browser login
- Root cause identified within 15 minutes of investigation start
What Went Wrong
- Migration checklist did NOT include "verify all POST endpoints have backend handlers"
- No E2E test protocol for forms (HTTP 200 assumed sufficient)
- No monitoring/alerting on email delivery rates (silent failure undetected for 40 hours)
- Cloudflare Tunnel routing too broad (
/*catch-all dangerous for multi-service proxy)
Prevention Actions
| Action | Owner | MC Task | Status |
|---|---|---|---|
| Update site migration checklist: "Verify form handlers migrated" | Skillforge | #8587 | DONE (this doc) |
| Create Forms E2E Testing Protocol (HTTP + inbox check required) | Skillforge | #8587 | DONE (BookStack QA section) |
Add Grafana alert: info@alai.no message rate < 1/week → notify #ops |
FlowForge | #8588 | OPEN |
Audit all CF Tunnel ingress rules for overly-broad /* patterns |
Securion | #8589 | OPEN |
| Migrate snowit.ba contact form (same silent failure risk) | CodeCraft | #8591 | OPEN |
| Add form submission logging to all contact handlers (track volume even if email fails) | CodeCraft | #8592 | OPEN |
Related Incidents
- snowit.ba contact form: Same root cause (Vercel pattern, no CF Pages handler). Bouncing to
info@snowit.ba(LumisCare side, not ALAI). MC #8591 tracks. - getdrop.no waitlist: Already migrated correctly (CF Pages Function + D1 storage). No issue.
References
- Email Pipeline Runbook
- Forms E2E Testing Protocol (new)
- Static Hosting Migration — Progress Log
- Himalaya setup:
~/.config/himalaya/config.toml(info@alai.noIMAP credentials in Bitwarden)