Test Strategy

Test Strategy

Project: Bilko Version: 1.1 Date: 2026-05-21 Author: Ops Architect / ALAI Documentation Team Status: Active Reviewers: Tech Lead, Alem Bašić

Document History

1. Testing Philosophy & Principles

Financial software has a higher correctness bar than typical web apps. A bug in VAT calculation or double-entry bookkeeping is not a UX inconvenience — it's a compliance failure that could expose Bilko users to tax liability or audit findings.

Core Principles:

1. Financial logic is P0 — VAT calculations, double-entry balance, NUMERIC precision are tested at >95% coverage before any feature ships
2. Tests are first-class code — reviewed, maintained, and refactored alongside production code
3. Test the behavior, not the implementation — tests enable safe refactoring of internals
4. Fast feedback — unit tests run in < 3 min; full suite < 10 min
5. No test = no ship — financial logic without a test is a P0 blocker for merging
6. Isolation — every test cleans up after itself; no test depends on another

Testing philosophy: Bilko follows an industry-standard layered strategy: focused unit coverage for financial calculations and business logic, strong integration/contract coverage for Ktor API + PostgreSQL behavior, and a small Playwright layer for critical user journeys. This is compatible with Spotify's public "testing honeycomb" direction: most confidence should come from service interaction tests and contracts, not from trying to automate every behavior through a browser. We do not aim for 100% E2E coverage.

2. Layered Test Strategy

                 Playwright E2E / Demo Smoke
        Critical browser journeys + deployed health evidence

              Integration + Contract Tests (dominant)
       Ktor routes, services, PostgreSQL/Testcontainers, auth,
       RBAC, multi-tenant isolation, frontend/backend API contract

                    Focused Unit Tests
       Financial engine, VAT, currency, validators, pure logic

Target distribution:

• Unit tests — financial logic and pure business rules, especially packages/core and accounting/tax code
• Integration/contract tests — API routes, DB behavior, auth/session boundaries, RBAC, org isolation, frontend/backend API compatibility
• Critical Playwright E2E — a small, maintained set for invoice, expense, report, auth, and settings flows
• Real-demo smoke — non-destructive deployed health check against https://bilko-demo.alai.no
• Full-demo rehearsal — resettable demo tenant/environment used for stakeholder demos; not the deploy gate

3. Testing Tools

Why Vitest (not Jest)

• ESM native, Vite-based → faster
• Compatible with Turborepo
• Watch mode with HMR
• Same API as Jest (easy migration)

Why Playwright (not Cypress)

• Multi-browser: Chromium, Firefox, WebKit (Safari)
• Auto-wait (no flaky tests from race conditions)
• Parallel execution (workers: 4)
• Video and trace on failure

4. Test Scope by Layer

4.1 Unit Tests (Vitest)

What to unit test:

• calculateVAT(amount, rate, country) — Serbia 20%, BiH 17%, Croatia 25%
• validateDoubleEntry(debit, credit) — must be equal, error on imbalance
• convertCurrency(amount, fromCurrency, toCurrency, exchangeRate) — NUMERIC(19,4)
• calculateInvoiceTotal(items) — subtotal, tax, discount, total
• lockExchangeRate(date, fromCurrency, toCurrency) — historical rate, not today's

What NOT to unit test:

• Framework internals (Ktor, Next.js, Exposed, JDBC)
• Simple property getters/setters with no logic
• Full browser journeys that belong in Playwright E2E or demo smoke

4.2 Integration + Contract Tests

What to integration test:

• Auth flow (register, login, refresh, logout)
• Invoice CRUD + status transitions (draft → sent → paid)
• Expense CRUD + approval flow
• Reports API (P&L, VAT, balance sheet)
• Organization scoping — org A cannot read org B's data (P0 security test)
• RBAC enforcement — viewer cannot create, owner can delete

4.3 E2E Tests (Playwright)

Critical journeys:

1. Auth/session Flow: Login → refresh/session validation → logout or session expiry behavior
2. Invoice Flow: Create draft in resettable staging/demo tenant → Preview → Send/Mark Paid where safe
3. Expense Flow: Add → Upload Receipt → Approve/Reject/Pay where safe
4. Report Flow: Generate P&L/VAT report → Export PDF/XLSX where safe
5. Settings Flow: Organization/settings/users page loads and key controls are visible

Rule: public real-demo tests must be non-destructive. Registration, deletion, rate-limit torture, invoice/expense creation, and expected-fail regressions belong in resettable staging/nightly suites, not the live demo smoke gate.

5. Test Data Management

Isolation rule: integration tests must create isolated organization/user fixtures and clean up deterministically. Cross-test dependence is forbidden.

Test org pattern: Each integration test creates a fresh bilko_test organization and user to prevent cross-test contamination.

6. Coverage Requirements

Coverage enforcement: Vitest coverage thresholds in vitest.config.ts. CI pipeline fails if below threshold.

7. Quality Gates

PR Merge Gate

• All unit tests pass
• All integration tests pass
• Coverage ≥ minimum thresholds
• Linting passes (ESLint + Prettier)
• Type checking passes (TypeScript strict)
• No new HIGH/CRITICAL security findings

Staging Deploy Gate

• All PR gates passed
• Build artifact created successfully

Production Deploy Gate

• Critical E2E gate passes on staging/resettable environment
• Real-demo smoke passes after deploy with screenshot/video evidence
• Performance baseline not degraded > 20% for relevant changes
• Manual approval in CI pipeline

8. Responsibility Matrix

9. Test Reporting & Metrics

10. Continuous Testing in CI/CD

Related Documents

• Test Plan
• E2E Test Plan
• Performance Test Plan
• Definition of Done
• CI/CD Pipeline
• TESTING-GUIDE.md
• TEST-INVENTORY.md
• Demo Testing Plan

Approval