# Integration Design

# Integration Design Document

> **Project:** {{PROJECT_NAME}}
> **Integration:** {{INTEGRATION_NAME}}
> **Version:** {{VERSION}}
> **Date:** {{DATE}}
> **Author:** {{AUTHOR}}
> **Status:** Draft | In Review | Approved
> **Reviewers:** {{REVIEWERS}}

## Document History
| Version | Date | Author | Changes |
|---------|------|--------|---------|
| 0.1     | {{DATE}} | {{AUTHOR}} | Initial draft |

---

## 1. Integration Overview & Context

<!-- GUIDANCE: Explain the business need for this integration. What business process does it enable? Who are the owners on both sides? What would break if this integration failed? -->

**Integration Name:** {{INTEGRATION_NAME}}
**Type:** Synchronous (REST/gRPC) | Asynchronous (Events/Queue) | Bidirectional | File-based

**Business Purpose:** {{WHY_THIS_INTEGRATION_EXISTS}}

**Criticality:** Critical | High | Medium | Low
- **Impact if down:** {{BUSINESS_IMPACT_IF_UNAVAILABLE}}
- **Acceptable downtime:** {{RTO}} | **Max data loss:** {{RPO}}

**Parties:**
| Party | System | Team | Contact |
|-------|--------|------|---------|
| Consumer (caller) | {{CONSUMER_SYSTEM}} | {{TEAM_A}} | {{CONTACT_A}} |
| Provider (server) | {{PROVIDER_SYSTEM}} | {{TEAM_B}} | {{CONTACT_B}} |

---

## 2. Integration Topology Diagram

<!-- GUIDANCE: Show all services/systems involved. For complex integrations, show the message flow between all parties. -->

```mermaid
flowchart LR
    subgraph ConsumerSide["Consumer — {{CONSUMER_SYSTEM}}"]
        C_SVC[{{ConsumerService}}]
        C_CB[Circuit Breaker]
        C_RETRY[Retry Handler]
    end

    subgraph Integration["Integration Layer"]
        GW[API Gateway / Load Balancer]
        Q[Message Queue\n{{QUEUE_NAME}}]
        DLQ[Dead Letter Queue\n{{DLQ_NAME}}]
    end

    subgraph ProviderSide["Provider — {{PROVIDER_SYSTEM}}"]
        P_SVC[{{ProviderService}}]
        P_DB[(Provider DB)]
        P_WORKER[Event Worker]
    end

    C_SVC --> C_CB
    C_CB --> C_RETRY
    C_RETRY -->|HTTPS REST| GW
    GW --> P_SVC
    P_SVC --> P_DB

    P_WORKER -->|Publish| Q
    Q -->|Consume| C_SVC
    Q -->|Failed| DLQ
    DLQ -->|Alert| AlertSystem[PagerDuty]
```

---

## 3. Service Contracts

<!-- GUIDANCE: Define the exact contract for each integration point. This is a binding agreement between teams. -->

### 3.1 Integration: {{INTEGRATION_NAME_1}}

**Protocol:** REST/HTTPS | gRPC | GraphQL | WebSocket | AMQP
**Direction:** {{CONSUMER}} → {{PROVIDER}}
**Idempotency:** YES — use `Idempotency-Key` header | NO

#### Authentication
| Method | Details |
|--------|---------|
| **Type** | Bearer JWT | API Key | OAuth2 Client Credentials | mTLS |
| **Header** | `Authorization: Bearer {{TOKEN}}` |
| **Key rotation** | Every {{ROTATION_PERIOD}} — coordinated via {{ROTATION_PROCESS}} |
| **Token endpoint** | `{{AUTH_ENDPOINT}}` (if OAuth2) |

#### Request Contract

**Endpoint:** `{{HTTP_METHOD}} {{BASE_URL}}/{{PATH}}`

**Headers:**
```http
Authorization: Bearer {{JWT_OR_API_KEY}}
Content-Type: application/json
Accept: application/json
X-Request-ID: {{UUID}}
X-Idempotency-Key: {{IDEMPOTENCY_KEY}}
```

**Request Body:**
```json
{
  "{{field1}}": "{{type}} — {{description}}",
  "{{field2}}": "{{type}} — {{description}}",
  "metadata": {
    "sourceSystem": "{{CONSUMER_SYSTEM_ID}}",
    "timestamp": "ISO8601"
  }
}
```

**Successful Response `200 / 201`:**
```json
{
  "{{responseField1}}": "{{type}}",
  "{{responseField2}}": "{{type}}",
  "requestId": "echo of X-Request-ID"
}
```

#### Error Handling

| HTTP Status | Error Code | Consumer Action |
|------------|-----------|----------------|
| `400` | `VALIDATION_ERROR` | Log error, do NOT retry — fix request |
| `401` | `UNAUTHORIZED` | Refresh token, retry once |
| `403` | `FORBIDDEN` | Alert engineering, do NOT retry |
| `404` | `NOT_FOUND` | Log, do NOT retry — check resource ID |
| `409` | `CONFLICT` | Log, skip (idempotent) |
| `422` | `BUSINESS_RULE` | Log error, do NOT retry — escalate |
| `429` | `RATE_LIMITED` | Backoff per `Retry-After` header |
| `500` | `INTERNAL_ERROR` | Retry with exponential backoff |
| `502/503` | `UNAVAILABLE` | Circuit breaker — fail fast |

#### Retry Policy
```
Max retries: {{MAX_RETRIES}} (retry only on 500, 502, 503, 429, network errors)
Strategy: Exponential backoff with jitter
Delays: [{{DELAY_1}}ms, {{DELAY_2}}ms, {{DELAY_3}}ms]
Timeout per attempt: {{TIMEOUT_MS}}ms
```

#### Circuit Breaker Configuration
```
Failure threshold: {{FAILURE_PERCENT}}% failures in {{WINDOW_SECONDS}}s window
Open duration: {{OPEN_DURATION_SECONDS}}s
Half-open test: 1 request
Alert on: Circuit open for > {{ALERT_THRESHOLD_SECONDS}}s
```

#### Rate Limiting
| Limit | Value | Window | Action when exceeded |
|-------|-------|--------|---------------------|
| Requests per minute | {{RPM}} | 60s sliding | HTTP 429, Retry-After |
| Burst limit | {{BURST}} | 1s | HTTP 429 immediately |
| Daily quota | {{DAILY}} | 24h | HTTP 429, contact support |

#### Timeout Configuration
| Timeout Type | Value | Notes |
|-------------|-------|-------|
| Connection timeout | {{CONN_TIMEOUT_MS}}ms | Time to establish connection |
| Read timeout | {{READ_TIMEOUT_MS}}ms | Time to receive first byte |
| Total request timeout | {{TOTAL_TIMEOUT_MS}}ms | End-to-end budget |

---

### 3.2 Integration: {{INTEGRATION_NAME_2}} (if applicable)

<!-- GUIDANCE: Repeat section 3.1 pattern for additional integration points. -->

**Protocol:** gRPC
**Service definition:**
```protobuf
service {{ServiceName}} {
  rpc {{MethodName}} ({{RequestMessage}}) returns ({{ResponseMessage}});
  rpc {{StreamMethodName}} ({{RequestMessage}}) returns (stream {{ResponseMessage}});
}

message {{RequestMessage}} {
  string id = 1;
  string tenant_id = 2;
  {{FieldType}} {{field_name}} = 3;
}

message {{ResponseMessage}} {
  string id = 1;
  {{FieldType}} {{field_name}} = 2;
  google.protobuf.Timestamp created_at = 3;
}
```

---

## 4. Event-Driven Integrations

<!-- GUIDANCE: For async integrations via message queues or event streaming. Follow CloudEvents spec for event envelopes. -->

### 4.1 Event Schemas (CloudEvents 1.0)

#### Event: `{{entity}}.{{ACTION}}`

**Published by:** `{{PUBLISHER_SYSTEM}}`
**Consumed by:** `{{CONSUMER_SYSTEM_1}}`, `{{CONSUMER_SYSTEM_2}}`

```json
{
  "specversion": "1.0",
  "type": "{{REVERSE_DNS_EVENT_TYPE}}",
  "source": "https://{{SYSTEM_DOMAIN}}/{{resource}}",
  "id": "{{UUID}}",
  "time": "2024-01-01T00:00:00Z",
  "datacontenttype": "application/json",
  "subject": "{{RESOURCE_ID}}",
  "data": {
    "entityId": "UUID of affected resource",
    "tenantId": "UUID of tenant",
    "actorId": "UUID of user who triggered event",
    "{{DOMAIN_FIELD_1}}": "domain-specific data",
    "{{DOMAIN_FIELD_2}}": "domain-specific data",
    "previousState": null,
    "newState": "{{STATE}}"
  }
}
```

### 4.2 Topics / Queues

| Topic/Queue | Partitions | Retention | Consumers | Producer |
|------------|-----------|---------|----------|---------|
| `{{TOPIC_NAME_1}}` | {{N}} | {{RETENTION}} | {{CONSUMER_GROUPS}} | {{PRODUCER_SERVICE}} |
| `{{TOPIC_NAME_2}}` | {{N}} | {{RETENTION}} | {{CONSUMER_GROUPS}} | {{PRODUCER_SERVICE}} |

### 4.3 Ordering Guarantees

| Integration | Ordering | Scope | Notes |
|------------|---------|-------|-------|
| {{INTEGRATION_1}} | Strict order | Per `tenantId` | Kafka partition by tenantId |
| {{INTEGRATION_2}} | Best-effort | Global | FIFO queue — no strict ordering |
| {{INTEGRATION_3}} | No ordering | N/A | Independent events |

### 4.4 Idempotency Strategy

<!-- GUIDANCE: Event consumers MUST be idempotent — events can be delivered more than once. -->

```
For each consumed event:
1. Check processed_events table: SELECT 1 WHERE event_id = $1 AND consumer_group = $2
2. If found: log "Duplicate event skipped" and ACK (do not reprocess)
3. If not found: process event
4. On success: INSERT INTO processed_events (event_id, consumer_group, processed_at)
5. ACK message

Deduplication window: {{DEDUP_WINDOW}} (keep processed_events for this duration)
```

---

## 5. Data Consistency Patterns

<!-- GUIDANCE: Distributed systems cannot guarantee ACID across services. Document the consistency model and the pattern used to achieve it. -->

### 5.1 Consistency Model

**Model:** Strong | Eventual | Causal
**Acceptable lag:** {{MAX_LAG_SECONDS}}s

### 5.2 Saga Pattern (if used for distributed transactions)

```mermaid
sequenceDiagram
    autonumber
    participant O as Orchestrator
    participant S1 as {{SERVICE_1}}
    participant S2 as {{SERVICE_2}}
    participant S3 as {{SERVICE_3}}

    O->>S1: Execute Step 1
    S1-->>O: Step 1 succeeded {result1}
    O->>S2: Execute Step 2 (with result1)
    S2-->>O: Step 2 succeeded {result2}
    O->>S3: Execute Step 3 (with result2)
    S3-->>O: Step 3 FAILED

    Note over O: Compensating transactions (reverse order)
    O->>S2: Compensate Step 2
    S2-->>O: Compensated
    O->>S1: Compensate Step 1
    S1-->>O: Compensated
    O-->>Client: Transaction rolled back
```

**Compensation strategies:**
| Step | Compensation | Notes |
|------|-------------|-------|
| {{STEP_1}} | {{COMPENSATION_1}} | {{NOTES}} |
| {{STEP_2}} | {{COMPENSATION_2}} | {{NOTES}} |

---

## 6. Integration Testing Strategy

<!-- GUIDANCE: How do we verify the integration works? What tests run in CI? What runs in staging? -->

### 6.1 Contract Testing (Pact)

- **Consumer-driven contracts:** Consumer writes tests defining expected provider behavior
- **Provider verification:** Provider CI runs consumer contracts on every build
- **Pact Broker:** `{{PACT_BROKER_URL}}`

### 6.2 Integration Test Environments

| Environment | Purpose | Trigger |
|-------------|---------|---------|
| Local | Dev testing with mocked provider | Manual |
| Staging | Full integration with staging provider | Every PR merge |
| Production | Synthetic monitoring | Every 5 minutes |

### 6.3 Test Scenarios

**Happy path:**
- [ ] {{SCENARIO_1}} — expected outcome
- [ ] {{SCENARIO_2}} — expected outcome

**Error scenarios:**
- [ ] Provider returns 500 — circuit breaker opens after threshold
- [ ] Provider times out — retry policy kicks in
- [ ] Auth token expired — token refresh flow works
- [ ] Rate limit exceeded — 429 handled, backoff applied
- [ ] Duplicate event consumed — idempotency key prevents double-processing

---

## 7. Monitoring & Alerting

<!-- GUIDANCE: Define what observability is in place for this integration. -->

### 7.1 Key Metrics

| Metric | Type | Alert Condition | Severity |
|--------|------|-----------------|---------|
| `integration_{{name}}_requests_total` | Counter | — | — |
| `integration_{{name}}_error_rate` | Gauge | > {{THRESHOLD}}% for 5m | HIGH |
| `integration_{{name}}_latency_p99_ms` | Histogram | > {{THRESHOLD}}ms for 5m | MEDIUM |
| `integration_{{name}}_circuit_open` | Gauge | == 1 | CRITICAL |
| `integration_{{name}}_dlq_depth` | Gauge | > 0 | HIGH |
| `integration_{{name}}_consumer_lag` | Gauge | > {{LAG_THRESHOLD}} | HIGH |

### 7.2 Distributed Tracing

- **Trace ID propagation:** `X-Request-ID` and `traceparent` headers forwarded
- **Sampling rate:** {{SAMPLE_RATE}}% in production, 100% in staging
- **Tracing tool:** {{TRACING_TOOL}} — dashboard: {{DASHBOARD_URL}}

### 7.3 Alert Routing

| Condition | Alert Channel | Escalation |
|----------|--------------|----------|
| Circuit breaker open | PagerDuty {{TEAM_A}} + Slack #{{CHANNEL}} | On-call engineer |
| DLQ depth > 0 | Slack #{{CHANNEL}} | Investigate within 1h |
| Error rate > {{THRESHOLD}}% | PagerDuty | On-call engineer |

---

## Approval
| Role | Name | Date | Signature |
|------|------|------|-----------|
| Author | | | |
| Consumer Team Lead | | | |
| Provider Team Lead | | | |
| Platform/Infra | | | |
| Approver | | | |