Skip to main content

Cross-Platform Advisory AI PR Reviewer — Architecture and Operations (MC #105620)

Cross-platform AI PR reviewer — shared core and adapters (WP1)

MC: #105621 Parent: #105620 State: implementation candidate only; no live repository, workflow, pipeline, policy, secret, PR comment, or deployment change is authorized by WP1.

Purpose

WP1 replaces drift-prone platform-specific review logic with one deterministic, platform-neutral planning core and thin Azure DevOps/GitHub payload adapters. It is advisory and comment-only. It cannot approve, merge, push, dismiss reviews, mutate repository content, or block native CI.

Components

  • tools/lib/ai-pr-reviewer-core.js
    • validates (platform, immutable repository identity, PR number, exact 40-hex head SHA);
    • bounds unified diffs by total bytes, file count, and per-file bytes;
    • excludes binary, generated, dependency-lock, vendor, and build artifacts;
    • redacts recognized/explicit secrets before prompt construction;
    • wraps untrusted diff text as a JSON data value and labels it non-authoritative;
    • validates model output against a closed schema and never forwards malformed raw output;
    • accepts inline findings only on exact or narrowly adjusted added lines in the same hunk;
    • creates exact-revision markers, idempotency plans, supersession plans, and metadata-only audit events.
  • tools/lib/ai-pr-reviewer-adapters.js
    • creates Azure DevOps thread request plans and GitHub COMMENT review plans;
    • binds the plan a second time to the adapter-observed platform, canonical repository identity, PR number, and exact head SHA;
    • reports missing context, drafts, fork safety, and external failures as neutral outcomes;
    • performs no HTTP request and reads no PR code.
  • tools/ai-pr-reviewer.js
    • local self-test, prompt inspection, and deterministic dry-run planning;
    • invokes the fixed Gemini executable through stdin, never argv;
    • uses a bounded timeout/output buffer and minimal child environment;
    • never logs provider stderr because it may contain prompt or diff fragments.

Model response contract

Only this closed JSON shape is accepted:

{
  "summary": "bounded plain text",
  "findings": [
    {
      "file": "repo/relative/path",
      "line": 123,
      "severity": "blocker|warning|nit",
      "comment": "bounded plain text"
    }
  ]
}

Known severity aliases are normalized. Unknown keys, unknown severities, absolute/backslash/traversal paths, reserved bot markers, controls, oversized text, non-integer lines, malformed JSON, and excess findings make the entire model response a neutral parse skip. Raw model text is never used as a fallback comment.

Exact-revision, idempotency, and supersession rules

  1. The caller supplies the complete immutable identity and the adapter independently observes platform, canonical repository identity, PR number, and exact PR head SHA.
  2. Any field mismatch or malformed identity is rejected before post, skip, or defer planning.
  3. The marker contains the exact SHA and a repository/PR scope hash. The runtime idempotency key also includes the exact SHA.
  4. Existing comments affect idempotency only when the caller has independently authenticated them as reviewer-bot authored (trusted: true). User-authored marker text cannot suppress a review.
  5. One trusted marker for the same scope and SHA yields skip; a new SHA yields one new review plus deterministic supersession requests for trusted older reviews.
  6. A stale result must be discarded if the head changes before posting. Live wiring must re-read the head immediately before each platform mutation.

Changed-line anchoring (MC #105102 gap)

The core derives anchors only from + lines in bounded unified-diff hunks. A finding may use an exact added line or adjust at most three lines to the nearest added line inside the same hunk. Wrong files, removed/context lines outside that bound, unrelated hunks, and out-of-range lines are rejected rather than moved to an arbitrary line. Azure uses threadContext.rightFileStart/rightFileEnd; GitHub uses line, side: RIGHT, and the exact commit_id.

Platform safety

Azure DevOps

Required runtime context is limited to collection/project/repository/PR/head identifiers and an access token. WP1 only builds thread payloads. A later rollout must use a tenant-specific, least-privilege identity and fetch diffs/comments through Azure APIs without executing PR code.

GitHub

Declared workflow permissions are exactly:

permissions:
  contents: read
  pull-requests: write

The review event is the fixed literal COMMENT; model output cannot select APPROVE or REQUEST_CHANGES. The adapter requires pull_request.head.sha from the event and never substitutes generic GITHUB_SHA, which may identify a base or merge commit. Draft PRs defer until ready for review only after full context binding. Fork PRs require bounded API diff retrieval only: privileged contexts must never checkout or execute the untrusted head while a token or secret is present.

Failure behavior

Identity/head failures reject the review. Empty or fully excluded diffs, malformed model responses, model timeout/nonzero exit, missing platform context, permission failures, diff API failures, and comment API failures are neutral/nonblocking outcomes with bounded classifications. They must not convert native CI failure to success and must not claim a review was posted. Audit output is metadata-only and excludes prompts, diffs, headers, tokens, provider stderr, and environments.

Deterministic verification

From the isolated system worktree:

node --check tools/lib/ai-pr-reviewer-core.js
node --check tools/lib/ai-pr-reviewer-adapters.js
node --check tools/ai-pr-reviewer.js
node --test tests/ai-pr-reviewer-core.test.js
node tools/ai-pr-reviewer.js self-test
git diff --check

The test suite covers strict identity/SHA validation, exact marker scope, trusted-author idempotency, supersession, hard diff bounds, binary/lockfile exclusion, injection isolation, secret redaction, closed-schema failures, changed-line anchoring, stale-head rejection, neutral failures, Azure/GitHub payloads, fork/draft behavior, fixed COMMENT authority, and stdin-only provider invocation.

Rollout prerequisites (not authorized by WP1)

  1. Independent Proveo PASS bound to the exact candidate SHA and a clean detached worktree.
  2. Explicit tenant authorization and tenant-specific credentials for client repositories.
  3. Immutable shared-core consumption: accepted absolute/package path for Azure; immutable commit SHA pin for GitHub.
  4. One separately approved Azure canary and one separately approved GitHub canary, comment-only and nonblocking.
  5. Canary proof for same-SHA retry, new-SHA update, stale-head race, valid inline thread, malformed response, provider/API failure, draft, and fork PR.
  6. Human review of false-positive/false-negative results before any request for blocking policy.

Mirrors and non-merge-authoritative surfaces remain excluded unless separately classified. No global rollout is implied by WP1 acceptance.

Rollback

Because WP1 has no live wiring, rollback is removal/revert of these candidate files. During a later canary, rollback must disable/remove only the reviewer trigger or workflow reference and revoke its tenant-specific credential; native build/test policies remain unchanged. Existing advisory comments may be left as historical exact-SHA evidence or marked superseded through platform APIs. Rollback must never require force-push, repository-history rewrite, policy bypass, or product deployment.

No comments to display

Back to top