A pull request dropped a required field from our own MLB signals API
— the gate caught it before it shipped
The full loop, end to end: a real API with real downstream consumers, a plausible breaking change in a PR, the merge gate catching it, and a signed, replayable attestation anyone can verify.
We ran delimit-action@v1 on a pull request that removed the required field confidence_score from the GameSignalResponse schema of our WireIntel MLB Signals API (OpenAPI 3.1, published on RapidAPI). One breaking change, classified major, next version 3.0.0 — and the verdict was signed into the public Rekor transparency log.
- WireIntel MLB Signals is our sister venture
wire.report's own API. We run the merge gate on our own spec; there is no third-party customer in this report and none is implied. - What makes the spec a fair test is that it is genuinely published — OpenAPI 3.1, listed on RapidAPI, with real external consumers reading the response contract. A break here reaches people we do not control.
- The removal is the kind of change an AI coding assistant produces on a "tidy up the schema" task. The PR, the signature, and the Rekor entry are all real and independently verifiable.
The subject: a live API with real consumers
The WireIntel MLB Signals API returns per-game signal objects. The one field this report turns on is confidence_score on the GameSignalResponse schema — a required numeric field that downstream consumers use to rank and filter signals. It is exactly the sort of field a client keys core logic on, and exactly the sort of field that disappears quietly when a schema gets regenerated.
Because the spec is OpenAPI 3.1 and published for outside consumers, "did this PR break the contract?" is not a rhetorical question. Someone downstream finds out the hard way if the answer is yes and nobody caught it.
The change under review
The pull request removed one required field. That is the whole break. It is two lines in the spec, the kind that reads as cleanup in a review skim:
# openapi.yaml — components/schemas/GameSignalResponse
GameSignalResponse:
type: object
- required: [game_id, signal, confidence_score]
+ required: [game_id, signal]
properties:
game_id:
type: string
signal:
type: string
- confidence_score:
- type: number
- description: Model confidence for this game signal (0-1).What the gate produced
One field_removed on a required field is enough to force the classifier to call the release major and suggest the next version 3.0.0. The verdict is deterministic: the same two specs always produce the same answer.
The four pre-merge checks
API Lint failed on the removed required field. Policy and Security Audit passed. Deploy Readiness came back blocked — the release is not safe to ship as a minor. This run was in advisory mode, so CI did not hard-fail; the block is recorded in the verdict rather than turned into a red X. Each consumer of a Delimit attestation decides whether a blocked deploy-readiness stops the merge or is just logged. Both modes produce the same signed bytes.
The finding
The finding cites the change-type from the taxonomy and the exact JSON Pointer. This is mechanical diff output, annotated.
- breakingfinding F1change type: field_removedsurface:
#/components/schemas/GameSignalResponse.confidence_scoreThe required field confidence_score was removed from the GameSignalResponse schema. Every downstream consumer reading response.confidence_score — to rank signals, gate a bet, or drive a UI — now reads undefined. Nothing in the request shape changes, so this break is invisible in the request contract and only surfaces when a live response comes back missing the field. On a spec published to external consumers on RapidAPI, that is a break that ships to people we do not control.
The migration guide the gate generates
When the gate flags a breaking change it also emits a migration guide keyed to it. This is the generated guidance for this diff:
# Migration Guide: WireIntel MLB Signals (2.x -> 3.0.0)
This release contains 1 breaking change(s).
Follow the steps below to update your integration.
### Step 1: Field Removed
Change: Required field 'confidence_score' removed at
#/components/schemas/GameSignalResponse
Impact: Consumers reading this field will break.
Action: Add it back or provide a migration path.The guide is generated from the classified diff, not written by hand. Its job is to turn "this is breaking" into "here is the consumer-side change."
The signed attestation
This is the part that makes the report worth reading twice. The verdict above is not a screenshot you have to trust. It is bound to a Sigstore keyless signature and recorded in the public Rekor transparency log. Anyone can verify it without trusting the action runner, the repo owner, or us.
The Rekor entry is queryable directly by log index — a public, append-only, independently operated log:
The GitHub Actions run that produced it, with the downloadable delimit-attestation-29d2fee40bf5ca10 artifact for local verification:
The same verdict can be verified locally with cosign. The certificate identity is the caller workflow that requested the OIDC token — pin to that exact path:
# Download the artifact from the workflow run above, then: cosign verify-blob \ --bundle delimit-attestation-29d2fee40bf5ca10.bundle \ --certificate-identity-regexp "^https://github\.com/delimit-ai/delimit-action-demo/\.github/workflows/.*\.yml@" \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ delimit-attestation-29d2fee40bf5ca10.json
When you run this on your own repo, replace the regexp with the workflow path that invokes delimit-action in your CI (the caller, not the Action). If verify fails, that itself is a finding worth raising — the whole point of a public Rekor entry is that anyone can call us on it.
Why this is the whole loop
A generic "review this PR" prompt can sometimes name a removed field, but it hands you an opinion with no ground truth behind it and nothing a third party can check. Two lines that read as cleanup are exactly where that opinion gets thin.
The merge gate closes the loop instead: it compares the two specs structurally, assigns the delta a change-type, applies a deterministic policy, and then signs the verdict so the answer outlives the CI run. A removed required field on a published API is caught, classified, explained, and — this is the part a review comment can never do — left as a signed, replayable record anyone downstream can verify for themselves.
Reproduce the diff
The classification is reproducible from the two specs — remove the required field from any copy of the schema and re-run:
# Install the CLI npm install -g delimit-cli # Run the merge gate on your before/after spec pair delimit lint /tmp/old.yaml /tmp/new.yaml
The signature and Rekor entry come from the CI run linked above; the classification comes out the same every time you diff the same two specs.
Free detects. Pro proves.
Free Delimit is the merge gate: it catches the removed required field on every pull request and surfaces the break before it ships. Install delimit-cli and run delimit lint <old> <new> against your own specs, or drop the delimit-ai/delimit-action into CI. Free for individual maintainers.
Pro turns the caught verdict into a signed, replayable attestation like the one above: a portable record of what this pull request changed, what was found, and how it was classified — verifiable against the public Rekor log without rerunning the gate or trusting the runner. Pro tier $10/month for teams.