The PR we shipped that broke our spec
— and the gate that caught it
A real signed, replayable attestation, end to end.
On 2026-04-25 we opened a public PR on delimit-ai/delimit-action-demo with two contract changes on purpose. The merge gate flagged both, signed the verdict via Sigstore, and recorded the entry in the public Rekor transparency log.
- We wrote the spec changes intentionally to exercise the signing flow end to end.
- The PR, the signature, the Rekor entry are all real and reproducible from the same two commits.
- Intentional breakage makes the transparency easier, not weaker.
What we did
Opened PR #3 on delimit-ai/delimit-action-demo. The diff is five lines of OpenAPI plus one workflow line.
CI ran delimit-action@v1.11.0 against the baseline and the new spec. The diff engine classified each change against the 27-type taxonomy, the semver classifier walked the partition, and the attestation engine signed the verdict via Sigstore keyless. Every artifact below is mechanical output of that run.
The diff
Five lines of OpenAPI change plus one workflow permission. The diff engine classified one of the OpenAPI lines as breaking (required-field added) and one as warning (type changed). The workflow line just lets the action sign the attestation.
# openapi.yaml
schemas:
User:
type: object
- required: [id, email, name]
+ required: [id, email, name, organization_id]
properties:
id:
- type: string
- maxLength: 36
+ type: integer
email:
type: string
format: email
@@ name:
minLength: 1
maxLength: 100
+ organization_id:
+ type: string
# .github/workflows/api-check.yml
permissions:
contents: read
pull-requests: write
+ id-token: write # enables Sigstore-signed attestationWhat the gate produced
The PR comment that landed under the diff classified each change, named the affected surface, offered a migration note, walked four pre-merge checks (lint, policy, security, deploy-readiness), and ended with the signed-attestation footer. The full comment is on the PR; the structured findings are reproduced below.
Findings
Each finding cites the exact change-type from the 27-type taxonomy, the OpenAPI surface affected, and the consumer impact a downstream integrator would hit.
- breakingfinding F1change type: required_field_addedsurface:
#/components/schemas/User.organization_idA new required field organization_id was added to the User schema. Every existing consumer that constructs a User object will fail validation against the new contract because the new required field is absent from their payload. This is the classic silent-break shape: the spec adds one line, the consumer tooling reports nothing, and the first signal anyone gets is a 422 at the integration boundary in production. A pre-merge gate exists to catch exactly this asymmetry before the spec ships.
- warningfinding F2change type: type_changed (string → integer)surface:
#/components/schemas/User.idThe User.id field changed type from string to integer. A consumer holding a stored User.id as a string (database column, cached object, URL path segment) will not deserialize against the new integer contract without an explicit migration. The diff engine flags this as a warning rather than a hard breaking change because some clients tolerate both shapes; the migration burden is real but the surface area is narrower than a required-field addition.
The signed attestation
The verdict above is bound to a Sigstore-signed bundle and recorded in the public Rekor transparency log. Anyone can verify it without trusting the action runner, the repo owner, or us. The attestation ID, Rekor log index, repo, PR number, and commit hash are encoded in the permalink:
The Rekor entry is queryable directly by log index:
And the same verdict can be verified locally with cosign. The certificate identity is the caller workflow that requested the OIDC token — for this attestation, that is delimit-ai/delimit-action-demo/.github/workflows/api-check.yml, not the Action's own repo. Pin to that exact path:
# Verify the signed attestation locally cosign verify-blob \ --bundle attestation-10d658b04e8e3bd2.bundle \ --certificate-identity-regexp "^https://github\.com/delimit-ai/delimit-action-demo/\.github/workflows/api-check\.yml@" \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ attestation-10d658b04e8e3bd2.json
When you run this on your own repo, replace the regexp with the workflow path that invokes delimit-action in your CI (the caller, not this Action). The bundle and JSON are downloadable from the attestation page above. If verify fails, that itself is a finding worth raising — the whole point of a public Rekor entry is that anyone can call us on it.
What this report is not
Not a demonstration that we caught a surprise. We wrote the breakage. The narrative discipline matters: a worked-example report that pretends to be a discovery when it is a dogfood is a credibility tax we are not paying. The point of this artifact is the signing and replay chain, not the detection difficulty.
Not an enforcement claim either. The PR ran in advisory mode (CI did not fail). Each consumer of a Delimit attestation decides whether to block a merge or just record the verdict; both modes produce the same signed bytes.
Reproduce locally
Anyone can re-run the diff against the same two commits and get the same classification:
# Install the CLI npm install -g delimit-cli # Clone the repo git clone https://github.com/delimit-ai/delimit-action-demo cd delimit-action-demo # Extract the two specs git show main:openapi.yaml > /tmp/old.yaml git show feat/dogfood-v1.11.0-attestation:openapi.yaml > /tmp/new.yaml # Run the merge gate delimit lint /tmp/old.yaml /tmp/new.yaml
If the bytes you get differ from the bytes in this report, that is itself a finding worth reporting; raise it on the Delimit repo and we will look.
For your own API surface
If you ship a public API and want pre-merge signed attestations in your CI, install delimit-cli and run delimit lint <old> <new> against your own specs. The GitHub Action is on the Marketplace at delimit-ai/delimit-action. Free for individual maintainers. Pro tier $10/month for teams.
The signed, replayable attestation is the artifact your reviewers, auditors, or downstream consumers can read without rerunning the gate.