CI: local verification
Every GitHub Actions job that fires on a PR, a push, or a schedule is gated off and will not run. bun run verify:pr replaced it. The manual and on-demand workflows are deliberately left ungated and are listed under Turning Actions back on; they will still fail if you trigger them.
This is a port of the same change in lacymorrow/shipkit (commit 5021f7ee), filed here as LAC-5101 after two agents read a PR's seven red checks as "green" on LAC-5093. Both readings were defensible, and that was the bug: the red meant nothing.
Why
GitHub Actions billing is a permanent constraint on this account, not an open bug. The decision is no paid GitHub billing, ever. On a private repo with no paid minutes, Actions does not degrade gracefully: every job refuses to start in about three seconds with steps=0, so the workflow reports failure without running a single step.
Measured on this repo on 2026-10-20, before this change:
- 46 failures and 4 successes across the last 100 runs. All four "successes" are Dependabot's internal
dynamicruns, which never claim a runner. Zero real jobs have passed. - A representative CI run: eight jobs, every one
steps=0, the whole run start to finish in 3 seconds. - All 7 open PRs carried red checks that said nothing about the code.
- The daily contact-form health check has reported shipkit.io's contact form as FAILED every day regardless of what the site was doing.
That is worse than having no CI. A red check that is always red trains everyone to ignore checks.
The constraint that shaped the fix
This repo has no branch protection at all. On a free plan, both endpoints refuse:
GET /repos/lacymorrow/shipkit-www/branches/main/protection -> 403
GET /repos/lacymorrow/shipkit-www/rulesets -> 403
"Upgrade to GitHub Pro or make this repository public to enable this feature."
So there are no required status checks, and there never were. Merges already happen with CI fully red.
That rules out the expensive options. A self-hosted runner would produce a trustworthy, GitHub-orchestrated check, but with no branch protection that check still cannot be required, so its enforcement value over a locally published status is zero. It would cost arbitrary PR code executing on a laptop, fork-PR gating discipline, and checks that hang pending whenever the machine is asleep. Paying a real security cost for zero enforcement gain is a bad trade.
Making the repo public would make minutes free, but this repo is a downstream of the premium shipkit framework and carries that code, so going public is permanently unavailable here.
So the goal is not the strongest gate. On this plan every check is advisory no matter what. The goal is an honest signal for the least cost.
How it works
bun run verify:pr runs the checks locally and publishes the result to the PR head commit through the GitHub commit Statuses API. That API needs no runner and burns no Actions minutes.
git push # the status binds to a pushed commit
bun run verify:pr # full run: build + route smoke
bun run verify:pr --no-build # faster; the status lists what was skipped
The PR then carries a local-verify check, plus one comment holding the full result table. The check's "Details" link points at that comment.
What it refuses to do
Two guards are the difference between a real signal and theater:
- Clean tree required. If tracked files differ from
HEAD, it refuses. A status naming a commit while testing uncommitted code is a lie.--allow-dirtyoverrides, and then the status and the comment both say[dirty tree], so the deviation is visible rather than hidden. Harness-local.claude/config is exempt because it cannot affect a build. HEADmust be the PR head. If you have not pushed, it refuses rather than certifying a commit GitHub has never seen.
A status is bound to one commit, so re-run it after every push. An older green check does not carry forward.
If the run is killed partway through (session teardown, Ctrl-C, an OOM on a long build), an EXIT trap lands an error state saying so. A pending status that never resolves would be worse than a red one: it reads as "still running" forever, and waiting never fixes it. An error here means the run died, not that the code is bad. Re-run it.
What a green local-verify actually means
The checks named in the PR comment passed on someone's machine, against that exact commit, with a clean tree. That is the whole claim.
Read the limits honestly:
- It is self-reported. Anyone with push access can post one by hand without running anything. The threat model that makes this acceptable: push access to this repo is the owner and the owner's agents, who are already trusted. Fork PRs cannot post a status at all because they have no token, which is the correct outcome rather than a gap.
- It is advisory. With no branch protection, nothing can be required. The check tells you the work was verified. It does not stop a merge.
- It is not complete.
verify.shprints what it did not cover, and you should read that line before trusting a pass. - It inherits the machine it ran on. This is the structural weakness of a local gate and it has no clean fix. A clean container runs the repo; your machine runs the repo plus whatever else is in the directory. Untracked and gitignored files can both cause failures the repo does not have and mask failures it does. Practical mitigations, in order of strength: run from a clean
git worktreefor anything load-bearing, checkgit status --porcelain --ignored=matchingwhen a failure looks unrelated to your diff, and distrust a failure whose file path you do not recognize before you distrust your own change.
Not covered locally
These gated jobs needed infrastructure the script does not stand up:
| Gated job | Why not local |
|---|---|
| Integration tests | needs a real Postgres service |
| Playwright e2e | needs a browser matrix against a production build |
| Docs link check | needs a Holocron build |
| Deployment Check smoke | hits the deployed Vercel URL, not a local server |
| Contact form health check | hits https://shipkit.io on a daily cron |
The last two leave a real gap, and a wider one here than upstream, because this repo is what deploys shipkit.io: nothing currently checks the live site. Do not read a green local-verify as evidence that production is up. Tracked in LAC-4955; LAC-3633 holds the Vercel-cron recipe for re-pointing /api/health/contact at a scheduler that is not GitHub Actions.
Turning Actions back on
Nothing was deleted. Every gated job carries one guard:
if: ${{ vars.ACTIONS_BILLING_OK == 'true' }}
A job whose if is false is never dispatched to a runner, so it costs nothing and renders neutral instead of red. Set the repo variable ACTIONS_BILLING_OK=true and the entire suite turns back on, minutes policy intact: cheap tier (typecheck, lint, unit) on every PR, expensive tier on pushes to main or PRs labeled ci:full.
Fully gated: ci.yml, gitleaks.yml, registry.yml, codeql.yml, deployment-check.yml, db-neon-branch-on-pr.yaml.
Scheduled firing gated, manual dispatch still allowed: contact-form-healthcheck.yml and sync-upstream.yml. Both are cron + workflow_dispatch here, which is a divergence from upstream shipkit where sync-upstream.yml is dispatch-only. The cron is what produced a red run nobody asked for; a dispatch is a deliberate act, so it is left alone and fails at the billing layer if you use it. Their guard reads:
if: ${{ vars.ACTIONS_BILLING_OK == 'true' || github.event_name == 'workflow_dispatch' }}
Not gated at all, on purpose: deploy-to-production.yml and init-upstream.yml are workflow_dispatch only, and claude.yml fires only on an explicit @claude mention. None of them fire on their own, so none of them produce a red check, and gating them would only hide a manual action you deliberately took. They do still need Actions minutes, so they fail if you trigger them. That includes the @claude PR-review path, so there is no working bot reviewer on this repo: agent or human review is the only review.
Because gitleaks.yml and registry.yml are off, their checks moved into verify.sh: a gitleaks scan of the branch delta against main, plus the registry check. Those run on plain bun run verify too.
Existing PRs stay red
Gating is a change to the workflow files, and GitHub evaluates a workflow from the version on the PR's head, not from main. So every PR opened before this lands keeps its red checks until it is rebased or merged past this commit. That is expected, not a leftover failure. Rebase the PR, or push to it, and the next run skips instead.
Route per repo
| Repo class | Route |
|---|---|
| shipkit-www, and every private repo on this account | local pre-merge gates, this page |
| bones and other already-public repos | public repo, Actions minutes are free there, nothing to do |
| any repo | self-hosted runners rejected, see above |