Something finally runs the tests
Explore this workspace.
Technical background
There was no test script in any package.json and no workflow ran any of the forty test files here — every one was run by hand, which is not a slow safety net but the absence of one. The suites are grouped into tiers, because they cost three orders of magnitude apart: the fast tier is pure logic, 818 checks in four seconds, and that is what CI gates on alongside typecheck, lint, the consistency checkers and a build. The tier that launches a hidden window stays out on purpose — a gate that goes red for windowing reasons teaches people to ignore it. A missing suite file fails rather than skips, since a test that is renamed and quietly stops running is the exact thing this exists to prevent.
The problem
There was no test script in any package.json, and no workflow ran any of the forty test files in the repository. Every one was run by hand, from memory, when someone thought of it. That is not a slow safety net; it is the absence of one, and it is the real gap behind wanting the project to be more stable.
How it works
The suites are grouped into tiers by what they need, because they cost three orders of magnitude apart. The fast tier needs nothing running — 6 suites, 818 checks, about four seconds — and that is what CI gates on, alongside typecheck, lint, the two consistency checkers and a build. build.yml no longer restates any of it; it calls the gate, and so does the release workflow, so the thing that runs is the same definition and it runs on the ref actually being shipped.
Settings
::::
Details
- A missing suite file fails rather than skips. A test that is renamed and quietly stops running is precisely what this exists to prevent, so it cannot be allowed to look like a pass.
- A fast-tier suite must stay dependency-free. One that quietly starts needing a compiled bundle turns the gate into a liability the first time somebody runs it on a clean checkout.
- A green run on
mainsays nothing about a tag: separate triggers, separate run histories. So the release workflow calls the gate rather than trusting that it passed at some point.
What this does not claim
- The CDP tier is not in CI, and that is a real gap, not a solved problem. Each of those suites launches a window and takes the better part of a minute; a gate that goes red for windowing reasons on a hosted runner teaches people to ignore it. They still have to be run deliberately.
- The same applies to the suites needing a WSL distro or Electron's native ABI.