A Committed Build Artifact That No Deploy Step Rebuilds

Some build outputs get committed to the repository on purpose. A compiled stylesheet, a generated API client, a bundled translation file. The pattern works until you notice that the command which regenerates the file lives in one script and the deploy path runs a different one. Then new inputs ship without a rebuild, and nothing tells you.

The shape I ran into: a stylesheet generated by a CSS framework CLI, committed to git, and embedded into a Go binary. The deploy script built a container image and never called the CSS CLI. Any new utility class written after the last local build would have shipped with no rule behind it.

The setup that hides the gap

Three decisions are individually reasonable and jointly dangerous.

First, commit the generated file. This is common for assets that are slow to build or need a toolchain the runtime image does not have. You get reproducible serving and one less dependency in the container.

Second, embed it into the binary. In Go this is //go:embed, and it is the reason the file has to exist at compile time rather than being fetched at boot.

Third, put the regeneration command in a build script that also does other things: signing, notarization, uploading desktop artifacts. That script needs credentials and a VPN, so it is not what CI runs.

Now trace the file. It is produced by script A. It is consumed by the compiler during script B. Nothing in script B checks whether script A ran since the last input change.

Why the failure is silent

The severity depends entirely on how the consumer reacts to a missing entry.

A generated API client that lost a method fails loudly. The call site does not compile, CI goes red, and you fix it before merge. Same for generated database query code and for protobuf stubs. The type system is doing the checking for you.

A stylesheet is the opposite. CSS has no notion of an undefined class. Write class="items-center" with no matching rule and the browser resolves it to nothing. The element renders, the page loads, no console warning appears. You get a layout that is subtly wrong in production and correct on the machine where the CLI last ran.

Translation bundles behave the same way when the lookup falls back to the key. Feature flag manifests behave the same way when an unknown flag reads as false. In each case the missing entry has a plausible default, and a plausible default is what makes the bug quiet.

So the rule of thumb: if the generated artifact is consumed by something with no schema, assume the gap exists until you have checked.

Find out which path rebuilds it

This takes about a minute and the answer is often uncomfortable.

Start by confirming the file is really tracked rather than ignored:

git ls-files --error-unmatch path/to/generated.css
git check-ignore -v path/to/generated.css   # expect no output

Then grep every file that participates in a deploy for the generator command:

grep -rn "tailwindcss\|protoc\|sqlc\|openapi-generator" \
  Dockerfile* .github/workflows/ script/ Makefile cloudbuild.yaml 2>/dev/null

In my case the hits were all in one place, the desktop build script, and there were zero hits in the Dockerfile and zero in the deploy script. That is the whole diagnosis. The container build copies the repository and compiles, so whatever is in git is what ships.

One more check that catches the reverse mistake, where the file is generated during deploy but also committed, which makes every deploy produce a spurious diff:

git status --porcelain path/to/generated.css   # after a local build

If a fresh build dirties the file with no input change, the generator is not deterministic and committing it will fight you forever.

Two ways to close it

Both are valid. Pick based on whether the runtime image can host the toolchain.

Move the generation into the deploy path. This is the honest fix. Add the generator to the container build or to the CI workflow that produces the image, and stop committing the output. The cost is a heavier build image and one more toolchain to pin. If the generator needs credentials or a signing key, this option is closed to you, which is exactly how the gap appeared in the first place.

Keep committing it, and verify the diff is empty. If the artifact must stay in git, then make the invariant explicit: after a normal edit, the generated file must not change. A check that runs the generator and fails on a dirty tree turns a silent gap into a red build:

make generate
git diff --exit-code path/to/generated.css \
  || { echo "generated asset is stale, run make generate and commit"; exit 1; }

Put that in CI, not in a local hook. Local hooks are skipped by the person who is in a hurry, which is the person most likely to add a new class.

There is a weaker variant worth knowing when neither fits. Keep a hard-coded list of the tokens the artifact currently supports and check new source against it. I did this as a review step: extract every class name present in the compiled stylesheet, extract every class attribute added by the change, and diff the two sets. It found that a candidate class was not in the generated set before the code shipped. It is a checklist item rather than a gate, but it is better than trusting memory.

What else has this shape

The pattern is "an artifact that is an input to deploy but an output of something else." Once you name it that way it shows up in a lot of places.

Compiled stylesheets and JS bundles when the framework CLI is not in the runtime image. Generated API clients from an OpenAPI spec, especially when the spec lives in a different repository. Database query code from a SQL generator. Protobuf and gRPC stubs. Compiled translation catalogs. Embedded migration bundles. Vendored dependency trees. Terraform lock files.

The loud ones, where a missing entry breaks compilation, mostly take care of themselves. The quiet ones are worth auditing on purpose: stylesheets, translation catalogs, flag manifests, icon sprite sheets, anything where a lookup miss has a reasonable fallback.

A second signal is worth watching for. If the generated file is also embedded into the binary, the consumer cannot fetch a fresh copy at runtime, so the gap cannot be papered over by a cache purge or a config reload. Embedding removes your escape hatch.

FAQ

Should I just stop committing generated files? It removes this class of bug, but it moves the toolchain into the image and makes the build slower and heavier. For assets that build in seconds with a widely available tool, yes, generate during deploy. For anything that needs credentials, native compilation, or a slow toolchain, committing plus a CI staleness check is a reasonable trade.

Is a pre-commit hook enough? No. Hooks are local, skippable, and easy to lose when someone clones fresh. Use a hook for speed if you like, but the authoritative check has to run where nobody can bypass it.

How do I know a class is actually missing rather than misspelled? Extract the identifiers from the generated artifact and search for the exact token. Escaping bites here: a utility named py-2.5 may appear in compiled CSS as .py-2\.5, and a naive grep for .py-2.5 reports it as missing. Match against the unescaped form, or check for both.

What if the diff check keeps failing on a clean tree? Then the generator is not deterministic. Common causes are an embedded timestamp, a version banner, or file ordering that depends on the filesystem. Pin the version, strip the banner, or sort the inputs before you make the check blocking. A flaky gate gets disabled within a week.

Does this apply to lock files? Partly. Lock files are committed artifacts too, but package managers usually verify them during install, so a stale lock file fails loudly. The risk with lock files is the opposite one: a deploy that regenerates them silently and ships dependency versions nobody reviewed.