Stopping Secrets Before They Get Committed to Git History

The moment an API key, a token, a password, or a private certificate lands in a commit, you have a problem that deletion cannot fix. Git history is durable by design. A force-removed secret still lives in every clone, every fork, every CI cache, and every editor that fetched the branch before you noticed. The value has to be treated as burned, which means rotation, not cleanup. So the real defense is not detecting a leaked secret after the fact. It is stopping the commit before the secret ever enters history. This post shows how to build that block in layers, why one layer is never enough, and what to do on the day a secret gets through anyway.

Why deleting a committed secret is already too late

A committed secret is a published secret. Between the push and the moment you notice, the value can be pulled by anyone with read access, mirrored by automated forks, and indexed by scanners that crawl public and private hosts alike. Rewriting history removes the value from the tip of your branch, but it does not reach the copies that already left your machine.

There is a second reason cleanup fails. Rewriting history changes commit hashes for every commit after the one you edited, which forces every collaborator to reset their local branches and breaks open pull requests. You pay a real operational cost, and the secret is still compromised, because you cannot prove no one copied it during the exposure window.

The correct mental model is simple. Once a secret reaches a shared history, its value is dead. The only safe response is to rotate it at the source and invalidate the old one. Everything else, including the history rewrite, is housekeeping that happens after the value is already worthless. That is why the effort belongs upstream, at the commit boundary, where the secret can still be kept out of history entirely.

The layered defense, at a glance

No single check catches everything. Local hooks are fast but can be skipped. Server scans are enforced but run after the push. File rules prevent whole classes of mistakes but cannot read intent. Each layer covers a gap the others leave open, so you run them together.

Read the table top to bottom as a funnel. The pre-commit hook stops the common case cheaply and early. The CI scan is the net under the hook, for the developer who ran git commit --no-verify or never installed the hook at all. The file rules shrink the surface both scanners have to reason about, the automation guard covers commits no human reviews, and rotation is the last layer, the one you hope never runs.

Blocking the commit with a pre-commit hook

The cheapest place to stop a secret is the developer's own machine, before the commit object exists. A pre-commit hook runs against the staged change, and if it finds something that looks like a credential, it exits non-zero and the commit never happens. Because it inspects only what is staged, it is fast enough to run on every commit without anyone noticing the delay.

Three signals catch most real leaks. First, the filename: files like .env, .pem, .key, .p12, and .pfx almost never belong in a repository. Second, known credential formats: many providers issue keys with a fixed prefix and length, and private keys carry a recognizable header line. Third, high entropy: a long run of random-looking characters assigned to a variable named token or password is a strong hint even when it matches no known format.

#!/usr/bin/env bash
# .git/hooks/pre-commit (or the equivalent managed by a hook framework)
set -euo pipefail

# Patterns for well-known credential shapes and private key headers.
SECRET_PATTERNS='(-----BEGIN [A-Z ]*PRIVATE KEY-----)|(secret_[A-Za-z0-9]{24,})|([A-Za-z0-9+/]{40,}={0,2})'

# Inspect only what is staged, and only added or changed lines.
staged=$(git diff --cached --name-only --diff-filter=ACM)
fail=0

for file in $staged; do
  # 1. Reject credential-bearing filenames outright.
  case "$file" in
    *.pem|*.key|*.p12|*.pfx|.env|.env.*)
      echo "blocked: $file looks like a credential file"
      fail=1
      continue
      ;;
  esac

  # 2. Scan the staged content of the file for secret-shaped values.
  if git show ":$file" | grep -nEq "$SECRET_PATTERNS"; then
    echo "blocked: $file contains a value matching a secret pattern"
    fail=1
  fi
done

if [ "$fail" -ne 0 ]; then
  echo "commit rejected. remove the secret, or add an allowlist entry if this is a false positive."
  exit 1
fi

Two details matter more than they look. The hook reads git show ":$file", which is the staged version of the content, not the working copy. That prevents a race where a developer unstages a fix but the working file still looks clean. And it filters with --diff-filter=ACM so deletions do not trigger the scan, since removing a file is never how a secret enters history.

Entropy is the pattern set's weak spot, because a base64 blob of legitimate binary or a long hash can look exactly like a key. The regex above uses a conservative length floor to keep the noise down, and the next sections cover the two things that make entropy usable in practice: a CI net for what it misses, and an allowlist for what it wrongly flags.

A second gate in CI

A local hook has one fatal weakness. It lives on the developer's machine, which means it can be skipped, uninstalled, or simply never set up on a fresh clone. git commit --no-verify bypasses it in one flag. Any defense that a single flag disables is not a control, it is a suggestion. So the same scan has to run again where no one can opt out: in CI, on every push.

The CI scan does more than repeat the hook. It scans the full diff of the push, including commits made on machines that never had the hook, and it can scan history depth on the first run to catch secrets committed before the control existed. Because it runs on the server, its result is authoritative. A failing scan blocks the merge, and no local flag can wave it through.

# A minimal CI job that fails the pipeline on a detected secret.
scan-secrets:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0          # full history, so the scan sees every commit in the push
    - name: scan for secrets
      run: |
        # Scan the range that this push introduced, not just the tip.
        base="${BEFORE_SHA:-HEAD~1}"
        if git diff "$base"..HEAD | grep -nE "$SECRET_PATTERNS"; then
          echo "secret detected in pushed commits. rotate the value and clean history."
          exit 1
        fi

The division of labor is the point. The hook is the fast path that keeps almost every secret out of history on the developer's machine, where fixing it costs nothing more than editing a file. The CI scan is the enforced path that assumes the hook was skipped and refuses to let a secret-bearing push become a merge. One is convenient, the other is authoritative, and you want both.

There is an important asymmetry when the CI scan fires. By the time it runs, the secret has already been pushed, which means it may already be exposed. So a CI catch is not a clean save. It is a signal to start the rotation procedure, treat the value as compromised, and then clean the history.

File and path rules that reduce false positives

Scanners get better when they have less to guess about. Two file-level rules do most of that work. The first is a strict .gitignore that keeps whole categories of secret files out of the staging area to begin with. If .env, key files, and local credential caches are ignored, a developer cannot accidentally git add them, and the scanners never have to reason about their contents.

# Never track local secret material.
.env
.env.*
!.env.example
*.pem
*.key
*.p12
*.pfx
credentials.json

The !.env.example line is worth calling out. You want a committed template that shows which variables exist, with placeholder values, so a scanner allowlist can carve out exactly that file. It documents the shape of the configuration without carrying a single real value.

The second rule is an explicit allowlist for the scanner itself, because false positives are inevitable. Test fixtures contain fake keys on purpose. Documentation shows an example token. A vendored dependency ships a sample certificate. Without a way to mark these as known-safe, every one of them blocks a legitimate commit, and developers learn to reach for --no-verify, which defeats the whole system. The allowlist should be narrow and reviewable, pinned to a specific path or a specific line, never a blanket "ignore this pattern everywhere."

# Scanner allowlist: narrow, path-scoped, reviewed in pull requests.
allowlist:
  - path: testdata/fake_keys.txt      # deliberately fake fixtures
  - path: docs/configuration.md       # example token in prose
    reason: "documented placeholder, not a live credential"

The discipline here is that every allowlist entry is a small, explicit exception with a reason, reviewed like any other change. A pattern-level mute is a hole you will forget you dug. A path-level exception with a stated reason is auditable, and someone reviewing the pull request can see exactly what was excused and why.

Defending automated commits

The most dangerous commits are the ones no human looks at. A bot that opens dependency-update pull requests, a script that regenerates a config file and commits the result, a release job that writes a build manifest: any of these can stage a secret without a person in the loop to notice the red flag in the diff. Automation moves fast and reviews nothing, so it needs the block wired directly into its own path.

The rule is that automated commit paths run the same scan the humans do, and treat a hit as a hard stop rather than a warning to log. If a bot's scan finds a secret in what it is about to commit, the commit must not happen. The bot fails its job loudly, so a human investigates, rather than quietly committing the secret and moving on. It is the same fail-closed posture as the human hook, applied where the stakes are higher because there is no reviewer as a backstop.

This is also where entropy checks earn their keep. A human writing code rarely pastes a real 40-character random string by accident in a way that surprises them. Automation that templates config files, on the other hand, can pull a live value from an environment variable and write it straight into a tracked file. The generic high-entropy check catches exactly that class of mistake, the one where the value matches no known provider format but is unmistakably a secret by its shape and its variable name.

When a secret does leak, rotate first

Sooner or later a secret gets through, and the order of your response decides how much damage it does. The instinct is to scrub the history immediately so the value disappears. That instinct is wrong, because the value is already out and the history rewrite does nothing to the copies that left your machine. The first move is always rotation.

1. Rotate or revoke the exposed value at its source, before anything else. Generate a replacement at the provider and invalidate the old one. The instant a secret is public, its value is dead, so kill it first.
2. Confirm the old value no longer works. Try to authenticate with it and verify the call is rejected. Until you have seen it fail, you have not actually revoked it.
3. Deploy the new value wherever the service needs it, through your normal secret store, so the system keeps running on the replacement.
4. Only now clean the history. Rewrite the commits that carried the secret and force-push, understanding that clones, forks, and caches still hold the old value. This step is hygiene, not containment.
5. Check the access logs for use of the old value between exposure and revocation. If the credential grants access to real data, assume it may have been used and investigate accordingly.

The ordering is the entire lesson. Rotation contains the incident. History cleanup only tidies up afterward. A team that rewrites history first and rotates second spends its fastest minutes on the step that changes nothing, while the live, compromised credential keeps working for whoever already copied it.

The false positive tradeoff, and why fail-closed wins

Every secret scanner faces the same tension. Tighten the patterns and you miss real secrets. Loosen them and you block legitimate commits with false alarms. There is no setting that eliminates both, so you have to choose which error you prefer, and the choice is not symmetric.

A false positive is an annoyance. A developer sees a blocked commit, checks the flagged line, confirms it is a test fixture or a documented placeholder, and adds a narrow allowlist entry. The cost is a few minutes and a reviewed exception. A false negative is a breach. A real credential slips through, reaches a shared history, and now you are rotating secrets and reading access logs under pressure. The two outcomes are not in the same league, which is why the scanner should fail closed: when in doubt, block, and let a human confirm the exception.

That choice only stays workable if the false positive path is genuinely cheap, which is why the allowlist matters so much. If clearing a false alarm is painful, developers route around the scanner with --no-verify, and a bypassed control protects nothing. So you keep the patterns aggressive enough to catch real secrets, and you invest in making exceptions fast, narrow, and reviewable. Fail closed on detection, stay cheap on exceptions, and you get a system that developers keep using instead of fighting.

Put the layers together and the shape is clear. The pre-commit hook keeps secrets out of history on the developer's machine. The CI scan enforces the same rule where no flag can skip it. File and allowlist rules shrink the guesswork and keep the false positive tax low. The automation guard covers the commits no one reviews. And rotation stands ready for the day something gets through, because the honest assumption behind the whole design is that eventually something will, and a committed secret is dead the moment it lands.