Cutting Secrets Manager Cost with Caching and Consolidation

A managed secrets manager charges you on two axes: the number of secrets you store, and the number of API calls you make to read them. Most of a surprising bill comes from the second axis, and most of that is avoidable. If a process fetches its secrets on every request, or on every cold start of a short-lived worker, the call count grows with traffic instead of staying flat. Fetch each secret once at startup, hold the value in process memory, consolidate related secrets into a single document, and the same protection costs a fraction to run. This post shows the access patterns that leak money and the two changes that stop the leak.

How a managed secrets manager bills you

A managed secrets manager is priced like a small metered database, not like an environment variable. Two things drive the bill.

The first is storage. You pay a monthly rate for each distinct secret you keep, often prorated by how long it exists in the billing period. Ten secrets cost ten times the storage line of one secret, whether or not anyone reads them.

The second is access. Every API call that reads or lists a secret is metered, usually counted in blocks of ten thousand calls. A call is a call whether the value changed or not, and whether you read it a moment ago or not. The provider has no idea that your last thousand reads all returned the same bytes. It bills each one.

Neither axis is expensive on its own. A single secret read once is close to free. The bill grows because naive access patterns turn a handful of secrets into millions of calls, and a handful of logical values into dozens of stored entries. Both are pattern problems, and both have direct fixes.

Where the cost leaks

Three habits account for most oversized secrets bills, and all three feel reasonable when you write them.

The first is fetching inside the request path. A handler needs a database password, so it reads the secret when the request arrives. That looks clean and local, but it ties your call count directly to your traffic. At a thousand requests per second, one secret read per request is a thousand metered calls per second, which is close to two and a half billion calls a month for a value that never changed.

The second is fetching on every cold start of a short-lived process. Serverless functions and job runners spin up, do one unit of work, and exit. If each instance reads its secrets on boot, a busy function that scales to thousands of concurrent instances and recycles them constantly turns startup reads into a steady, high call rate. The value is stable, but the process lifetime is short, so the read repeats endlessly.

The third is splitting one logical configuration into many stored secrets. A service needs a database URL, an API token, a signing key, and a webhook URL, so you store four secrets. Multiply that by a few services and a few environments and the stored-secret count climbs into the dozens. Now you pay storage on every one of them, and any code that reads the full set makes one call per secret.

None of these is a security flaw. They are access-pattern flaws, and you fix them without weakening anything.

Fetch once at startup, then cache in memory

The core move is to treat a secret the way you treat any expensive value that rarely changes: read it once, keep it, and reuse it. Load every secret the process needs during startup, store the values in process memory, and let the hot path read from memory instead of the network. The request handler never calls the secrets manager at all. It reads a field.

This collapses the access axis from "one call per request" to "one call per secret per process start." A server that boots once and runs for days makes a few reads at boot and then zero for the rest of its life, no matter how much traffic it serves.

Here is the pattern in Go. A small config struct holds the resolved values, a loader fetches each secret once, and the rest of the program takes the struct by reference and reads fields.

// Secrets holds resolved values in process memory. Fetched once at startup,
// read from memory on every request after that.
type Secrets struct {
	DatabaseURL string
	APIToken    string
	SigningKey  []byte
}

// Load fetches every secret the process needs, one call each, at startup.
// If any required secret is missing, the process fails fast instead of
// discovering the gap on the first request.
func Load(ctx context.Context, sm SecretClient) (*Secrets, error) {
	dbURL, err := sm.Get(ctx, "database-url")
	if err != nil {
		return nil, fmt.Errorf("load database-url: %w", err)
	}
	token, err := sm.Get(ctx, "api-token")
	if err != nil {
		return nil, fmt.Errorf("load api-token: %w", err)
	}
	key, err := sm.Get(ctx, "signing-key")
	if err != nil {
		return nil, fmt.Errorf("load signing-key: %w", err)
	}
	return &Secrets{
		DatabaseURL: dbURL,
		APIToken:    token,
		SigningKey:  []byte(key),
	}, nil
}

func main() {
	ctx := context.Background()
	secrets, err := Load(ctx, newSecretClient())
	if err != nil {
		log.Fatalf("startup: %v", err)
	}
	// Handlers close over secrets and read fields. No network call per request.
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		use(secrets.APIToken)
	})
	log.Fatal(http.ListenAndServe(":8080", nil))
}

Two properties make this safe as well as cheap. Loading at startup means a missing or malformed secret crashes the process immediately, at deploy time, instead of failing the first user request an hour later. And because the value lives only in memory, it never touches disk and disappears when the process exits.

For short-lived workers the same idea applies with one adjustment. A function that handles many invocations per instance should load secrets once per instance, in the initialization code that runs before the handler, not inside the handler body. Most function runtimes keep a warm instance alive across invocations, so a per-instance load amortizes across every invocation that instance serves. The read count drops from once per invocation to once per instance lifetime.

Consolidate related secrets into one document

The storage axis and part of the access axis shrink together when you stop splitting one configuration across many secrets. A secrets manager stores opaque bytes. Nothing stops those bytes from being a JSON object that holds several related values at once.

Instead of four secrets named database-url, api-token, signing-key, and webhook-url, store one secret named service-config whose value is a small JSON document.

{
  "database_url": "postgres://user:pass@host:5432/app",
  "api_token": "tok_live_abc123",
  "signing_key": "base64keymaterial",
  "webhook_url": "https://hooks.example.com/abc"
}

The loader now makes one call, parses the JSON, and fills the same struct.

func LoadBundle(ctx context.Context, sm SecretClient) (*Secrets, error) {
	raw, err := sm.Get(ctx, "service-config")
	if err != nil {
		return nil, fmt.Errorf("load service-config: %w", err)
	}
	var b struct {
		DatabaseURL string `json:"database_url"`
		APIToken    string `json:"api_token"`
		SigningKey  string `json:"signing_key"`
	}
	if err := json.Unmarshal([]byte(raw), &b); err != nil {
		return nil, fmt.Errorf("parse service-config: %w", err)
	}
	return &Secrets{
		DatabaseURL: b.DatabaseURL,
		APIToken:    b.APIToken,
		SigningKey:  []byte(b.SigningKey),
	}, nil
}

This cuts the stored-secret count from four to one, so the storage line drops by roughly three quarters for that service. It also cuts startup reads from four calls to one, which matters most for the short-lived workers that read on every instance. The two changes stack. Consolidation lowers the per-start cost, and caching makes each start rare.

Group by trust boundary and by rotation cadence, not by how many keys happen to exist. Values that belong to the same service, that the same set of principals may read, and that tend to change together belong in one document. Values with genuinely different access rules or independent rotation schedules stay separate, because merging them would force a coarser grant or a clumsier rotation. Consolidation is about the values that naturally travel together.

The billing axes, side by side

The table makes the leak and the fix concrete. It compares the naive pattern (four separate secrets, read once per request) against the fixed pattern (one consolidated secret, fetched once at startup) for a service holding four logical values and serving a thousand requests per second.

The access row is where the money is. Moving reads out of the request path and into startup changes the call count from a function of traffic to a function of deploy frequency. Traffic can be millions of requests an hour. Deploys are a handful a day. That is the whole difference between a large bill and a rounding error.

Handling rotation when the value is cached

Caching a secret in memory raises one fair objection. If you rotate a secret, a process that cached the old value keeps using it until something makes it reload. The cache trades freshness for cost, and rotation is where that trade shows up. There are three ways to keep rotation working, and you can combine them.

The simplest is a restart. Most rotation workflows already redeploy or recycle the affected services, and a fresh process loads the new value at startup by definition. If your rotation runbook ends with a rolling restart, cached secrets refresh for free and you need nothing else.

The second is a bounded cache lifetime. Instead of holding a value forever, attach a time to live and reload when it expires. A one hour lifetime means a rotated secret is picked up within an hour without any restart, at the cost of one extra read per secret per hour per process. That is still a tiny call count compared to per-request reads, and it caps how stale a value can get. Pick the lifetime from how fast a rotation must propagate, not from habit.

The third is an explicit signal. Have the rotation step notify running processes to reload, through a message channel, a config-reload endpoint, or a POSIX signal that the process traps to re-run its loader. This gives near-immediate propagation with no polling, at the cost of wiring up the notification path. Use it when rotations must take effect in seconds rather than minutes.

For most services the first two are enough. Rotations are infrequent and restarts are routine, so a restart-driven refresh plus a modest time to live covers the common case without building a notification system.

What to keep out of the secrets manager

Part of the bill comes from storing things that are not secrets. A secrets manager is for values that would cause harm if disclosed: passwords, private keys, tokens, connection strings with embedded credentials. It is not a general configuration store, and using it as one inflates both the stored-secret count and the read count.

Non-sensitive configuration belongs in plain environment variables or a config file baked at build time. A feature flag, a region name, a page-size limit, a public base URL, a log level: none of these needs protection, and none of them should occupy a metered secret slot. Move them to environment variables and the secrets manager holds only what actually needs guarding, which shrinks storage and removes reads you were paying for.

A useful test: if a value appearing in a build log or a stack trace would be a security incident, it is a secret. If it would be merely untidy, it is configuration. Store each in the place priced for it.

The tradeoffs, stated plainly

These techniques change cost, not security, but they do have tradeoffs worth naming before you adopt them.

An in-memory cache means each process holds its own copy of a secret for the life of that process, and a rotated value is not seen until the cache expires, the process restarts, or a reload signal arrives. On a service with long-lived processes and infrequent rotation, that lag is harmless. On a system that rotates secrets every few minutes and expects instant propagation everywhere, plan the reload path deliberately rather than relying on restarts.

Consolidating into one JSON document means you lose per-value rotation granularity. Rotating any field means writing a new version of the whole document, and every reader picks up all fields together. If two values genuinely need independent rotation schedules or different access grants, keep them apart. Consolidation is for values that share a trust boundary and a rotation rhythm, which is most of them, but not all.

The underlying idea is small. A managed secrets manager is safe storage for sensitive bytes, not a low-latency store you query on every request. Once you treat it that way, reading rarely, grouping what belongs together, and keeping non-secrets out, the security stays exactly where it was and the bill mostly disappears.