Which AI Agent Commands Do You Actually Use? Audit Them

Every custom command you write for a coding agent starts as a good idea. Two dozen of them later, the folder is a museum. I counted actual invocations across a few thousand session transcripts and found that half my commands had not been called once in two months. The cleanup was easy. Reading the numbers correctly was not.

Commands accumulate, usage does not

An agent command is a packaged instruction set: a folder with a definition file that the agent loads when you type a slash name. Writing one costs an afternoon. Retiring one costs nothing, so nobody does it.

The pile is not just clutter. Each definition contributes its name and description to the agent's context on every session, and a crowded namespace makes the model pick the wrong tool. But the real cost showed up as a different shape. My 26 commands were not 26 of the same thing. They were three different kinds of object sitting in one flat folder.

Count from session logs, not memory

Agents write conversation transcripts to disk, usually one JSON-per-line file per session. Every tool call the model made is in there as a structured block. That file tree is a usage database nobody thinks to query.

The invocation of a command appears as a tool-use block with the command name in its input. One grep over the whole tree gives you a ranked list:

# adjust the JSON shape to match your agent's transcript format
rg -o -I '"name":"<ToolName>","input":\{"<field>":"([^"]+)"' -r '$1' \
  --glob '*.jsonl' "$TRANSCRIPT_DIR" | sort | uniq -c | sort -rn

Match the block, not the bare name. A loose grep for the command string also catches every time the name was merely mentioned in prose, which inflates rarely-used commands the most. That is exactly backwards from what you want.

The single most important step is splitting the count by time. Run it twice, once over everything and once over files touched in the last 60 days:

find "$TRANSCRIPT_DIR" -name '*.jsonl' -mtime -60 > recent.txt
rg -o -I '"name":"<ToolName>","input":\{"<field>":"([^"]+)"' -r '$1' $(cat recent.txt) \
  | sort | uniq -c | sort -rn

Lifetime totals lie. They flatten a command you used heavily last spring and abandoned in June into the same bucket as one you ran this morning. In my data the two columns disagreed on roughly a third of the list. One command had a few hundred lifetime calls and zero recent ones. Another had barely a dozen lifetime calls and was still in weekly use.

Zero calls does not always mean dead

Three of my commands showed near-zero invocations and were the most load-bearing files in the whole set.

They were the body of a larger pipeline command. That pipeline does not invoke them as tools. It reads their definition files off disk and follows the steps inside. From the logs, those three look abandoned. Delete them and the pipeline breaks in the middle of a run.

This is a general trap. Usage telemetry only sees the entry points your instrumentation knows about. Composition through file reads, shell-outs, or documentation references is invisible to it. Before you act on a zero, grep the rest of your tooling for the file path, not just the command name:

# does anything else read the definition file directly?
rg -n '<commands-dir>/[a-z-]+/<definition-file>' .

That one command turned a delete list into a keep list. The correct read of "hundreds of direct calls last year, zero this quarter" was not that the command died. It was that a newer pipeline had absorbed it as an entry point, and the underlying steps still ran on every invocation of the parent.

So the three kinds in that flat folder were: things I call directly, things the pipeline reads, and things nobody touches. Only the third group is a cleanup candidate. For the second group I changed their descriptions to say the pipeline is the intended entry point, and left the files exactly where they were, because the pipeline hardcodes those paths.

Deactivate by depth, not by deletion

For the 13 dormant ones, deleting felt wrong. They work. They are documented. I might want one back in six months.

Most agents discover commands by scanning a directory exactly one level deep: for each child directory, check for a definition file. There is no recursion. Read your own agent's loader rather than trusting the docs. In the one I checked the behavior was unambiguous. If a directory has no definition file of its own, its children are never examined.

That gives you a free deactivation mechanism. Move the folder one level deeper:

mkdir -p commands/_attic
git mv commands/old-command commands/_attic/old-command

The definition now sits at depth two. The scanner never reaches it. The command disappears from the menu and stops consuming context, and every byte of it is still on disk and still in version control. Restoring is one git mv back.

Note what actually does the work here. It is not the underscore in the folder name, and it is not a naming convention the tool promises to honor. It is the depth change. Verify the scan behavior for your own agent before relying on this, because a loader that globs definition files recursively would silently keep every archived command alive. Two reviewers flagged exactly that risk for me, and only reading the loader settled it.

One more thing fell out of reading that code. The scanner also accepts symlinks as commands. I had a symlink pointing at another command folder as a language alias, which meant that command was being registered twice, every session. The alias was already handled by the description text. The symlink was pure duplication, invisible until I read how discovery worked.

What broke during the cleanup

The moves were trivial. The documentation was where things went wrong, and the same defect surfaced three times in different files.

Line numbers shift as you edit. My plan listed edits as absolute line ranges, top to bottom. Delete a line at 496 and every later range is off by one. In one file the target section header ended up one line above where the plan said to start, so the delete took the body and left the heading orphaned under a horizontal rule. Then the identical bug turned up in a second file, and then in a third with nine ranges in it. The fix is boring and absolute: edit from the bottom of the file upward, or anchor on strings instead of numbers.

Verification scope has to match edit scope. I wrote a check for zero stale references across the repo, then listed only four files to edit. A fifth file had a reference. The check was guaranteed to fail before I ran it. If your acceptance criterion sweeps a directory, every file in that directory is in scope, or the criterion needs an explicit exclusion list.

Do not update paths inside archived items. The archived definitions contain their own old paths. Rewriting those to the new archive location looks tidy and is wrong: restoring the folder moves it back to the original path, and the rewritten references would then point at nothing. Stale-looking paths inside an archive are correct paths for the state the archive expects to be restored into.

There was one more decision worth recording. I had planned to delete two whole sections from a config file, and it turned out one of them documented infrastructure that was still running. The commands it mentioned were retired, but the hooks and scripts underneath were not. Deleting the section would have removed the only description of live machinery. I cut the two lines about the commands and kept the rest.

FAQ

How many sessions do I need before the numbers mean anything? Enough that your recent window contains a normal mix of work. Two months of daily use was plenty for me. If your recent column has fewer than a couple hundred total invocations, treat low counts as noise rather than evidence.

Should I delete instead of archiving? If the repo has history, deletion is recoverable and leaves the cleanest tree. Archiving wins when you expect to consult the content without reactivating it, which is the common case for a command whose steps you might still follow by hand. Both are reversible. Pick one and be consistent.

What if a command is dormant but documentation still tells people to use it? Fix the documentation in the same change, and check the whole repo, not just the obvious file. Scripts, READMEs in subprojects, and workflow guides all accumulate references. Mine were spread across five files, and one of them told the reader to invoke a command I was about to archive.

Does trimming the list actually improve anything measurable? The context savings are real but small. The bigger effect is selection accuracy: fewer near-duplicate descriptions means fewer wrong picks. I would not do this for the token count alone. I would do it because a folder where three different kinds of object look identical is a folder that will eventually mislead you.