Recursively Creating Our Own Perforce Agent

For longer than I want to admit, our production Perforce client ran p4 describe -du -dl <changelist>. It worked, mostly. The intent was “unified diff, ignore line-ending noise.” What -dl actually means to p4 describe is “ignore line endings,” full stop, and it takes no argument, so the changelist number after it got parsed as a second changelist. We never noticed, because the command still returned output. We found it the day an AI agent, running against a real server, tried the same flags and read the result more carefully than we had.

I promised this post in the launch announcement, and the title is literal. We used agents to build ButterStack. One of those agents is a Perforce agent. We pointed it at the kind of server our product integrates with, and the corrections flowed in both directions: into the agent’s skills, and into the Ruby client the product ships. Then we pointed a security agent at the Perforce agent. Then we made all of it public. This is that loop.

Why point an agent at a real server

Part 6 of the Agentic Game Dev series explains why we build a real game, Butter Up!, on the real stack we sell: a seeded demo cannot answer “show me a real problem and show me the tool catching it.” The same logic applies one level down. An agent’s Perforce knowledge written from the docs is a hypothesis. A general-purpose coding agent, asked to change a file in a depot, reaches for Git muscle memory and either fails or strips the read-only bit and quietly desyncs the workspace. You do not find out which of your own assumptions are like that by reading more docs. You find out by running the commands.

So the validation rig was a dockerized Helix Core P4D 2026.1 with three classic depots, roughly 8,700 changelists of history, and real .fbx, .png, and .wav assets. Later, Butter Up!’s own depot at //sample-game. The agent was told to observe, analyze, and report, and we read what it got wrong.

What reality corrected

The describe bug above stung because it had been copied into more than one internal client. It was not the only one.

p4 -Mj on its own is not structured output. It wraps each text line as {"data": ..., "level": 0}, which looks like JSON and carries none of the fields you wanted. You need p4 -ztag -Mj to get change, user, time, and headType as real keys. The agent’s first draft used the first form.

p4 -ztag describe truncates the description to its first line. If your submit messages carry task keys and tags in the body, which ours do, you lose them. Plain p4 describe -s and read the tab-indented body.

p4 info succeeds without a ticket. Almost nothing else does. An agent that uses p4 info as its “am I connected” check will report healthy on an expired session, which is the common real-world state. p4 login -s is the check.

Streams were authored from domain knowledge and flagged for validation. The live server confirmed the direction: the default -S <stream> points toward the parent, so merge-down is bare p4 merge on the child and copy-up is p4 copy -S <child> from the parent, and the server enforces merge-down before copy-up. Deleting a stream only tombstones it; p4 stream --obliterate -y is the real purge, which is exactly the kind of thing an agent should know and never run.

Exclusive locks: opened -a shows the +l holder directly, and fstat exposes the same state as structured fields, so “who is blocking me” is a single read.

Each of those became a line in the agent’s skills and, where it applied, a fix in the product client. None of them were in the plan.

Then we turned the loop on the agent

The agent ships read-first: it observes and analyzes, and the write operations (checkout, submit, shelve, reconcile, resolve) are gated. Underneath the gating is a hook, guard-p4.sh, that hard-blocks server-lifecycle and history-rewriting verbs (obliterate, admin, archive, restore, unload, unsubmit and the rest) before the command runs, regardless of how permissive the session’s allowlist is.

Before the repo went public we had our security-review agent audit it the way it audits our app: every blob in history swept, every hook reproduced in a sandbox. It found no secrets. It also found that p4 info && p4 obliterate -y //depot/x/... passed the guard. The script located the first p4 token in the command, read the verb after it, saw info, and allowed the whole line. The Lore agent’s guard in the same repo already handled chained commands, so the fix was known: emit a verb for every p4 occurrence and block if any of them is on the list. That shipped before anyone outside the company could install the plugin.

I include this because “we have a safety hook” is the kind of sentence you should distrust until someone tries to get past it. The honest boundary, which the README says in bold, is still the Perforce account: run the agent as a normal or read-only user, never as super. Protections on the server stop a destructive command. A hook is a backstop.

The same loop, on Unreal

We ran the identical process against Unreal, because a pipeline agent that cannot read a cook log is half an agent. Against real 5.6 and 5.8 engines it caught things the docs do not say. A headless 5.8 run against 5.6-built binaries does not prompt “rebuild modules?”; it silently drops the module and reports a misleading “plugin could not be found, consider disabling it.” Unreal Build Tool’s exit codes are 8 for a rules error and 6 for a compilation error, not what we had guessed. And our own cook-log parser, written against a pre-Zen engine, matched almost nothing in 5.6, where there is no DDC Hit Rate line and cache effectiveness lives in FShaderJobCache stats. That last one is a product bug we would not have found by reading our own code. Butter Up!’s matched 5.8 build and cook came back green, 496 of 503 packages, with a clean doctor report first.

It is public now

The agents live at github.com/ButterStack/gamedev-agents, MIT licensed, as a Claude Code plugin marketplace:

/plugin marketplace add ButterStack/gamedev-agents
/plugin install perforce@gamedev-agents

Perforce, Unreal, Unity, Lore, Godot, and Jenkins agents are there today. Each carries a dated LEARNINGS.md of what real servers keep teaching us, and a section listing what is still authored from docs and needs validating on your setup. The repo is distilled from the private one we run in production, on a standing basis, with internal paths, hostnames, and names stripped; the process document is in the repo too. The most useful contribution you can make is a report from a real depot or an engine version we have not touched.

What the recursion is for

The managed version of this loop is the ButterStack Build Investigator, which I wrote up two weeks ago. Same principle: the agent reads, hands you evidence, and never gets a tool it should not have. The open agents trail what the product learns. The product is where the learning compounds across every depot it touches, without any studio’s content leaving their control.

Point the agent at reality. Let reality correct it. Ship the correction. Repeat.

Get started with ButterStack

Thanks!
Ryan L’Italien
Founder and CEO of ButterStack

Want to see what pipeline observability looks like? Try ButterStack free and connect your first integration in minutes.

Or just email me at: ryan@butterstack.com.