One Caveman test shows why “be concise” belongs in the baseline

Image credit clarification: The hero is an unmodified Caveman project screenshot by Julius Brussee / Caveman contributors, licensed under Apache-2.0. It is not a photograph and was not sourced from Unsplash. The screenshot's displayed figures are product example data, not the reported experiment.

A developer writing as Naze Code has published a small comparison of Caveman, an open-source toolkit that aims to reduce the tokens coding agents use. The useful finding is narrower than a headline percentage: in one coding task, asking the agent to be concise already reduced its output substantially.

In the 16 October DEV Community post, the author reports running four variants three times each on a C# dictionary-key bug. Mean output fell from 845 tokens without a brevity instruction to 603 with “Be concise.” The reported averages were 482 for the Caveman rules and 308 for the more aggressive Ultracave mode.

That makes the two-word prompt about 29% shorter than the author's plain baseline, compared with about 43% and 64% for the two skill modes. Those are reported output-token reductions on that task, not independently established performance figures for coding agents generally.

A small experiment with important limits

Three runs on one problem cannot establish how a prompt performs across models, languages or long projects. The article describes a single model but does not clearly name it in the methodology. Auratro has not reproduced the experiment, inspected its raw runs or independently checked the claimed correctness of the returned fixes.

The post also estimates the input overhead of sending the skill instructions. That overhead matters when comparing total usage: a shorter answer can still come after a longer prompt. Prompt caching and how often instructions are sent can change the accounting, so the author's break-even estimate should not be treated as a universal rule.

Output savings and input compression answer different questions

Caveman's current README distinguishes shorter replies from compressing material before a model reads it. It also includes a concise-prompt baseline and warns that its reported session-compression results come from a particular evaluation setup. Those figures measure different workloads and token categories; they should not be combined into a single promised saving.

For a useful comparison on your own work, keep the task and model settings fixed, count input and output separately, and check whether the answer still satisfies the task. Include an ordinary brevity instruction as a baseline before attributing every reduction to an installed toolkit. These are evaluation suggestions, not a tutorial Auratro has tested.

Image: the project-supplied Caveman Learn interface screenshot shows product example data. Its displayed numbers are not the results of Naze Code's experiment. Screenshot copyright Julius Brussee / Caveman contributors, reproduced unmodified under Apache-2.0; source and licence are linked below.