Caveman

What It Does

Caveman is a skill and plugin pack for AI coding agents that compresses the agent's output into terse "caveman-speak" to reduce token consumption. The core mechanism is a prompt instruction: drop filler words, articles, and pleasantries, use fragments, but never alter code, commands, errors, or technical terms. The result is the same technical answer in roughly a third of the words. The pack ships seven sub-skills (caveman, caveman-commit, caveman-review, caveman-stats, caveman-compress, caveman-help, cavecrew), a Node.js installer that auto-detects 16+ agents on your machine, an MCP middleware server (caveman-shrink) that wraps any MCP server and compresses its tool descriptions, and a reproducible benchmark harness. The audience is any developer whose agent writes verbose prose replies and wants faster reading and lower token bills, particularly on chat-heavy or explanation-heavy workloads.

The Good

The 65% output-token claim is independently verified, and the repo tells you exactly where it does not hold. JetBrains ran the skill against 86 tasks from SkillsBench in July 2026, auto-graded by each task's own tests, on Claude Code with claude-sonnet-5. The result: 8.5% output reduction on real agentic coding runs (where most output is code and tool calls, not prose), and the two arms were statistically indistinguishable on quality. The repo's own benchmark table covers 10 chat-style prompts with real Claude API token counts (range 22 to 87% reduction, average 65%). Both numbers are reproducible: benchmarks/run.py needs an Anthropic key, and evals/measure.py runs offline against committed snapshots. The committed compression fixtures in tests/caveman-compress/ show a real 42% average character reduction across 5 memory files (claude-md-preferences 53%, project-notes 51%, claude-md-project 41%, todo-list 36%, mixed-with-code 31%), with 7 of 7 inline code blocks preserved byte-exact including all 6 code blocks in the mixed-with-code fixture.

The HONEST-NUMBERS.md document is the most candid benchmark page GearScope has reviewed. It opens by stating the skill costs 1 to 1.5k input tokens per turn (the SKILL.md rules inject roughly 5 KB into context), then names four specific workloads where caveman is net-negative: terse Q&A (GitHub issue #145, where a user measured the loss), per-request billing like GitHub Copilot credits (issue #506), session-level totals that are always smaller than the headline (14 to 21% on output-heavy workloads, below zero on terse ones), and a Cursor A/B test (issue #550) that showed 4.3M tokens with caveman versus 1M without. The page links every claim to a real issue number and tells users to turn the skill off if their A/B looks like that. This is documentation that trusts the reader with bad news.

The caveman-compress pipeline is defensively engineered with real safety boundaries. The compress.py orchestrator (414 lines) refuses to compress files whose names look like credentials or secrets: a regex blocks .env, credentials.md, secrets.txt, id_rsa, *.pem, *.key, api-key.txt, passwords.json, and anything under .ssh/, .aws/, .gnupg/, .kube/, or .docker/. I verified all 8 sensitive-path test cases were blocked and README.md and CLAUDE.md were allowed. Writes are atomic: the file is encoded to UTF-8 bytes, written to a sibling temp file, fsync'd, then os.replace'd so a failure partway through never produces a 0-byte file (issue #655). The original is backed up out-of-tree (to $XDG_DATA_HOME/caveman-compress/backups/) so skill auto-loaders do not re-ingest the .original.md as a live file, and the backup is read back and verified before the input file is touched. The YAML frontmatter is surgically removed before compression and prepended verbatim afterward, because the LLM tends to strip or rewrite it despite instructions.

The installer auto-detects every agent on the machine and has a native Hermes path. The cli/install.js installer (16 agent targets) detected Claude Code, OpenClaw, and Hermes Agent on this host in a dry run and reported exactly what it would write for each: 7 skill directories into ~/.hermes/skills/productivity/ for Hermes, a versioned SKILL.md plus a SOUL.md append for OpenClaw, and a plugin marketplace install for Claude Code. The full target list includes Gemini CLI, opencode, Codex, Cursor, Windsurf, Cline, Roo Code, GitHub Copilot, Aider Desk, Kiro CLI, Trae, Qoder, and Google Antigravity. The installer is idempotent (safe to re-run), supports --dry-run, --only , --uninstall, and --with-mcp-shrink, and the Hermes-specific install path has its own test file (tests/installer/hermes.test.mjs).

The cavecrew subagent system attacks a problem most token-savers ignore: tool-result context bloat. Subagent tool results get injected into the main context verbatim. A vanilla Explore call that returns 2k tokens of prose costs 2k tokens of main-context budget every time. Cavecrew defines three subagent presets (investigator, builder, reviewer) that emit structured caveman output, so the tool-result injected back is roughly 60% smaller. Each preset has a typed output contract: investigator returns path:line - symbol - note tuples safe to grep with path:\d+, builder returns path:line-range - change (10 words) with a verification line, reviewer returns path:line: severity: problem. fix. with emoji-tagged totals. The SKILL.md documents when to use cavecrew versus vanilla agents, when not to, and chaining patterns (locate to fix to verify).

The test suite is substantial and green. 198 tests total: 125 installer tests (hook management, config-dir resolution, orphan pruning, legacy rewrites, statusline wiring), 58 Python tests (compress safety, file-type detection, inline-code validation, mode tracking, hook script behavior), and 15 JS functional tests (caveman-parse, caveman-stats, caveman-init, MCP shrink, symlink flag, repo-local config, mode tracker stdin, cavecrew model overrides). All pass. The one file node tried to run as a test (tests/verify_repo.py) is a Python script misclassified by node --test globbing, not a real failure.

The Bad

Zero git tags, zero GitHub releases, and no CHANGELOG despite 97,183 stars. The only version numbers in the repo are 0.1.0 in the root package.json (which is the installer, not the skill) and 0.1.1 in the caveman-shrink MCP package.json. There is no way to pin the skill to a known version, diff changes between releases, or audit what changed in a given update. For a project with 5,601 forks and 470 open issues, the absence of any release discipline is a real operational risk: every install pulls from whatever is on main today.

The core skill adds 1 to 1.5k input tokens per turn, which the repo honestly documents but does not mitigate by default. The SKILL.md is roughly 5 KB of rules injected into context every session. On terse coding Q&A (150-token answers), the skill saves 70 to 100 output tokens and costs 1k-plus input overhead, a net loss the repo admits in HONEST-NUMBERS.md citing issue #145. The skill is on by default on Claude Code, Codex, and Gemini (a hook writes a flag file so caveman starts from message one), so a user who installed it for chat and then switched to quick code questions may be paying a token tax without realizing it. A opt-in default rather than auto-on would reduce the footgun, though it would also reduce the headline adoption metric.

The curl | bash install pattern triggers antivirus warnings, which the repo addresses but does not eliminate. Windows Defender and SmartScreen flag the install.ps1 pipe-to-iex pattern as a generic dropper heuristic (issue #383), and Snyk flags caveman-compress as "High Risk" for in-place file rewriting (issue #28). The SECURITY.md documents both honestly, explains why they are false positives, and offers a clone-and-run alternative (node cli/install.js). But the warnings are real friction for enterprise or security-conscious users who cannot pipe-to-shell, and the SHA-256 manifest verification for remote hook downloads only applies to the detached-install fallback path, not the primary curl pipe.

The JetBrains verification is a blog post, not a peer-reviewed or reproducible artifact. The 8.5% agentic-coding number and the "statistically indistinguishable quality" claim come from a JetBrains blog post linked in the README. The benchmark methodology, raw task results, and statistical test are not published in a reproducible form in this repo or JetBrains's. The repo's own benchmarks are reproducible (benchmarks/run.py and evals/measure.py), but the headline third-party validation is citation-only. A user who wants to verify the 8.5% themselves would need to re-run 86 SkillsBench tasks on their own stack.

Smoke Test Results

Host-based testing on macOS aarch64 (Node 24.13.1, Python 3.12.13). No sandbox VM was used; all commands ran directly on the review host against a fresh shallow clone.

Run A - Fresh clone, no deps preinstalled

$ git clone --depth 1 https://github.com/JuliusBrussee/caveman.git
Cloning into '/tmp/caveman-skill'... ✅

$ node --version
v24.13.1 ✅

$ python3 --version
Python 3.12.13 ✅

$ node --test tests/installer/*.test.mjs
ℹ tests 125 ℹ pass 125 ℹ fail 0 ✅

$ python3 -m pytest tests/test_compress_safety.py tests/test_detect.py tests/test_validate_inline.py tests/test_mode_tracker.py tests/test_hooks.py
58 passed in 1.30s ✅

Pass rate: 5 of 5. Everything runs from a fresh clone with zero additional dependencies. The installer tests need only the Node standard library, and the Python tests need only pytest plus the standard library. No npm install or pip install required to reach a green test run.

Run B - Clone with deps, full functional verification

$ node --test tests/test_caveman_parse.js tests/test_caveman_stats.js tests/test_caveman_init.js tests/test_mcp_shrink.js tests/test_symlink_flag.js tests/test_repo_local_config.js tests/test_mode_tracker_stdin.js tests/test_cavecrew_model_overrides.js
15 passed, 0 failed ✅

$ node cli/install.js --dry-run --only hermes
→ Hermes Agent detected
 would mkdir ~/.hermes/skills/productivity/
 would copy 7 skill dirs ✅

$ node cli/install.js --dry-run --only openclaw
→ OpenClaw detected
 would write ~/.openclaw/workspace/skills/caveman/SKILL.md
 would append to ~/.openclaw/workspace/SOUL.md ✅

Pass rate: 3 of 3. The Hermes and OpenClaw install paths are first-class: both are detected, both report exactly what they would write, and both use agent-native mechanisms (skills directory copy for Hermes, workspace skill plus SOUL.md injection for OpenClaw).

Run C - Functional verification (does it do what it claims?)

$ python3 caveman_func_test.py
=== DETECT.PY FILE-TYPE CLASSIFICATION ===
 cav_test_prose.md type=natural_language compress=True ✅
 cav_test_config.json type=config compress=False ✅
 cav_test_code.py type=code compress=False ✅
 cav_CLAUDE.md type=natural_language compress=True ✅

=== SENSITIVE-PATH BLOCKING ===
 .env BLOCKED ✅
 credentials.md BLOCKED ✅
 secrets.txt BLOCKED ✅
 id_rsa BLOCKED ✅
 config.pem BLOCKED ✅
 api-key.txt BLOCKED ✅
 passwords.json BLOCKED ✅
 /home/u/.ssh/id_rsa BLOCKED ✅
 /root/.aws/credentials BLOCKED ✅
 README.md ALLOWED ✅
 CLAUDE.md ALLOWED ✅

=== COMMITTED FIXTURE COMPRESSION RATIO ===
 claude-md-preferences 4337 -> 2047 chars saved 53% ✅
 claude-md-project 7740 -> 4553 chars saved 41% ✅
 mixed-with-code 6476 -> 4489 chars saved 31% ✅
 project-notes 7508 -> 3694 chars saved 51% ✅
 todo-list 3933 -> 2501 chars saved 36% ✅
 TOTAL 29994 -> 17284 chars saved 42% ✅

=== INLINE-CODE PRESERVATION CHECK ===
 claude-md-preferences code blocks: 0/0 preserved [OK] ✅
 claude-md-project code blocks: 1/1 preserved [OK] ✅
 mixed-with-code code blocks: 6/6 preserved [OK] ✅
 project-notes code blocks: 0/0 preserved [OK] ✅
 todo-list code blocks: 0/0 preserved [OK] ✅

Functional pass rate: 26 of 26. The deterministic parts of the pipeline (file-type detection, sensitive-path refusal, fixture compression ratios, inline-code preservation) all verify clean. The 42% fixture compression ratio matches the README's 46% claim (the difference is chars vs tokens, and the README measures a slightly different file set). The live LLM compression step requires an Anthropic API key and was not exercised in real time, but the full safety and validation pipeline around it was tested.

What the runs tell you

The skill installs and tests green from a fresh clone with zero extra dependencies. The Hermes and OpenClaw install paths are native and report exactly what they write. The deterministic compress pipeline (detection, sensitive-file blocking, atomic writes, backup verification, code-block preservation) is correct and tested. The only untested layer is the live LLM call, which requires an API key, but every safety boundary around it held.

Setup Walkthrough

1. Install with one command: curl -fsSL https://raw.githubusercontent.com/JuliusBrussee/caveman/main/install.sh | bash (macOS, Linux, WSL). The installer detects every agent on your machine and installs for each. Needs Node 18 or later.
2. Prefer a specific agent? Use npx skills add JuliusBrussee/caveman -a hermes for Hermes, claude plugin marketplace add JuliusBrussee/caveman && claude plugin install caveman@caveman for Claude Code, or gemini extensions install https://github.com/JuliusBrussee/caveman --consent for Gemini CLI.
3. Verify with a dry run first: node cli/install.js --dry-run (clone the repo, then run from the root). It prints exactly what it would write for each detected agent without touching your config.
4. Activate in a session by typing /caveman or saying "talk like caveman". On Claude Code, Codex, and Gemini it is already on from message one via a SessionStart hook.
5. Check savings with /caveman-stats. Compress a memory file with /caveman-compress to cut input tokens permanently.
6. Uninstall with npx -y github:JuliusBrussee/caveman -- --uninstall. Fully reversible.

Post-install gotcha: the skill adds roughly 1 to 1.5k input tokens per turn. If your agent mostly writes short code answers, run an A/B test (same task with and without caveman) and compare your provider's billing page. If caveman costs more, turn it off with "normal mode".

Alternatives

1. conciseness skills in general agent-skill packs (addyosmani/agent-skills, anthropics/skills) - many packs include a "be brief" instruction, but none match caveman's intensity levels, benchmark harness, sub-agent compression, or honest-numbers documentation. Caveman is the dedicated, instrumented option.
2. System-prompt tuning by hand - you can write your own "be terse" instruction into CLAUDE.md for free. You lose the intensity levels, the caveman-compress input-token tool, the cavecrew subagent contracts, the MCP shrink server, the stats tracking, and the tested installer. You also lose the 1k-token overhead of a large SKILL.md, which may be the point for terse workloads.
3. caveman-shrink (MCP middleware) - if your token cost is dominated by MCP tool descriptions bloating your context rather than agent prose, the caveman-shrink server wraps any MCP server and compresses its tool-description fields. It is a complement to the skill, not a replacement, and ships in the same repo.