SkillCorpus

What It Does

SkillCorpus is EverMind AI's open-source answer to a problem every heavy skills user hits: too many SKILL.md files, no way to pick the right one at the right moment. The repo ships two halves. The producer is a curation pipeline (aggregate, curate, export) that clones skill repos, parses SKILL.md files, scores quality on three facets, dedups by embedding and name hash, gates by a GREEN license whitelist, and exports a parquet corpus with attachments. The consumer is skillsearch, a retrieval engine that fuses local BM25 over your skills directory with three remote catalogs, optional model-driven rewriting and gating, and injection of the winning skill bodies before the agent answers. Host adapters wire that engine into Hermes (as a memory-provider prefetch hook), OpenClaw 1.x and 2.0 (separate packages, since 2.0 dropped the 1.x hook), WorkBuddy, Raven, and DeepSeek Harness, plus an HTTP adapter for anything else. An arXiv paper (2607.15557, verified live) reports the payoff: pooled SkillsBench pass rate gains of +7.5 points when skills are retrieved this way.

The Good

The whole stack is testable without a key, and all of it passed. On a clean macOS host with a fresh Python 3.11 venv, four suites ran green: the producer suite (70 passed), the engine-python suite (103 passed), the Hermes plugin suite (24 passed), and the TypeScript engine suite (47 passed via the same locked tsx invocation plugin CI uses). That is 244 passing tests for a repo whose marquee feature is an integration surface, not a library. The TS suite even covers the exact failure mode a reviewer would probe: "EverMind lexical guard rejects forced unrelated Top K results" and "Chinese queries are segmented and matched without a rewriter".

The demo build runs the full curation chain with zero credentials and degrades loudly, not silently. python -m skillcorpus.cli build cloned the four demo sources (anthropics/skills, vercel-labs/skills, addyosmani/agent-skills, K-Dense-AI/scientific-agent-skills), ingested 209 skills, found 4 name-hash collision pairs in dedup, activated all 209 rows through the GREEN license whitelist, and exported a 1.57MB skills.parquet plus a 9.5MB attachments.tar.zst with a dataset card. With no LLM endpoint the safety gate printed an explicit warning: 209 active skills have no LLM judgment, the hard gate was NOT applied, the active set is NOT fully safety-vetted. Degradation that names its own gap is rarer than it should be.

INSTALL.agent.md is the best agent-facing install document GearScope has reviewed. It gives the installing agent five enforced rules (show diffs before writing, back up as .bak-skillsearch, merge never replace, stop on ambiguity, finish with verification), per-host playbooks for all six hosts, a plain-language network disclosure (three remote catalogs receive every retrieval query by default), and a definition of done with both a positive probe and a nonsense-string negative probe. It even explains why a weather question is a bad negative probe: real weather skills exist, so a hit there is correct behavior. An uninstall section closes it out. We reused its pdf-tables positive probe and its "zxqv-7319" negative probe verbatim in our functional run, and both behaved as documented offline.

The Hermes adapter is real, not aspirational. The plugin README documents the honest constraint up front: prefetch routes through the memory provider slot in Hermes, so skill retrieval and a memory backend cannot hold the slot at the same time. The suite ships a fallback base class so it runs anywhere, and the README instructs running it against a real hermes-agent checkout too. We did, with PYTHONPATH pointed at this machine's live checkout: 24 passed in 5.03s against the host's actual MemoryProvider ABC, which means an unimplemented abstract method would have failed at instantiation rather than hiding.

Dependency hygiene and license honesty are both above the tier norm. The producer installs exactly 8 dependencies (torch and transformers are isolated in an eval extra), and the retrieval engine has a single runtime dependency, httpx. Exported rows carry each skill's upstream license string verbatim: our demo corpus contained GPL-2.0, PolyForm-Noncommercial, and Proprietary rows even though all four sources are GREEN-gated, which matches the README's exact warning that per-row license and source-level gating are different things. A GREEN/RED/YELLOW policy doc with opt-out instructions backs this up.

The Bad

The README overstates the no-LLM fallback for classification. It says LLM classification "degrades gracefully to rules when no model endpoint is reachable". It does not: classify.py falls back to category=OTHER with method=fallback, and all 209 demo skills landed in OTHER. Only tag extraction is rule-based. Quality scoring does degrade to rules (avg_quality 0.694 was computed), so the sentence is half right, and the pipeline does keep running end to end, but a reader expecting rule-based 16-class taxonomy output without an endpoint will not get it.

The evaluate toolkit's license claim is only half implemented. The README says the match/ and evaluate/ toolkits "are each MIT, see their own LICENSE". match/LICENSE exists and is MIT, and skillsbench has an MIT LICENSE, but the gdpval and qwenclawbench sub-benchmarks ship no LICENSE file at all. For a repo whose entire value proposition includes license governance of other people's skills, two unlicensed directories are a self-consistency gap.

Remote catalogs are on by default, and the hosted catalog matches loosely without a model gate. All three endpoints (EverMind SkillHub, ClawHub, skillhub.cn) receive every retrieval query unless you empty three separate config fields. Local-only operation works (we verified a planted skill directory with all remotes disabled), but privacy defaults to opt-out. Related: our nonsense probe "zxqv-7319 frobnicate the quux" against the hosted catalog returned 3 hits, topped by a play-dumb skill at q=0.593. The docs are honest that fusion ranks by position and the model gate is the countermeasure for unvetted catalogs, but the zero-install demo path has no gate, so first impressions can include visibly wrong matches.

The big corpus is hosted-only and the public artifacts are thin so far. The production source registry (62 sources) is gitignored, the downloadable corpus is a 1,000-skill demo, and the full 114,190-skill catalog exists only behind the hosted API; the roadmap marks the full HF release as pending. HuggingFace traction matches: 87 dataset downloads, 36 and 16 for the two retrieval models. Small blemishes round it out: a leftover internal TODO comment sits in the README roadmap ("edit to match your plan"), the producer claims Python 3.10 while the engine requires 3.11, and a demo build writes roughly 565MB of cloned sources and index data into ~/.skillcorpus by default.

Smoke Test Results

Host-based hands-on testing on macOS 26.5.2 aarch64, Node 24.13.1, fresh venvs from Python 3.11. Producer installed from a shallow clone (3.4MB). No API keys were used at any point.

Run A (fresh venv, producer only)

$ git clone --depth 1 https://github.com/EverMind-AI/SkillCorpus
✅ clone ok, 3.4MB
$ python3.11 -m venv .venv && pip install -e .
✅ 8 declared deps installed (pyyaml, numpy, click, faiss-cpu, sqlite-vec, openai, pyarrow, zstandard), import ok
$ python -m skillcorpus.cli --help
✅ build / export / init / stats commands listed
$ python -m skillcorpus.cli build
✅ exit 0: 209 skills ingested (anthropics 18, vercel 1, addyosmani 25, K-Dense 165), dedup 4 collision pairs, license audit activated 209, export 209 rows / 181 with attachments
$ python -m skillcorpus.cli stats
✅ JSON stats: by_source, 27 distinct raw license strings, avg_quality 0.694
$ python -m skillcorpus.cli export --out ./corpus
✅ skills.parquet 1.57MB + attachments.tar.zst 9.5MB + README dataset card

Pass rate: 6 of 6. One script-side note: the build step was rerun directly because GNU timeout is absent on macOS; every command itself succeeded first try.

Full fresh log →

Run B (sandbox with deps preinstalled)

$ pip install -e .[dev] && pytest skillcorpus/tests
✅ 70 passed in 2.86s
$ pip install -e skillcorpus_plugin/engine-python[dev] && pytest tests
✅ 103 passed in 0.28s
$ pytest plugin-hermes/tests
✅ 24 passed in 7.82s (standalone fallback base class)
$ npm ci --prefix plugin-openclaw && tsx --test engine-typescript/tests/*.test.ts
✅ 47 pass, 0 fail in 0.56s (CI's own locked-tsx invocation)

Pass rate: 4 of 4 suites, 244 of 244 tests. This is the canonical install-and-run verdict for the badge: every suite the repo ships for the surfaces we could reach passed without modification.

Full with-deps log →

Run C (functional verification)

$ python functional.py # offline engine probes per INSTALL.agent.md
✅ local-only retrieve runs with all remote endpoints disabled
✅ positive probe returns pdf-tables with its camelot guidance (275 chars)
✅ negative probe "zxqv-7319..." injects nothing (0 chars)
$ PYTHONPATH=~/.hermes/hermes-agent pytest plugin-hermes/tests
✅ 24 passed in 5.03s against the real hermes-agent MemoryProvider ABC
$ python examples/skillhub_demo.py "extract tables from a scanned PDF invoice"
✅ live no-key search: top hit extract-tables-from-pdf, q=0.708, DOC-PROC, MIT
$ python examples/skillhub_demo.py "zxqv-7319 frobnicate the quux"
✅ no crash; hosted catalog still returned 3 loose hits (gate absent on the demo path)

Functional pass rate: 8 of 8. The engine's core contract (inject the right local skill, inject nothing on nonsense) is verified with assertions on output, and the live API works without a key.

Full functional log →

What the runs tell you

The pipeline and the engine both do their jobs offline and keyless, and the plugin layer holds up against the real Hermes host contract. What you cannot get without infrastructure is the full experience: the 114K-skill catalog is hosted, the model-backed gate needs an LLM key, and the last mile into a live Hermes session requires giving up the memory provider slot, which is a user decision rather than a test step.

Setup Walkthrough

1. Clone the repo and create a venv: git clone --depth 1 https://github.com/EverMind-AI/SkillCorpus && cd SkillCorpus && python3.11 -m venv .venv && . .venv/bin/activate.
2. Install the producer: pip install -e . (8 deps, no torch).
3. Build the demo corpus: python -m skillcorpus.cli build. This clones 4 public skill repos and writes about 565MB to ~/.skillcorpus on first run. No LLM or embedding endpoint is needed; classification lands in OTHER and the tool tells you so.
4. For Hermes: pip install ./skillcorpus_plugin/engine-python, then cp -r skillcorpus_plugin/plugin-hermes "$HERMES_HOME/plugins/skillsearch", then hermes memory setup and pick skillsearch. Know that it takes the memory slot.
5. For local-only retrieval (no query leaves the machine), set hub_endpoint, clawhub_endpoint, and skillhub_cn_endpoint each to an empty string in the plugin config.
6. To just try the hosted catalog: python examples/skillhub_demo.py "", stdlib only.

Gotchas: Python 3.11 is the practical floor (the engine requires it even though the producer says 3.10), GNU timeout is absent on macOS so script the build directly, and pip-installing plugin-hermes fails by design since it is meant to be copied, not installed.

Alternatives

1. vercel-labs/skills (npx skills find) - registry-side search at install time rather than per-turn retrieval. Prefer it when you want to pick skills manually and keep context fully static.
2. thedotmack/claude-mem - occupies the same every-turn injection surface but for persistent memory rather than skill selection; the two solve adjacent problems and, on Hermes, compete for the same slot.
3. The hosted SkillHub API alone - curl https://skillhub.evermind.ai/openapi/v1/skills?q=... with no install gives you the same catalog the plugin queries. Prefer it for a zero-commitment evaluation, which is exactly how this review's functional probes used it.