NVIDIA SkillSpector

What It Does

SkillSpector is a command-line security scanner that analyzes AI agent skills (Claude Code, Codex CLI, Cursor, Gemini CLI) for vulnerabilities before installation. You point it at a directory, Git URL, zip file, or single SKILL.md file, and it runs a two-stage analysis pipeline: fast static pattern matching (regex, AST analysis, YARA signatures, taint tracking) followed by an optional LLM-powered semantic evaluation. The tool outputs terminal, JSON, Markdown, or SARIF 2.1.0 reports with a 0-100 risk score and a clear SAFE/CAUTION/DO NOT INSTALL recommendation. It detects 64 vulnerability patterns across 16 categories including prompt injection, data exfiltration, supply chain risks, MCP tool poisoning, and MCP least privilege violations. Built on LangGraph with a 600-test suite, it is positioned as the CI/CD-ready gatekeeper for the agent skill ecosystem.

The Good

600 passing tests validate the detection engine end to end. The test suite covers unit tests (patterns, providers, model info, OSV client, SARIF, CLI), node tests (LLM analyzer, report generation, context building), and analyzer-specific tests for every pattern category. The test-to-source ratio is nearly 1:1: 9,460 lines of tests against 9,930 lines of source code. Running pytest -m "not integration" tests/ completed in 1.33 seconds with 600 passed, 11 skipped, 22 deselected. This is the strongest test suite of any agent skill tool we have reviewed.

Live OSV.dev CVE lookups work out of the box with no API key. The SC4 analyzer queries the OSV.dev vulnerability database in a single batch HTTP call for all dependencies. In our malicious test skill, it found 10 advisories for the unpinned requests package, including CVE-2024-47081 (.netrc credential leak) and CVE-2024-35195 (Session TLS bypass). The lookup completed in under two seconds. When the network is unavailable, it falls back to a small built-in static list. No registration or API key needed.

MCP-specific vulnerability detection fills a real gap. Two analyzer categories target MCP servers specifically: MCP Least Privilege (LP1-LP4) detects capability mismatches between declared permissions and actual code behavior, and MCP Tool Poisoning (TP1-TP4) catches hidden instructions in tool metadata, Unicode deception (homoglyphs, RTL overrides), parameter description injection, and description-behavior mismatches. No other agent skill scanner in our review set covers MCP tool poisoning at this depth.

All four output formats produce valid, usable reports. Terminal output uses Rich formatting with a risk assessment table and color-coded severity badges. JSON output is clean and parseable. Markdown output includes severity emoji indicators and per-finding remediation guidance. SARIF 2.1.0 output is schema-valid for CI/CD integration (GitHub Code Scanning, Azure DevOps). Exit codes are correct: 0 for clean skills, 1 for findings.

The Bad

Python 3.12+ is required, excluding older environments. The requires-python constraint in pyproject.toml is >=3.12,<3.14. macOS ships Python 3.9 by default. Users on older systems must install Python 3.12+ via Homebrew, pyenv, or uv before they can use SkillSpector. The PEP 604 union type syntax (str | None) in the codebase makes backporting non-trivial.

No PyPI package. Installation is git-clone-only. Running pip install skillspector returns "No matching distribution found." You must clone the repository, create a venv, and run make install. For a security tool that teams might want to integrate into CI/CD pipelines, the lack of pip install skillspector or a pre-built binary adds friction. The pyproject.toml has all the configuration for publishing (twine, build in dev deps, SPDX headers), but no release has been published yet.

Static-only mode produces duplicate findings on the same code line. In our test, the SC2 (External Script Fetching) pattern fired three times on line 15 of the malicious script, each with a different confidence value (0.90, 0.90, 0.80). The TM1 (Tool Parameter Abuse) pattern also fired twice on line 11. This inflates the issue count and could cause alert fatigue in CI/CD. The LLM semantic analysis stage is designed to filter these duplicates, but without an API key, users get the raw static output.

No GitHub Actions CI pipeline. The repository has pre-commit hooks (ruff check, ruff format) but no .github/workflows/ directory. There are no tagged releases. The pyproject.toml classifies the project as "Development Status :: 3 - Alpha." While the test suite is comprehensive, there is no automated CI enforcement on pull requests.

Smoke Test Results

Hands-on testing on macOS (host) with Python 3.12.13 and uv 0.10.2. Installed from source via uv pip install -e ".[dev]". Tested two crafted skills (malicious and clean) plus all four output formats.

Run A. Fresh install, no deps preinstalled

$ cd /tmp/skillspector-review
$ uv venv .venv-ss && source .venv-ss/bin/activate
$ uv pip install -e ".[dev]"
✅ Virtual environment created with CPython 3.13.12
✅ All dependencies installed (typer, rich, httpx, pydantic, openai, langgraph, yara-python, etc.)

$ skillspector --version
✅ Prints version banner

$ skillspector scan --help
✅ Full help with examples, env vars, provider credentials

Pass rate: 3 of 3. Clean install from source with zero configuration issues.

Run B. With deps preinstalled

$ skillspector scan /tmp/malicious-test-skill/ --no-llm --format json
✅ Returns JSON with score 100/100, CRITICAL, DO_NOT_INSTALL
✅ 16 findings across 8 categories (AST1, AST4, E1, E2, EA1, LP3, P1, SC1, SC2×3, SC3, SC4, TM1×3)
✅ SC4 live OSV.dev lookup: 10 CVEs found for requests package
✅ Exit code 1 (security findings present)

$ skillspector scan /tmp/clean-test-skill/ --no-llm --format json
✅ Returns JSON with score 0/100, LOW, SAFE
✅ 0 findings
✅ Exit code 0

$ skillspector scan /tmp/malicious-test-skill/ --no-llm --format terminal
✅ Rich-formatted terminal report with risk table, component table, issue list
$ skillspector scan /tmp/malicious-test-skill/ --no-llm --format sarif -o /tmp/test.sarif
✅ Valid SARIF 2.1.0 output (321 lines, schema-compliant)
$ skillspector scan /tmp/malicious-test-skill/ --no-llm --format markdown -o /tmp/test.md
✅ Clean Markdown with severity emoji, per-finding remediation

Pass rate: 6 of 6. All output formats produce valid results. Both malicious and clean skills scored correctly.

Run C. Functional verification (does it do what it claims?)

$ python -m pytest tests/ -m "not integration" -q
✅ 600 passed, 11 skipped, 22 deselected in 1.33s

Malicious skill detections verified:
✅ P1 (Prompt Injection): "Ignore all previous instructions" caught at SKILL.md:8
✅ E2 (Env Variable Harvesting): os.environ.items() loop caught at scripts/exfil.py:6
✅ AST1 (exec() Call): exec(code) caught at scripts/exfil.py:12
✅ SC2 (External Script Fetching): curl | bash caught at scripts/exfil.py:14-15
✅ SC4 (Known Vulnerable Dependency): 10 OSV.dev CVEs for requests
✅ TM1 (Tool Parameter Abuse): shell=True and rm -rf / caught

Clean skill:
✅ Zero false positives on a pure-doc skill with no scripts

Functional pass rate: 7 of 7. Every planted vulnerability was detected. No false positives on the clean skill.

What the runs tell you

SkillSpector installs cleanly from source and its static analysis engine works as documented. The 600-test suite, live CVE lookups, and four output formats all function correctly. The only untested path is the optional LLM semantic analysis, which requires an API key. Static-only mode (--no-llm) is a fully supported standalone mode and produces high-recall results, though with some duplicate findings that the LLM stage would filter.

Setup Walkthrough

1. Install Python 3.12+ if not already available: brew install python@3.12
2. Clone the repo: git clone https://github.com/NVIDIA/SkillSpector.git && cd SkillSpector
3. Create and activate a venv: uv venv .venv && source .venv/bin/activate
4. Install: make install (uses uv if available, else pip)
5. Scan a skill: skillspector scan ./my-skill/ --no-llm
6. For LLM-enhanced analysis, set a provider (export SKILLSPECTOR_PROVIDER=openai and export OPENAI_API_KEY), then scan without the --no-llm flag

The heavy dependency footprint (langgraph, langchain-core, langchain-openai, langsmith, yara-python, pydantic, openai, rich, typer) means the install pulls about 40 packages. This is normal for a LangGraph-based tool but heavier than a standalone scanner. YARA rules ship bundled (malware.yar, webshells.yar, cryptominers.yar, hacktools.yar) so no separate download is needed.

Alternatives

1. snyk/agent-scan - Snyk's agent security scanner focuses on broader Snyk-platform integration and SBOM analysis. Prefer SkillSpector for dedicated agent-skill pattern coverage (64 patterns vs. Snyk's platform-dependent checks). Prefer Snyk if you already use the Snyk ecosystem for dependency scanning.
2. yegor256/dogent - A lightweight SKILL.md and CLAUDE.md linter focused on format and structure validation, not security. Use alongside SkillSpector for quality checks that SkillSpector does not cover (frontmatter format, description conventions).
3. Manual review - Reading every file in a skill before installation. Still necessary for semantic understanding, but SkillSpector automates the repetitive pattern-matching that humans miss.