NVIDIA/SkillEvaluator
- Stars
- 1,634
- Language
- Python
- License
- Apache-2.0
- Daily
- rank 12 · +205 stars gained
Repository metrics were observed on this edition date and may have changed.
Key details A CLI that evaluates agent skills in stages, covering static quality, instruction duplication, and their impact on actual task outcomes. It distinguishes comparisons before and after applying a skill from comparisons between versions, and records evaluation budgets, incomplete evidence, and failure causes in reports. Use cases First inspect the static quality of a skill directory, then optionally add duplication checks and evaluations using actual agents in a sandbox. GitHub and GitLab CI configurations are also provided for checking skills changed in PRs. Costs and limitations The basic static quality-check example is documented to run without an API key, GPU, or Docker. Model-based evaluations and managed sandboxes may incur separate charges, and self-managed execution resource costs should also be considered. NVIDIA Build's free prototyping is subject to eligibility, access conditions, and call limits. Adoption considerations Check Python 3.12–3.14 requirements and the dependencies for each stage. Full static checks require external scanners, while actual evaluations require provider authentication, a supported agent, and a sandbox. Support is experimental, and static scores alone do not establish security or real-world effectiveness. Why it stands out It was observed at No. 12 on daily Trending, with an increase of 205. Version 0.9.0, listed on 2026-11-13, expands reusable baseline runs, version comparisons, token, judging-call, and cost budgets, and reporting of evaluation evidence.
View on GitHub