skill-up
An evaluation and evolution tool for Agent Skills.
skill-up is an evaluation and evolution framework for Agent Skills that includes skill-upper, a conversational agent skill for building and iterating eval suites. It automates the complete evaluation workflow: agents use skill-upper to generate declarative YAML test cases, execute them across multiple agent engines (Claude Code, Codex, Qoder CLI), grade outputs with rule-based, script, or AI judges, and automatically repair failing cases or expand coverage. The tool bridges one-off testing into a CI-ready, repeatable improvement loop that measures skill quality and drives iterative refinement.
Key Features
Use Cases
- 01Automated quality assurance for Agent Skills with repeatable test cases that run locally and in CI
- 02Conversational eval development where the agent reads SKILL.md, generates realistic test scenarios, and iterates on failures
- 03Cross-engine compatibility testing to validate skills work correctly in Claude Code, Codex, and other clients
- 04Regression testing by automatically adding coverage for fixed bugs during the evolution loop
- 05CI/CD integration for pull request skill validation with structured pass/fail reports
- 06Migration of existing Anthropic evaluation suites into declarative YAML workflows
Related Skills
View moresuperpowers
An agentic skills framework & software development methodology that works.
skills
Skills for Real Engineers. Straight from my .agents directory.
skills
Public repository for Agent Skills
ponytail
Makes your AI agent think like the laziest senior dev in the room. The best code is the code you never wrote.
skill-up — FAQ
What is skill-up and skill-upper?+
skill-up is a CLI evaluation framework for Agent Skills, while skill-upper is an Agent Skill bundled with it that lets AI agents create evals, run skill-up, analyze failures, and iteratively improve skills through conversation. You typically interact with skill-upper in your agent client, which handles running the CLI for you.
How do I install skill-upper?+
Install globally in Codex or Claude Code using 'npx skills add https://github.com/alibaba/skill-up/tree/main/skills/skill-upper -g -a codex -y' (replace 'codex' with 'claude-code' as needed). skill-upper will guide your agent to install the skill-up CLI automatically when first needed.
Which agent clients work with skill-upper?+
skill-upper works with Codex, Claude Code, and any agent client compatible with the Agent Skills standard. The underlying skill-up CLI supports Claude Code, Codex, Qoder CLI, and Qwen Code engines for running evaluations.
Do I need API keys to use skill-up?+
You need API keys for the agent engine you're evaluating (e.g., Anthropic API key for Claude Code) and for AI judge modes if you choose agent-based evaluation instead of rule-based or script judges.
Is skill-up free and open source?+
Yes, skill-up is released under the Apache License 2.0 and is freely available on GitHub. However, running evaluations may incur costs from the underlying AI model providers (Anthropic, OpenAI, etc.) depending on your engine choice.
Can I run skill-up evaluations in GitHub Actions?+
Yes, the repository includes a GitHub Action (action.yml) that runs skill evaluations in CI on Linux runners. You configure the engine, model, and API key as workflow inputs, and it produces structured reports for each pull request.
How do I install skill-up?+
Open the source repository on GitHub and follow its README. skill-up is a skill — MCP Agents Market links you directly to the official repo.
Is skill-up free?+
skill-up is an open-source project hosted on GitHub. Check the repository for its license and any usage requirements.