anakin
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up
Anakin is an open-source web scraping agent skill that converts websites into LLM-ready markdown or structured JSON. It features intelligent handler chains (HTTP → anti-detect browser → API fallback), per-domain scraping strategies, and ML-powered proxy selection using Thompson Sampling. Developers can self-host the entire stack with a single command or integrate it as an OpenClaw skill to give AI agents robust web data extraction capabilities for RAG pipelines, research tasks, and data collection workflows.
Key Features
Use Cases
- 01Powering RAG pipelines with clean, LLM-ready content from any website
- 02Enabling AI research agents to gather structured data from multiple sources autonomously
- 03Building data extraction workflows that automatically handle JavaScript-heavy sites and anti-bot protection
- 04Creating self-hosted web scraping infrastructure with no recurring API costs
- 05Extracting structured product, pricing, or contact information from competitor websites for analysis
- 06Augmenting AI assistants with real-time web data retrieval capabilities
Related Skills
View moresuperpowers
An agentic skills framework & software development methodology that works.
skills
Skills for Real Engineers. Straight from my .agents directory.
skills
Public repository for Agent Skills
ponytail
Makes your AI agent think like the laziest senior dev in the room. The best code is the code you never wrote.
anakin — FAQ
What is the Anakin agent skill?+
Anakin is a self-hosted web scraping agent skill that converts websites into clean markdown or structured JSON for AI agents. It uses intelligent fallback chains and anti-detect browsers to handle both static and JavaScript-heavy sites, making web data accessible to LLM-powered workflows.
How do I install Anakin for use with AI agents?+
Clone the repository and run 'make up' to launch the full Docker stack, or install it as an OpenClaw skill by copying the openclaw-skill directory to ~/.openclaw/workspace/skills/anakinscraper. For minimal setup, just run the Go server with 'go run cmd/server/main.go'.
Which AI clients and frameworks work with Anakin?+
Anakin works with OpenClaw as a native skill and can be integrated into any AI agent framework via its REST API. The Anakin CLI also supports direct command-line access for terminal-based workflows.
Do I need API keys to use Anakin?+
No API keys are required for basic web scraping. Optional: set GEMINI_API_KEY for AI-powered JSON extraction, ANAKIN_API_KEY for hosted API fallback on hard-to-scrape sites, or configure your own proxy pool for geo-targeted scraping.
Is Anakin free to use?+
Yes, Anakin is open-source under AGPL-3.0 and free to self-host. The code, Docker setup, and OpenClaw skill are all included at no cost.
What are the prerequisites for running Anakin?+
Minimal setup requires only Go 1.25+. For full functionality, install Docker and Docker Compose to run the browser service (Camoufox) and optional PostgreSQL persistence. The browser service requires Python 3.11+ if running outside Docker.
How do I install anakin?+
Open the source repository on GitHub and follow its README. anakin is a skill — MCP Agents Market links you directly to the official repo.
Is anakin free?+
anakin is an open-source project hosted on GitHub. Check the repository for its license and any usage requirements.