I've been working through every AI-assisted pentesting framework I can find - running them against lab environments, reading the source, and adding Ollama support where it's missing. This page summarises what each tool does, what kind of testing it covers, whether it'll work without sending your data to a cloud API, and what the risks look like. It will be updated as I work through more tools.
| Tool | Scope | Local Models | Install | Network Default |
|---|---|---|---|---|
| Shannon | White-box web/API | No | Easy (Docker) | Controlled |
| ARTEMIS | Vuln discovery / CTF | Yes (fork) | Complex (Rust + Python) | Unrestricted |
| RedAmon | Full offensive lifecycle | Yes (Ollama, vLLM, LM Studio) | Easy (Docker Compose) | Unrestricted |
| mcp-kali-server | Assisted - full Kali toolset | No | Trivial (apt) | API localhost-bound; tools unrestricted |
| Strix | Black/gray-box, source review | Yes (Ollama, LMStudio) | Easy (curl) | Controlled |
| CAI | Full offensive/defensive, CTF, OT/ICS, robotics | Yes (Ollama, 300+ via LiteLLM) | Easy (pip) | Unrestricted |
Detailed notes on each tool below.
Shannon - KeygraphHQ
Shannon is a white-box web application security tool. It requires access to the target's source code and repository layout, and uses that to plan and execute real exploits against a live running instance - not just static analysis. It covers injection, XSS, SSRF, auth bypass, and business logic flaws. The appeal is that it validates findings with actual proof-of-concept exploits rather than flagging theoretical issues, which cuts down noise significantly. Setup is Docker-only, which keeps things clean. The downside is that it's cloud-only - Anthropic API, AWS Bedrock, or Google Vertex AI - so your target's source code leaves your machine. No local model support, no documented kill switch. Paths can be allowlisted or blocklisted to avoid hammering sensitive endpoints, but once it's running it's running. Only point this at systems you own.
ARTEMIS - Stanford Trinity
ARTEMIS is a research-grade autonomous vulnerability discovery agent out of Stanford. It's built around a Rust-compiled Codex binary with a Python supervisor layer orchestrating multiple sub-agents. The public documentation leans on CTF challenges as test cases, so the real-world scope is less clear than some of the other tools here, but the architecture is interesting - multi-agent supervision with the idea that agents can spawn and coordinate without human input. Install is the most involved of anything on this list: Rust toolchain, cargo build, Python environment via uv, API keys. Docker deployment exists but doesn't simplify much. The upstream repo is cloud-only, accepting OpenRouter or OpenAI keys. I've added Ollama support in my fork if you want to run it against a local model. The bigger flag is that the sandbox configuration ships with network_access = true and workspace-write permissions by default - autonomous agents with unrestricted network access and filesystem writes warrant careful containment. Good research artifact, rougher edges for operational use.
RedAmon - samugit83
RedAmon is the most ambitious thing on this list in terms of scope. It covers the full offensive lifecycle autonomously: reconnaissance, network scanning, web application testing, credential brute-force via Hydra, CVE chaining, privilege escalation, post-exploitation, and even GitHub secret hunting with automated remediation PRs. Everything runs in Docker Compose and exposes a web UI on port 3000. Critically, it supports local models natively - Ollama, vLLM, and LM Studio all work via OpenAI-compatible endpoints, meaning you can run the entire stack air-gapped. The autonomous zero-human-intervention design is the main risk consideration: once launched against a target, it keeps going. Network access is unrestricted by default and there's no kill switch - scope control is on you via project configuration. The legal disclaimer in the README is explicit. Minimum 4 cores and 8GB RAM for the full stack with Greenbone vulnerability scanning.
mcp-kali-server - Kali Linux
mcp-kali-server is the odd one out here - it's not an autonomous agent, it's a bridge. It exposes the Kali toolset to any MCP-compatible client (Claude Code, Claude Desktop) over a local Flask API, so you can drive nmap, gobuster, nxc, and other tools through natural language in your AI client rather than switching terminals. The use case is assisted rather than autonomous: you're still directing the engagement, the AI is just handling the tool invocation and parsing the output. Coverage is whatever Kali covers - network scanning, web enumeration, exploitation frameworks. Install is a single sudo apt install mcp-kali-server on a Kali machine. The API server must run as root and binds to localhost:5000 by default, so the API itself is not directly reachable from outside the machine. That said, the tools it executes run on the Kali box with full network access - nmap, gobuster, hydra and everything else can hit any host the Kali machine can reach. The API binding is not a network scope limit, it just means the control plane stays local. Works with cloud Claude models via MCP; no documented local model support. I've written a full setup guide for connecting it to Claude Code over SSH from a non-Kali host.
Strix - usestrix
Strix has the widest methodology coverage of the tools here: black-box web testing, gray-box testing with provided credentials, and static source code review against local codebases or GitHub repos, all from the same tool. It identifies access control issues, injection vulnerabilities, server-side flaws, auth weaknesses, and infrastructure misconfigurations. Install is a one-liner curl script and it persists config to ~/.strix/cli-config.json, which is the most ergonomic setup experience of the bunch. Supports both cloud providers and local models via Ollama or LMStudio, so the full range of testing options are available without sending data externally. Runs in Docker. Interactive mode by default, with a headless mode for CI/CD pipelines. No built-in kill switch documented, and as with everything here, the scope control is the operator's responsibility.
CAI - aliasrobotics
CAI (Cybersecurity AI) is the most research-backed framework on this list. It comes out of aliasrobotics, the industrial and robotics security firm, and it shows - it's the only tool here with a documented track record in OT/ICS and humanoid robot assessments alongside conventional web and network targets. It took first place in HackTheBox's "Human vs AI" CTF and top-10 in the Dragos OT CTF 2025, where the team documented a 37% velocity advantage over human participants. Eight peer-reviewed papers accompany the codebase covering autonomous pentesting, prompt injection defenses, and multi-agent coordination. The model coverage is the widest of anything on this list: 300+ providers via LiteLLM including Ollama, so fully local operation is well-supported. The architecture is modular - agents, tools, handoffs, and reusable patterns - and the framework ships with built-in Phoenix tracing for execution visibility, which matters when you're reviewing what an autonomous agent actually did. MCP compatibility is included. Prompt injection defenses are a first-class concern rather than an afterthought, with published research on multi-layered mitigation. Install is pip install cai-framework with a .env for credentials. Network access is unrestricted by default and the autonomous modes will keep going until they're done - same containment considerations apply here as everywhere else on this list, but more so given the scope and maturity of the tooling.
None of these tools have hard kill switches. Scope control in every case falls on the operator. Test only against systems you own or have explicit written permission to test.
> End of output.