ArtemisKit Documentation

Ship AI You Trust

ArtemisKit is the open-source toolkit for testing LLM applications. One CLI. Three capabilities:

Quick Start

From zero to your first test in 5 minutes. Install, configure, run.

Core Concepts

Understand scenarios, expectations, providers, and evaluators.

CLI (@artemiskit/cli)

Run evaluations from the terminal, integrate with CI/CD, generate reports. Most teams start here.

SDK (@artemiskit/sdk)

Embed evaluations in your code. Includes Guardian Mode for runtime protection and Jest/Vitest matchers.

# Install
npm install -g @artemiskit/cli

# Run a scenario
akit run scenarios/hello-world.yaml

Package	npm Name	Description
CLI	`@artemiskit/cli`	Command-line interface (`artemiskit`, `akit`)
SDK	`@artemiskit/sdk`	Programmatic API, Guardian Mode, test matchers
Core	`@artemiskit/core`	Core evaluation engine
Reports	`@artemiskit/reports`	HTML/JSON report generation
Red Team	`@artemiskit/redteam`	Security testing and mutations
OpenAI Adapter	`@artemiskit/adapter-openai`	OpenAI and Azure OpenAI adapter
Anthropic Adapter	`@artemiskit/adapter-anthropic`	Anthropic Claude adapter
Vercel AI Adapter	`@artemiskit/adapter-vercel-ai`	Vercel AI SDK adapter
LangChain Adapter	`@artemiskit/adapter-langchain`	LangChain chains and agents
DeepAgents Adapter	`@artemiskit/adapter-deepagents`	DeepAgents multi-agent systems

Examples & Cookbook

CI/CD integration, security testing, regression detection recipes.

Guardian Mode

Runtime protection with semantic validation and injection detection.