Introduction to ArtemisKit

Welcome to ArtemisKit

ArtemisKit is an open-source toolkit for evaluating LLM applications. It provides systematic testing for:

• Quality Assurance — Test prompts and responses with scenario-based testing
• Security — Red team your LLM for vulnerabilities
• Performance — Stress test under load

Packages

ArtemisKit is available as multiple packages:

Package	npm Name	Description
CLI	@artemiskit/cli	Command-line interface (artemiskit, akit)
Core	@artemiskit/core	Core library (coming soon)

Which Should I Use?

CLI (@artemiskit/cli)

Best for most users. Run evaluations from the command line, integrate with CI/CD, and generate reports.

Get started →

API (@artemiskit/core)

For embedding evaluations in your own code, test frameworks (Jest, Vitest), or custom tooling. Coming in v0.2.0.

Learn more →

Quick Example (CLI)

# Install
npm install -g @artemiskit/cli

# Run a scenario
akit run scenarios/hello-world.yaml

Next Steps

• CLI Documentation — Get started with the CLI
• Providers — Configure LLM providers (OpenAI, Anthropic, Azure, Vercel AI)
• Storage — Store results locally or in Supabase
• API Documentation — Programmatic usage (coming soon)