CLI Overview

ArtemisKit CLI

The ArtemisKit CLI (@artemiskit/cli) is the primary way to run LLM evaluations.

Installation

npm install -g @artemiskit/cli

Commands

Command	Description
artemiskit run	Run scenario-based evaluations
artemiskit redteam	Security red team testing
artemiskit stress	Load and stress testing
artemiskit compare	Compare two evaluation runs
artemiskit report	Generate reports from saved runs
artemiskit history	View run history
artemiskit init	Initialize configuration

You can use akit as a shorter alias for artemiskit.

Quick Start

# Set your API key
export OPENAI_API_KEY="sk-..."

# Run a scenario (results are saved by default)
akit run scenarios/my-test.yaml

# Run without saving results
akit run scenarios/my-test.yaml --no-save

Configuration

Create artemis.config.yaml for default settings:

provider: openai
model: gpt-5

providers:
  openai:
    apiKey: ${OPENAI_API_KEY}
    timeout: 60000

output:
  format: json
  dir: ./artemis-output

Next Steps

• Installation Guide
• Getting Started
• Scenario Format
• Command Reference
• Storage Backends
• Providers