OpenAI

OpenAI Provider

Use ArtemisKit with OpenAI’s GPT models.

Setup

1. Get API Key

Get your API key from the OpenAI Dashboard.

2. Set Environment Variable

export OPENAI_API_KEY="sk-..."

3. Configure Scenario

name: my-test
provider: openai
model: gpt-5

cases:
  - id: example
    prompt: "Hello"
    expected:
      type: contains
      values: ["hello"]
      mode: any

Available Models

GPT-5 Family (Latest)

Model	Context	Description
`gpt-5`	400K	Latest flagship model, best overall performance
`gpt-5.1`	400K	Enhanced version with improved reasoning
`gpt-5.2`	400K	Premium variant with extended capabilities
`gpt-5-mini`	128K	Fast and cost-effective
`gpt-5-nano`	32K	Ultra-lightweight for simple tasks

GPT-4.1 Family

Model	Context	Description
`gpt-4.1`	1M	Extended context specialist
`gpt-4.1-mini`	256K	Balanced performance and cost
`gpt-4.1-nano`	64K	Lightweight variant

GPT-4o Family

Model	Context	Description
`gpt-4o`	128K	Multimodal, versatile
`gpt-4o-mini`	128K	Fast and highly cost-effective

O-Series (Reasoning Models)

Model	Description
`o1`	Advanced reasoning, complex problem solving
`o3`	Balanced reasoning model
`o3-mini`	Fast reasoning for simpler tasks
`o4-mini`	Latest fast reasoning model

Note: O-series models use internal “thinking tokens” that are billed as output tokens but not visible in responses. Actual costs may be higher than estimated.

Config File

Configure OpenAI in your artemis.config.yaml:

provider: openai
model: gpt-5

providers:
  openai:
    apiKey: ${OPENAI_API_KEY}
    organization: ${OPENAI_ORG_ID}  # Optional
    timeout: 60000
    maxRetries: 2

Scenario-Level Options

Override settings in your scenario file:

name: my-test
provider: openai
model: gpt-5
temperature: 0
seed: 42
maxTokens: 1000

cases:
  - id: example
    prompt: "Hello"
    expected:
      type: contains
      values: ["hello"]
      mode: any

Organization ID

If you belong to multiple organizations:

export OPENAI_ORG_ID="org-..."

Or in your config:

providers:
  openai:
    apiKey: ${OPENAI_API_KEY}
    organization: ${OPENAI_ORG_ID}