> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixus.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluation System Overview

> Test AI agents with human-in-the-loop verification at critical checkpoints

## What is the evaluation system?

The mixus evaluation system enables you to test and measure AI agent performance with optional human verification at critical checkpoints. This unique approach ensures agents work correctly step-by-step before executing actions.

## How it works

The evaluation system accepts tasks via two methods:

<CardGroup cols={2}>
  <Card title="Email Submission" icon="envelope">
    Send tasks to `agent@mixus.com` with subject starting with "Eval:"
  </Card>

  <Card title="API Submission" icon="code">
    POST to `/api/eval/create-task-agent` for programmatic access
  </Card>
</CardGroup>

### Process flow

```mermaid theme={null}
graph LR
    A[Submit Task] --> B[Create Agent]
    B --> C[Execute Steps]
    C --> D{Checkpoint?}
    D -->|Yes| E[Human Verifies]
    D -->|No| F[Continue]
    E --> F
    F --> G{More Steps?}
    G -->|Yes| C
    G -->|No| H[Complete]
```

1. **Task Submission** - Submit via email or API
2. **Agent Creation** - System converts task to executable agent
3. **Execution** - Agent runs through steps using AI and tools
4. **Verification** - Human reviews and approves critical steps
5. **Results** - Metrics and outcomes returned

## Test modes

### With Verification (HITL)

Agent pauses at checkpoints for human approval. Ideal for:

* Testing agent reliability
* High-stakes operations
* Learning agent behavior

**Checkpoint Detection:**

* **Auto-detect** - AI determines optimal verification points
* **Manual** - You specify exact checkpoints

### Without Verification (Baseline)

Agent runs autonomously without human oversight. Ideal for:

* Speed benchmarks
* Comparing with/without human
* Low-risk tasks

## Key capabilities

<CardGroup cols={2}>
  <Card title="AI Checkpoint Detection" icon="brain">
    Automatically identifies where verification is needed based on task risk
  </Card>

  <Card title="Email Integration" icon="envelope">
    Submit tasks and receive notifications via email
  </Card>

  <Card title="API Access" icon="plug">
    Programmatic integration for automation and batch processing
  </Card>

  <Card title="Metrics Tracking" icon="chart-line">
    Detailed metrics on performance, cost, and human verification
  </Card>

  <Card title="Webhook Callbacks" icon="webhook">
    Real-time notifications when evaluations complete
  </Card>

  <Card title="Dashboard View" icon="gauge">
    Visual tracking of all evaluations at app.mixus.ai/eval
  </Card>
</CardGroup>

## Use cases

### Evaluation & Testing

Test AI agents systematically:

* Run benchmark suites
* Compare different approaches
* Measure reliability improvements
* Validate agent behavior

### Quality Assurance

Ensure agents work correctly:

* Verify calculations before execution
* Review communications before sending
* Approve transactions before processing
* Validate data changes before committing

### Benchmarking

Compare performance:

* With vs without human oversight
* Different models or configurations
* Cost vs accuracy tradeoffs
* Speed vs reliability metrics

### Integration Testing

Test external integrations:

* Custom evaluation frameworks
* CI/CD pipelines
* Automated testing workflows

## Why human-in-the-loop?

Traditional AI evaluation measures final outcomes but misses critical failures during execution:

<Warning>
  **Without Verification:** - Sending wrong emails before catching mistakes -
  Making purchases with incorrect amounts - Deleting data that shouldn't be
  removed
</Warning>

<Check>
  **With Verification:** - Review agent work BEFORE execution - Approve or
  reject critical steps - Provide hints to guide agents - Prevent costly
  mistakes
</Check>

## Getting started

<Steps>
  <Step title="Choose submission method">
    Email for quick tests, API for automation
  </Step>

  <Step title="Prepare your task">
    Write clear task descriptions with expected outcomes
  </Step>

  <Step title="Submit evaluation">
    Send via email or API with test mode and reviewer
  </Step>

  <Step title="Verify checkpoints">
    Review and approve agent work at critical steps
  </Step>

  <Step title="Track results">View metrics in dashboard or via API</Step>
</Steps>

## Next steps

<CardGroup cols={2}>
  <Card title="Quick Start" icon="rocket" href="/evaluation/getting-started">
    Submit your first evaluation task
  </Card>

  <Card title="Task Preparation" icon="pen" href="/evaluation/task-preparation">
    Learn how to write effective tasks
  </Card>

  <Card title="API Reference" icon="code" href="/eval-api">
    Complete API documentation
  </Card>

  <Card title="Examples" icon="lightbulb" href="/evaluation/examples">
    Ready-to-use example tasks
  </Card>
</CardGroup>

## Support

Questions about the evaluation system?

* **Email:** [support@mixus.ai](mailto:support@mixus.ai)
* **Dashboard:** [app.mixus.ai/eval](https://app.mixus.ai/eval)
* **API Keys:** [app.mixus.ai/integrations/api-keys](https://app.mixus.ai/integrations/api-keys)