Task Preparation

To: [email protected]
Subject: Eval: [Your Task Name]

[Clear description of what you want done]

Test type: with-verification
Auto-detect: true
Reviewer: [email protected]

To: [email protected]
Subject: Eval: Market Research

Research the top 3 AI agent platforms by market share.
For each, find: pricing model, key features, and target market.

Test type: with-verification
Auto-detect: true
Reviewer: [email protected]

To: [email protected]
Subject: Eval: [Your Task Name]

[Task description]

Checkpoints:
1. [First verification point]
2. [Second verification point]
3. [Third verification point]

Test type: with-verification
Reviewer: [email protected]

To: [email protected]
Subject: Eval: Financial Calculation

Calculate the R&D tax credit for $120,000 in qualified expenses.

Checkpoints:
1. Verify the base amount calculation is correct
2. Verify the final tax credit amount
3. Confirm the IRS form is properly filled

Test type: with-verification
Reviewer: [email protected]

To: [email protected]
Subject: Eval: [Your Task Name]

[Task description]

Test type: without-verification
Reviewer: [email protected]

To: [email protected]
Subject: Eval: Simple Calculation

Calculate 15% of $50,000

Test type: without-verification
Reviewer: [email protected]

{
  "taskName": "Market Research",
  "taskDescription": "Research top 3 AI agent platforms, find pricing, features, and target market",
  "autoDetectCheckpoints": true,
  "testMode": "with-verification",
  "assignedReviewer": "[email protected]"
}

{
  "taskName": "Financial Calculation",
  "taskDescription": "Calculate R&D tax credit for $120,000 QRE",
  "checkpoints": [
    {
      "stage": "base_calculation",
      "description": "Calculate base amount",
      "verificationQuestion": "Is the base amount $45,000?"
    },
    {
      "stage": "final_calculation",
      "description": "Calculate final tax credit",
      "verificationQuestion": "Is the tax credit $10,500?"
    }
  ],
  "testMode": "with-verification",
  "assignedReviewer": "[email protected]"
}

{
  "taskName": "Automated Test",
  "taskDescription": "Calculate 15% commission on $50,000 sale",
  "autoDetectCheckpoints": true,
  "testMode": "with-verification",
  "assignedReviewer": "[email protected]",
  "webhookUrl": "https://your-system.com/webhook/eval-complete",
  "externalId": "test-001"
}

Good: "Calculate 15% commission on $50,000"
Bad: "Calculate some commission"

Good: "Expected result should be $7,500 (15% of $50,000)"
Bad: "Calculate it"

Good: "Get Bitcoin price from CoinMarketCap"
Bad: "Get crypto price"

Good: "1) Research competitors 2) Analyze pricing 3) Create table 4) Email summary"
Bad: "Do competitor analysis"

Bad: "Do some research about AI"
Good: "Research top 3 AI companies: OpenAI, Anthropic, Google DeepMind"

Bad: Missing "Reviewer:" field
Good: "Reviewer: [email protected]"

Bad: Both "Checkpoints: 1, 2, 3" AND "Auto-detect: true"
Good: Choose ONE approach