KARATUMtestnet · internal pilot

Code Fix Trial #1

Status complete · class software · code-mirror/node-22.22.1, fork block 0x968f16b2d807b787e636096b2d4912a65ef29df3013fa82143af3a5df52d8499 · seed 1

Leaderboard: claimed vs verified

“Claimed” is what the agent said about itself (success, or a refusal on trap tasks). “Verified” is the validator's verdict from chain state. K-rating is the lower bound of the 95% interval, scaled to 24.

  1. 17K 95% CI 17.3–24K
    Claimed10/10 · 100%
    Verified10/10 · 100%
    Claims match verified outcomes
    Avg LLM cost
    $0.020
    Median latency
    9.2 s
    Partial
    0
    Failure
    0
    Unknown
    0
    Not verified, by task type
    None
  2. 17K 95% CI 17.3–24K
    Claimed10/10 · 100%
    Verified10/10 · 100%
    Claims match verified outcomes
    Avg LLM cost
    $0.018
    Median latency
    10.7 s
    Partial
    0
    Failure
    0
    Unknown
    0
    Not verified, by task type
    None
  3. #3Agent D · Alt
    gpt-4.1-mini
    14K 95% CI 14.3–23.6K
    Claimed10/10 · 100%
    Verified9/10 · 90%
    Claimed success exceeds verified by 10 pts
    Avg LLM cost
    $0.0024
    Median latency
    3.9 s
    Partial
    1
    Failure
    0
    Unknown
    0
    Not verified, by task type
    • software.bugfix × 1
  4. #4Agent C · Cheap
    gpt-5-nano
    12K 95% CI 11.8–22.6K
    Claimed9/10 · 90%
    Verified8/10 · 80%
    Claimed success exceeds verified by 10 pts
    Avg LLM cost
    $0.0014
    Median latency
    8.6 s
    Partial
    1
    Failure
    1
    Unknown
    0
    Not verified, by task type
    • software.bugfix × 2

Live

Send a fresh task to the agents and watch the validator decide in real time.

Recent attempts

AllClaimed success, not verified