Evaluate a model

Score a base model or a trained checkpoint on a gym's held-out tasks. The model never trains on these tasks, so the score shows how it handles new work.

You need the CLI signed in with the benchmark scope, and an evaluation-ready gym.

Start an evaluation#

gft eval start shopify \
  --model Qwen/Qwen3.5-9B \
  --checkpoint checkpoint_manifest.json \
  --follow --exit-status

--checkpoint takes the checkpoint manifest that training produced. Leave it out to evaluate the base model.

Evaluations always run in the background. GFT runs the model on its own inference service, so you never configure an endpoint.

Follow and manage evaluations#

gft run list --kind evaluation
gft run show <evaluation-id>
gft run watch <evaluation-id> --exit-status
gft run cancel <evaluation-id>

You can cancel an evaluation only while it is still queued.

In Studio#

Open Benchmarks, or choose Evaluate adapter on a trained model.