Rollout Generation

Runs the agent over a Harbor task index with the policy frozen. Nothing is trained and no weights are written; the run exists to produce data about the tasks and about the model's behaviour on them.

base task index ─▶ [ rollout generation ] ─▶ filtered index + trajectory corpus ─▶ training

Two things come out of it:

• A per-instance difficulty measurement. Each instance is attempted N_TRIALS times and scored by the same verifier training uses. <RESULTS_DIR>/summary.csv records n_pass, n_fail and pass_rate per instance, and OUTPUT_INDEX is the subset with 0 < pass_rate < 1: a task index like any other. This is stage 4 of data preparation.
• A trajectory corpus. Every trial's full transcript is kept under RESULTS_DIR, for behaviour analysis or as SFT data.

Prerequisites

• A base task index covering every candidate instance.
• Sandbox images for those instances, pullable from the cluster.
• A checkpoint to sample from — the model whose difficulty you want measured.
• A sampling temperature that matches the one training will use. Measure at a temperature you will not train at and the resulting index describes a policy that does not exist.

Setup

cp scripts/infer/_template.env scripts/infer/configs/my_infer.env

Identity and model

# scripts/infer/configs/my_infer.env
PROJECT_NAME=infer-qwen36-27b
SCAFFOLD=ohsdk
BACKEND=k8s
MODEL_PATH=/models/Qwen3.6-27B      # or a checkpoint: /models/global_step_N_hf

Data and output

INDEX_FILE=/data/base_index.parquet      # every candidate instance
# INSTANCES_FILE=/data/shard0.txt         # this node's shard (empty = whole index)
RESULTS_DIR=/data/results/node0
OUTPUT_INDEX=/data/selected_node0.parquet

RESULTS_DIR holds the trajectories and summary.csv; OUTPUT_INDEX is the filtered index written at the end. Both are per-node paths — two nodes writing the same directory will overwrite each other's summary.

Sampling

N_TRIALS=4                                # attempts per instance (the k in pass@k)
N_CONCURRENT=80
TEMPERATURE=1.0

N_TRIALS sets the resolution of the measurement: at 4 attempts a task can only land on 0, 0.25, 0.5, 0.75 or 1, and only the middle three survive filtering.

Serving

GEN_TP=8
GPUS_PER_NODE=8
VLLM_MAX_MODEL_LEN=128000

One vLLM server per node, sized by GEN_TP. There is no cross-node tensor or data parallelism in this runner — more nodes means more independent servers, each working its own shard.

Check

bash scripts/infer/infer.sh --dry-run scripts/infer/configs/my_infer.env
# or:  /rl:check scripts/infer/configs/my_infer.env
# PREFLIGHT_ONLY (scripts/lib/preflight.sh) is wired into the train runner only.

Run

nohup setsid bash scripts/infer/infer.sh scripts/infer/configs/my_infer.env \
  > logs/infer_$(date +%m%d).out 2>&1 &
# or:  /rl:run scripts/infer/configs/my_infer.env

Cost intuition. The work is instances × N_TRIALS trials, drained N_CONCURRENT at a time, so a full pool over a large index is a multi-day job. Runs resume automatically — completed trials are detected and skipped, so a job killed at hour six restarts where it stopped, and sharding across nodes is the way to make it finish sooner.

Sharding across nodes

Each node runs the same command against its own copy of the config, differing in three lines. Nothing coordinates them, so a node that dies takes only its own shard down:

# node 0's config                     # node 1's config
INSTANCES_FILE=/data/shard0.txt       # /data/shard1.txt
RESULTS_DIR=/data/results/node0       # /data/results/node1
OUTPUT_INDEX=/data/selected0.parquet  # /data/selected1.parquet

# on each node:
bash scripts/infer/infer.sh scripts/infer/configs/my_infer.env

Union the per-node OUTPUT_INDEX files afterwards; see Output.

Output

logs/<exp>.log                        progress, then the final paths
<RESULTS_DIR>/summary.csv             n_pass / n_fail / pass_rate per instance
<RESULTS_DIR>/<instance>/<trial>/     full transcript per attempt
<OUTPUT_INDEX>                        the filtered index

tail -F logs/<exp>.log
# === inference done: results=/data/results/node0  selected=/data/selected_node0.parquet ===

OUTPUT_INDEX is the TRAIN_FILES of a Training run. Because it carries the same three pointer fields as any other index, unioning shards across nodes and re-banding by difficulty are both plain dataframe operations:

import pandas as pd
idx = pd.concat([pd.read_parquet(p) for p in shard_paths], ignore_index=True)
idx.to_parquet("/data/train_index.parquet")

Dashboard

A rollout-generation run writes no step metrics and does not appear on the board. Read it through summary.csv and the per-trial transcripts under RESULTS_DIR.

Cleanup

bash scripts/cleanup_before_run.sh

Keep RESULTS_DIR — it is what makes the next attempt resume instead of re-running trials you have already paid for.

Parameters