Lego-RL

Configuration

The values you must supply before anything can run, and how to verify them

A run is one .env file passed to one runner. That file is the whole configuration: the runner sources it, then composes the template modules it names. Everything on this page is required. Optional accelerations and the knobs that change how a run trains are in the Configuration Reference.

The runners do not read site.env

scripts/lib/site.env is sourced by scripts/setup_env.sh and the older single-file scripts, not by scripts/{train,eval,infer}/*.sh. Cluster values such as K8S_KUBECONFIG belong in the run config, which is what the configs in scripts/train/configs/ do. To pull a site file in anyway, see Config layers.

1. Run config

Start from the demo config rather than the bare template; it already carries a working value for everything except your own inputs.

cp scripts/train/examples/demo.env scripts/train/configs/demo.env
$EDITOR scripts/train/configs/demo.env

Four values are marked CHANGEME. They are the inputs Lego-RL cannot invent:

VariableSet it to
MODEL_PATHthe policy checkpoint on local disk, typically SFT-trained
TRAIN_FILESthe training task index — a parquet pointer table
VAL_FILESthe held-out task index
K8S_KUBECONFIGthe kubeconfig for the cluster that will run the sandboxes

Two more values are not CHANGEME, but check them before launching:

VariableSet it to
TOOL_CALL_PARSERthe parser matching the checkpoint's chat template. qwen3_coder for qwen3.5/3.6 (XML); a mismatch produces tool calls the runtime cannot read
HARBOR_K8S_INLINE_BUILDtrue in demo.env: pods build each task image on first use. No registry needed, but slow. For a real run set it false and point HARBOR_OPENSWE_IMAGE_REGISTRY at prebuilt images

k8s needs exactly one image source

KubernetesEnvironment only ever pulls images, never builds them. Setting both HARBOR_OPENSWE_IMAGE_REGISTRY and HARBOR_K8S_INLINE_BUILD=true, or neither, is a validation FATAL — with no image source every pod ends in ImagePullBackOffenv_setup_failed → reward 0.

Everything else in demo.env — the four template axes, topology, batch scale, cadence — is a working default. The Configuration Reference covers changing them.

Task indexes are not the tasks

TRAIN_FILES and VAL_FILES are thin parquet tables pointing at Harbor task directories, not the repositories themselves. Build one with utils/create_task_index.py, or follow Data Preparation.

2. Verify

This resolves the config, composes its templates and runs every assertion that reads config values. It needs no GPU, no cluster and no data, so it is worth running before the inputs above exist:

bash scripts/train/train.sh --structure-only scripts/train/configs/demo.env
═════════════════════ run configuration (train) ═════════════════════
  config       .../scripts/train/configs/demo.env
  experiment   project=lego-rl-demo  exp=demo
  axes         mode=sync engine=veomni scaffold=ohsdk backend=k8s
  model        /models/Qwen3.5-35B-A3B  [unknown]
               served=vllm_model  tool_parser=qwen3_coder
  data         train=/data/harbor_indexes/train.parquet
               val  =/data/harbor_indexes/val.parquet
  topology     1 nodes × 8 gpu (colocated)  SP=8 → dp=1  vllm_TP=4
  batch        8 prompts × 4 resp = 32 trials/step  mini=8  dynamic_bsz=True
  context      prompt=30000 + resp=58304 = 88304
  algorithm    grpo  lr=1e-6  lr_sched=constant  kl_loss_coef=0.001
  rollout      temp=1.0 top_p=1.0  val_temp=0.7  val_n=1
               R3=False (model is unknown)  rollout_is=null  max_num_seqs=<verl default>
  schedule     save_freq=5  test_freq=5  val_before_train=False  epochs=1
  logs         train=.../logs/demo.log
               trials=.../harbor_trials/lego-rl-demo/demo
══════════════════════════════════════════════════════════════════════

───────────────────────── preflight config check ─────────────────────────
  ✓ OK     tool_parser=qwen3_coder ✓ matches qwen3.5/3.6 (XML template)
  ✓ OK     topology consistent: NNODES=1 = train 1 + rollout 0
  ✓ OK     device-mesh valid: dp=1 (train_world 8 / sp 8)
  ✓ OK     35B VRAM: train_node=1 window=88304 in safe zone
  ✓ OK     veomni: fused_kernels=False ✓
  ✓ OK     veomni: activation_offload=False ✓
  ⊘ SKIP   R3 × MoE check: needs the checkpoint's config.json on disk
  ✓ OK     AGENT_NAME=null ✓ (ohsdk uses import_path)
  ✓ OK     image: registry empty + inline_build=true ✓ (vanilla build-on-demand, slow but portable)
  ⊘ SKIP   K8S_KUBECONFIG existence: needs the cluster's kubeconfig on disk
  ⚠ WARN   NYDUS_MIRROR empty: if val uses official swebench images and pod has no
           Docker Hub egress → val reward=0
  ✓ OK     lr_scheduler=constant ✓ (avoids the total_training_steps=-1 → lr=0 collapse)
  ✓ OK     val-specific timeout=4800s ✓
  ⊘ SKIP   path existence (MODEL_PATH TRAIN_INDEX VAL_INDEX): structure-only run
──────────────────────────────────────────────────────────────────────
[preflight] structure-only: all config checks passed (1 WARN). Filesystem checks skipped.
[structure-only] config and install look sound; not launching.

The [unknown] next to the model and R3=False (model is unknown) are expected here: --structure-only does not open the checkpoint. Both resolve once the path exists.

Exit code 0 with no ✗ FATAL means the venv imports, the config resolves, the template modules compose and the structural rules hold.

What the SKIP lines mean

--structure-only says nothing about whether your checkpoint is loadable, your task images are pullable or your cluster is reachable. Those are checked by the full run validation that Demo Run performs before launching.

Common failures

OutputMeaningFix
ModuleNotFoundError: veomnithe venv build did not finishre-run bash scripts/setup_env.sh; see Setup FAQ
[FATAL] config not foundthe path resolved against your shell's working directorypass the repo-root-relative path, from the repo root
⚠ WARN model 'CHANGEME' not in known rulesMODEL_PATH is still a placeholderexpected until you fill it in
K8S_KUBECONFIG: K8S_KUBECONFIG is requiredthe k8s template refuses to compose without it, before validation runsset it in the config; the runners do not read site.env
✗ FATAL IMAGE_REGISTRY empty and INLINE_BUILD=falseno image sourceset HARBOR_K8S_INLINE_BUILD=true, or point HARBOR_OPENSWE_IMAGE_REGISTRY at prebuilt images
✗ FATAL on a rulethe config violates itthe line names the variable and the failure it prevents

Next

Demo Run — launch it.

On this page