Setup FAQ

The questions that come up most often on a first run. They are ordinary setup friction rather than failures; when something is genuinely broken mid-run, go to the Failure Playbook, which is indexed by symptom.

Q1: I opened Claude Code but there are no /rl:* commands

The plugin is declared in this repository's .claude/settings.json, which registers .claude/plugins/ as a local marketplace and enables rl@harbor-rl. That file is project-scoped, so Claude Code only sees it when it starts from the repository root:

cd /path/to/harbor-verl-train
claude

If the session was already open, run /reload-plugins, then type / and search for rl. To load the plugin from somewhere else, point at it directly:

claude --plugin-dir /path/to/harbor-verl-train/.claude/plugins/rl-plugin

Q2: /rl:check printed ✗ FATAL and refused to launch. Is something broken?

No — that is the job. Validation runs before any GPU is touched precisely so a mistake costs seconds instead of a multi-hour run. Read the assertion it named, fix that line in the config, and run the same check again.

The nine assertion classes and what each one means are in Run Validation.

Q3: Where do cluster addresses and secrets go?

In scripts/lib/site.env, which is gitignored, never in a run config. Copy the example and edit it once per cluster:

cp scripts/lib/site.example.env scripts/lib/site.env

Keep API keys, kubeconfig paths and registry credentials there or in environment variables. Because no run config holds a cluster address, a config is portable to a colleague's cluster as soon as they write their own site.env. The full variable surface is in Site variables.

Q4: Every trial fails with env_setup_failed on the first step

Almost always the sandbox cannot get an image. Check, in this order:

1. HARBOR_OPENSWE_IMAGE_REGISTRY points at a registry every sandbox node can pull from, and the nodes trust it (insecure registries need a hosts.toml).
2. If no registry exists, INLINE_BUILD=true — slower, and it needs egress from the task pod.
3. agent_name and SCAFFOLD agree; a mismatch produces empty trajectories that look identical to an image failure (Run Validation §6).

Q5: The run is training, but the dashboard shows no curves

The dashboard scans a log directory. Most configs override HARBOR_LOG_DIR to a per-experiment directory under the shared trials tree, so the run's log never lands in the logs/ the board is serving — the run is healthy and simply invisible.

Point the board at both:

python3 webui/server.py --port 8090 --log-dir logs --extra-log-dir <that dir>

/rl:dashboard probes for this and says which directories it found. See Monitoring.

Q6: Do I need a Kubernetes cluster?

No. Kubernetes is the production backend, but a single Docker daemon — local or remote — runs the same trials on one machine. Set BACKEND=docker and see Sandbox Backends. Expect lower concurrency: one host's CPU and disk cap how many sandboxes can run at once.

Q7: Where do checkpoints, logs and trajectories end up?

Results & Artifacts covers what each one contains, including which files the merger needs and which are safe to delete.

Q8: My val score is low or zero. Is the policy bad?

Check the infrastructure before believing the number. A zero or depressed val score is more often a broken image pull, a grading path without network access, or a coverage shortfall than a real regression, and each has a distinct fingerprint — start by reading num_turns. See Low val scores that are not the model.

Q9: My SSH session dropped and the run died

Launch long runs detached. A training run lasts hours, so a disconnect should not be able to end it:

tmux new -s rl
# or
nohup setsid bash scripts/train/train.sh scripts/train/configs/my_run.env &

/rl:run already launches in the background for this reason.