Training Startup

Two issues that bite right at launch: a crash on step 0, and a slow-import tax paid by every launch.

Tool-call parser mismatch

Issue. Training crashes on the very first step with:

response_mask must contain at least one valid token (1)

vLLM is healthy and the model clearly generates text, but the entire batch has zero tool calls (e.g. 0/1024 trials) and the trajectories are single-turn.

Reason. The vLLM rollout is configured with the wrong tool-call parser for the model's output format. A Qwen3-Coder model emits XML tool calls:

<function=EnterWorktree>...</function>

but the rollout config sets tool-call-parser: hermes, which expects JSON. vLLM's hermes parser runs json.loads() on the XML, throws JSONDecodeError on every turn, recognizes zero tool calls, so the agent never takes an action. No assistant tokens are produced and response_mask is all zeros.

Fix. Match the parser to the model. The runtime exposes a single variable, HARBOR_TOOL_PARSER (default qwen3_coder), which drives every config point (lego_rl_*.yaml → tool-call-parser, agent_loop_config_cc.yaml → tool_parser):

Verify. The vLLM logs stop printing JSONDecodeError, and the first one or two steps' proxy_trajectory.json show non-zero tool_calls with finish_reason: tool_calls.

Full write-up: troubleshooting/training_env/tool-call-parser-mismatch.md

Slow venv imports on a network filesystem

Issue. Every launch pays a cold-import tax: import torch takes ~8.7 s against ~1.8 s on local disk.

Reason. The venv lives on a network filesystem (CPFS / fuse.aliyun-alinas-efc), so the small-file metadata I/O of an import walk goes over the network.

Fix. Copy the venv onto local NVMe once and point VENV_PATH at it:

cp -a .venv /root/venvs/rl-venv
VENV_PATH=/root/venvs/rl-venv bash scripts/sync_1node_cc.sh

/root is per-node local storage, so each node needs its own copy. Re-sync when pyproject.toml changes or packages are added: rsync -a --delete <src>/.venv/ /root/venvs/rl-venv/.

Full write-up: troubleshooting/training_env/local-venv-cache.md