Compatibility
What Lego-RL supports, at which level of validation
Support here means one of four things:
| Mark | Meaning |
|---|---|
| ✅ Tested | exercised end-to-end on our reference cluster, and covered by run validation |
| 🧪 Experimental | implemented and selectable, not validated end-to-end |
| 🔌 Inherited | provided by an upstream dependency, not exercised through Lego-RL |
| ❌ Not wired | present in the codebase, but no current config path reaches it |
Agent scaffolds
Selected with SCAFFOLD; each value maps to a module in
scripts/templates/scaffold/.
| Scaffold | SCAFFOLD | Status |
|---|---|---|
| OpenHands SDK | ohsdk | ✅ Tested — the default |
| Claude Code | cc | ✅ Tested |
| OpenCode | oc | ✅ Tested |
| OpenHands (full app) | oh | 🧪 Experimental — drops rejected actions, which corrupts token-level capture |
| Terminus 2 | — | ❌ Not wired — implemented in harbor and src/harbor_patch/agents/terminus_2/, but no scaffold module selects it |
Other harnesses connect through a custom adapter; see Environment.
Sandbox backends
Selected with BACKEND.
| Backend | BACKEND | Status |
|---|---|---|
| Kubernetes | k8s | ✅ Tested — the production path |
| Docker (local or remote daemon) | docker | ✅ Tested with ohsdk / oh |
Docker + cc / oc | docker | ❌ Not wired — only agent_loop_config_oh_docker.yaml exists; run validation warns |
| Daytona, E2B, Modal, Runloop | — | 🔌 Inherited from Harbor, never exercised in a Lego-RL run |
Training engines
Selected with MODEL_ENGINE.
| Engine | MODEL_ENGINE | Status |
|---|---|---|
| VeOmni | veomni | ✅ Tested — required for MoE policies with routing replay |
| FSDP | fsdp | ✅ Tested — dense policies; R3 requires SP_SIZE=1 |
| Megatron | — | ❌ Not wired — src/verl_patch/config/lego_rl_fully_async_megatron.yaml exists, but no runner path sets VERL_CONFIG_NAME to it |
Rollout placement
Selected with TRAINING_MODE.
| Mode | TRAINING_MODE | Status |
|---|---|---|
| Synchronous (colocated) | sync | ✅ Tested — one pool, simplest to run |
| Fully asynchronous | async | ✅ Tested — separate train and rollout pools, partial rollout |
Algorithms
ADV_ESTIMATOR (default grpo) is passed to verl's algorithm.adv_estimator.
| Algorithm | Status |
|---|---|
| GRPO | ✅ Tested — the default |
| GSPO | ✅ Tested |
| PPO | 🔌 Inherited from verl; no Lego-RL reference run |
Models
Run validation knows the tool-parser rules for these families; anything else gets a warning telling you to verify the pairing yourself.
| Model | Parser rule | Status |
|---|---|---|
| Qwen3.5-35B-A3B | qwen3_coder (XML template) | ✅ Tested |
| Qwen3.6-27B | qwen3_coder | ✅ Tested |
| Qwen3-30B-A3B | qwen3_coder under cc, hermes under ohsdk / oh | ✅ Tested |
| Other MoE | verified against config.json for routing replay | 🧪 Experimental |
| Dense models | R3 must be off | ✅ Tested on the FSDP path |
Reference deployment
Every performance number in these docs was measured on this cluster:
| Nodes | 8× GPU per node, 200k-context runs on 3–4 nodes |
| Policies | Qwen3.5-35B-A3B, Qwen3.6-27B |
| Serving | vLLM, TP=4 per replica |
| Sandboxes | Kubernetes, prebuilt task images from a local registry |
They are observations for capacity planning, not performance guarantees.