Scale RL training with VeRL

Train LLMs with RL at scale using verl

Production math/reasoning tasks (GSM8K, MATH) where you need proven RL algorithms at scale.

• · Base model
• · Training dataset (parquet with prompts)
• · Reward function or model
• · Config YAML

• · Trained policy model
• · Training curves/metrics

• · verl>=0.3.0
• · torch>=2.0.0
• · ray>=2.41.0
• · vllm>=0.8.2
• · transformers>=4.40.0

8+ GPUs; H100 or A100 recommended; models must be HuggingFace compatible

• · Memory explosion on 671B+ models
• · Misaligned reward function
• · Distributed training synchronization failures

• · Powers Doubao-1.5-pro
• · EuroSys 2025 HybridFlow paper
• · Tested up to 671B parameters