# Multi-Reward RL, Part 3 — evidence (20261004)

## Environment A: holdout score by checkpoint (selected = best train-side core score)

| Configuration | Runs | step 0 | step 100 | step 200 | step 300 | step 400 | step 500 | step 600 | Selected |
|---|---|---|---|---|---|---|---|---|---|
| DAPO · LoRA · 500 prompts | 1790661651 → 1790738119 | -1.366 | -0.911 | -0.500 | -0.587 | -0.626 | -0.555 | -0.474 | -0.474 @ 600 |
| DAPO · DoRA · 500 prompts | 1790888478 | -1.366 | -0.592 | -0.287 | -0.181 | -0.135 | -0.316 | -0.279 | -0.287 @ 200 |
| CISPO · LoRA · 500 prompts | 1790891170 | -1.366 | -0.877 | -0.844 | -0.357 | -0.411 | -0.073 | -0.024 | -0.024 @ 600 |
| CISPO + REPO-R · LoRA · 500 prompts | 1790891672 → 1790920657 | -1.366 | -0.480 | +0.056 | +0.406 | +0.827 | +1.322 | +1.327 | +1.327 @ 600 |
| CISPO + REPO-R · LoRA · 226 prompts | 1790958861 | -1.366 | -0.469 | -0.131 | +0.846 | +0.506 | +1.800 | +1.581 | +1.800 @ 500 |
| CISPO + REPO-R · DoRA · 226 prompts | 1790967377 | -1.366 | -0.376 | -0.122 | +0.601 | +1.101 | +1.333 | +1.170 | +1.333 @ 500 |
| DAPO + REPO-R · DoRA · 226 prompts | 1790973987 | -1.366 | -0.287 | -0.162 | +0.481 | +0.779 | +0.694 | +0.680 | +0.694 @ 500 |

## Environment B: share of samples with a non-zero advantage

| Arm | Run | Steps | Non-zero advantage share | Last-20 KL | Last-20 reward | Last-20 core |
|---|---|---|---|---|---|---|
| Advantage floor at 0 (first slice) | 1791045171 | 427 | 0.042 | 0.018 | -10.06 | -3.29 |
| No floor (first slice) | 1791130455 | 339 | 0.999 | 0.304 | -2.58 | -0.74 |
| Advantage floor at 0 (second slice) | 1791044621 | 449 | 0.035 | 0.012 | -12.09 | -3.97 |
| No floor (second slice) | 1791065555 | 161 | 0.998 | 0.171 | -3.34 | -1.02 |
| Advantage floor at 0 (third slice) | 1791044830 | 143 | 0.031 | 0.013 | -12.84 | -4.21 |
| No floor (third slice) | 1791130204 | 369 | 0.999 | 0.130 | -3.85 | -1.21 |
| No floor (fourth slice) | 1791130610 | 393 | 0.999 | 0.183 | -1.93 | -0.49 |

## Per-step training logs

Full per-step values are in the JSON file next to this one.