Initial upload: 2000-step MixGRPO LoRA checkpoints + trend figure + README

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+figures/w3030_trends.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,119 @@
+---
+license: cc-by-nc-4.0
+tags:
+- diffusion
+- flow-matching
+- music
+- lora
+- grpo
+- rl
+- diffrhythm
+base_model: ASLP-lab/DiffRhythm2
+library_name: peft
+---
+
+# DiffRhythm2 · MixGRPO LoRA (checkpoints, MAAP)
+
+LoRA adapter checkpoints from a **block-wise dense-reward GRPO** run on top of
+the [ASLP-lab/DiffRhythm2](https://huggingface.co/ASLP-lab/DiffRhythm2)
+continuous-time flow-matching music model.  Released as **research artifacts
+for a negative-result study** on RL fine-tuning of long-form audio flow models.
+
+## What is here
+
+- **`adapter_model.safetensors` / `adapter_config.json`** at the repo root — the
+  final `ckpt_final_002000` (2000 optimization steps).  Load directly via
+  `PeftModel.from_pretrained("m-a-a-p/DiffRhythm2-MixGRPO-LoRA", …)`.
+- **`checkpoints/step_XXXXXX/`** — intermediate LoRA adapters at every 100
+  training steps (Step 100, 200, …, 1900) for anyone who wants to reproduce
+  the reward trajectory or run ablations.
+- **`figures/w3030_trends.png`** — 3-panel trend plot for the final sliding
+  window (`window=[30-30]`), showing reward, KL, and clip-fraction over the
+  1840 steps spent in that window.
+- **`config/config_used.py`** — the exact `mixgrpo_lora_diffrhythm2()` config
+  section used for this run, for reproducibility.
+
+## Training setup
+
+Adapter is inserted into the DiT attention layers of DiffRhythm2 (dim 2048,
+depth 16, 16 heads, mel_dim 64).
+
+| item | value |
+|---|---|
+| base model | `ASLP-lab/DiffRhythm2` |
+| optimizer | AdamW, `lr = 5e-6`, `weight_decay = 1e-4`, `eps = 1e-8` |
+| LoRA | `r = 32`, `alpha = 64` |
+| RL algorithm | Block-wise MixGRPO (sliding window over 4 blocks, 30 s each) |
+| group size | 12 rollouts per prompt |
+| PPO-style clip | `clip_range = 0.1` |
+| KL coefficient | `0.01` |
+| gradient accumulation | 3 |
+| inner epochs | 1 |
+| EMA | **disabled** (OOM mitigation) |
+| CUDA alloc | `expandable_segments:True` |
+| total steps | 2000 |
+| hardware | 1 × RTX PRO 6000 (96 GB), Slurm job 1738062 |
+
+**Rewards** are a 50/50 blend of the audiobox aesthetics score and an
+ACE-Step-style Audio Alignment Score (AAS), aggregated over each 30 s block
+window.
+
+## Result: no measurable improvement
+
+Across 2000 steps in the final sliding window (`[30-30]`, 1840 steps, Steps
+160→1999), reward exhibits a **statistically significant regression**:
+
+- early-half mean reward: **0.18669** (n = 920)
+- late-half mean reward:  **0.18592** (n = 920)
+- Δ = **−7.7 × 10⁻⁴**, t ≈ **−26.7**
+- linear slope: **−8.9 × 10⁻⁴ per 1000 steps**
+- KL(π ∥ π_ref) simultaneously *decreases* from 0.018 to 0.013
+
+The full reward range across all 1840 steps is only **0.006** (≈ 0.3 %),
+i.e. the same magnitude as the per-step reward noise.  Hyperparameter sweeps
+in the accompanying paper (across `num_generations ∈ {4, 12, 16}`,
+`clip_range ∈ {1e-5, 1e-4, 0.1}`, `lr ∈ {5e-6, 1e-5, 5e-5}`) do not escape
+this plateau.
+
+Interpretation: the log-probability signal that GRPO needs — computed here
+through the surrogate ratio between the current and the pre-update policy on
+the ODE / SDE trajectory — is **noise-dominated** at the scale of the block-
+wise dense rewards used here, so the policy gradient does not consistently
+point in the reward-increasing direction.  See the accompanying paper for the
+full analysis (including the diagnosis of the `1 / (1−t)` score-head
+singularity in the underlying flow-matching formulation).
+
+## Intended use
+
+These adapters are released as **research artifacts** to accompany the
+negative-result study, not as a recommended production LoRA on top of
+DiffRhythm2.  In particular:
+
+- Do not expect audio-quality gains over the base DiffRhythm2 model.
+- If you use them as a *baseline* or *sanity check* for a new RL algorithm,
+  please cite the accompanying paper.
+
+## Quick start
+
+```python
+from peft import PeftModel
+from safetensors.torch import load_file
+
+# Final (2000-step) adapter, loaded via peft on top of your DiffRhythm2 DiT.
+adapter = load_file("adapter_model.safetensors")
+
+# Intermediate:
+# hf_hub_download("m-a-a-p/DiffRhythm2-MixGRPO-LoRA",
+#                 "checkpoints/step_001000/adapter_model.safetensors")
+```
+
+## License
+
+Released under **CC BY-NC 4.0**.  Non-commercial use only; academic use
+encouraged.  Base model (`ASLP-lab/DiffRhythm2`) is subject to its own
+license.
+
+## Citation
+
+If you use these checkpoints or the accompanying analysis, please cite the
+associated paper (details to be added when the paper appears).

adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5db4ed3e1c6383171896b3d1825e67eebf2083a72c8472a5c80f940e1126cb81
+size 96500080

checkpoints/step_000100/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_000100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52744bfc41f52db9ff346abc2a7b91427da9cbf6ee0636a8b20ef7f176a9fbe7
+size 96500080

checkpoints/step_000200/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_000200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a4f1a74aff8f953e7676ea7e965f593941377ba8c19c2edb7a835d569485a33
+size 96500080

checkpoints/step_000300/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_000300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63ac7ca80251b7e799a70a07033b5e0d4cd65493b66f2f854ddda1a708d2d05f
+size 96500080

checkpoints/step_000400/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_000400/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69d70bb08a6a9b16f2a40cd7a371cb253ebf8b8a86fffca3a1e20a7ad8c2300b
+size 96500080

checkpoints/step_000500/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_000500/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d471cfa6f1c3a8674b03ea0b597410dca38d5e9ad3ea3b110f5749988c36eaa0
+size 96500080

checkpoints/step_000600/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_000600/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39f76807bebff8e0ee2c392c6d93c36b22f9a27e53b880698af12481cdda5a0f
+size 96500080

checkpoints/step_000700/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_000700/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df422f37b6fa56bc5fc5a8a2b6913b3354810ba747b4de5f2b9cb5d6114df50d
+size 96500080

checkpoints/step_000800/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_000800/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4da09b6f08a790d5d7fc34d37993fc111fa84ab0546827f68c324e6a116e46a8
+size 96500080

checkpoints/step_000900/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_000900/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67af9bd9676855f0dde493321a85a2b062c09f535e0c658f1f8a27f93c10bb44
+size 96500080

checkpoints/step_001000/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_001000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ca2745d9bf31d9796caa4ad20a1df18980c4c72881febabaa9ae18c47979f93
+size 96500080

checkpoints/step_001100/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_001100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76444092033155f152cb7a6d8afe92a90e48b3af17a74f00ba067cdbb5cde45b
+size 96500080

checkpoints/step_001200/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_001200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd43f94c2399b1d8a5e77c3f1d417b68d8c2c29a0c8220716c5988b003985bb9
+size 96500080

checkpoints/step_001300/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_001300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c65abc7fe386fc396b72ec8bdef85b976884d7e5556e64e6b60d4cbdf46bbbe
+size 96500080

checkpoints/step_001400/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_001400/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6964ab97526f2aba842c20cc05a910200b2b050f093b62ab0e18df43a3738a52
+size 96500080

checkpoints/step_001500/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_001500/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4cc9cd908e84900cea784ca5a5b5d77075d5696182796b21bd1b80b07de14ee5
+size 96500080

checkpoints/step_001600/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_001600/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15395553df630c84c8724e5357509459b95274392711755866105f2b238ec57a
+size 96500080

checkpoints/step_001700/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_001700/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49d4e88181dffa2218f35b495df75e38b80c5577f8a034120c88778276a1f89a
+size 96500080

checkpoints/step_001800/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_001800/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50dc0cafab9da75cc4e73920ed85f0357c3bbf19998e3464b7f0ed3cbf98d830
+size 96500080

checkpoints/step_001900/adapter_config.json

@@ -0,0 +1,49 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "DiT",
+    "parent_library": "diffrhythm2.backbones.dit"
+  },
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": "gaussian",
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoints/step_001900/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b83c6067da807a5a8979dedb9e6f19a2885cf91dd7cf3b1c8db247b1e8dafc12
+size 96500080

config/config_used.py

@@ -0,0 +1,97 @@
+def mixgrpo_lora_diffrhythm2():
+    config = get_config()
+    gpu_number = 1
+    if torch.cuda.is_available():
+        gpu_number = torch.cuda.device_count()
+
+    # --- Dataset ---
+    config.dataset = os.path.join(config.root_dir, "_dataset")
+    config.prompt_fn = "genius"
+    config.json_path = os.path.join(config.dataset, "Genius_local.json")
+
+    # --- Reward (placeholder; will be replaced with ACE-Step AAS later) ---
+    config.reward_fn = {"placeholder": 1.0}
+    config.eval_reward_fn = {"placeholder": 1.0}
+
+    # --- MixGRPO Sliding Window ---
+    config.training_strategy = "part"  # "part" = MixGRPO, "all" = DanceGRPO
+    config.grpo = ml_collections.ConfigDict()
+    config.grpo.iters_per_group = 20
+    config.grpo.group_size = 4
+    config.grpo.sample_strategy = "progressive"
+    config.grpo.prog_overlap = False
+    config.grpo.prog_overlap_step = 1
+    config.grpo.roll_back = False
+    config.grpo.max_iters_per_group = 10
+    config.grpo.min_iters_per_group = 1
+
+    # --- GRPO Training Hyperparameters ---
+    config.train.total_steps = 2000
+    config.train.clip_range = 0.1
+    config.train.adv_clip_max = 5
+    config.train.kl_coeff = 0.01
+    config.train.learning_rate = 5e-6
+    config.train.max_grad_norm = 1.0
+    config.train.ema = False  # OOM mitigation: EMA model copy holds ~model size in VRAM
+    config.train.gradient_accumulation_steps = 3
+    config.train.num_inner_epochs = 1
+    config.train.cfg = False
+    config.train.num_generations = 12
+    config.train.init_same_noise = True
+    # Reward interpolation: reward = (1-λ)*audiobox + λ*AAS
+    # λ=0 → pure Audiobox Aesthetics, λ=1 → pure AAS, λ=0.5 → equal mix
+    config.train.reward_lambda = 0.5
+
+    # --- Sampling ---
+    config.sample.use_sde = True
+    config.sample.num_steps = 32
+    config.sample.guidance_scale = 4.5
+    config.sample.noise_level = 1.0  # eta: 0.0 = ODE, 1.0 = fully stochastic SDE
+    config.sample.rollout_duration_sec = 30
+    config.sample.train_batch_size = 1
+    config.sample.num_batches_per_epoch = 1
+    config.sample.num_audio_per_prompt = 12
+    config.sample.mini_num_audio_per_prompt = 1
+
+    # --- LoRA ---
+    config.use_lora = True
+    config.train.lora_rank = 32
+    config.train.lora_alpha = 64
+    config.train.lora_path = None
+
+    # --- Score Head (SDE) ---
+    config.score_head_path = "_train/checkpoints_score_head/score_head_step_4000.pt"
+
+    # --- Logging & Save Overrides ---
+    config.save_freq = 100
+
+    # --- Run name ---
+    config.run_name = (
+        f"mixgrpo_lora_diffrhythm_G{gpu_number}"
+        f"_r{config.train.lora_rank}"
+        f"_grp{config.train.num_generations}"
+        f"_win{config.grpo.group_size}"
+        f"_steps{config.sample.num_steps}"
+    )
+
+    # --- Debug Mode ---
+    config.debug = False
+    if config.debug:
+        config.run_name = "DEBUG_" + config.run_name
+        config.sample.num_batches_per_epoch = 1
+        config.sample.train_batch_size = 1
+        config.sample.num_audio_per_prompt = 2
+        config.sample.mini_num_audio_per_prompt = 2
+        config.train.num_generations = 2
+        config.sample.rollout_duration_sec = 6
+        config.sample.num_steps = 8
+        config.grpo.group_size = 2
+        config.grpo.iters_per_group = 3
+        config.train.total_steps = 10
+        config.save_freq = 1
+        config.wandb_init = False
+        config.per_prompt_stat_tracking = False
+
+    config.save_dir = os.path.join(config.output_dir, config.run_name)
+    config.case_name = config.run_name
+    return config