--- license: cc-by-nc-4.0 base_model: - m-a-p/YuE2-3B - Comfy-Org/YuE2 pipeline_tag: text-to-audio language: - fr tags: - lora - yue2 - music-generation - text-to-music - song-generation - french - chanson - chanson-francaise - orchestral-pop - ye-ye - spoken-word - comfyui - fs_audio --- # CHNSN — Chanson Française LoRAs for YuE2 Two checkpoints of one artist-style LoRA that push [YuE2-3B](https://huggingface.co/m-a-p/YuE2-3B) into **classic French chanson**: theatrical baritone and soprano leads with real vibrato, sweeping strings, accordion, piano, brass swells, upright bass, brushed drums, and the mid-century orchestral sound behind them. Vocals are sung in French. Both male and female leads are in the data, and the prompt decides which one you get. It also does something the base model is not built for: a **spoken passage** inside a song (see *A read letter* below). Each file patches **both halves** of YuE2 in one go: the autoregressive planner (writes the score, decides the arrangement and the vocal lines) and the flow-matching decoder (the sound). Trigger word for both: **`chnsn`**. | | LoRA | File | Character | Start here | |---|---|---|---|---| | 🎻 | **CHNSN Rive Gauche** | `chnsn_rive_gauche.safetensors` | Step 200, the decoder-loss minimum. The more supple of the two: acoustic narrative, yé-yé, female leads, waltz meters, and the sung-letter duet all came from this file. | clip 1.0 / model 1.0, cfg 1.0 | | 🎺 | **CHNSN Grand Boulevard** | `chnsn_grand_boulevard.safetensors` | Step 350, the final checkpoint. Tighter, more orchestral, writes shorter chanson-length songs. The orchestral ballad demo. | clip 1.0 / model 1.0, cfg 1.0 | ## Listen All demos use an original French lyric (nine tagged sections, a Paris-in-the-rain chanson), seed 7, baseline recipe, 32 steps `dpm_2` / `sgm_uniform`, 360 s cap, no post-processing. **CHNSN Rive Gauche — a read letter** (duet), prompt `prompts/duet_spoken_letter.txt`. Female lead on the verses and chorus, then a male baritone takes a letter written as prose and delivers it as a declaimed verse, then the chorus returns. 6/8, 90 BPM, D minor, 3:06: **CHNSN Rive Gauche — yé-yé baroque**, prompt `prompts/female_yeye_baroque.txt`: youthful female lead, harpsichord, fuzz bass, mellotron flutes, brass stabs (125 BPM, C major, 4:09): **CHNSN Rive Gauche — acoustic narrative**, prompt `prompts/acoustic_narrative.txt`: warm baritone storytelling over nylon-string guitar, upright bass and soft strings (105 BPM, D minor, 3:44): **CHNSN Grand Boulevard — orchestral ballad**, prompt `prompts/orchestral_ballad.txt`: dramatic baritone with vibrato, sweeping strings, piano, brass swells (86 BPM, D minor, 3:26): ## Quick start (ComfyUI) The LoRAs are in the fused-key layout that Comfy's YuE2 implementation uses (`text_encoders.*` for the planner, `diffusion_model.*` for the decoder). They were trained with, and load through, the **[FS_Audio Suite](https://github.com/KytraScript/ComfyUI-FS_Audio_Suite)** node pack. 1. ComfyUI ≥ v0.36.0 (native YuE2 support) and the FS_Audio Suite custom node pack. 2. Base model: `yue2_3b_bf16.safetensors` from [Comfy-Org/YuE2](https://huggingface.co/Comfy-Org/YuE2) in `models/checkpoints/`. 3. Drop one `chnsn_*.safetensors` into `models/loras/`. 4. Chain the nodes: ``` 🧩 FS_Audio Lora Loader ──loras──▶ 🎤 FS_Audio Model Loader ──pipe──▶ 🎵 FS_Audio Sampler ──▶ 💿 FS_Audio Output lora_name = chnsn_rive_gauche.safetensors yue2_checkpoint = yue2_3b_bf16 style = strength_clip = 1.0 (planner) melody_transcriber = none lyrics = strength_model = 1.0 (decoder) score_mode = full ``` Sampler settings used for every demo: | widget | value | |---|---| | steps / sampler / scheduler | 32 / `dpm_2` / `sgm_uniform` | | score_mode | `full` | | song_length_cap | 360 (chanson-length lyrics plan to 3–5 minutes; the cap rarely triggers) | | repetition_penalty | 1.2 | Two recipes: | | Baseline (all demos) | Wild | |---|---|---| | strength_clip (planner) | 1.0 | 1.0 | | strength_model (decoder) | 1.0 | 1.0 | | Weirdness (cfg) | 1.0 | **1.4** | | score_temperature | 0.7 | **0.9** | | music_temperature | 1.0 | **1.2** | Headless, the same graph as an API prompt: ```python prompt = { "1": {"class_type": "FSAudioLoraLoader", "inputs": {"lora_name": "chnsn_rive_gauche.safetensors", "strength_model": 1.0, "strength_clip": 1.0}}, "2": {"class_type": "FSAudioModelLoader", "inputs": {"yue2_checkpoint": "yue2_3b_bf16.safetensors", "melody_transcriber": "none", "loras": ["1", 0]}}, "3": {"class_type": "FSAudioSampler", "inputs": {"pipe": ["2", 0], "style": STYLE, "lyrics": LYRICS, "seed": 7, "song_length_cap": 360, "score_mode": "full", "steps": 32, "sampler": "dpm_2", "scheduler": "sgm_uniform", "Weirdness (cfg)": 1.0, "score_temperature": 0.7, "music_temperature": 1.0, "repetition_penalty": 1.2}}, "4": {"class_type": "FSAudioOutput", "inputs": {"song": ["3", 0], "score": ["3", 1], "info": ["3", 2], "filename_prefix": "chnsn"}}, } ``` ## Prompting ### Style prompt Start with the trigger, then write **one descriptive sentence** in this order: language → genre → vocal → instruments → mood → production → BPM. This is the format the planner was trained on. Bare tag lists produce odd plans. ``` chnsn, French, classic French chanson, dramatic expressive male baritone vocal with rich vibrato and theatrical phrasing, sweeping orchestral strings, acoustic piano, brass swells, upright bass, soft brushed drums, poignant, romantic and deeply melancholic, grand cinematic arrangement with emotional crescendos, 81 BPM ``` Ready-made prompts in `prompts/`: | file | what it gets you | |---|---| | `orchestral_ballad.txt` | the Grand Boulevard demo: dramatic baritone, strings, piano, brass | | `acoustic_narrative.txt` | warm baritone storytelling, nylon guitar, upright bass, soft strings | | `swing_bigband.txt` | 1960s big-band swing chanson; rendered but the planner leaves long instrumental stretches (only a third of vocal bars sung), so treat it as an instrumental-leaning prompt | | `female_yeye_baroque.txt` | the yé-yé demo: youthful female lead, harpsichord, fuzz bass, mellotron | | `female_dark_waltz.txt` | dramatic female with vibrato over a dark 6/8 waltz, accordion, dissonant strings, tape loops (rendered, not in the demos) | | `female_torch_minimal.txt` | smoky low-register female torch song, felt piano, glass harmonica, long silences; use the wild recipe (rendered, not in the demos) | | `duet_spoken_letter.txt` | the duet demo: female lead plus a male baritone narrator reading a letter | "Female vocal" in the prompt reliably gives a female lead even though only three of the fifteen training songs have one. Waltz words ("slow waltz", "dark waltz") put the planner in 6/8; "yé-yé", "harpsichord", "fuzz bass" push it to 125 BPM in a major key. ### A read letter (spoken passages) YuE2 has no spoken-word mode: the planner writes a pitched melody for every vocal bar. But two things together get a passage *delivered* rather than sung: 1. Write the passage as **prose**: long lines, no rhyme, no repetition, second person, like a letter. Put it in its own section. 2. In the style prompt, name the narrator and say what he does: `a deep low male baritone narrator who does not sing but reads a letter aloud in a calm monotone spoken voice over the bridge, recited parlé passage with no melody ... the spoken passage close-miked and dry`. The section tag then picks the flavour. Under a normal **`[Bridge]`** tag the planner writes the letter as a declaimed verse on a simple melody and hands it to the male voice: that is the duet demo. Under an unfamiliar **`[Spoken]`** tag the planner writes that stretch as an *instrumental interlude with no vocal melody at all*, and the decoder still voices every word over the band, at about 2.5 words per second: proper recitation. The `[Spoken]` render carried all of the letter; the `[Bridge]` render sang about half of it before returning to the chorus. Check the `.abc` next to the render: a `% interlude` with rest-only `V: Vocal` bars at that spot means it worked. ### Lyrics Tagged French blocks. Use `[Intro]`, `[Verse 1]`, `[Verse 2]`, `[Chorus]`, `[Bridge]`, `[Outro]` (and `[Spoken]` for narration); number the verses and write a repeated chorus out again where you want it sung. Standard orthography with accents and apostrophes. Shape that made the demos: ``` [Intro] two short lines, the image the song returns to [Verse 1] four lines, ten to twelve syllables, rhymed in pairs [Verse 2] four more lines [Chorus] four lines, the hook phrase at the top of the first two [Verse 3] four lines [Chorus] (repeat, written out) [Bridge] four lines, or a prose letter for the narrator [Chorus] (repeat, written out) [Outro] the intro image again, trailing off ``` The planner sometimes drops the bridge on the orchestral prompt and goes verse–chorus–verse–chorus–outro; the acoustic and swing prompts keep it. Chanson-length lyrics like this plan to 3–5 minutes. ### Which knob does what - **Checkpoint** is the first choice: Rive Gauche (step 200) for anything acoustic, female, experimental or spoken; Grand Boulevard (step 350) for the big orchestral ballad. - **strength_model (decoder)** 1.0 → 1.5 makes the voice rougher and more present without touching the writing. **strength_clip (planner) above 1.0 collapses the vocal** on YuE2 artist LoRAs in general; keep it at 1.0. - **Weirdness (cfg)** changes only the sound. **music_temperature / score_temperature** change the writing; 1.2 / 0.9 is the wild recipe used for the torch-song prompt. - **Seeds** decide length as much as the cap does; the same lyric planned 3:26 on one checkpoint and 4:28 on the other. ## Training Trained on a single RTX 5090 with the FS_Audio Suite **Artist Trainer** on Comfy's own YuE2 weights. One run trains the planner LoRA and the decoder LoRA together; the file also carries full `vae2llm` / `llm2vae` projection diffs. Both published files come from the same run. | | Rive Gauche | Grand Boulevard | |---|---|---| | dataset | 15 songs, 45 minutes: French chanson from the 1950s to the 1980s, mostly orchestral male-baritone chanson with a few acoustic guitar narratives and three female-led songs | same | | checkpoint | **step 200** | step 350 (final) | | planner / decoder steps | 350 / 600 | 350 / 600 | | artist (planner) loss | 5.220 → 4.606 | 5.220 → 4.582 | | decoder loss at checkpoint | **1.148** (the minimum) | 1.162 | | wall time | 21 minutes for the whole run, dataset build included | | Hyper-parameters: rank 64 planner / rank 32 decoder, LR 3e-5 planner / 4e-5 decoder / 2e-5 I/O projections, artist fraction 0.5 vs regularizer pack, KL 0.1, batch 2 songs, 8192 max tokens, 750-frame windows, EMA 0.99, one song held out, score-first 0, no transcription. **Why two checkpoints.** The trainer names its `_best` file by planner loss, which keeps falling to the end. The decoder loss bottomed at step 200 (1.148) and drifted back up to 1.162 by step 350 on this 15-song set, a mild version of the overfit seen on other small sets. Step 200 is the pick by the decoder curve, and three of the four demos are from it. The step-350 file is published because the orchestral ballad rendered from it was preferred by ear, and the drift is small enough that both are usable. ### Captioning your own dataset The planner learns from the captions as much as from the audio, so the caption format decides whether your prompts work later. What worked here: - **One descriptive sentence per song, trigger first**, in the order you will prompt in: language → genre → vocal → instruments → mood → production → BPM. Decade words ("1960s French chanson") are useful descriptors. - **No tag lists, no section scaffolding** in the caption. - **Measure BPM, don't guess**, and sanity-check it: beat trackers double slow ballads and halve fast ones (a lively yé-yé track came out as 72 and is 144; a piano ballad came out as 136 and is 68). Compare the number with the mood words in the same caption. - A vision-language model can draft the captions from the audio, but **check the vocal gender by hand**: a high male tenor got labelled "female lead" here. The three real female leads were labelled correctly, so this is a grep-and-listen pass, not a blind find-and-replace. - **Lyrics as tagged blocks** with standard orthography, accents and apostrophes. Transcribe at high confidence; a mis-heard lyric teaches the planner the wrong syllable count for the bar. - **Audio prep**: FLAC 44.1 kHz / 16-bit, ≤ 320 s. MP3 rips with a damaged leading frame crash the dataset builder ("Header missing"); transcode with `ffmpeg -err_detect ignore_err`. - **Trainer flags**: `score-first 0` and no automatic transcription. ## Known limitations - French vocals only. - Female leads come from three training songs; they render well on the prompts above but the range of female timbres is narrower than the male ones. - The big-band swing prompt leaves most bars instrumental. Add "sung throughout" and a denser lyric, or change the seed. - The planner may drop a `[Bridge]` on orchestral prompts. If the bridge matters, try the acoustic prompt family or the `[Spoken]` tag. - No instrumental-only mode is baked in; the LoRAs assume a lyric. ## Files ``` chnsn_rive_gauche.safetensors 177 MB step 200, planner + decoder LoRA (bf16 weights, fp32 projection diffs) chnsn_grand_boulevard.safetensors 177 MB step 350, same layout demos/ mp3 renders (192 kbps from the FLAC masters), seed 7 throughout prompts/ style prompts ``` ## License and credits Weights are released under **CC BY-NC 4.0**, inherited from the YuE2-3B base model. Non-commercial use only; attribute "CHNSN LoRAs by becausereasons". - [YuE2](https://huggingface.co/m-a-p/YuE2-3B) by the Multimodal Art Projection (m-a-p) team; ComfyUI repack by [Comfy-Org](https://huggingface.co/Comfy-Org/YuE2). - [ComfyUI-FS_Audio_Suite](https://github.com/KytraScript/ComfyUI-FS_Audio_Suite) by KytraScript / The Fixed Seed Company: inference nodes and the artist trainer. - Sister releases: [CNZN canzone italiana](https://huggingface.co/becausereasons/yue2-cnzn-canzone-italiana), [MLTNT militant roots reggae](https://huggingface.co/becausereasons/yue2-mltnt-militant-reggae). - Trained and documented by becausereasons, September 2026.