adityalakhani/htdemucs-6s-guitar-ft

Guitar Domain Expert HTDemucs 6s

A domain-specialized fine-tune of Meta AI's htdemucs_6s trained exclusively to extract clean, isolated guitar stems from dense, real-world studio mixes.

TL;DR: Drop in any song. Get back a clean guitar stem. Works on clean riffs, distorted rock, acoustic, and layered studio productions.

---

Results

Evaluated on a 24-track held-out split of MoisesDB v0.1 using full-track BSS metrics (deterministic shifts=0):

Model	SDR ↑ (dB)	SIR ↑ (dB)	SAR ↑ (dB)
Stock htdemucs_6s (Meta AI baseline)	8.42	20.41	6.28
htdemucs-6s-guitar-ft (this model)	8.73	21.30	6.65
Δ improvement	+0.31	+0.89	+0.37

The +0.89 dB SIR gain is the headline result — it directly measures guitar clarity and bleed rejection. Less drum transient, less bass rumble, less cymbal wash bleeding into your extracted guitar stem.

---

Quick Start

from demucs.pretrained import get_model
from demucs.apply import apply_model
from huggingface_hub import hf_hub_download
import torch
import soundfile as sf

GUITAR_STEM_INDEX = 4  # htdemucs_6s: [drums, bass, other, vocals, guitar, piano]

# Download fine-tuned weights from HuggingFace
checkpoint_path = hf_hub_download(
    repo_id="adityalakhani/htdemucs-6s-guitar-ft", 
    filename="guitar_htdemucs_6s.pt"
)

# Load base model architecture
model = get_model("htdemucs_6s")

# Load and unwrap fine-tuned weights (training dict)
checkpoint = torch.load(checkpoint_path, map_location="cpu", weights_only=True)
state_dict = checkpoint["model_state_dict"]

# Remap keys to match demucs' BagOfModels wrapper
state_dict = {f"models.0.{k}": v for k, v in state_dict.items()}
model.load_state_dict(state_dict)
model.eval()

# Load audio (must be 44100 Hz. soundfile avoids torchaudio Windows DLL issues)
audio_np, sr = sf.read("your_song.wav", always_2d=True, dtype="float32")
waveform = torch.from_numpy(audio_np.T)  # (C, T)
if waveform.shape[0] == 1:
    waveform = waveform.repeat(2, 1)

# Separate using apply_model for proper overlap-add chunking
with torch.no_grad():
    stems = apply_model(model, waveform.unsqueeze(0), shifts=1, overlap=0.25)
    guitar = stems[0, GUITAR_STEM_INDEX]

sf.write("guitar_stem.wav", guitar.T.numpy(), samplerate=44100, subtype="PCM_24")
print("Done.")

Or use the included CLI script for any audio file format (mp3, flac, wav, etc.):

python inference.py --input song.mp3 --output guitar_stem.wav

---

Model Details

Architecture

This model is a fine-tuned checkpoint of htdemucs_6s — Meta AI's Hybrid Transformer Demucs (6-source variant). The architecture processes audio simultaneously in two domains:

• Time domain: Raw waveform → 1D Conv encoder → Cross-Domain Transformer → 1D ConvTranspose decoder
• Frequency domain: STFT spectrogram → 2D Conv encoder → Cross-Domain Transformer → 2D ConvTranspose → iSTFT

Both decoder outputs are summed to produce each separated stem. The model outputs 6 stems: drums, bass, other, vocals, guitar (index 4), piano.

~80M total parameters.

What makes this different from stock htdemucs_6s?

The pre-trained htdemucs_6s is a general-purpose separator trained on MUSDB18-HQ — a broad mix of musical genres. This fine-tuned version was subsequently trained on MoisesDB (240 real multitrack songs with professionally isolated guitar stems), which produces a model that has developed a much stronger prior for real guitar acoustic and spectral characteristics.

The result is a model that:

• Rejects drum transient bleed more aggressively (+0.89 dB SIR)
• More precisely follows guitar sustain curves vs. background noise
• Is better calibrated for the full guitar frequency range (clean highs to distorted lows)

---

Training Details

Dataset

MoisesDB v0.1 — 240 real commercial multitrack songs with professionally isolated stems:

• Each song contains ground-truth isolated guitar WAV files
• Covers Rock, Singer-Songwriter, Pop, Latin, and Electronic genres
• Guitar stems are harmonically and rhythmically interlocked with their actual mix — no synthetic mix construction

Split: 216 songs training / 24 songs validation (stratified by genre via fixed-seed shuffle).

Fine-Tuning Strategy

Training was split into two phases to prevent catastrophic forgetting:

Phase	Layers Active	LR	Epochs
Phase 1 (Warmup)	Decoders only (Encoder + Transformer frozen)	3×10⁻⁴	20
Phase 2 (Fine-tune)	Full network unfrozen	1×10⁻⁴	~55 (early stopped)

The GradScaler was explicitly reset at the Phase 1 → Phase 2 transition to prevent the sudden encoder gradient regime from destabilizing the decoder weights learned in Phase 1.

Loss Function

L_total = L1(pred, target)
        + 0.5 × MultiResolutionSTFT(pred, target)
        + SI-SDR(pred, target)    # weighted to zero on silent chunks (RMS < 1e-4)

• L1 provides waveform fidelity
• Multi-Resolution STFT adds spectral phase and magnitude matching across 3 FFT sizes
• SI-SDR (Scale-Invariant SDR) directly optimizes the deployment metric — added in V4 to break through the +5.3 dB plateau

Data Augmentation

• Remix augmentation (80%): For each training chunk, non-guitar stems (drums, bass, vocals, etc.) are independently swapped from random other songs in the dataset. This prevents the model from memorising song-specific mix signatures.
• Guitar gain randomisation: Guitar stem scaled randomly in [-6 dB, +6 dB] before mixing.
• Chunk length: 7.8 seconds (matches the model's effective receptive field).

Hyperparameters

Parameter	Value
Batch size	1
Gradient accumulation	4 steps (effective batch = 4)
Mixed precision	fp16 (torch.autocast + GradScaler)
Early stopping patience	10 epochs (on val loss)
Gradient clipping	max_norm = 1.0
Total training epochs	75 (early stopped)

---

Limitations

• Heavy multi-guitar polyphony: When a mix contains 3+ overlapping guitars (lead + rhythm + acoustic), the model extracts all of them into a single stem — it does not separate individual guitar layers from each other. This is by design for the source separation stage of a larger pipeline.
• Extreme distortion + cymbal bleed: Very high-gain metal tones (wall-of-sound production) with heavy cymbal wash remain the hardest cases. The model reduces bleed significantly but does not eliminate it.
• Piano / keyboard ambiguity: Synth pads in the upper mid-frequency range can occasionally bleed into the guitar output on dense electronic productions. This is inherited from the base htdemucs_6s architecture.
• Mono input: Mono audio is automatically upmixed to stereo before inference; results may be slightly less precise than native stereo input.

---

How to Cite

If you use this model in research or a project, please cite:

@misc{lakhani2026htdemucs_guitar_ft,
  title     = {Guitar Domain Expert HTDemucs 6s: A Domain-Specialized Fine-Tune of HTDemucs for Guitar Stem Extraction},
  author    = {Lakhani, Aditya},
  year      = {2026},
  url       = {https://huggingface.co/aditya-lakhani/htdemucs-6s-guitar-ft},
}

---

Acknowledgements

• Meta AI Research — HTDemucs architecture and pre-trained weights (demucs==4.0.1)
• Moises.ai — MoisesDB v0.1 multitrack dataset used for domain fine-tuning
• auraloss — Multi-Resolution STFT loss implementation (auraloss==0.4.0)

---

License

This model is released under the Apache 2.0 license. The base htdemucs_6s weights (Meta AI) are released under MIT. MoisesDB data is used in compliance with Moises.ai's research licensing terms; raw audio data is not redistributed.