---
license: cc-by-nc-sa-4.0
tags:
  - speech
  - audio
  - vocoder
  - speech-synthesis
  - neural-vocoder
  - firnet

gated: true

extra_gated_heading: "Agreement to the Terms of Use"
extra_gated_prompt: >
  Please read the **The Terms of Use in Model Card** before requesting access to the models and source code.

extra_gated_fields:
  I have read and agree to The Terms of Use in Model Card: checkbox

extra_gated_button_content: "Agree and request access"

---

# FIRNet: Fast Neural Vocoder With Fundamental Frequency Control Using Finite Impulse Response Filters
We release official pretrained models of [FIRNet](https://ieeexplore.ieee.org/document/10976410), a DSP-based fast neural vocoder with voice factor control. FIRNet can control pitch bias and scale, voice depth (formant shift), and aperiodicity by modifying vocoder parameters. We provide two pretrained models for 24-kHz and 48-kHz sampling rates. FIRNet can be used not only for speech generation but also for data augmentation and perturbation during training.

## Requirements
- Python 3.10+
- PyTorch (`torch >=2.7`)

Install project dependencies:
```bash
pip install -r ./requirements.txt
```

## Repository layout
- checksum: checksum information for .pt files of the pretrained models
- models: pretrained model weights and model definition files
- src: Python script set
  - Network blocks for FIRNet
  - Waveform analysis and synthesis
  - sample script for voice factor control

## Examples of usage for sample script
- Analysis-synthesis (24 kHz sampling rate)
```bash
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav
```

- Analysis-synthesis (48 kHz sampling rate)
```bash
python3 ./src/bin/sample_anasyn.py --sample_rate 48000 -I input48kHz.wav -O output48kHz.wav
```

- Pitch-modified voice
```bash
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --f0_shift 1.2 --f0_emph 0.7
```

- Constant F0 voice
```bash
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --fixed_f0 100
```

- Deeper voice
```bash
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --alpha -0.15
```

- Hoarse voice
```bash
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --beta 0.20
```

## Model specifications
### Model architecture
These pretrained models are parallel FIRNet models based on [ASRU2025](https://ieeexplore.ieee.org/abstract/document/11434739) version. The differences from [the original version](https://ieeexplore.ieee.org/document/10976410) are as follows:
- Use ConvNeXt blocks instead of its core blocks in shared encoder.
- Frame shift size (hop size) is changed to 10 ms from 5 ms.
- The number of filter taps is changed to 384 (24 kHz) and 768 (48 kHz).
- Use a combination of spectral criteria instead of the mel-spectrogram loss for the generator loss.

### Training data
For traning the pretrained models, we employ the following speech datasets:
- A part of [commonvoice22_sidon](https://huggingface.co/datasets/sarulab-speech/commonvoice22_sidon), which includes 134 languages.
- Hi-Fi-Captain (currently unpublished)
<!-- - [Hi-Fi-Captain](https://ast-astrec.nict.go.jp/release/hi-fi-captain/) -->

## The Terms of Use
### Source code
- The FIRNet source code is distributed under the BSD 3-Clause license.
- The BSD license applies only to the source code.
- See `LICENSE_FOR_PYTHON_CODES` for the full license terms. 

### Pretrained models
- The pretrained models are distributed under CC BY-NC-SA 4.0, in accordance with the licensing conditions of the training corpora.
- The pretrained models are intended for research and educational purposes.
- For commercial use, please contact us in advance at astl-service@ml.nict.go.jp. Higher-quality pretrained models are available for commercial applications, particularly for 48-kHz speech synthesis.
- See `LICENSE_FOR_MODELS` for the full license terms. 

## References
When you cite the FIRNet in your paper, please cite the following papers:

[1] <u>Yamato Ohtani</u>, Takuma Okamoto, Tomoki Toda and Hisashi Kawai, "Fast Neural Vocoder with Fundamental Frequency Control Using Finite Impulse Response Filters," in IEEE Transactions on Audio, Speech and Language Processing, vol. 33, pp. 1893-1906, 2025. [[link](https://ieeexplore.ieee.org/document/10976410)]

[2] <u>Yamato Ohtani</u>, Takuma Okamoto, Tomoki Toda, Hisashi Kawai, "Voice factor control using FIR-based fast neural vocoder for speech generation applications," Proc. ASRU, pp. 1-4, Dec. 2025. [[link](https://ieeexplore.ieee.org/abstract/document/11434739)]

[3] Takuma Okamoto, <u>Yamato Ohtani</u>, Sota Shimizu, Tomoki Toda, Hisashi Kawai, "Challenge of Singing Voice Synthesis Using Only Text-To-Speech Corpus With FIRNet Source-Filter Neural Vocoder," Proc. Interspeech 2024, pp. 1870-1874, September 2024. [[link](https://www.isca-archive.org/interspeech_2024/okamoto24_interspeech.html)]

[4] <u>Yamato Ohtani</u>, Takuma Okamoto, Tomoki Toda and Hisashi Kawai, "FIRNet: Fundamental frequency controllable fast neural vocoder with trainable finite impulse response filter," Proc. ICASSP 2024, pp. 10871-10875, April 2024. [[link](https://ieeexplore.ieee.org/document/10446960)]