Instructions to use Guilherme34/Firefly-V3.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Guilherme34/Firefly-V3.2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Guilherme34/Firefly-V3.2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Guilherme34/Firefly-V3.2")
model = AutoModelForCausalLM.from_pretrained("Guilherme34/Firefly-V3.2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Guilherme34/Firefly-V3.2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Guilherme34/Firefly-V3.2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Guilherme34/Firefly-V3.2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Guilherme34/Firefly-V3.2

SGLang

How to use Guilherme34/Firefly-V3.2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Guilherme34/Firefly-V3.2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Guilherme34/Firefly-V3.2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Guilherme34/Firefly-V3.2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Guilherme34/Firefly-V3.2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Guilherme34/Firefly-V3.2 with Docker Model Runner:
```
docker model run hf.co/Guilherme34/Firefly-V3.2
```

Firefly-V3.2 / Firefly-V3.2-README.md

Guilherme34

Upload Firefly-V3.2-README.md

d86da03 verified 11 days ago

preview code

Raw

History Blame Contribute Delete

11 kB

metadata

base_model:
  - Guilherme34/Firefly-V2.5
tags:
  - merge
  - mergekit
  - Sakana-ai
  - Research-Made-Model
  - lazymergekit
  - finetuned
  - further-trained
  - roleplay
  - creative-writing
  - storytelling
  - character-ai
  - Guilherme34/Firefly
  - SicariusSicariiStuff/Impish_LLAMA_3B
library_name: transformers

Guilherme34 · Firefly series

Firefly-V3.2

A 3.2B roleplay and creative-writing model built for character presence, natural dialogue, and scenes that keep moving.

This is my follow-up to Firefly-V3: same direction, another round of refinement, and fewer rough edges in my testing.

Get the GGUF quantizations

Firefly-V3.2 logo and key art.

A quick trailer for Firefly-V3.2.

What this model is

Firefly-V3.2 is the current version of my small-model roleplay project. It started from the Firefly line, with Firefly-V2.5 as the base and Impish LLAMA 3B as part of the merge foundation. I then applied more roleplay-focused training and an additional refinement pass inspired by the Text-to-LoRA direction from Sakana AI's research.

The goal is not to turn a 3B model into something it is not. The goal is to make a compact model more enjoyable at the things I actually use it for: characters, dialogue, atmosphere, and long interactive scenes.

The short version: if you liked Firefly-V3, this is the version I would try first. In my own tests it is more coherent, steadier in character, and a little less likely to lose the scene.

What changed in V3.2

I focused on behavior rather than chasing a benchmark headline:

Cleaner roleplay flow. Replies tend to connect more naturally to what just happened instead of feeling like isolated blocks of prose.
Stronger character presence. Voices and reactions have been more distinct in my testing, especially when the character card gives the model something useful to work with.
Better scene awareness. It has been more reliable at tracking the immediate situation and carrying details into the next response.
A modest reasoning pass. I included material aimed at coherence and basic math. This is still a roleplay model, not a math specialist.

These are observations from my own prompts, not benchmark claims. Different frontends, character cards, samplers, and quantizations can change the result a lot.

Where it works best

Roleplay chats

Character-led conversations, emotional beats, and interactive scenes.

Creative writing

Dialogue, prose, atmosphere, and continuing an existing scene.

Character cards

Detailed personas with a clear voice, situation, and relationship to the user.

Local use

A compact model for people who want fast iteration and accessible quantizations.

Limits and expectations

Firefly-V3.2 is a 3.2B model. It can still contradict earlier details, misread an ambiguous card, repeat a phrase, or confidently invent information. Long scenes are a strength I aimed for, not a guarantee of perfect memory.

It is also not the model I would choose first for factual research, difficult math, production code, or anything where a plausible mistake could cause harm. Use it as a creative model and judge it on your own prompts.

I am not publishing benchmark numbers for V3.2 here because I do not have a result set I am comfortable presenting as meaningful. I would rather be clear about that than dress up a few hand-picked examples as proof.

J-Lens observations

The images below come from exploratory J-Lens sessions. They are interesting clues about how a response may be represented during generation; they are not evidence of consciousness, private thoughts, or a benchmark result.

01 · The response taking shape

The visualization showed information related to the later response while the opening was still being generated. My reading is that the model had already formed part of the direction it was taking.

J-Lens observation from a prompt about an unspoken fear

02 · An unspoken detail

I asked a character about its greatest fear while instructing it not to say the fear directly. The visualization appeared to track that hidden constraint while the visible reply stayed in character.

J-Lens observation from a humorous response

03 · Humor in the generation

This run surfaced a humorous association that did not appear verbatim in the final answer. I found it interesting, but I treat it as a visualization artifact to investigate—not as an inner monologue.

Model lineage

Base: Guilherme34/Firefly-V2.5

Merge component: SicariusSicariiStuff/Impish_LLAMA_3B

Author: Guilherme34

The merge is the foundation, not the whole story. Firefly-V3 and V3.2 were further trained and refined around the behavior I wanted from roleplay: stronger voices, better dialogue, and more continuity from one turn to the next.

Use responsibly

Uncensored model: Firefly-V3.2 may produce explicit or otherwise sensitive material and does not have reliable built-in refusals. It is intended for lawful creative fiction and roleplay between consenting adults. You are responsible for how you use its output and for following the laws that apply to you.

I made V3.2 because I still thought there was more to get out of this little model. Try it with a character you know well—that is where the differences are easiest to notice.

— Guilherme34