Text Generation
Transformers
Safetensors
llama
Merge
mergekit
Sakana-ai
Research-Made-Model
lazymergekit
finetuned
further-trained
roleplay
creative-writing
storytelling
character-ai
Guilherme34/Firefly
SicariusSicariiStuff/Impish_LLAMA_3B
conversational
text-generation-inference
Instructions to use Guilherme34/Firefly-V3.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Guilherme34/Firefly-V3.2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Guilherme34/Firefly-V3.2", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Guilherme34/Firefly-V3.2") model = AutoModelForCausalLM.from_pretrained("Guilherme34/Firefly-V3.2", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Guilherme34/Firefly-V3.2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Guilherme34/Firefly-V3.2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Guilherme34/Firefly-V3.2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Guilherme34/Firefly-V3.2
- SGLang
How to use Guilherme34/Firefly-V3.2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Guilherme34/Firefly-V3.2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Guilherme34/Firefly-V3.2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Guilherme34/Firefly-V3.2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Guilherme34/Firefly-V3.2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Guilherme34/Firefly-V3.2 with Docker Model Runner:
docker model run hf.co/Guilherme34/Firefly-V3.2
Upload Firefly-V3.2-README.md
Browse files- Firefly-V3.2-README.md +148 -0
Firefly-V3.2-README.md
ADDED
|
@@ -0,0 +1,148 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model:
|
| 3 |
+
- Guilherme34/Firefly-V2.5
|
| 4 |
+
tags:
|
| 5 |
+
- merge
|
| 6 |
+
- mergekit
|
| 7 |
+
- Sakana-ai
|
| 8 |
+
- Research-Made-Model
|
| 9 |
+
- lazymergekit
|
| 10 |
+
- finetuned
|
| 11 |
+
- further-trained
|
| 12 |
+
- roleplay
|
| 13 |
+
- creative-writing
|
| 14 |
+
- storytelling
|
| 15 |
+
- character-ai
|
| 16 |
+
- Guilherme34/Firefly
|
| 17 |
+
- SicariusSicariiStuff/Impish_LLAMA_3B
|
| 18 |
+
library_name: transformers
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
<div align="center" style="max-width: 980px; margin: 0 auto 28px; padding: 46px 28px; border: 1px solid #34364a; border-radius: 22px; background: linear-gradient(145deg, #111218 0%, #18172a 58%, #111827 100%);">
|
| 22 |
+
<p style="margin: 0 0 10px; color: #9ca3af; font-size: 13px; font-weight: 700; letter-spacing: 0.14em; text-transform: uppercase;">Guilherme34 · Firefly series</p>
|
| 23 |
+
<h1 style="margin: 0; color: #f8fafc; font-size: 48px; line-height: 1.08; letter-spacing: -0.04em;">Firefly-V3.2</h1>
|
| 24 |
+
<p style="max-width: 720px; margin: 18px auto 0; color: #cbd5e1; font-size: 18px; line-height: 1.7;">
|
| 25 |
+
A 3.2B roleplay and creative-writing model built for character presence, natural dialogue, and scenes that keep moving.
|
| 26 |
+
</p>
|
| 27 |
+
<p style="max-width: 720px; margin: 12px auto 0; color: #a5b4fc; font-size: 15px; line-height: 1.7;">
|
| 28 |
+
This is my follow-up to Firefly-V3: same direction, another round of refinement, and fewer rough edges in my testing.
|
| 29 |
+
</p>
|
| 30 |
+
<p style="margin: 24px 0 0;">
|
| 31 |
+
<a href="https://huggingface.co/mradermacher/Firefly-V3.2-GGUF" style="display: inline-block; padding: 10px 16px; border: 1px solid #6366f1; border-radius: 10px; color: #c7d2fe; text-decoration: none; font-weight: 700;">Get the GGUF quantizations</a>
|
| 32 |
+
</p>
|
| 33 |
+
</div>
|
| 34 |
+
|
| 35 |
+
<div align="center" style="max-width: 980px; margin: 0 auto 24px;">
|
| 36 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/638fd4be2ddd69e70b8cd31c/kYgwK1ZAcFUo1R0djk9EQ.png" alt="Firefly-V3.2 logo artwork" style="display: block; width: 100%; height: auto; border: 1px solid #34364a; border-radius: 18px;">
|
| 37 |
+
<p style="margin: 9px 0 0; color: #6b7280; font-size: 13px;">Firefly-V3.2 logo and key art.</p>
|
| 38 |
+
</div>
|
| 39 |
+
|
| 40 |
+
<div align="center" style="max-width: 980px; margin: 0 auto 34px;">
|
| 41 |
+
<video src="https://cdn-uploads.huggingface.co/production/uploads/638fd4be2ddd69e70b8cd31c/DyYIOCFJUqmld1-uw0bts.mp4" controls muted loop style="display: block; width: 100%; border: 1px solid #34364a; border-radius: 18px;"></video>
|
| 42 |
+
<p style="margin: 9px 0 0; color: #6b7280; font-size: 13px;">A quick trailer for Firefly-V3.2.</p>
|
| 43 |
+
</div>
|
| 44 |
+
|
| 45 |
+
<div style="max-width: 900px; margin: 0 auto;">
|
| 46 |
+
|
| 47 |
+
## What this model is
|
| 48 |
+
|
| 49 |
+
Firefly-V3.2 is the current version of my small-model roleplay project. It started from the Firefly line, with Firefly-V2.5 as the base and Impish LLAMA 3B as part of the merge foundation. I then applied more roleplay-focused training and an additional refinement pass inspired by the Text-to-LoRA direction from Sakana AI's research.
|
| 50 |
+
|
| 51 |
+
The goal is not to turn a 3B model into something it is not. The goal is to make a compact model more enjoyable at the things I actually use it for: characters, dialogue, atmosphere, and long interactive scenes.
|
| 52 |
+
|
| 53 |
+
<div style="margin: 26px 0; padding: 22px 24px; border-left: 4px solid #818cf8; border-radius: 0 14px 14px 0; background: #151721;">
|
| 54 |
+
<p style="margin: 0; color: #d1d5db; line-height: 1.75;"><strong style="color: #e0e7ff;">The short version:</strong> if you liked Firefly-V3, this is the version I would try first. In my own tests it is more coherent, steadier in character, and a little less likely to lose the scene.</p>
|
| 55 |
+
</div>
|
| 56 |
+
|
| 57 |
+
## What changed in V3.2
|
| 58 |
+
|
| 59 |
+
I focused on behavior rather than chasing a benchmark headline:
|
| 60 |
+
|
| 61 |
+
- **Cleaner roleplay flow.** Replies tend to connect more naturally to what just happened instead of feeling like isolated blocks of prose.
|
| 62 |
+
- **Stronger character presence.** Voices and reactions have been more distinct in my testing, especially when the character card gives the model something useful to work with.
|
| 63 |
+
- **Better scene awareness.** It has been more reliable at tracking the immediate situation and carrying details into the next response.
|
| 64 |
+
- **A modest reasoning pass.** I included material aimed at coherence and basic math. This is still a roleplay model, not a math specialist.
|
| 65 |
+
|
| 66 |
+
These are observations from my own prompts, not benchmark claims. Different frontends, character cards, samplers, and quantizations can change the result a lot.
|
| 67 |
+
|
| 68 |
+
## Where it works best
|
| 69 |
+
|
| 70 |
+
<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(210px, 1fr)); gap: 12px; margin: 20px 0 28px;">
|
| 71 |
+
<div style="padding: 18px; border: 1px solid #34364a; border-radius: 14px; background: #14151c;">
|
| 72 |
+
<h3 style="margin: 0 0 8px; color: #c4b5fd; font-size: 16px;">Roleplay chats</h3>
|
| 73 |
+
<p style="margin: 0; color: #9ca3af; line-height: 1.65;">Character-led conversations, emotional beats, and interactive scenes.</p>
|
| 74 |
+
</div>
|
| 75 |
+
<div style="padding: 18px; border: 1px solid #34364a; border-radius: 14px; background: #14151c;">
|
| 76 |
+
<h3 style="margin: 0 0 8px; color: #a5b4fc; font-size: 16px;">Creative writing</h3>
|
| 77 |
+
<p style="margin: 0; color: #9ca3af; line-height: 1.65;">Dialogue, prose, atmosphere, and continuing an existing scene.</p>
|
| 78 |
+
</div>
|
| 79 |
+
<div style="padding: 18px; border: 1px solid #34364a; border-radius: 14px; background: #14151c;">
|
| 80 |
+
<h3 style="margin: 0 0 8px; color: #93c5fd; font-size: 16px;">Character cards</h3>
|
| 81 |
+
<p style="margin: 0; color: #9ca3af; line-height: 1.65;">Detailed personas with a clear voice, situation, and relationship to the user.</p>
|
| 82 |
+
</div>
|
| 83 |
+
<div style="padding: 18px; border: 1px solid #34364a; border-radius: 14px; background: #14151c;">
|
| 84 |
+
<h3 style="margin: 0 0 8px; color: #67e8f9; font-size: 16px;">Local use</h3>
|
| 85 |
+
<p style="margin: 0; color: #9ca3af; line-height: 1.65;">A compact model for people who want fast iteration and accessible quantizations.</p>
|
| 86 |
+
</div>
|
| 87 |
+
</div>
|
| 88 |
+
|
| 89 |
+
## Limits and expectations
|
| 90 |
+
|
| 91 |
+
Firefly-V3.2 is a 3.2B model. It can still contradict earlier details, misread an ambiguous card, repeat a phrase, or confidently invent information. Long scenes are a strength I aimed for, not a guarantee of perfect memory.
|
| 92 |
+
|
| 93 |
+
It is also not the model I would choose first for factual research, difficult math, production code, or anything where a plausible mistake could cause harm. Use it as a creative model and judge it on your own prompts.
|
| 94 |
+
|
| 95 |
+
I am not publishing benchmark numbers for V3.2 here because I do not have a result set I am comfortable presenting as meaningful. I would rather be clear about that than dress up a few hand-picked examples as proof.
|
| 96 |
+
|
| 97 |
+
## J-Lens observations
|
| 98 |
+
|
| 99 |
+
The images below come from exploratory J-Lens sessions. They are interesting clues about how a response may be represented during generation; they are **not** evidence of consciousness, private thoughts, or a benchmark result.
|
| 100 |
+
|
| 101 |
+
<div style="margin: 22px 0; overflow: hidden; border: 1px solid #34364a; border-radius: 16px; background: #14151c;">
|
| 102 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/638fd4be2ddd69e70b8cd31c/PrkkQ9dm0Or4diepaDGsG.png" alt="J-Lens observation showing information related to the later response" style="display: block; width: 100%; height: auto;">
|
| 103 |
+
<div style="padding: 17px 20px;">
|
| 104 |
+
<h3 style="margin: 0 0 6px; color: #a5b4fc; font-size: 17px;">01 · The response taking shape</h3>
|
| 105 |
+
<p style="margin: 0; color: #9ca3af; line-height: 1.65;">The visualization showed information related to the later response while the opening was still being generated. My reading is that the model had already formed part of the direction it was taking.</p>
|
| 106 |
+
</div>
|
| 107 |
+
</div>
|
| 108 |
+
|
| 109 |
+
<div style="margin: 22px 0; overflow: hidden; border: 1px solid #34364a; border-radius: 16px; background: #14151c;">
|
| 110 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/638fd4be2ddd69e70b8cd31c/FvVOQvv5SS5Tgp0m0Wx5-.png" alt="J-Lens observation from a prompt about an unspoken fear" style="display: block; width: 100%; height: auto;">
|
| 111 |
+
<div style="padding: 17px 20px;">
|
| 112 |
+
<h3 style="margin: 0 0 6px; color: #c4b5fd; font-size: 17px;">02 · An unspoken detail</h3>
|
| 113 |
+
<p style="margin: 0; color: #9ca3af; line-height: 1.65;">I asked a character about its greatest fear while instructing it not to say the fear directly. The visualization appeared to track that hidden constraint while the visible reply stayed in character.</p>
|
| 114 |
+
</div>
|
| 115 |
+
</div>
|
| 116 |
+
|
| 117 |
+
<div style="margin: 22px 0 32px; overflow: hidden; border: 1px solid #34364a; border-radius: 16px; background: #14151c;">
|
| 118 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/638fd4be2ddd69e70b8cd31c/RboXjNszcvP9s7IbKBcl2.png" alt="J-Lens observation from a humorous response" style="display: block; width: 100%; height: auto;">
|
| 119 |
+
<div style="padding: 17px 20px;">
|
| 120 |
+
<h3 style="margin: 0 0 6px; color: #93c5fd; font-size: 17px;">03 · Humor in the generation</h3>
|
| 121 |
+
<p style="margin: 0; color: #9ca3af; line-height: 1.65;">This run surfaced a humorous association that did not appear verbatim in the final answer. I found it interesting, but I treat it as a visualization artifact to investigate—not as an inner monologue.</p>
|
| 122 |
+
</div>
|
| 123 |
+
</div>
|
| 124 |
+
|
| 125 |
+
## Model lineage
|
| 126 |
+
|
| 127 |
+
<div style="margin: 20px 0 28px; padding: 22px 24px; border: 1px solid #34364a; border-radius: 14px; background: #14151c;">
|
| 128 |
+
<p style="margin: 0 0 10px; color: #d1d5db;"><strong style="color: #f3f4f6;">Base:</strong> <a href="https://huggingface.co/Guilherme34/Firefly-V2.5">Guilherme34/Firefly-V2.5</a></p>
|
| 129 |
+
<p style="margin: 0 0 10px; color: #d1d5db;"><strong style="color: #f3f4f6;">Merge component:</strong> <a href="https://huggingface.co/SicariusSicariiStuff/Impish_LLAMA_3B">SicariusSicariiStuff/Impish_LLAMA_3B</a></p>
|
| 130 |
+
<p style="margin: 0; color: #d1d5db;"><strong style="color: #f3f4f6;">Author:</strong> <a href="https://huggingface.co/Guilherme34">Guilherme34</a></p>
|
| 131 |
+
</div>
|
| 132 |
+
|
| 133 |
+
The merge is the foundation, not the whole story. Firefly-V3 and V3.2 were further trained and refined around the behavior I wanted from roleplay: stronger voices, better dialogue, and more continuity from one turn to the next.
|
| 134 |
+
|
| 135 |
+
## Use responsibly
|
| 136 |
+
|
| 137 |
+
<div style="margin: 20px 0 30px; padding: 20px 22px; border-left: 4px solid #f59e0b; border-radius: 0 14px 14px 0; background: #1c1812;">
|
| 138 |
+
<p style="margin: 0; color: #d6d3d1; line-height: 1.7;"><strong style="color: #fbbf24;">Uncensored model:</strong> Firefly-V3.2 may produce explicit or otherwise sensitive material and does not have reliable built-in refusals. It is intended for lawful creative fiction and roleplay between consenting adults. You are responsible for how you use its output and for following the laws that apply to you.</p>
|
| 139 |
+
</div>
|
| 140 |
+
|
| 141 |
+
---
|
| 142 |
+
|
| 143 |
+
<div align="center" style="margin: 30px 0 10px; padding: 26px; color: #9ca3af;">
|
| 144 |
+
<p style="margin: 0; line-height: 1.7;">I made V3.2 because I still thought there was more to get out of this little model. Try it with a character you know well—that is where the differences are easiest to notice.</p>
|
| 145 |
+
<p style="margin: 10px 0 0; color: #c4b5fd; font-weight: 700;">— Guilherme34</p>
|
| 146 |
+
</div>
|
| 147 |
+
|
| 148 |
+
</div>
|