---
base_model:
  - meta-llama/Llama-3.2-1B-Instruct
datasets:
  - WithinUsAI/Llama_4_Maverick_Distilled_5k
  - WithinUsAI/Meta_Muse_Spark_Distilled_5k
license: llama3.2
language:
  - en
tags:
  - llama
  - llama3.2
  - distill
  - text-generation
  - conversational
  - WithinUsAI
  - 11-47
  - Muse-Spark
  - Llama-4-Maverick
  - 1B
---

# Llama-3.2-Mavrick.Spark-1B

**By 11-47 / WithinUsAI**

A fast, edge-friendly 1.23B parameter model that distills the **Spark-style persona** (concise, helpful, agentic vibes) and **Maverick long-form reasoning traces** into Meta's Llama 3.2 1B Instruct base.

> Built with Llama 3.2. This is a community research distillation. Not affiliated with Meta. Llama 3.2 is licensed under the [Llama 3.2 Community License](https://www.llama.com/llama3_2/license).

## Model Details

- **Parameters**: 1.23B
- **Context Length**: 128k
- **Model Size**: ~2.49 GB (FP16 / Safetensors)
- **Base Model**: `meta-llama/Llama-3.2-1B-Instruct`
- **Tensor Type**: FP16

### Training Data
- **Dataset 1**: [WithinUsAI/Meta_Muse_Spark_Distilled_5k](https://huggingface.co/WithinUsAI/Meta_Muse_Spark_Distilled_5k) — 5k Spark-style instructions, persona prompts, and tool-use traces.
- **Dataset 2**: [WithinUsAI/Llama_4_Maverick_Distilled_5k](https://huggingface.co/WithinUsAI/Llama_4_Maverick_Distilled_5k) — 5k long-form Chain-of-Thought / reasoning traces from Maverick.
- **Total**: 10k high-quality distilled conversations.
- **Method**: Supervised Fine-Tuning (SFT) on assistant tokens only (ShareGPT / ChatML format with system + user + assistant roles).

### Why This Model Rocks
Llama 3.2 1B is perfect for phones, laptops, and edge devices but can feel shallow on reasoning. This distillation compresses the clever, agentic flavor of **Muse Spark** (Meta's closed frontier model) and **Llama 4 Maverick** into a runnable local model. Great for quick inference while keeping a spark of that frontier reasoning style.

## Quick Start

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "11-47/Llama-3.2-Mavrick.Spark-1B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto"
)

messages = [
    {"role": "system", "content": "You are Mavrick.Spark, a concise, helpful assistant built by WithinUsAI."},
    {"role": "user", "content": "Explain how distillation works in 3 bullets."}
]

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=400,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)

print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
The repo includes a chat_template.jinja, so chat interfaces and HF Inference work out of the box.
Intended Uses
	•	Local / on-device inference (phones, laptops, edge devices)
	•	Conversational agents with Spark-like personality
	•	Research on knowledge distillation & style transfer
	•	Lightweight reasoning tasks
Limitations
	•	Still a 1B model — will hallucinate more than full-size Maverick/Spark models.
	•	Distilled style can sometimes over-explain.
	•	Knowledge cutoff inherited from base Llama 3.2 (~Dec 2023) + synthetic traces.
	•	Not production-hardened for heavy tool use or code execution.
License & Attribution
	•	Base weights: Llama 3.2 Community License — keep the “Llama” prefix and display “Built with Llama”.
	•	Datasets are from WithinUsAI.
	•	Please cite the original repo and base model if you build on this.
Citation
@misc{llama-3.2-mavrick-spark-1b,
  author = {11-47 / WithinUsAI},
  title = {Llama-3.2-Mavrick.Spark-1B},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/11-47/Llama-3.2-Mavrick.Spark-1B}
}