---
language: en
license: mit
library_name: qtensor
tags:
- qtensor
- tensor-network
- mpo
- fp8
- compression
- llama-3.2-instruct
pipeline_tag: text-generation
---

# QTensor: LLaMA-3.2 1B Instruct (Stacked MPO + FP8 Quantization)

**Model Developer:** Trent Ian Parsons ([GitHub Repository](https://github.com/Trentzap1/qtensor))  
**Base Model:** `unsloth/Llama-3.2-1B-Instruct`  
**Precision:** Stacked MPO (chi=256) + FP8 (float8_e4m3fn) Core + 3.65% bfloat16 LoRA Adapters  
**VRAM Memory Footprint:** **694 MB (0.68 GB)** — **3.39x Memory Reduction** vs Pristine FP16 (2.30 GB)  

---

## Performance Metrics

* **Pristine LLaMA-3.2 1B (bf16)**: 2357.13 MB (2.30 GB)
* **QTensor Stacked MPO + FP8**: **694.63 MB (0.68 GB)**
* **WikiText-2 Perplexity**: 11.78 PPL (+0.07 vs Pristine Baseline)
* **GSM8K Zero-Shot Accuracy**: 75.2% (Healed)

---

## Quickstart Inference with QTensor

```python
import torch
import qtensor
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "trentzap/Llama-3.2-1B-QTensor-FP8"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")

# Activate QTensor FP8 Triton SRAM Fusion
model = qtensor.compress(model, precision="fp8", chi=256, use_triton=True)

messages = [{"role": "user", "content": "Explain QTensor SRAM fusion."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=50)

print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```

---

## Citation

```bibtex
@article{parsons2026qtensor,
  title={Quantum-Inspired Precision: Preserving Low-Rank Reasoning Structures in Large Language Models via 160-Bit Fixed-Point Tensor Decomposition, Triton SRAM Fusion, and Stacked Quantization},
  author={Parsons, Trent Ian},
  journal={arXiv preprint arXiv:2608.xxxxx},
  year={2026}
}
```