---
language:
- en
license: apache-2.0
library_name: llama.cpp
pipeline_tag: text-generation
tags:
- gguf
- llama.cpp
- smollm2
- text-generation
- conversational
base_model: HuggingFaceTB/SmolLM2-135M-Instruct
---

# SmolLM2 135M Instruct — Unofficial GGUF Conversions

> **Original source model and publisher:** [HuggingFaceTB/SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct) by **HuggingFaceTB**. This repository contains file-format conversions only. It does **not** claim authorship, ownership, training, or creation of the underlying SmolLM2 model.

This repository provides local CPU-oriented GGUF conversions of the upstream SmolLM2 135M Instruct model for use with `llama.cpp` and compatible runtimes. The source model's Apache-2.0 license is retained; no separate or replacement license is asserted for the underlying model.

## Available files

| File | Format | Intended use | File size |
|---|---|---|---:|
| `SmolLM2-135M-Instruct-f16.gguf` | F16 GGUF | Quality-preserving local inference | 258 MiB |
| `SmolLM2-135M-Instruct-Q6_K.gguf` | Q6_K GGUF | Lower-memory CPU inference | 132 MiB |

The F16 variant preserves the original converted weight precision. The Q6_K variant is provided for systems where memory usage is more important than retaining the F16 representation. Use the F16 file for the highest-fidelity local behavior.

## Quick start with llama.cpp

```bash
./llama-cli \
  --model SmolLM2-135M-Instruct-f16.gguf \
  --conversation \
  --n-gpu-layers 0
```

For persistent local serving, start `llama-server` once and send OpenAI-compatible requests to the local endpoint:

```bash
./llama-server \
  --model SmolLM2-135M-Instruct-f16.gguf \
  --host 127.0.0.1 \
  --port 8080 \
  --ctx-size 2048 \
  --n-gpu-layers 0
```

## Integrity verification

Verify downloaded files with:

```bash
sha256sum -c CHECKSUMS.sha256
```

## Conversion details

The model was converted from the official Hugging Face checkpoint with the official `llama.cpp` Hugging Face-to-GGUF converter. The F16 GGUF output was retained as the quality-preserving variant, then quantized with `llama-quantize` to produce the Q6_K variant. See `conversion-metadata.json` for the artifact metadata.

## Upstream source, ownership, and license

The underlying model is published by **HuggingFaceTB** at [HuggingFaceTB/SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct). The upstream model card identifies its license as **Apache-2.0**. The conversion distributor claims no ownership or authorship of the original model and does not impose a new license on it. Retain the upstream attribution and consult the source model card for limitations, training details, and citation information.