HuggingFaceBio/Carbon-3B-GGUF

Carbon-3B GGUF

Running with llama.cpp

Carbon is a base DNA model (not instruction-tuned). Use the raw completion front-end so llama.cpp does not apply any chat templating:

llama-completion \
  -hf HuggingFaceBio/Carbon-3B-GGUF:Q8_0 \
  --prompt "<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG" \
  -n 128 --temp 0 --no-display-prompt

If you use llama-cli instead, pass -no-cnv so it does not wrap the prompt in assistant-style markers (otherwise the model may emit annotation tags such as <protein_coding_region> instead of DNA).

Available files

Quant	File	Size
BF16	carbon-3b-bf16.gguf	6.4 GB
Q8_0	carbon-3b-q8_0.gguf	3.4 GB
Q6_K	carbon-3b-q6_k.gguf	2.8 GB
Q5_K_M	carbon-3b-q5_k_m.gguf	2.4 GB
Q4_K_M	carbon-3b-q4_k_m.gguf	2.1 GB

K-quants are produced with --token-embedding-type Q8_0 --output-tensor-type Q8_0 so the sensitive embedding/output tensors stay at Q8_0 precision (important for this model's tight DNA-vs-text logit margins).

Requires a recent llama.cpp

HybridDNATokenizer support was merged in ggml-org/llama.cpp#23410, so any build from master after that works:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build -j

Usage

Download

hf download HuggingFaceBio/Carbon-3B-GGUF carbon-3b-bf16.gguf --local-dir .

Basic DNA completion

./build/bin/llama-completion -m carbon-3b-bf16.gguf \
    -p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
    -n 64 --temp 0 -no-cnv

Metadata-conditioned generation

./build/bin/llama-completion -m carbon-3b-bf16.gguf \
    -p '<vertebrate_mammalian><protein_coding_region><dna>ATGCGCTAG' \
    -n 64 --temp 0 -no-cnv

Speculative decoding with Carbon-500M draft (~1.35x speedup)

Carbon-500M shares the HybridDNA vocab, so it works as a drop-in draft model. Grab it first:

hf download HuggingFaceBio/Carbon-500M-GGUF carbon-500m-bf16.gguf --local-dir .

Then run the standalone tool with --model-draft:

./build/bin/llama-speculative \
    -m  carbon-3b-bf16.gguf \
    -md carbon-500m-bf16.gguf \
    -p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
    -n 256 --temp 0

Or serve the 3B with the 500M draft (llama-server accepts the same -md flag):

./build/bin/llama-server \
    -m  carbon-3b-bf16.gguf \
    -md carbon-500m-bf16.gguf \
    --draft-max 16 --draft-min 1 \
    --port 8080

curl -s http://localhost:8080/completion -d '{
  "prompt": "<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG",
  "n_predict": 256,
  "temperature": 0
}' | jq -r .content

Likelihood scoring

The source card's Python score() function computes mean log-prob per DNA token. In llama.cpp the closest tools are llama-perplexity for corpus-level perplexity (perplexity = exp(-mean_logprob)):

# one prompt per line in dna_corpus.txt, each wrapped in <dna>...</dna>
./build/bin/llama-perplexity -m carbon-3b-bf16.gguf -f dna_corpus.txt --ppl-stride 0

Or llama-server with logprobs for per-token log-probabilities:

./build/bin/llama-server -m carbon-3b-bf16.gguf --port 8080 &
curl -s http://localhost:8080/completion -d '{
  "prompt": "<dna>GGGCTATAAAGGCCATCGATCGATCGATCGATCGATCGATCG</dna>",
  "n_predict": 0,
  "n_probs": 1
}' | jq '.completion_probabilities'

Long context with YaRN (65k tokens ≈ 393 kbp)

Mirrors the source card's rope_scaling = {type: yarn, factor: 4.0, original_max_position_embeddings: 32768}:

./build/bin/llama-completion -m carbon-3b-bf16.gguf \
    -c 65536 --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 32768 \
    -p '<dna>...' -n 64 --temp 0 -no-cnv

Base-pair-level generation (FNS branch) — not supported

The revision="fns" example from the source card needs custom modeling code (factorized nucleotide supervision head), which only the Python transformers path can load. llama.cpp can't run that branch.

Tokenization parity

llama.cpp produces byte-for-byte identical token IDs to the Python HybridDNATokenizer (loaded with trust_remote_code=True) on the standard <dna>/metadata/edge-case fixtures shipped in ggml-org/vocabs.

See also

• Source weights: HuggingFaceBio/Carbon-3B
• Other GGUF variants: 500M · 3B · 8B

License

Apache-2.0, inherited from the source model.