---
library_name: llama-cpp
base_model: perplexity-ai/pplx-embed-v1-0.6b
tags:
  - embedding
  - gguf
  - qwen3
  - bidirectional
license: apache-2.0
---

# pplx-embed-v1-0.6b GGUF (F16)

GGUF conversion of [perplexity-ai/pplx-embed-v1-0.6b](https://huggingface.co/perplexity-ai/pplx-embed-v1-0.6b).

| File | Quant | Size |
|---|---|---|
| `pplx-embed-v1-0.6b-f16.gguf` | F16 | 1.2 GB |

## Usage

> **This model requires non-causal (bidirectional) attention.** Without it, outputs will be incorrect.

```python
from llama_cpp import Llama, llama_cpp

llm = Llama(model_path="pplx-embed-v1-0.6b-f16.gguf", embedding=True, pooling_type=1)
llama_cpp.llama_set_causal_attn(llm._ctx.ctx, False)

raw = llm.embed("your text here")
```

> **Note:** The GGUF outputs raw float embeddings only. The original model natively produces int8/binary quantized embeddings via a post-processing step (`st_quantize.FlexibleQuantizer`). To match that behavior, apply the quantization manually:

```python
import numpy as np

# Int8: tanh → scale → round → clamp (matches Int8TanhQuantizer)
int8_emb = np.clip(np.round(np.tanh(raw) * 127), -128, 127).astype(np.int8)

# Binary: sign (matches BinaryTanhQuantizer)
binary_emb = np.where(np.array(raw) >= 0, 1, -1).astype(np.int8)
```

CLI:
```bash
llama-embedding -m pplx-embed-v1-0.6b-f16.gguf --attention non-causal --pooling mean -p "your text here"
```

## Verification

Cosine similarity vs original: **> 0.99999** across all test inputs. Residual diffs are from F32 → F16 precision.