---
license: other
base_model: nvidia/LocateAnything-3B
base_model_relation: quantized
quantized_by: blockblockblock
library_name: exllamav3
pipeline_tag: text-generation
tags:
  - exl3
  - exllamav3
  - quantized

quantization_format: exl3
bits_per_weight: 6.0
---

<div align="center">

# LocateAnything · 3B

<sub><code>EXL3</code> &nbsp;·&nbsp; <b>6.0&nbsp;bpw</b> &nbsp;·&nbsp; 3.9&nbsp;GB &nbsp;·&nbsp; Dense</sub>

<br/>

[![format](https://img.shields.io/badge/format-EXL3-c63010?style=for-the-badge&labelColor=14120e)](https://github.com/turboderp-org/exllamav3)
[![bpw](https://img.shields.io/badge/bpw-6.0-6b8a76?style=for-the-badge&labelColor=14120e)](#quants)
[![size](https://img.shields.io/badge/size-3.9_GB-6b8a76?style=for-the-badge&labelColor=14120e)](#quants)
[![arch](https://img.shields.io/badge/arch-Dense_3B-c63010?style=for-the-badge&labelColor=14120e)](https://huggingface.co/nvidia/LocateAnything-3B)

[![base model](https://img.shields.io/badge/Base-nvidia%2FLocateAnything--3B-2a2620?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/nvidia/LocateAnything-3B)
[![quantized by](https://img.shields.io/badge/Quantized_by-blockblockblock-2a2620?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/blockblockblock)
[![collection](https://img.shields.io/badge/All_bpws-Collection-c63010?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/collections/blockblockblock/locateanything-3b-exl3-6a205fbdd9aee1d0eba42b33)

</div>

---

> [!NOTE]
> An [ExLlamaV3](https://github.com/turboderp-org/exllamav3) build of [`nvidia/LocateAnything-3B`](https://huggingface.co/nvidia/LocateAnything-3B) at **6.0 bits per weight**. See [Quants](#quants) for sibling repos at other bit&#8209;widths or browse the [collection](https://huggingface.co/collections/blockblockblock/locateanything-3b-exl3-6a205fbdd9aee1d0eba42b33).

## Quants

<div align="center">

| BPW &nbsp; | &nbsp; Head bits &nbsp; | &nbsp; Calibration rows &nbsp; | &nbsp; Size &nbsp; | &nbsp; Status |
| :---: | :---: | :---: | ---: | :--- |
| 4.0 | 8 | 250 | 3.2&nbsp;GB | [link](https://huggingface.co/blockblockblock/LocateAnything-3B-exl3-4.0bpw) |
| **6.0** | 8 | 250 | **3.9&nbsp;GB** | <kbd>this repo</kbd> |

</div>

## Inference

<table>
  <thead>
    <tr>
      <th align="left" width="32%">Loader</th>
      <th align="left">Use it for</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><a href="https://github.com/theroyallab/tabbyAPI"><b>TabbyAPI</b></a></td>
      <td>OpenAI&#8209;compatible HTTP server. Drop&#8209;in for OpenAI clients.</td>
    </tr>
    <tr>
      <td><a href="https://github.com/oobabooga/text-generation-webui"><b>text&#8209;generation&#8209;webui</b></a></td>
      <td>Local chat UI. Pick the <i>ExLlamaV3</i> loader from the model dropdown.</td>
    </tr>
    <tr>
      <td><a href="https://github.com/turboderp-org/exllamav3"><b>ExLlamaV3</b></a></td>
      <td>Direct Python API for embedding the model in your own code or pipeline.</td>
    </tr>
  </tbody>
</table>

## Download

```bash
pip install -U huggingface_hub

hf download \
  blockblockblock/LocateAnything-3B-exl3-6.0bpw \
  --local-dir ./LocateAnything-3B-exl3-6.0bpw
```

<details>
<summary><b>Quantization recipe</b> &nbsp;<sub>(advanced, embedded in <code>quantization_config.json</code>)</sub></summary>

<br/>

| Setting | Value |
| :--- | :--- |
| Format | `EXL3` |
| Bits per weight | `6.0` |
| Head bits | `8` |
| Calibration rows | `250` |
| Codebook | `MCG` |
| Out&#8209;scales | `always` |
| Parallel mode | `enabled` |

Loaded automatically by every ExLlamaV3 loader; reproduced here for searchability.

</details>

## License &amp; use

> [!IMPORTANT]
> Use and license **follow the [base model](https://huggingface.co/nvidia/LocateAnything-3B)**.
> Quantization adds no additional restrictions. Refer to the upstream repository for terms, citation, and safety documentation.

---

<div align="center">
  <sub><i>Quantized with <a href="https://github.com/Honkware/blockquant"><b>BlockQuant</b></a> &nbsp;·&nbsp; convention&nbsp;<code>{org}/{model}-exl3-{bpw}bpw</code></i></sub>
</div>