---
library_name: transformers.js
base_model: eulogik/Bharat-Tiny-LLM-v3
tags:
- transformers.js
- onnx
- webgpu
- text-generation
- hindi
- qwen3
license: apache-2.0
---

# Bharat-Tiny-LLM v3 ONNX (WebGPU)

ONNX weights for [eulogik/Bharat-Tiny-LLM-v3](https://huggingface.co/eulogik/Bharat-Tiny-LLM-v3),
a 1.7B bilingual Hindi/Hinglish/English edge LLM (Qwen3-1.7B-Base + 326
Devanagari tokens, total vocab 151,969), converted for in-browser inference
with Transformers.js on WebGPU.

## Recipe

1. FP32 ONNX export of the merged model via `optimum-cli export onnx`
   (`text-generation-with-past`), validated against PyTorch logits.
2. 4-bit symmetric weight quantization, block size 32
   (`MatMulNBitsQuantizer`), same settings as onnx-community conversions.
3. Embedding table rounded to FP16 (1.24 GB -> 0.62 GB).
4. Verified: `onnx.checker` passes and ONNX Runtime loads the model
   (59 inputs / 57 outputs: `input_ids`, `attention_mask`, past key/values).

Layout follows the onnx-community convention (`onnx/model_q4f16.onnx`).

## Usage

```js
// npm i @huggingface/transformers
import { pipeline } from '@huggingface/transformers';

const generator = await pipeline(
  'text-generation',
  'eulogik/Bharat-Tiny-LLM-v3-ONNX',
  { device: 'webgpu', dtype: 'q4f16' },
);

const out = await generator(
  [{ role: 'user', content: 'नमस्ते, आप कैसे हैं?' }],
  { max_new_tokens: 200, temperature: 0.4, top_p: 0.9 },
);
console.log(out[0].generated_text.at(-1).content);
```

Live demo:
[eulogik/bharat-tiny-llm-v3-demo](https://huggingface.co/spaces/eulogik/bharat-tiny-llm-v3-demo)

License: Apache-2.0.