iAcloud/Qwen3-Embedding-4B-RBLN-0.11.2

Qwen3-Embedding-4B — RBLN (ATOM CA25) compiled

Qwen/Qwen3-Embedding-4B pre-compiled for Rebellions RBLN-CA25 NPUs with optimum-rbln, ready to serve with vllm-rbln. This repository contains the compiled artifact only — RBLN NPUs and the RBLN SDK are required to run it.

Hardware & software

Component	Version / configuration
NPU	2× RBLN-CA25 chiplets (32 GiB total; half of one 64 GiB CA25 card)
Compiler	rebel-compiler==0.11.2
Runtime	optimum-rbln==0.11.2, vllm-rbln==0.11.2.post1
Framework	transformers==5.8.1, vllm==0.24.0+cpu

Compile configuration

Option	Value
max_seq_len	40,960
batch_size	1
num_devices	2
attn_impl	flash_attn
kvcache_partition_len	8,192
prefill_chunk_size	128
kvcache_num_blocks	5 (one complete 40,960-token sequence)
phases	prefill only

The batch size is intentionally one. In the current Optimum pooling path, larger batches are dispatched through a Python-side per-item prefill loop and do not improve NPU throughput. A 64 GiB CA25 card can host two independent 2-chiplet replicas for real request-level parallelism.

Serving

vllm serve <this-repo-or-local-path> \
  --runner pooling \
  --max-num-seqs 1 \
  --max-model-len 40960 \
  --no-enable-prefix-caching

Do not pass --tensor-parallel-size 2; the two-device split is already baked into the compiled model and vllm-rbln uses tensor parallel size 1 for the serving process.

The OpenAI-compatible embedding endpoint is /v1/embeddings. Embeddings have 2,560 dimensions and are returned L2-normalized.

License

Derived from Qwen/Qwen3-Embedding-4B; use is subject to the original model's license and terms.