BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models
Paper • 2602.04163 • Published • 10
How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "goodman20241017/Llama-2-7B-BPD3-G64" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "goodman20241017/Llama-2-7B-BPD3-G64",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
BPDQ: Llama-2-7b-hf 3-bit
This model is a 3-bit quantized version of meta-llama/Llama-2-7b-hf using the Bit-Plane Decomposition Quantization (BPDQ) algorithm.
- Paper: BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models
- GitHub: https://github.com/KingdalfGoodman/BPDQ
Description
BPDQ is a post-training quantization (PTQ) method that constructs a variable quantization grid via bit-plane decomposition and scalar coefficients. By iteratively refining these using second-order information, BPDQ expands the feasible set for error minimization, allowing for high-fidelity performance in extreme low-bit regimes (2–3 bits) where conventional fixed-grid PTQ methods typically degrade.
Installation and Usage
BPDQ is implemented as a patch on top of GPTQModel at version 5.7.0. To use this model, you need to apply the patch provided in the official repository:
- Clone GPTQModel and check out v5.7.0:
git clone https://github.com/ModelCloud/GPTQModel.git cd GPTQModel git checkout v5.7.0 - Apply the BPDQ patch (available in the BPDQ GitHub repo):
git apply /path/to/BPDQ/bpdq.patch - Install the modified library:
pip install -e .
For detailed quantization and evaluation workflows, please refer to the official repository.
Citation
If you find BPDQ useful in your research, please cite:
@article{chen2026bpdq,
title={BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models},
author={Chen, Junyu and Li, Jungang and Xiong, Jing and Wang, Wenjie and Yang, Qingyao and Xiao, He and Li, Zhen and Wu, Taiqiang and Chen, Mengzhao and Peng, Zhen and others},
journal={arXiv preprint arXiv:2602.04163},
year={2026}
}
- Downloads last month
- 85
Model tree for goodman20241017/Llama-2-7B-BPD3-G64
Base model
meta-llama/Llama-2-7b-hf
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "goodman20241017/Llama-2-7B-BPD3-G64" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "goodman20241017/Llama-2-7B-BPD3-G64", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'