goodman20241017/Llama-2-7B-BPD3-G64

BPDQ: Llama-2-7b-hf 3-bit

This model is a 3-bit quantized version of meta-llama/Llama-2-7b-hf using the Bit-Plane Decomposition Quantization (BPDQ) algorithm.

• Paper: BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models
• GitHub: https://github.com/KingdalfGoodman/BPDQ

Description

BPDQ is a post-training quantization (PTQ) method that constructs a variable quantization grid via bit-plane decomposition and scalar coefficients. By iteratively refining these using second-order information, BPDQ expands the feasible set for error minimization, allowing for high-fidelity performance in extreme low-bit regimes (2–3 bits) where conventional fixed-grid PTQ methods typically degrade.

Installation and Usage

BPDQ is implemented as a patch on top of GPTQModel at version 5.7.0. To use this model, you need to apply the patch provided in the official repository:

1. Clone GPTQModel and check out v5.7.0:

   git clone https://github.com/ModelCloud/GPTQModel.git
   cd GPTQModel
   git checkout v5.7.0
   

2. Apply the BPDQ patch (available in the BPDQ GitHub repo):

   git apply /path/to/BPDQ/bpdq.patch
   

3. Install the modified library:

   pip install -e .
   

For detailed quantization and evaluation workflows, please refer to the official repository.

Citation

If you find BPDQ useful in your research, please cite:

@article{chen2026bpdq,
  title={BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models},
  author={Chen, Junyu and Li, Jungang and Xiong, Jing and Wang, Wenjie and Yang, Qingyao and Xiao, He and Li, Zhen and Wu, Taiqiang and Chen, Mengzhao and Peng, Zhen and others},
  journal={arXiv preprint arXiv:2602.04163},
  year={2026}
}