Initial Upload

LICENSE.md

@@ -0,0 +1,70 @@
+LLAMA 3.2 COMMUNITY LICENSE AGREEMENT
+
+Llama 3.2 Version Release Date: September 25, 2024
+
+“Agreement” means the terms and conditions for use, reproduction, distribution and modification of the Llama Materials set forth herein.
+
+
+“Documentation” means the specifications, manuals and documentation accompanying Llama 3.2 distributed by Meta at https://www.llama.com/docs/overview.
+
+
+“Licensee” or “you” means you, or your employer or any other person or entity (if you are entering into this Agreement on such person or entity’s behalf), of the age required under applicable laws, rules or regulations to provide legal consent and that has legal authority to bind your employer or such other person or entity if you are entering in this Agreement on their behalf.
+
+
+“Llama 3.2” means the foundational large language models and software and algorithms, including machine-learning model code, trained model weights, inference-enabling code, training-enabling code, fine-tuning enabling code and other elements of the foregoing distributed by Meta at https://www.llama.com/llama-downloads.
+
+
+“Llama Materials” means, collectively, Meta’s proprietary Llama 3.2 and Documentation (and any portion thereof) made available under this Agreement.
+
+
+“Meta” or “we” means Meta Platforms Ireland Limited (if you are located in or, if you are an entity, your principal place of business is in the EEA or Switzerland) and Meta Platforms, Inc. (if you are located outside of the EEA or Switzerland).
+
+
+By clicking “I Accept” below or by using or distributing any portion or element of the Llama Materials, you agree to be bound by this Agreement.
+
+
+1. License Rights and Redistribution.
+
+
+    a. Grant of Rights. You are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under Meta’s intellectual property or other rights owned by Meta embodied in the Llama Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the Llama Materials.
+
+
+    b. Redistribution and Use.
+
+
+        i. If you distribute or make available the Llama Materials (or any derivative works thereof), or a product or service (including another AI model) that contains any of them, you shall (A) provide a copy of this Agreement with any such Llama Materials; and (B) prominently display “Built with Llama” on a related website, user interface, blogpost, about page, or product documentation. If you use the Llama Materials or any outputs or results of the Llama Materials to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, you shall also include “Llama” at the beginning of any such AI model name.
+
+
+        ii. If you receive Llama Materials, or any derivative works thereof, from a Licensee as part of an integrated end user product, then Section 2 of this Agreement will not apply to you.
+
+
+        iii. You must retain in all copies of the Llama Materials that you distribute the following attribution notice within a “Notice” text file distributed as a part of such copies: “Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.”
+
+
+        iv. Your use of the Llama Materials must comply with applicable laws and regulations (including trade compliance laws and regulations) and adhere to the Acceptable Use Policy for the Llama Materials (available at https://www.llama.com/llama3_2/use-policy), which is hereby incorporated by reference into this Agreement.
+
+2. Additional Commercial Terms. If, on the Llama 3.2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights.
+
+
+3. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE LAW, THE LLAMA MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND META DISCLAIMS ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE LLAMA MATERIALS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE LLAMA MATERIALS AND ANY OUTPUT AND RESULTS.
+
+
+4. Limitation of Liability. IN NO EVENT WILL META OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
+
+
+5. Intellectual Property.
+
+
+    a. No trademark licenses are granted under this Agreement, and in connection with the Llama Materials, neither Meta nor Licensee may use any name or mark owned by or associated with the other or any of its affiliates, except as required for reasonable and customary use in describing and redistributing the Llama Materials or as set forth in this Section 5(a). Meta hereby grants you a license to use “Llama” (the “Mark”) solely as required to comply with the last sentence of Section 1.b.i. You will comply with Meta’s brand guidelines (currently accessible at https://about.meta.com/brand/resources/meta/company-brand/). All goodwill arising out of your use of the Mark will inure to the benefit of Meta.
+
+
+    b. Subject to Meta’s ownership of Llama Materials and derivatives made by or for Meta, with respect to any derivative works and modifications of the Llama Materials that are made by you, as between you and Meta, you are and will be the owner of such derivative works and modifications.
+
+
+    c. If you institute litigation or other proceedings against Meta or any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Llama Materials or Llama 3.2 outputs or results, or any portion of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable by you, then any licenses granted to you under this Agreement shall terminate as of the date such litigation or claim is filed or instituted. You will indemnify and hold harmless Meta from and against any claim by any third party arising out of or related to your use or distribution of the Llama Materials.
+
+
+6. Term and Termination. The term of this Agreement will commence upon your acceptance of this Agreement or access to the Llama Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Meta may terminate this Agreement if you are in breach of any term or condition of this Agreement. Upon termination of this Agreement, you shall delete and cease use of the Llama Materials. Sections 3, 4 and 7 shall survive the termination of this Agreement.
+
+
+7. Governing Law and Jurisdiction. This Agreement will be governed and construed under the laws of the State of California without regard to choice of law principles, and the UN Convention on Contracts for the International Sale of Goods does not apply to this Agreement. The courts of California shall have exclusive jurisdiction of any dispute arising out of this Agreement.

README.md

@@ -0,0 +1,73 @@
+---
+language:
+- en
+- de
+- fr
+- it
+- pt
+- hi
+- es
+- th
+license: llama3.2
+library_name: transformers
+tags:
+- woq
+- intel-neural-compressor
+- inc
+- neural-compressor
+- intel
+- teq
+- meta
+- pytorch
+- llama
+- llama-3
+model_name: Llama 3.2 1B Instruct
+base_model: meta-llama/Llama-3.2-1B-Instruct
+inference: false
+model_creator: meta-llama
+pipeline_tag: text-generation
+prompt_template: '{prompt}
+  '
+quantized_by: fbaldassarri
+---
+
+## Model Information
+
+Quantized version of [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) using torch.float32 for quantization tuning.
+- 4 bits (INT4)
+- group size = 128
+- Symmetrical Quantization
+- Algorith method: TEQ (Trainable Equivalent Transformation for Quantization of LLMs)
+
+Quantization framework: [Intel Neural Compressor](https://github.com/intel/neural-compressor/) version 3.3.1
+
+Note: this INT4 version of Llama-3.2-1B-Instruct has been quantized to run inference through CPU.
+
+## Disclaimer
+
+This quantized model comes with no warrenty. It has been developed experimetally only for research purposes. 
+
+This repository only contains contains two files: quantized_model.pt (weights structure) and qconfig.json, and the generated model is a quantized model. 
+It needs to be used in combination with the base model meta-llama/Llama-3.2-1B-Instruct.
+
+## Replication Recipe
+
+```
+$ conda create --name neural-compressor-3.3.1 --file requirements_conda_neural-compressor-3.3.1
+
+$ python meta-llama_Llama-3.2-1B-Instruct-TEQ-int4-gs128-sym.py
+```
+
+## Run Inference
+
+To run inference you can use [fbaldassarri/woq-inference](https://github.com/fbaldassarri/woq-inference).
+
+```
+python teq_inference.py --base meta-llama/Llama-3.2-1B-Instruct --model_dir ./meta-llama_Llama-3.2-1B-Instruct-TEQ-int4-gs128-sym --weights_file quantized_weight.pt --config_file qconfig.json --prompt "What If you have got superpowers?" --device cpu
+```
+
+Note: You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+
+## License
+
+[Llama 3.2 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE)

meta-llama_Llama-3.2-1B-Instruct-TEQ-int4-gs128-sym.py

@@ -0,0 +1,350 @@
+import os
+import sys
+import time
+import random
+import torch
+
+from collections import UserDict
+from packaging.version import Version
+from datasets import load_dataset
+from torch.utils.data import DataLoader
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+from neural_compressor.common import logger
+from neural_compressor.torch.utils import is_hpex_available, get_torch_version
+
+# ====== utils.py content inlined and fixed ======
+
+class DataloaderPreprocessor:
+    def __init__(self, dataloader_original, use_max_length=False, max_seq_length=2048, nsamples=128) -> None:
+        self.dataloader_original = dataloader_original
+        self.use_max_length = use_max_length
+        self.max_seq_length = max_seq_length
+        self.nsamples = nsamples
+        self.dataloader = []
+        self.is_ready = False
+
+    def get_prepared_dataloader(self):
+        if not self.is_ready:
+            self.prepare_dataloader()
+        return self.dataloader
+
+    def prepare_dataloader(self):
+        if self.use_max_length:
+            self.obtain_first_n_samples_fulllength()
+        else:
+            self.obtain_first_n_samples()
+        self.is_ready = True
+
+    def obtain_first_n_samples(self, seed=0):
+        """Get first nsample data as the real calibration dataset."""
+        self.dataloader.clear()
+        random.seed(seed)
+        for batch in self.dataloader_original:
+            if len(self.dataloader) == self.nsamples:
+                logger.info(f"Successfully collect {self.nsamples} calibration samples.")
+                break
+            # list, tuple
+            if isinstance(batch, list) or isinstance(batch, tuple):
+                if batch[0].shape[-1] > self.max_seq_length:
+                    i = random.randint(0, batch[0].shape[-1] - self.max_seq_length - 1)
+                    j = i + self.max_seq_length
+                    batch_final = []
+                    for item in batch:
+                        if isinstance(item, torch.Tensor) and item.ndim == 2:
+                            batch_final.append(item[:, i:j])
+                        else:
+                            batch_final.append(item)
+                else:
+                    batch_final = batch[:]
+            # dict
+            elif isinstance(batch, dict):
+                try:
+                    length = batch["input_ids"].shape[-1]
+                except Exception:
+                    logger.warning("Please make sure your dict'like data contains key of 'input_ids'.")
+                    continue
+                batch_final = {}
+                if length > self.max_seq_length:
+                    i = random.randint(0, length - self.max_seq_length - 1)
+                    j = i + self.max_seq_length
+                    for key in batch.keys():
+                        if isinstance(batch[key], torch.Tensor):
+                            batch_final[key] = batch[key][:, i:j]
+                        else:
+                            batch_final[key] = batch[key]
+                else:
+                    batch_final = batch
+            # tensor
+            else:
+                if batch.shape[-1] > self.max_seq_length:
+                    i = random.randint(0, batch.shape[-1] - self.max_seq_length - 1)
+                    j = i + self.max_seq_length
+                    batch_final = batch[:, i:j]
+                else:
+                    batch_final = batch
+            self.dataloader.append(batch_final)
+        if len(self.dataloader) < self.nsamples:
+            logger.warning(f"Try to use {self.nsamples} data, but entire dataset size is {len(self.dataloader)}.")
+
+    def obtain_first_n_samples_fulllength(self, seed=0):
+        self.dataloader.clear()
+        random.seed(seed)
+        unified_length = self.max_seq_length
+        for batch in self.dataloader_original:
+            if len(self.dataloader) == self.nsamples:
+                logger.info(f"Successfully collect {self.nsamples} calibration samples.")
+                break
+            # list & tuple
+            if isinstance(batch, list) or isinstance(batch, tuple):
+                if batch[0].shape[-1] == unified_length:
+                    batch_final = batch[:]
+                elif batch[0].shape[-1] > unified_length:
+                    i = random.randint(0, batch[0].shape[-1] - unified_length - 1)
+                    j = i + unified_length
+                    batch_final = []
+                    for item in batch:
+                        if isinstance(item, torch.Tensor) and item.ndim == 2:
+                            batch_final.append(item[:, i:j])
+                        else:
+                            batch_final.append(item)
+                else:
+                    continue
+            # dict
+            elif isinstance(batch, dict):
+                try:
+                    length = batch["input_ids"].shape[-1]
+                except Exception:
+                    logger.warning("Please make sure your dict'like data contains key of 'input_ids'.")
+                    continue
+                batch_final = {}
+                if length == self.max_seq_length:
+                    batch_final = batch
+                elif length > self.max_seq_length:
+                    i = random.randint(0, length - self.max_seq_length - 1)
+                    j = i + self.max_seq_length
+                    for key in batch.keys():
+                        if isinstance(batch[key], torch.Tensor):
+                            batch_final[key] = batch[key][:, i:j]
+                        else:
+                            batch_final[key] = batch[key]
+                else:
+                    continue
+            # tensor
+            else:
+                if batch.shape[-1] == unified_length:
+                    batch_final = batch
+                elif batch.shape[-1] > unified_length:
+                    i = random.randint(0, batch.shape[-1] - unified_length - 1)
+                    j = i + unified_length
+                    batch_final = batch[:, i:j]
+                else:
+                    continue
+            self.dataloader.append(batch_final)
+        if len(self.dataloader) < self.nsamples:
+            logger.warning(
+                f"Trying to allocate {self.nsamples} data with fixed length {unified_length}, "
+                f"but only {len(self.dataloader)} samples are found. Please use smaller 'self.max_seq_length' value."
+            )
+
+def get_example_inputs(model, dataloader):
+    version = get_torch_version()
+    from neural_compressor.torch.algorithms.smooth_quant import move_input_to_device
+    if dataloader is None:
+        return None
+    device = next(model.parameters()).device
+    try:
+        for idx, (input, label) in enumerate(dataloader):
+            input = move_input_to_device(input, device)
+            if isinstance(input, (dict, UserDict)):
+                assert version.release >= Version("1.12.0").release, "INC support IPEX version >= 1.12.0"
+                if "label" in input.keys():
+                    input.pop("label")
+                if version.release <= Version("2.0.1").release:
+                    return tuple(input.values())
+                else:
+                    return dict(input)
+            if isinstance(input, (list, tuple)):
+                return tuple(input)
+            if isinstance(input, torch.Tensor):
+                return input
+            break
+    except Exception as e:
+        for idx, input in enumerate(dataloader):
+            input = move_input_to_device(input, device)
+            if isinstance(input, (dict, UserDict)):
+                assert version.release >= Version("1.12.0").release, "INC support IPEX version >= 1.12.0"
+                if "label" in input.keys():
+                    input.pop("label")
+                if version.release <= Version("2.0.1").release:
+                    return tuple(input.values())
+                else:
+                    return dict(input)
+            if isinstance(input, list) or isinstance(input, tuple):
+                return tuple(input)
+            if isinstance(input, torch.Tensor):
+                return input
+            break
+        if idx == 0:
+            assert False, "Please checkout the example_inputs format."
+
+# ====== End of utils.py content ======
+
+# ====== Hardcoded arguments ======
+class Args:
+    model = "meta-llama/Llama-3.2-1B-Instruct"
+    trust_remote_code = True
+    revision = None
+    dataset = "neuralmagic/LLM_compression_calibration"
+    output_dir = "meta-llama_Llama-3.2-1B-Instruct-TEQ-int4-gs128-sym"
+    quantize = True
+    seed = 42
+    load = False
+    accuracy = False
+    performance = False
+    iters = 100
+    batch_size = 1
+    pad_max_length = 512
+    calib_iters = 512
+    tasks = "lambada_openai,hellaswag,winogrande,piqa"
+    peft_model_id = None
+
+    # Weight-only quantization configs
+    woq_algo = "TEQ"
+    woq_bits = 4
+    woq_dtype = "int"
+    woq_group_size = 128
+    woq_group_dim = 1
+    woq_scheme = "sym"
+    woq_use_mse_search = False
+    woq_use_full_range = False
+    quant_lm_head = True
+    use_hf_format = False
+
+    # TEQ/AWQ configs
+    use_auto_scale = False
+    use_auto_clip = False
+    folding = False
+    absorb_layer_dict = {}
+
+    # DoubleQuant configs
+    double_quant_type = None
+    double_quant_dtype = "fp32"
+    double_quant_bits = 8
+    double_quant_use_sym = True
+    double_quant_group_size = 256
+
+args = Args()
+calib_size = 1
+
+if is_hpex_available():
+    import habana_frameworks.torch.core as htcore
+    htcore.hpu_set_inference_env()
+    device = "hpu"
+else:
+    device = "cpu"
+
+# ====== Helper functions ======
+def get_user_model():
+    torchscript = False
+    if args.woq_algo in ["AWQ", "TEQ"]:
+        torchscript = True
+    user_model = AutoModelForCausalLM.from_pretrained(
+        args.model,
+        torchscript=torchscript,
+        trust_remote_code=args.trust_remote_code,
+        revision=args.revision,
+    )
+    tokenizer = AutoTokenizer.from_pretrained(args.model)
+    user_model = user_model.float()
+    user_model = user_model.to(memory_format=torch.channels_last)
+    user_model.eval()
+    return user_model, tokenizer
+
+def calib_func(prepared_model):
+    for i, calib_input in enumerate(calib_dataloader):
+        if i > args.calib_iters:
+            break
+        prepared_model(calib_input[0])
+
+# ====== Main quantization logic ======
+if args.quantize:
+    user_model, tokenizer = get_user_model()
+    calib_dataset = load_dataset(args.dataset, split="train")
+    calib_dataset = calib_dataset.shuffle(seed=args.seed)
+
+    class Evaluator:
+        def __init__(self, dataset, tokenizer, batch_size=8, pad_val=1, pad_max=196, is_calib=False):
+            self.dataset = dataset
+            self.tokenizer = tokenizer
+            self.batch_size = batch_size
+            self.pad_val = pad_val
+            self.pad_max = pad_max
+            self.is_calib = is_calib
+            self.dataset = self.dataset.map(self.tokenize_function, batched=True)
+            self.dataset.set_format(type="torch", columns=["input_ids"])
+
+        @torch.no_grad()
+        def tokenize_function(self, examples):
+            if args.woq_algo in ['TEQ']:
+                if self.tokenizer.pad_token is None:
+                    self.tokenizer.pad_token = self.tokenizer.eos_token
+                example = self.tokenizer(examples["text"], padding="max_length", max_length=self.pad_max)
+            else:
+                example = self.tokenizer(examples["text"])
+            return example
+
+        @torch.no_grad()
+        def collate_batch(self, batch):
+            input_ids_padded = []
+            last_ind = []
+            for text in batch:
+                input_ids = text["input_ids"]
+                pad_len = self.pad_max - input_ids.shape[0]
+                last_ind.append(input_ids.shape[0] - 1)
+                input_ids = input_ids[:self.pad_max] if len(input_ids) > self.pad_max else input_ids
+                input_ids = torch.nn.functional.pad(input_ids, (0, pad_len), value=self.pad_val)
+                input_ids_padded.append(input_ids)
+            return (torch.vstack(input_ids_padded), torch.tensor(last_ind))
+
+    calib_evaluator = Evaluator(calib_dataset, tokenizer, args.batch_size, pad_max=args.pad_max_length, is_calib=True)
+    calib_dataloader = DataLoader(
+        calib_evaluator.dataset,
+        batch_size=calib_size,
+        shuffle=False,
+        collate_fn=calib_evaluator.collate_batch,
+    )
+
+    # === TEQ quantization ===
+    from neural_compressor.torch.quantization import TEQConfig, prepare, convert
+
+    weight_sym = True if args.woq_scheme == "sym" else False
+    quant_config = TEQConfig(
+        dtype=args.woq_dtype,
+        bits=args.woq_bits,
+        use_sym=weight_sym,
+        group_size=args.woq_group_size,
+        group_dim=args.woq_group_dim,
+        folding=args.folding,
+        quant_lm_head=args.quant_lm_head,
+    )
+
+    example_inputs = torch.ones([1, args.pad_max_length], dtype=torch.long)
+    run_fn = calib_func
+
+    user_model = prepare(model=user_model, quant_config=quant_config, example_inputs=example_inputs)
+    run_fn(user_model)
+    user_model = convert(user_model)
+
+    # === Save quantized model ===
+    os.makedirs(args.output_dir, exist_ok=True)
+    print("Saving weight-only quantized model to", args.output_dir)
+    if args.use_hf_format:
+        user_model.save(args.output_dir, format="huggingface")
+        tokenizer.save_pretrained(args.output_dir)
+    else:
+        user_model.save(args.output_dir)
+    print("Saved weight-only quantized model.")
+
+else:
+    print("Quantization not enabled. Exiting.")

qconfig.json

@@ -0,0 +1,2262 @@
+{
+    "('model.layers.0.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.0.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.0.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.0.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.0.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.0.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.0.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.1.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.1.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.1.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.1.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.1.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.1.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.1.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.2.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.2.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.2.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.2.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.2.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.2.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.2.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.3.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.3.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.3.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.3.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.3.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.3.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.3.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.4.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.4.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.4.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.4.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.4.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.4.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.4.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.5.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.5.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.5.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.5.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.5.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.5.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.5.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.6.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.6.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.6.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.6.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.6.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.6.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.6.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.7.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.7.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.7.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.7.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.7.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.7.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.7.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.8.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.8.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.8.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.8.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.8.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.8.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.8.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.9.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.9.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.9.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.9.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.9.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.9.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.9.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.10.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.10.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.10.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.10.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.10.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.10.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.10.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.11.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.11.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.11.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.11.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.11.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.11.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.11.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.12.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.12.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.12.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.12.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.12.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.12.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.12.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.13.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.13.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.13.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.13.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.13.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.13.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.13.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.14.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.14.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.14.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.14.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.14.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.14.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.14.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.15.self_attn.q_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.15.self_attn.k_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.15.self_attn.v_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.15.self_attn.o_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.15.mlp.gate_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.15.mlp.up_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('model.layers.15.mlp.down_proj.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    },
+    "('lm_head.orig_layer', 'Linear')": {
+        "teq": {
+            "dtype": "int",
+            "bits": 4,
+            "use_sym": true,
+            "group_size": 128,
+            "group_dim": 1,
+            "use_full_range": false,
+            "use_mse_search": false,
+            "use_layer_wise": false,
+            "use_double_quant": false,
+            "double_quant_bits": 8,
+            "double_quant_dtype": "int",
+            "double_quant_use_sym": true,
+            "double_quant_group_size": 256,
+            "quant_lm_head": true,
+            "absorb_to_layer": {},
+            "folding": false
+        }
+    }
+}

quantized_weight.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1bd9d3a6968d5668423a9526bc0cdf80e6a2bbaadcc3909850a3bbd0944940e
+size 1695497302

requirements_conda_neural-compressor-3.3.1.txt

@@ -0,0 +1,164 @@
+# This file may be used to create an environment using:
+# $ conda create --name <env> --file <this file>
+# platform: linux-64
+# created-by: conda 24.11.3
+_libgcc_mutex=0.1=main
+_openmp_mutex=5.1=1_gnu
+absl-py=2.2.2=pypi_0
+accelerate=1.6.0=pypi_0
+aiohappyeyeballs=2.6.1=pypi_0
+aiohttp=3.11.18=pypi_0
+aiosignal=1.3.2=pypi_0
+annotated-types=0.7.0=pypi_0
+astunparse=1.6.3=pypi_0
+attrs=25.3.0=pypi_0
+auto-round=0.5.1=pypi_0
+bzip2=1.0.8=h5eee18b_6
+ca-certificates=2025.2.25=h06a4308_0
+certifi=2025.1.31=pypi_0
+chardet=5.2.0=pypi_0
+charset-normalizer=3.4.1=pypi_0
+click=8.1.8=pypi_0
+colorama=0.4.6=pypi_0
+contourpy=1.3.2=pypi_0
+cycler=0.12.1=pypi_0
+dataproperty=1.1.0=pypi_0
+datasets=3.5.0=pypi_0
+deprecated=1.2.18=pypi_0
+dill=0.3.8=pypi_0
+docker-pycreds=0.4.0=pypi_0
+einops=0.8.1=pypi_0
+evaluate=0.4.3=pypi_0
+filelock=3.13.1=pypi_0
+flatbuffers=25.2.10=pypi_0
+fonttools=4.57.0=pypi_0
+frozenlist=1.6.0=pypi_0
+fsspec=2024.6.1=pypi_0
+gast=0.6.0=pypi_0
+gitdb=4.0.12=pypi_0
+gitpython=3.1.44=pypi_0
+google-pasta=0.2.0=pypi_0
+grpcio=1.71.0=pypi_0
+h5py=3.13.0=pypi_0
+huggingface-hub=0.30.2=pypi_0
+idna=3.10=pypi_0
+iniconfig=2.1.0=pypi_0
+jinja2=3.1.4=pypi_0
+joblib=1.4.2=pypi_0
+jsonlines=4.0.0=pypi_0
+keras=3.9.2=pypi_0
+kiwisolver=1.4.8=pypi_0
+ld_impl_linux-64=2.40=h12ee557_0
+libclang=18.1.1=pypi_0
+libffi=3.4.4=h6a678d5_1
+libgcc-ng=11.2.0=h1234567_1
+libgomp=11.2.0=h1234567_1
+libstdcxx-ng=11.2.0=h1234567_1
+libuuid=1.41.5=h5eee18b_0
+llvmlite=0.44.0=pypi_0
+lm-eval=0.4.3=pypi_0
+lxml=5.4.0=pypi_0
+markdown=3.8=pypi_0
+markdown-it-py=3.0.0=pypi_0
+markupsafe=2.1.5=pypi_0
+matplotlib=3.10.1=pypi_0
+mbstrdecoder=1.1.4=pypi_0
+mdurl=0.1.2=pypi_0
+ml-dtypes=0.5.1=pypi_0
+more-itertools=10.7.0=pypi_0
+mpmath=1.3.0=pypi_0
+multidict=6.4.3=pypi_0
+multiprocess=0.70.16=pypi_0
+namex=0.0.9=pypi_0
+ncurses=6.4=h6a678d5_0
+networkx=3.3=pypi_0
+neural-compressor=3.3.1=pypi_0
+nltk=3.9.1=pypi_0
+numba=0.61.2=pypi_0
+numexpr=2.10.2=pypi_0
+numpy=1.26.4=pypi_0
+opencv-python-headless=4.11.0.86=pypi_0
+openssl=3.0.16=h5eee18b_0
+opt-einsum=3.4.0=pypi_0
+optree=0.15.0=pypi_0
+packaging=25.0=pypi_0
+pandas=2.2.3=pypi_0
+pathvalidate=3.2.3=pypi_0
+peft=0.15.2=pypi_0
+pillow=11.0.0=pypi_0
+pip=25.0=py311h06a4308_0
+platformdirs=4.3.7=pypi_0
+pluggy=1.5.0=pypi_0
+portalocker=3.1.1=pypi_0
+prettytable=3.16.0=pypi_0
+propcache=0.3.1=pypi_0
+protobuf=5.29.4=pypi_0
+psutil=7.0.0=pypi_0
+py-cpuinfo=9.0.0=pypi_0
+pyarrow=19.0.1=pypi_0
+pybind11=2.13.6=pypi_0
+pycocotools=2.0.8=pypi_0
+pydantic=2.11.3=pypi_0
+pydantic-core=2.33.1=pypi_0
+pygments=2.19.1=pypi_0
+pyparsing=3.2.3=pypi_0
+pytablewriter=1.2.1=pypi_0
+pytest=8.3.5=pypi_0
+python=3.11.11=he870216_0
+python-dateutil=2.9.0.post0=pypi_0
+pytz=2025.2=pypi_0
+pyyaml=6.0.2=pypi_0
+readline=8.2=h5eee18b_0
+regex=2024.11.6=pypi_0
+requests=2.32.3=pypi_0
+rich=14.0.0=pypi_0
+rouge-score=0.1.2=pypi_0
+sacrebleu=2.5.1=pypi_0
+safetensors=0.5.3=pypi_0
+schema=0.7.7=pypi_0
+scikit-learn=1.6.1=pypi_0
+scipy=1.15.2=pypi_0
+sentencepiece=0.2.0=pypi_0
+sentry-sdk=2.27.0=pypi_0
+setproctitle=1.3.5=pypi_0
+setuptools=75.8.0=py311h06a4308_0
+six=1.17.0=pypi_0
+smmap=5.0.2=pypi_0
+sqlite=3.45.3=h5eee18b_0
+sqlitedict=2.1.0=pypi_0
+sympy=1.13.3=pypi_0
+tabledata=1.3.4=pypi_0
+tabulate=0.9.0=pypi_0
+tbb=2022.1.0=pypi_0
+tcmlib=1.3.0=pypi_0
+tcolorpy=0.1.7=pypi_0
+tensorboard=2.19.0=pypi_0
+tensorboard-data-server=0.7.2=pypi_0
+tensorflow-cpu=2.19.0=pypi_0
+tensorflow-io-gcs-filesystem=0.37.1=pypi_0
+termcolor=3.0.1=pypi_0
+threadpoolctl=3.6.0=pypi_0
+tk=8.6.14=h39e8969_0
+tokenizers=0.21.1=pypi_0
+torch=2.7.0+cpu=pypi_0
+torchaudio=2.7.0+cpu=pypi_0
+torchvision=0.22.0+cpu=pypi_0
+tqdm=4.67.1=pypi_0
+tqdm-multiprocess=0.0.11=pypi_0
+transformers=4.51.3=pypi_0
+typepy=1.3.4=pypi_0
+typing-extensions=4.12.2=pypi_0
+typing-inspection=0.4.0=pypi_0
+tzdata=2025.2=pypi_0
+urllib3=2.4.0=pypi_0
+wandb=0.19.10=pypi_0
+wcwidth=0.2.13=pypi_0
+werkzeug=3.1.3=pypi_0
+wheel=0.45.1=py311h06a4308_0
+word2number=1.1=pypi_0
+wrapt=1.17.2=pypi_0
+xxhash=3.5.0=pypi_0
+xz=5.6.4=h5eee18b_1
+yarl=1.20.0=pypi_0
+zlib=1.2.13=h5eee18b_1
+zstandard=0.23.0=pypi_0