| /opt/conda/envs/py310/bin/python -m mlc_llm gen_config /models/Qwen2-72B-Instruct --quantization q4f16_1 --conv-template chatml --output /models/mlc-delivery/hf/mlc-ai/Qwen2-72B-Instruct-q4f16_1-MLC |
| [2024-06-07 00:12:29] INFO auto_config.py:116: [92mFound[0m model configuration: /models/Qwen2-72B-Instruct/config.json |
| [2024-06-07 00:12:29] INFO auto_config.py:154: [92mFound[0m model type: [1mqwen2[0m. Use `--model-type` to override. |
| [2024-06-07 00:12:29] INFO qwen2_model.py:49: [1mcontext_window_size[0m not found in config.json. Falling back to [1mmax_position_embeddings[0m (32768) |
| [2024-06-07 00:12:29] INFO qwen2_model.py:66: [1mprefill_chunk_size[0m defaults to 2048 |
| [2024-06-07 00:12:29] INFO config.py:107: Overriding [1mmax_batch_size[0m from 1 to 80 |
| [2024-06-07 00:12:29] INFO gen_config.py:143: [generation_config.json] Setting [1mbos_token_id[0m: 151643 |
| [2024-06-07 00:12:29] INFO gen_config.py:143: [generation_config.json] Setting [1mpad_token_id[0m: 151643 |
| [2024-06-07 00:12:29] INFO gen_config.py:143: [generation_config.json] Setting [1meos_token_id[0m: [151645, 151643] |
| [2024-06-07 00:12:29] INFO gen_config.py:143: [generation_config.json] Setting [1mrepetition_penalty[0m: 1.05 |
| [2024-06-07 00:12:29] INFO gen_config.py:143: [generation_config.json] Setting [1mtemperature[0m: 0.7 |
| [2024-06-07 00:12:29] INFO gen_config.py:143: [generation_config.json] Setting [1mtop_p[0m: 0.8 |
| [2024-06-07 00:12:29] INFO gen_config.py:157: [91mNot found[0m tokenizer config: /models/Qwen2-72B-Instruct/tokenizer.model |
| [2024-06-07 00:12:29] INFO gen_config.py:155: [92mFound[0m tokenizer config: /models/Qwen2-72B-Instruct/tokenizer.json. Copying to [1m/models/mlc-delivery/hf/mlc-ai/Qwen2-72B-Instruct-q4f16_1-MLC/tokenizer.json[0m |
| [2024-06-07 00:12:29] INFO gen_config.py:155: [92mFound[0m tokenizer config: /models/Qwen2-72B-Instruct/vocab.json. Copying to [1m/models/mlc-delivery/hf/mlc-ai/Qwen2-72B-Instruct-q4f16_1-MLC/vocab.json[0m |
| [2024-06-07 00:12:30] INFO gen_config.py:155: [92mFound[0m tokenizer config: /models/Qwen2-72B-Instruct/merges.txt. Copying to [1m/models/mlc-delivery/hf/mlc-ai/Qwen2-72B-Instruct-q4f16_1-MLC/merges.txt[0m |
| [2024-06-07 00:12:30] INFO gen_config.py:157: [91mNot found[0m tokenizer config: /models/Qwen2-72B-Instruct/added_tokens.json |
| [2024-06-07 00:12:30] INFO gen_config.py:155: [92mFound[0m tokenizer config: /models/Qwen2-72B-Instruct/tokenizer_config.json. Copying to [1m/models/mlc-delivery/hf/mlc-ai/Qwen2-72B-Instruct-q4f16_1-MLC/tokenizer_config.json[0m |
| [2024-06-07 00:12:30] INFO gen_config.py:216: Detected tokenizer info: {'token_postproc_method': 'byte_level', 'prepend_space_in_encode': False, 'strip_space_in_decode': False} |
| [2024-06-07 00:12:30] INFO gen_config.py:32: [System default] Setting [1mpresence_penalty[0m: 0.0 |
| [2024-06-07 00:12:30] INFO gen_config.py:32: [System default] Setting [1mfrequency_penalty[0m: 0.0 |
| [2024-06-07 00:12:30] INFO gen_config.py:223: Dumping configuration file to: [1m/models/mlc-delivery/hf/mlc-ai/Qwen2-72B-Instruct-q4f16_1-MLC/mlc-chat-config.json[0m |
| /opt/conda/envs/py310/bin/python -m mlc_llm convert_weight /models/Qwen2-72B-Instruct --quantization q4f16_1 --output /models/mlc-delivery/hf/mlc-ai/Qwen2-72B-Instruct-q4f16_1-MLC |
| [2024-06-07 00:12:31] INFO auto_config.py:116: [92mFound[0m model configuration: /models/Qwen2-72B-Instruct/config.json |
| [2024-06-07 00:12:33] INFO auto_device.py:79: [92mFound[0m device: cuda:0 |
| [2024-06-07 00:12:34] INFO auto_device.py:88: [91mNot found[0m device: rocm:0 |
| [2024-06-07 00:12:35] INFO auto_device.py:88: [91mNot found[0m device: metal:0 |
| [2024-06-07 00:12:37] INFO auto_device.py:79: [92mFound[0m device: vulkan:0 |
| [2024-06-07 00:12:37] INFO auto_device.py:79: [92mFound[0m device: vulkan:1 |
| [2024-06-07 00:12:37] INFO auto_device.py:79: [92mFound[0m device: vulkan:2 |
| [2024-06-07 00:12:37] INFO auto_device.py:79: [92mFound[0m device: vulkan:3 |
| [2024-06-07 00:12:39] INFO auto_device.py:88: [91mNot found[0m device: opencl:0 |
| [2024-06-07 00:12:39] INFO auto_device.py:35: Using device: [1mcuda:0[0m |
| [2024-06-07 00:12:39] INFO auto_weight.py:71: Finding weights in: /models/Qwen2-72B-Instruct |
| [2024-06-07 00:12:39] INFO auto_weight.py:137: [91mNot found[0m Huggingface PyTorch |
| [2024-06-07 00:12:39] INFO auto_weight.py:144: [92mFound[0m source weight format: huggingface-safetensor. Source configuration: /models/Qwen2-72B-Instruct/model.safetensors.index.json |
| [2024-06-07 00:12:39] INFO auto_weight.py:107: Using source weight configuration: [1m/models/Qwen2-72B-Instruct/model.safetensors.index.json[0m. Use `--source` to override. |
| [2024-06-07 00:12:39] INFO auto_weight.py:111: Using source weight format: [1mhuggingface-safetensor[0m. Use `--source-format` to override. |
| [2024-06-07 00:12:39] INFO auto_config.py:154: [92mFound[0m model type: [1mqwen2[0m. Use `--model-type` to override. |
| [2024-06-07 00:12:39] INFO qwen2_model.py:49: [1mcontext_window_size[0m not found in config.json. Falling back to [1mmax_position_embeddings[0m (32768) |
| [2024-06-07 00:12:39] INFO qwen2_model.py:66: [1mprefill_chunk_size[0m defaults to 2048 |
| [1mWeight conversion with arguments:[0m |
| [1m--config[0m /models/Qwen2-72B-Instruct/config.json |
| [1m--quantization[0m GroupQuantize(name='q4f16_1', kind='group-quant', group_size=32, quantize_dtype='int4', storage_dtype='uint32', model_dtype='float16', linear_weight_layout='NK', quantize_embedding=True, quantize_final_fc=True, num_elem_per_storage=8, num_storage_per_group=4, max_int_value=7) |
| [1m--model-type[0m qwen2 |
| [1m--device[0m cuda:0 |
| [1m--source[0m /models/Qwen2-72B-Instruct/model.safetensors.index.json |
| [1m--source-format[0m huggingface-safetensor |
| [1m--output[0m /models/mlc-delivery/hf/mlc-ai/Qwen2-72B-Instruct-q4f16_1-MLC |
| Start storing to cache /models/mlc-delivery/hf/mlc-ai/Qwen2-72B-Instruct-q4f16_1-MLC |
|
0%| | 0/563 [00:00<?, ?it/s]
[2024-06-07 00:12:46] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00037-of-00037.safetensors |
|
0%| | 0/563 [00:00<?, ?it/s]
[2024-06-07 00:12:57] INFO group_quantization.py:217: Compiling quantize function for key: ((152064, 8192), float16, cuda, axis=1, output_transpose=False) |
|
0%| | 0/563 [00:11<?, ?it/s]
[2024-06-07 00:12:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mlm_head.q_weight[0m", shape: (152064, 1024), dtype: uint32 |
|
0%| | 0/563 [00:12<?, ?it/s]
[2024-06-07 00:13:00] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mlm_head.q_scale[0m", shape: (152064, 256), dtype: float16 |
|
0%| | 0/563 [00:14<?, ?it/s]
0%| | 1/563 [00:14<2:16:05, 14.53s/it]
[2024-06-07 00:13:00] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.79.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
0%| | 1/563 [00:14<2:16:05, 14.53s/it]
[2024-06-07 00:13:01] INFO group_quantization.py:217: Compiling quantize function for key: ((8192, 29568), float16, cuda, axis=1, output_transpose=False) |
|
0%| | 1/563 [00:15<2:16:05, 14.53s/it]
[2024-06-07 00:13:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.79.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
0%| | 1/563 [00:15<2:16:05, 14.53s/it]
[2024-06-07 00:13:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.79.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
0%| | 1/563 [00:16<2:16:05, 14.53s/it]
1%| | 3/563 [00:16<41:00, 4.39s/it]
[2024-06-07 00:13:02] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00036-of-00037.safetensors |
|
1%| | 3/563 [00:16<41:00, 4.39s/it]
[2024-06-07 00:13:10] INFO group_quantization.py:217: Compiling quantize function for key: ((59136, 8192), float16, cuda, axis=1, output_transpose=False) |
|
1%| | 3/563 [00:24<41:00, 4.39s/it]
[2024-06-07 00:13:10] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.79.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
1%| | 3/563 [00:24<41:00, 4.39s/it]
[2024-06-07 00:13:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.79.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
1%| | 3/563 [00:25<41:00, 4.39s/it]
1%| | 4/563 [00:25<56:52, 6.10s/it]
[2024-06-07 00:13:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.79.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
1%| | 4/563 [00:25<56:52, 6.10s/it]
[2024-06-07 00:13:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.norm.weight[0m", shape: (8192,), dtype: float16 |
|
1%| | 4/563 [00:25<56:52, 6.10s/it]
[2024-06-07 00:13:11] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00036-of-00037.safetensors |
|
1%| | 4/563 [00:25<56:52, 6.10s/it]
[2024-06-07 00:13:12] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00037-of-00037.safetensors |
|
1%| | 4/563 [00:26<56:52, 6.10s/it]
[2024-06-07 00:13:12] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00001-of-00037.safetensors |
|
1%| | 4/563 [00:26<56:52, 6.10s/it]
[2024-06-07 00:13:27] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.embed_tokens.q_weight[0m", shape: (152064, 1024), dtype: uint32 |
|
1%| | 4/563 [00:40<56:52, 6.10s/it]
[2024-06-07 00:13:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.embed_tokens.q_scale[0m", shape: (152064, 256), dtype: float16 |
|
1%| | 4/563 [00:42<56:52, 6.10s/it]
1%| | 7/563 [00:42<54:30, 5.88s/it]
[2024-06-07 00:13:34] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.0.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
1%| | 7/563 [00:48<54:30, 5.88s/it]
[2024-06-07 00:13:34] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.0.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
1%| | 7/563 [00:48<54:30, 5.88s/it]
1%|▏ | 8/563 [00:48<54:48, 5.92s/it]
[2024-06-07 00:13:34] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.0.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
1%|▏ | 8/563 [00:48<54:48, 5.92s/it]
[2024-06-07 00:13:35] INFO group_quantization.py:217: Compiling quantize function for key: ((10240, 8192), float16, cuda, axis=1, output_transpose=False) |
|
1%|▏ | 8/563 [00:49<54:48, 5.92s/it]
[2024-06-07 00:13:35] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.0.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
1%|▏ | 8/563 [00:49<54:48, 5.92s/it]
[2024-06-07 00:13:35] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.0.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
1%|▏ | 8/563 [00:49<54:48, 5.92s/it]
2%|▏ | 10/563 [00:49<35:07, 3.81s/it]
[2024-06-07 00:13:36] INFO group_quantization.py:217: Compiling quantize function for key: ((8192, 8192), float16, cuda, axis=1, output_transpose=False) |
|
2%|▏ | 10/563 [00:50<35:07, 3.81s/it]
[2024-06-07 00:13:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.0.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
2%|▏ | 10/563 [00:50<35:07, 3.81s/it]
[2024-06-07 00:13:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.0.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
2%|▏ | 10/563 [00:50<35:07, 3.81s/it]
2%|▏ | 11/563 [00:50<29:03, 3.16s/it]
[2024-06-07 00:13:36] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00001-of-00037.safetensors |
|
2%|▏ | 11/563 [00:50<29:03, 3.16s/it]
[2024-06-07 00:13:37] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00002-of-00037.safetensors |
|
2%|▏ | 11/563 [00:51<29:03, 3.16s/it]
[2024-06-07 00:13:45] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.0.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
2%|▏ | 11/563 [00:58<29:03, 3.16s/it]
2%|▏ | 12/563 [00:58<40:22, 4.40s/it]
[2024-06-07 00:13:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.0.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
2%|▏ | 12/563 [01:00<40:22, 4.40s/it]
[2024-06-07 00:13:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.0.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
2%|▏ | 12/563 [01:00<40:22, 4.40s/it]
2%|▏ | 13/563 [01:00<34:29, 3.76s/it]
[2024-06-07 00:13:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.0.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
2%|▏ | 13/563 [01:00<34:29, 3.76s/it]
[2024-06-07 00:13:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.1.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
2%|▏ | 13/563 [01:00<34:29, 3.76s/it]
[2024-06-07 00:13:48] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.1.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
2%|▏ | 13/563 [01:02<34:29, 3.76s/it]
[2024-06-07 00:13:48] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.1.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
2%|▏ | 13/563 [01:02<34:29, 3.76s/it]
3%|▎ | 16/563 [01:02<19:18, 2.12s/it]
[2024-06-07 00:13:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.1.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
3%|▎ | 16/563 [01:08<19:18, 2.12s/it]
[2024-06-07 00:13:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.1.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
3%|▎ | 16/563 [01:08<19:18, 2.12s/it]
3%|▎ | 17/563 [01:08<26:35, 2.92s/it]
[2024-06-07 00:13:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.1.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
3%|▎ | 17/563 [01:08<26:35, 2.92s/it]
[2024-06-07 00:13:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.1.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
3%|▎ | 17/563 [01:08<26:35, 2.92s/it]
[2024-06-07 00:13:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.1.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
3%|▎ | 17/563 [01:09<26:35, 2.92s/it]
[2024-06-07 00:13:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.1.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
3%|▎ | 17/563 [01:09<26:35, 2.92s/it]
4%|▎ | 20/563 [01:09<15:17, 1.69s/it]
[2024-06-07 00:13:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.1.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
4%|▎ | 20/563 [01:09<15:17, 1.69s/it]
[2024-06-07 00:13:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.1.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
4%|▎ | 20/563 [01:09<15:17, 1.69s/it]
4%|▎ | 21/563 [01:09<13:03, 1.45s/it]
[2024-06-07 00:13:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.2.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
4%|▎ | 21/563 [01:09<13:03, 1.45s/it]
[2024-06-07 00:13:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.2.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
4%|▎ | 21/563 [01:10<13:03, 1.45s/it]
[2024-06-07 00:13:57] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.2.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
4%|▎ | 21/563 [01:11<13:03, 1.45s/it]
4%|▍ | 23/563 [01:11<10:27, 1.16s/it]
[2024-06-07 00:14:00] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.2.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
4%|▍ | 23/563 [01:14<10:27, 1.16s/it]
[2024-06-07 00:14:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.2.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
4%|▍ | 23/563 [01:15<10:27, 1.16s/it]
4%|▍ | 24/563 [01:15<15:50, 1.76s/it]
[2024-06-07 00:14:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.2.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
4%|▍ | 24/563 [01:15<15:50, 1.76s/it]
[2024-06-07 00:14:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.2.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
4%|▍ | 24/563 [01:15<15:50, 1.76s/it]
[2024-06-07 00:14:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.2.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
4%|▍ | 24/563 [01:15<15:50, 1.76s/it]
[2024-06-07 00:14:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.2.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
4%|▍ | 24/563 [01:15<15:50, 1.76s/it]
5%|▍ | 27/563 [01:15<09:16, 1.04s/it]
[2024-06-07 00:14:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.2.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
5%|▍ | 27/563 [01:16<09:16, 1.04s/it]
[2024-06-07 00:14:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.2.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
5%|▍ | 27/563 [01:16<09:16, 1.04s/it]
5%|▍ | 28/563 [01:16<08:12, 1.09it/s]
[2024-06-07 00:14:02] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00002-of-00037.safetensors |
|
5%|▍ | 28/563 [01:16<08:12, 1.09it/s]
[2024-06-07 00:14:02] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00006-of-00037.safetensors |
|
5%|▍ | 28/563 [01:16<08:12, 1.09it/s]
[2024-06-07 00:14:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.10.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
5%|▍ | 28/563 [01:25<08:12, 1.09it/s]
5%|▌ | 29/563 [01:25<22:32, 2.53s/it]
[2024-06-07 00:14:12] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.10.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
5%|▌ | 29/563 [01:26<22:32, 2.53s/it]
[2024-06-07 00:14:12] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.10.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
5%|▌ | 29/563 [01:26<22:32, 2.53s/it]
5%|▌ | 30/563 [01:26<20:49, 2.34s/it]
[2024-06-07 00:14:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.10.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
5%|▌ | 30/563 [01:30<20:49, 2.34s/it]
[2024-06-07 00:14:17] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.10.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
5%|▌ | 30/563 [01:31<20:49, 2.34s/it]
6%|▌ | 31/563 [01:31<25:42, 2.90s/it]
[2024-06-07 00:14:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.10.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
6%|▌ | 31/563 [01:31<25:42, 2.90s/it]
[2024-06-07 00:14:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.10.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
6%|▌ | 31/563 [01:31<25:42, 2.90s/it]
[2024-06-07 00:14:17] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.10.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
6%|▌ | 31/563 [01:31<25:42, 2.90s/it]
[2024-06-07 00:14:18] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.10.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
6%|▌ | 31/563 [01:31<25:42, 2.90s/it]
6%|▌ | 34/563 [01:31<13:11, 1.50s/it]
[2024-06-07 00:14:18] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.10.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
6%|▌ | 34/563 [01:32<13:11, 1.50s/it]
[2024-06-07 00:14:18] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.10.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
6%|▌ | 34/563 [01:32<13:11, 1.50s/it]
6%|▌ | 35/563 [01:32<11:10, 1.27s/it]
[2024-06-07 00:14:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.11.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
6%|▌ | 35/563 [01:32<11:10, 1.27s/it]
[2024-06-07 00:14:19] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.11.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
6%|▌ | 35/563 [01:33<11:10, 1.27s/it]
[2024-06-07 00:14:19] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.11.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
6%|▌ | 35/563 [01:33<11:10, 1.27s/it]
7%|▋ | 37/563 [01:33<09:04, 1.03s/it]
[2024-06-07 00:14:23] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.11.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
7%|▋ | 37/563 [01:37<09:04, 1.03s/it]
[2024-06-07 00:14:23] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.11.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
7%|▋ | 37/563 [01:37<09:04, 1.03s/it]
7%|▋ | 38/563 [01:37<15:03, 1.72s/it]
[2024-06-07 00:14:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.11.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
7%|▋ | 38/563 [01:37<15:03, 1.72s/it]
[2024-06-07 00:14:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.11.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
7%|▋ | 38/563 [01:37<15:03, 1.72s/it]
[2024-06-07 00:14:24] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.11.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
7%|▋ | 38/563 [01:38<15:03, 1.72s/it]
[2024-06-07 00:14:24] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.11.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
7%|▋ | 38/563 [01:38<15:03, 1.72s/it]
7%|▋ | 41/563 [01:38<08:37, 1.01it/s]
[2024-06-07 00:14:24] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.11.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
7%|▋ | 41/563 [01:38<08:37, 1.01it/s]
[2024-06-07 00:14:24] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.11.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
7%|▋ | 41/563 [01:38<08:37, 1.01it/s]
7%|▋ | 42/563 [01:38<07:33, 1.15it/s]
[2024-06-07 00:14:24] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.9.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
7%|▋ | 42/563 [01:38<07:33, 1.15it/s]
[2024-06-07 00:14:25] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.9.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
7%|▋ | 42/563 [01:39<07:33, 1.15it/s]
[2024-06-07 00:14:25] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.9.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
7%|▋ | 42/563 [01:39<07:33, 1.15it/s]
8%|▊ | 44/563 [01:39<06:45, 1.28it/s]
[2024-06-07 00:14:25] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.9.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
8%|▊ | 44/563 [01:39<06:45, 1.28it/s]
[2024-06-07 00:14:25] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00006-of-00037.safetensors |
|
8%|▊ | 44/563 [01:39<06:45, 1.28it/s]
[2024-06-07 00:14:26] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00007-of-00037.safetensors |
|
8%|▊ | 44/563 [01:40<06:45, 1.28it/s]
[2024-06-07 00:14:33] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.12.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
8%|▊ | 44/563 [01:47<06:45, 1.28it/s]
8%|▊ | 46/563 [01:47<15:15, 1.77s/it]
[2024-06-07 00:14:35] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.12.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
8%|▊ | 46/563 [01:49<15:15, 1.77s/it]
[2024-06-07 00:14:35] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.12.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
8%|▊ | 46/563 [01:49<15:15, 1.77s/it]
8%|▊ | 47/563 [01:49<15:33, 1.81s/it]
[2024-06-07 00:14:41] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.12.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
8%|▊ | 47/563 [01:55<15:33, 1.81s/it]
[2024-06-07 00:14:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.12.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
8%|▊ | 47/563 [01:56<15:33, 1.81s/it]
9%|▊ | 48/563 [01:56<25:13, 2.94s/it]
[2024-06-07 00:14:42] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.12.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
9%|▊ | 48/563 [01:56<25:13, 2.94s/it]
[2024-06-07 00:14:42] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.12.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
9%|▊ | 48/563 [01:56<25:13, 2.94s/it]
[2024-06-07 00:14:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.12.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
9%|▊ | 48/563 [01:56<25:13, 2.94s/it]
[2024-06-07 00:14:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.12.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
9%|▊ | 48/563 [01:57<25:13, 2.94s/it]
9%|▉ | 51/563 [01:57<13:57, 1.64s/it]
[2024-06-07 00:14:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.12.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
9%|▉ | 51/563 [01:57<13:57, 1.64s/it]
[2024-06-07 00:14:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.12.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
9%|▉ | 51/563 [01:57<13:57, 1.64s/it]
9%|▉ | 52/563 [01:57<11:51, 1.39s/it]
[2024-06-07 00:14:43] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.13.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
9%|▉ | 52/563 [01:57<11:51, 1.39s/it]
[2024-06-07 00:14:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.13.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
9%|▉ | 52/563 [01:58<11:51, 1.39s/it]
[2024-06-07 00:14:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.13.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
9%|▉ | 52/563 [01:58<11:51, 1.39s/it]
10%|▉ | 54/563 [01:58<09:32, 1.12s/it]
[2024-06-07 00:14:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.13.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
10%|▉ | 54/563 [02:03<09:32, 1.12s/it]
[2024-06-07 00:14:50] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.13.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
10%|▉ | 54/563 [02:03<09:32, 1.12s/it]
10%|▉ | 55/563 [02:04<16:33, 1.96s/it]
[2024-06-07 00:14:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.13.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
10%|▉ | 55/563 [02:04<16:33, 1.96s/it]
[2024-06-07 00:14:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.13.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
10%|▉ | 55/563 [02:04<16:33, 1.96s/it]
[2024-06-07 00:14:50] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.13.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
10%|▉ | 55/563 [02:04<16:33, 1.96s/it]
[2024-06-07 00:14:50] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.13.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
10%|▉ | 55/563 [02:04<16:33, 1.96s/it]
10%|█ | 58/563 [02:04<09:29, 1.13s/it]
[2024-06-07 00:14:50] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.13.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
10%|█ | 58/563 [02:04<09:29, 1.13s/it]
[2024-06-07 00:14:50] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.13.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
10%|█ | 58/563 [02:04<09:29, 1.13s/it]
10%|█ | 59/563 [02:04<08:22, 1.00it/s]
[2024-06-07 00:14:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.14.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
10%|█ | 59/563 [02:04<08:22, 1.00it/s]
[2024-06-07 00:14:51] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.14.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
10%|█ | 59/563 [02:05<08:22, 1.00it/s]
[2024-06-07 00:14:51] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.14.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
10%|█ | 59/563 [02:05<08:22, 1.00it/s]
11%|█ | 61/563 [02:05<06:09, 1.36it/s]
[2024-06-07 00:14:51] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.14.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
11%|█ | 61/563 [02:05<06:09, 1.36it/s]
[2024-06-07 00:14:51] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.14.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
11%|█ | 61/563 [02:05<06:09, 1.36it/s]
11%|█ | 62/563 [02:05<05:29, 1.52it/s]
[2024-06-07 00:14:51] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00007-of-00037.safetensors |
|
11%|█ | 62/563 [02:05<05:29, 1.52it/s]
[2024-06-07 00:14:52] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00008-of-00037.safetensors |
|
11%|█ | 62/563 [02:06<05:29, 1.52it/s]
[2024-06-07 00:14:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.14.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
11%|█ | 62/563 [02:13<05:29, 1.52it/s]
11%|█ | 63/563 [02:13<18:23, 2.21s/it]
[2024-06-07 00:15:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.14.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
11%|█ | 63/563 [02:14<18:23, 2.21s/it]
[2024-06-07 00:15:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.14.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
11%|█ | 63/563 [02:15<18:23, 2.21s/it]
11%|█▏ | 64/563 [02:15<17:39, 2.12s/it]
[2024-06-07 00:15:07] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.14.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
11%|█▏ | 64/563 [02:21<17:39, 2.12s/it]
[2024-06-07 00:15:07] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.14.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
11%|█▏ | 64/563 [02:21<17:39, 2.12s/it]
12%|█▏ | 65/563 [02:21<27:03, 3.26s/it]
[2024-06-07 00:15:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.14.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
12%|█▏ | 65/563 [02:21<27:03, 3.26s/it]
[2024-06-07 00:15:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.15.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
12%|█▏ | 65/563 [02:21<27:03, 3.26s/it]
[2024-06-07 00:15:09] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.15.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
12%|█▏ | 65/563 [02:23<27:03, 3.26s/it]
[2024-06-07 00:15:09] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.15.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
12%|█▏ | 65/563 [02:23<27:03, 3.26s/it]
12%|█▏ | 68/563 [02:23<15:03, 1.82s/it]
[2024-06-07 00:15:13] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.15.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
12%|█▏ | 68/563 [02:27<15:03, 1.82s/it]
[2024-06-07 00:15:14] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.15.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
12%|█▏ | 68/563 [02:28<15:03, 1.82s/it]
12%|█▏ | 69/563 [02:28<20:22, 2.48s/it]
[2024-06-07 00:15:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.15.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
12%|█▏ | 69/563 [02:28<20:22, 2.48s/it]
[2024-06-07 00:15:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.15.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
12%|█▏ | 69/563 [02:28<20:22, 2.48s/it]
[2024-06-07 00:15:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.15.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
12%|█▏ | 69/563 [02:28<20:22, 2.48s/it]
[2024-06-07 00:15:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.15.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
12%|█▏ | 69/563 [02:28<20:22, 2.48s/it]
13%|█▎ | 72/563 [02:28<11:30, 1.41s/it]
[2024-06-07 00:15:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.15.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
13%|█▎ | 72/563 [02:29<11:30, 1.41s/it]
[2024-06-07 00:15:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.15.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
13%|█▎ | 72/563 [02:29<11:30, 1.41s/it]
13%|█▎ | 73/563 [02:29<09:54, 1.21s/it]
[2024-06-07 00:15:15] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00009-of-00037.safetensors |
|
13%|█▎ | 73/563 [02:29<09:54, 1.21s/it]
[2024-06-07 00:15:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.16.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
13%|█▎ | 73/563 [02:43<09:54, 1.21s/it]
[2024-06-07 00:15:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.16.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
13%|█▎ | 73/563 [02:43<09:54, 1.21s/it]
13%|█▎ | 74/563 [02:43<32:04, 3.94s/it]
[2024-06-07 00:15:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.16.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
13%|█▎ | 74/563 [02:43<32:04, 3.94s/it]
[2024-06-07 00:15:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.16.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
13%|█▎ | 74/563 [02:44<32:04, 3.94s/it]
[2024-06-07 00:15:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.16.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
13%|█▎ | 74/563 [02:44<32:04, 3.94s/it]
13%|█▎ | 76/563 [02:44<21:01, 2.59s/it]
[2024-06-07 00:15:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.16.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
13%|█▎ | 76/563 [02:44<21:01, 2.59s/it]
[2024-06-07 00:15:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.16.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
13%|█▎ | 76/563 [02:44<21:01, 2.59s/it]
14%|█▎ | 77/563 [02:44<17:15, 2.13s/it]
[2024-06-07 00:15:30] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.16.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
14%|█▎ | 77/563 [02:44<17:15, 2.13s/it]
[2024-06-07 00:15:31] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.16.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
14%|█▎ | 77/563 [02:45<17:15, 2.13s/it]
[2024-06-07 00:15:32] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.16.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
14%|█▎ | 77/563 [02:45<17:15, 2.13s/it]
14%|█▍ | 79/563 [02:46<12:36, 1.56s/it]
[2024-06-07 00:15:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.16.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
14%|█▍ | 79/563 [02:46<12:36, 1.56s/it]
[2024-06-07 00:15:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.17.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
14%|█▍ | 79/563 [02:46<12:36, 1.56s/it]
[2024-06-07 00:15:33] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.17.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
14%|█▍ | 79/563 [02:46<12:36, 1.56s/it]
[2024-06-07 00:15:33] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.17.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
14%|█▍ | 79/563 [02:47<12:36, 1.56s/it]
15%|█▍ | 82/563 [02:47<08:26, 1.05s/it]
[2024-06-07 00:15:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.17.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
15%|█▍ | 82/563 [02:51<08:26, 1.05s/it]
[2024-06-07 00:15:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.17.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
15%|█▍ | 82/563 [02:51<08:26, 1.05s/it]
15%|█▍ | 83/563 [02:51<13:23, 1.67s/it]
[2024-06-07 00:15:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.17.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
15%|█▍ | 83/563 [02:51<13:23, 1.67s/it]
[2024-06-07 00:15:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.17.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
15%|█▍ | 83/563 [02:51<13:23, 1.67s/it]
[2024-06-07 00:15:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.17.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
15%|█▍ | 83/563 [02:52<13:23, 1.67s/it]
[2024-06-07 00:15:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.17.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
15%|█▍ | 83/563 [02:52<13:23, 1.67s/it]
15%|█▌ | 86/563 [02:52<08:08, 1.02s/it]
[2024-06-07 00:15:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.17.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
15%|█▌ | 86/563 [02:52<08:08, 1.02s/it]
[2024-06-07 00:15:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.17.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
15%|█▌ | 86/563 [02:52<08:08, 1.02s/it]
15%|█▌ | 87/563 [02:52<07:11, 1.10it/s]
[2024-06-07 00:15:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.18.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
15%|█▌ | 87/563 [02:56<07:11, 1.10it/s]
[2024-06-07 00:15:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.18.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
15%|█▌ | 87/563 [02:57<07:11, 1.10it/s]
16%|█▌ | 88/563 [02:57<13:11, 1.67s/it]
[2024-06-07 00:15:43] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.18.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
16%|█▌ | 88/563 [02:57<13:11, 1.67s/it]
[2024-06-07 00:15:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.18.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
16%|█▌ | 88/563 [02:57<13:11, 1.67s/it]
[2024-06-07 00:15:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.18.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
16%|█▌ | 88/563 [02:57<13:11, 1.67s/it]
16%|█▌ | 90/563 [02:57<09:05, 1.15s/it]
[2024-06-07 00:15:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.18.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
16%|█▌ | 90/563 [02:58<09:05, 1.15s/it]
[2024-06-07 00:15:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.18.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
16%|█▌ | 90/563 [02:58<09:05, 1.15s/it]
16%|█▌ | 91/563 [02:58<07:51, 1.00it/s]
[2024-06-07 00:15:44] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00008-of-00037.safetensors |
|
16%|█▌ | 91/563 [02:58<07:51, 1.00it/s]
[2024-06-07 00:15:44] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00009-of-00037.safetensors |
|
16%|█▌ | 91/563 [02:58<07:51, 1.00it/s]
[2024-06-07 00:15:45] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00010-of-00037.safetensors |
|
16%|█▌ | 91/563 [02:59<07:51, 1.00it/s]
[2024-06-07 00:15:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.18.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
16%|█▌ | 91/563 [03:07<07:51, 1.00it/s]
16%|█▋ | 92/563 [03:07<22:24, 2.86s/it]
[2024-06-07 00:15:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.18.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
16%|█▋ | 92/563 [03:08<22:24, 2.86s/it]
[2024-06-07 00:15:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.18.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
16%|█▋ | 92/563 [03:08<22:24, 2.86s/it]
17%|█▋ | 93/563 [03:08<19:12, 2.45s/it]
[2024-06-07 00:15:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.18.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
17%|█▋ | 93/563 [03:08<19:12, 2.45s/it]
[2024-06-07 00:15:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.19.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
17%|█▋ | 93/563 [03:08<19:12, 2.45s/it]
[2024-06-07 00:15:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.19.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
17%|█▋ | 93/563 [03:09<19:12, 2.45s/it]
[2024-06-07 00:15:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.19.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
17%|█▋ | 93/563 [03:09<19:12, 2.45s/it]
17%|█▋ | 96/563 [03:10<10:58, 1.41s/it]
[2024-06-07 00:15:59] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.19.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
17%|█▋ | 96/563 [03:13<10:58, 1.41s/it]
[2024-06-07 00:16:00] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.19.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
17%|█▋ | 96/563 [03:14<10:58, 1.41s/it]
17%|█▋ | 97/563 [03:14<15:23, 1.98s/it]
[2024-06-07 00:16:00] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.19.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
17%|█▋ | 97/563 [03:14<15:23, 1.98s/it]
[2024-06-07 00:16:00] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.19.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
17%|█▋ | 97/563 [03:14<15:23, 1.98s/it]
[2024-06-07 00:16:00] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.19.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
17%|█▋ | 97/563 [03:14<15:23, 1.98s/it]
[2024-06-07 00:16:00] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.19.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
17%|█▋ | 97/563 [03:14<15:23, 1.98s/it]
18%|█▊ | 100/563 [03:14<08:52, 1.15s/it]
[2024-06-07 00:16:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.19.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
18%|█▊ | 100/563 [03:15<08:52, 1.15s/it]
[2024-06-07 00:16:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.19.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
18%|█▊ | 100/563 [03:15<08:52, 1.15s/it]
18%|█▊ | 101/563 [03:15<07:44, 1.01s/it]
[2024-06-07 00:16:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.20.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
18%|█▊ | 101/563 [03:15<07:44, 1.01s/it]
[2024-06-07 00:16:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.20.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
18%|█▊ | 101/563 [03:16<07:44, 1.01s/it]
[2024-06-07 00:16:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.20.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
18%|█▊ | 101/563 [03:16<07:44, 1.01s/it]
18%|█▊ | 103/563 [03:16<06:43, 1.14it/s]
[2024-06-07 00:16:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.20.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
18%|█▊ | 103/563 [03:18<06:43, 1.14it/s]
[2024-06-07 00:16:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.20.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
18%|█▊ | 103/563 [03:19<06:43, 1.14it/s]
18%|█▊ | 104/563 [03:19<09:38, 1.26s/it]
[2024-06-07 00:16:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.20.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
18%|█▊ | 104/563 [03:19<09:38, 1.26s/it]
[2024-06-07 00:16:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.20.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
18%|█▊ | 104/563 [03:19<09:38, 1.26s/it]
[2024-06-07 00:16:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.20.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
18%|█▊ | 104/563 [03:19<09:38, 1.26s/it]
[2024-06-07 00:16:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.20.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
18%|█▊ | 104/563 [03:19<09:38, 1.26s/it]
19%|█▉ | 107/563 [03:19<05:45, 1.32it/s]
[2024-06-07 00:16:06] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.20.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
19%|█▉ | 107/563 [03:19<05:45, 1.32it/s]
[2024-06-07 00:16:06] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.20.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
19%|█▉ | 107/563 [03:20<05:45, 1.32it/s]
19%|█▉ | 108/563 [03:20<05:09, 1.47it/s]
[2024-06-07 00:16:06] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00010-of-00037.safetensors |
|
19%|█▉ | 108/563 [03:20<05:09, 1.47it/s]
[2024-06-07 00:16:06] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00011-of-00037.safetensors |
|
19%|█▉ | 108/563 [03:20<05:09, 1.47it/s]
[2024-06-07 00:16:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.21.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
19%|█▉ | 108/563 [03:28<05:09, 1.47it/s]
19%|█▉ | 109/563 [03:28<17:48, 2.35s/it]
[2024-06-07 00:16:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.21.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
19%|█▉ | 109/563 [03:29<17:48, 2.35s/it]
[2024-06-07 00:16:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.21.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
19%|█▉ | 109/563 [03:30<17:48, 2.35s/it]
20%|█▉ | 110/563 [03:30<15:53, 2.10s/it]
[2024-06-07 00:16:21] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.21.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
20%|█▉ | 110/563 [03:35<15:53, 2.10s/it]
[2024-06-07 00:16:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.21.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
20%|█▉ | 110/563 [03:35<15:53, 2.10s/it]
20%|█▉ | 111/563 [03:36<22:40, 3.01s/it]
[2024-06-07 00:16:22] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.21.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
20%|█▉ | 111/563 [03:36<22:40, 3.01s/it]
[2024-06-07 00:16:22] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.21.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
20%|█▉ | 111/563 [03:36<22:40, 3.01s/it]
[2024-06-07 00:16:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.21.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
20%|█▉ | 111/563 [03:36<22:40, 3.01s/it]
[2024-06-07 00:16:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.21.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
20%|█▉ | 111/563 [03:36<22:40, 3.01s/it]
20%|██ | 114/563 [03:36<11:36, 1.55s/it]
[2024-06-07 00:16:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.21.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
20%|██ | 114/563 [03:36<11:36, 1.55s/it]
[2024-06-07 00:16:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.21.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
20%|██ | 114/563 [03:36<11:36, 1.55s/it]
20%|██ | 115/563 [03:36<09:47, 1.31s/it]
[2024-06-07 00:16:22] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.22.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
20%|██ | 115/563 [03:36<09:47, 1.31s/it]
[2024-06-07 00:16:23] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.22.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
20%|██ | 115/563 [03:37<09:47, 1.31s/it]
[2024-06-07 00:16:24] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.22.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
20%|██ | 115/563 [03:38<09:47, 1.31s/it]
21%|██ | 117/563 [03:38<07:55, 1.07s/it]
[2024-06-07 00:16:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.22.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
21%|██ | 117/563 [03:42<07:55, 1.07s/it]
[2024-06-07 00:16:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.22.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
21%|██ | 117/563 [03:43<07:55, 1.07s/it]
21%|██ | 118/563 [03:43<14:33, 1.96s/it]
[2024-06-07 00:16:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.22.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
21%|██ | 118/563 [03:43<14:33, 1.96s/it]
[2024-06-07 00:16:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.22.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
21%|██ | 118/563 [03:43<14:33, 1.96s/it]
[2024-06-07 00:16:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.22.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
21%|██ | 118/563 [03:43<14:33, 1.96s/it]
[2024-06-07 00:16:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.22.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
21%|██ | 118/563 [03:44<14:33, 1.96s/it]
21%|██▏ | 121/563 [03:44<08:14, 1.12s/it]
[2024-06-07 00:16:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.22.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
21%|██▏ | 121/563 [03:44<08:14, 1.12s/it]
[2024-06-07 00:16:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.22.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
21%|██▏ | 121/563 [03:44<08:14, 1.12s/it]
22%|██▏ | 122/563 [03:44<07:10, 1.03it/s]
[2024-06-07 00:16:30] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.23.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
22%|██▏ | 122/563 [03:44<07:10, 1.03it/s]
[2024-06-07 00:16:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.23.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
22%|██▏ | 122/563 [03:44<07:10, 1.03it/s]
[2024-06-07 00:16:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.23.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
22%|██▏ | 122/563 [03:44<07:10, 1.03it/s]
22%|██▏ | 124/563 [03:44<05:15, 1.39it/s]
[2024-06-07 00:16:31] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.23.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
22%|██▏ | 124/563 [03:45<05:15, 1.39it/s]
[2024-06-07 00:16:31] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.23.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
22%|██▏ | 124/563 [03:45<05:15, 1.39it/s]
22%|██▏ | 125/563 [03:45<04:45, 1.53it/s]
[2024-06-07 00:16:31] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00011-of-00037.safetensors |
|
22%|██▏ | 125/563 [03:45<04:45, 1.53it/s]
[2024-06-07 00:16:31] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00012-of-00037.safetensors |
|
22%|██▏ | 125/563 [03:45<04:45, 1.53it/s]
[2024-06-07 00:16:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.23.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
22%|██▏ | 125/563 [03:54<04:45, 1.53it/s]
22%|██▏ | 126/563 [03:54<19:11, 2.63s/it]
[2024-06-07 00:16:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.23.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
22%|██▏ | 126/563 [03:56<19:11, 2.63s/it]
[2024-06-07 00:16:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.23.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
22%|██▏ | 126/563 [03:56<19:11, 2.63s/it]
23%|██▎ | 127/563 [03:56<17:36, 2.42s/it]
[2024-06-07 00:16:47] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.23.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
23%|██▎ | 127/563 [04:01<17:36, 2.42s/it]
[2024-06-07 00:16:48] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.23.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
23%|██▎ | 127/563 [04:02<17:36, 2.42s/it]
23%|██▎ | 128/563 [04:02<23:54, 3.30s/it]
[2024-06-07 00:16:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.23.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
23%|██▎ | 128/563 [04:02<23:54, 3.30s/it]
[2024-06-07 00:16:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.24.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
23%|██▎ | 128/563 [04:02<23:54, 3.30s/it]
[2024-06-07 00:16:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.24.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
23%|██▎ | 128/563 [04:03<23:54, 3.30s/it]
[2024-06-07 00:16:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.24.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
23%|██▎ | 128/563 [04:03<23:54, 3.30s/it]
23%|██▎ | 131/563 [04:03<12:57, 1.80s/it]
[2024-06-07 00:16:53] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.24.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
23%|██▎ | 131/563 [04:07<12:57, 1.80s/it]
[2024-06-07 00:16:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.24.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
23%|██▎ | 131/563 [04:08<12:57, 1.80s/it]
23%|██▎ | 132/563 [04:08<16:46, 2.33s/it]
[2024-06-07 00:16:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.24.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
23%|██▎ | 132/563 [04:08<16:46, 2.33s/it]
[2024-06-07 00:16:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.24.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
23%|██▎ | 132/563 [04:08<16:46, 2.33s/it]
[2024-06-07 00:16:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.24.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
23%|██▎ | 132/563 [04:08<16:46, 2.33s/it]
[2024-06-07 00:16:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.24.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
23%|██▎ | 132/563 [04:08<16:46, 2.33s/it]
24%|██▍ | 135/563 [04:08<09:30, 1.33s/it]
[2024-06-07 00:16:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.24.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
24%|██▍ | 135/563 [04:08<09:30, 1.33s/it]
[2024-06-07 00:16:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.24.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
24%|██▍ | 135/563 [04:09<09:30, 1.33s/it]
24%|██▍ | 136/563 [04:09<08:12, 1.15s/it]
[2024-06-07 00:16:55] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00013-of-00037.safetensors |
|
24%|██▍ | 136/563 [04:09<08:12, 1.15s/it]
[2024-06-07 00:17:08] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.25.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
24%|██▍ | 136/563 [04:22<08:12, 1.15s/it]
[2024-06-07 00:17:09] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.25.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
24%|██▍ | 136/563 [04:23<08:12, 1.15s/it]
24%|██▍ | 137/563 [04:23<26:55, 3.79s/it]
[2024-06-07 00:17:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.25.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
24%|██▍ | 137/563 [04:23<26:55, 3.79s/it]
[2024-06-07 00:17:09] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.25.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
24%|██▍ | 137/563 [04:23<26:55, 3.79s/it]
[2024-06-07 00:17:09] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.25.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
24%|██▍ | 137/563 [04:23<26:55, 3.79s/it]
25%|██▍ | 139/563 [04:23<17:37, 2.49s/it]
[2024-06-07 00:17:10] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.25.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
25%|██▍ | 139/563 [04:23<17:37, 2.49s/it]
[2024-06-07 00:17:10] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.25.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
25%|██▍ | 139/563 [04:23<17:37, 2.49s/it]
25%|██▍ | 140/563 [04:24<14:28, 2.05s/it]
[2024-06-07 00:17:10] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.25.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
25%|██▍ | 140/563 [04:24<14:28, 2.05s/it]
[2024-06-07 00:17:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.25.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
25%|██▍ | 140/563 [04:24<14:28, 2.05s/it]
[2024-06-07 00:17:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.25.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
25%|██▍ | 140/563 [04:25<14:28, 2.05s/it]
25%|██▌ | 142/563 [04:25<10:38, 1.52s/it]
[2024-06-07 00:17:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.25.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
25%|██▌ | 142/563 [04:25<10:38, 1.52s/it]
[2024-06-07 00:17:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.26.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
25%|██▌ | 142/563 [04:25<10:38, 1.52s/it]
[2024-06-07 00:17:12] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.26.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
25%|██▌ | 142/563 [04:26<10:38, 1.52s/it]
[2024-06-07 00:17:12] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.26.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
25%|██▌ | 142/563 [04:26<10:38, 1.52s/it]
26%|██▌ | 145/563 [04:26<07:11, 1.03s/it]
[2024-06-07 00:17:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.26.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
26%|██▌ | 145/563 [04:30<07:11, 1.03s/it]
[2024-06-07 00:17:17] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.26.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
26%|██▌ | 145/563 [04:31<07:11, 1.03s/it]
26%|██▌ | 146/563 [04:31<11:48, 1.70s/it]
[2024-06-07 00:17:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.26.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
26%|██▌ | 146/563 [04:31<11:48, 1.70s/it]
[2024-06-07 00:17:17] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.26.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
26%|██▌ | 146/563 [04:31<11:48, 1.70s/it]
[2024-06-07 00:17:17] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.26.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
26%|██▌ | 146/563 [04:31<11:48, 1.70s/it]
[2024-06-07 00:17:17] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.26.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
26%|██▌ | 146/563 [04:31<11:48, 1.70s/it]
26%|██▋ | 149/563 [04:31<07:08, 1.04s/it]
[2024-06-07 00:17:18] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.26.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
26%|██▋ | 149/563 [04:32<07:08, 1.04s/it]
[2024-06-07 00:17:18] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.26.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
26%|██▋ | 149/563 [04:32<07:08, 1.04s/it]
27%|██▋ | 150/563 [04:32<06:17, 1.09it/s]
[2024-06-07 00:17:21] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.27.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
27%|██▋ | 150/563 [04:35<06:17, 1.09it/s]
[2024-06-07 00:17:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.27.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
27%|██▋ | 150/563 [04:36<06:17, 1.09it/s]
27%|██▋ | 151/563 [04:36<10:24, 1.52s/it]
[2024-06-07 00:17:22] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.27.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
27%|██▋ | 151/563 [04:36<10:24, 1.52s/it]
[2024-06-07 00:17:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.27.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
27%|██▋ | 151/563 [04:36<10:24, 1.52s/it]
[2024-06-07 00:17:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.27.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
27%|██▋ | 151/563 [04:36<10:24, 1.52s/it]
27%|██▋ | 153/563 [04:36<07:12, 1.05s/it]
[2024-06-07 00:17:23] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.27.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
27%|██▋ | 153/563 [04:36<07:12, 1.05s/it]
[2024-06-07 00:17:23] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.27.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
27%|██▋ | 153/563 [04:36<07:12, 1.05s/it]
27%|██▋ | 154/563 [04:37<06:15, 1.09it/s]
[2024-06-07 00:17:23] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00013-of-00037.safetensors |
|
27%|██▋ | 154/563 [04:37<06:15, 1.09it/s]
[2024-06-07 00:17:23] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00012-of-00037.safetensors |
|
27%|██▋ | 154/563 [04:37<06:15, 1.09it/s]
[2024-06-07 00:17:24] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00014-of-00037.safetensors |
|
27%|██▋ | 154/563 [04:37<06:15, 1.09it/s]
[2024-06-07 00:17:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.27.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
27%|██▋ | 154/563 [04:46<06:15, 1.09it/s]
28%|██▊ | 155/563 [04:46<18:36, 2.74s/it]
[2024-06-07 00:17:33] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.27.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
28%|██▊ | 155/563 [04:46<18:36, 2.74s/it]
[2024-06-07 00:17:33] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.27.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
28%|██▊ | 155/563 [04:47<18:36, 2.74s/it]
28%|██▊ | 156/563 [04:47<16:02, 2.37s/it]
[2024-06-07 00:17:33] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.27.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
28%|██▊ | 156/563 [04:47<16:02, 2.37s/it]
[2024-06-07 00:17:33] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.28.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
28%|██▊ | 156/563 [04:47<16:02, 2.37s/it]
[2024-06-07 00:17:34] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.28.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
28%|██▊ | 156/563 [04:48<16:02, 2.37s/it]
[2024-06-07 00:17:34] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.28.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
28%|██▊ | 156/563 [04:48<16:02, 2.37s/it]
28%|██▊ | 159/563 [04:48<09:12, 1.37s/it]
[2024-06-07 00:17:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.28.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
28%|██▊ | 159/563 [04:51<09:12, 1.37s/it]
[2024-06-07 00:17:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.28.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
28%|██▊ | 159/563 [04:51<09:12, 1.37s/it]
28%|██▊ | 160/563 [04:51<11:56, 1.78s/it]
[2024-06-07 00:17:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.28.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
28%|██▊ | 160/563 [04:51<11:56, 1.78s/it]
[2024-06-07 00:17:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.28.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
28%|██▊ | 160/563 [04:51<11:56, 1.78s/it]
[2024-06-07 00:17:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.28.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
28%|██▊ | 160/563 [04:52<11:56, 1.78s/it]
[2024-06-07 00:17:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.28.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
28%|██▊ | 160/563 [04:52<11:56, 1.78s/it]
29%|██▉ | 163/563 [04:52<06:54, 1.04s/it]
[2024-06-07 00:17:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.28.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
29%|██▉ | 163/563 [04:52<06:54, 1.04s/it]
[2024-06-07 00:17:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.28.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
29%|██▉ | 163/563 [04:52<06:54, 1.04s/it]
29%|██▉ | 164/563 [04:52<06:02, 1.10it/s]
[2024-06-07 00:17:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.29.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
29%|██▉ | 164/563 [04:52<06:02, 1.10it/s]
[2024-06-07 00:17:39] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.29.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
29%|██▉ | 164/563 [04:53<06:02, 1.10it/s]
[2024-06-07 00:17:40] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.29.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
29%|██▉ | 164/563 [04:53<06:02, 1.10it/s]
29%|██▉ | 166/563 [04:53<05:22, 1.23it/s]
[2024-06-07 00:17:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.29.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
29%|██▉ | 166/563 [04:55<05:22, 1.23it/s]
[2024-06-07 00:17:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.29.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
29%|██▉ | 166/563 [04:56<05:22, 1.23it/s]
30%|██▉ | 167/563 [04:56<07:41, 1.17s/it]
[2024-06-07 00:17:42] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.29.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
30%|██▉ | 167/563 [04:56<07:41, 1.17s/it]
[2024-06-07 00:17:42] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.29.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
30%|██▉ | 167/563 [04:56<07:41, 1.17s/it]
[2024-06-07 00:17:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.29.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
30%|██▉ | 167/563 [04:56<07:41, 1.17s/it]
[2024-06-07 00:17:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.29.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
30%|██▉ | 167/563 [04:57<07:41, 1.17s/it]
30%|███ | 170/563 [04:57<04:36, 1.42it/s]
[2024-06-07 00:17:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.29.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
30%|███ | 170/563 [04:57<04:36, 1.42it/s]
[2024-06-07 00:17:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.29.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
30%|███ | 170/563 [04:57<04:36, 1.42it/s]
30%|███ | 171/563 [04:57<04:08, 1.58it/s]
[2024-06-07 00:17:43] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00014-of-00037.safetensors |
|
30%|███ | 171/563 [04:57<04:08, 1.58it/s]
[2024-06-07 00:17:43] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00003-of-00037.safetensors |
|
30%|███ | 171/563 [04:57<04:08, 1.58it/s]
[2024-06-07 00:17:52] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.3.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
30%|███ | 171/563 [05:06<04:08, 1.58it/s]
31%|███ | 172/563 [05:06<15:38, 2.40s/it]
[2024-06-07 00:17:53] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.3.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
31%|███ | 172/563 [05:07<15:38, 2.40s/it]
[2024-06-07 00:17:53] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.3.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
31%|███ | 172/563 [05:07<15:38, 2.40s/it]
31%|███ | 173/563 [05:07<13:52, 2.14s/it]
[2024-06-07 00:17:57] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.3.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
31%|███ | 173/563 [05:10<13:52, 2.14s/it]
[2024-06-07 00:17:57] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.3.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
31%|███ | 173/563 [05:11<13:52, 2.14s/it]
31%|███ | 174/563 [05:11<16:26, 2.54s/it]
[2024-06-07 00:17:57] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.3.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
31%|███ | 174/563 [05:11<16:26, 2.54s/it]
[2024-06-07 00:17:57] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.3.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
31%|███ | 174/563 [05:11<16:26, 2.54s/it]
[2024-06-07 00:17:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.3.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
31%|███ | 174/563 [05:12<16:26, 2.54s/it]
[2024-06-07 00:17:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.3.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
31%|███ | 174/563 [05:12<16:26, 2.54s/it]
31%|███▏ | 177/563 [05:12<08:31, 1.32s/it]
[2024-06-07 00:17:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.3.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
31%|███▏ | 177/563 [05:12<08:31, 1.32s/it]
[2024-06-07 00:17:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.3.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
31%|███▏ | 177/563 [05:12<08:31, 1.32s/it]
32%|███▏ | 178/563 [05:12<07:14, 1.13s/it]
[2024-06-07 00:17:58] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.4.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
32%|███▏ | 178/563 [05:12<07:14, 1.13s/it]
[2024-06-07 00:17:59] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.4.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
32%|███▏ | 178/563 [05:13<07:14, 1.13s/it]
[2024-06-07 00:17:59] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.4.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
32%|███▏ | 178/563 [05:13<07:14, 1.13s/it]
32%|███▏ | 180/563 [05:13<06:01, 1.06it/s]
[2024-06-07 00:18:03] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.4.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
32%|███▏ | 180/563 [05:17<06:01, 1.06it/s]
[2024-06-07 00:18:03] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.4.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
32%|███▏ | 180/563 [05:17<06:01, 1.06it/s]
32%|███▏ | 181/563 [05:17<10:00, 1.57s/it]
[2024-06-07 00:18:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.4.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
32%|███▏ | 181/563 [05:17<10:00, 1.57s/it]
[2024-06-07 00:18:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.4.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
32%|███▏ | 181/563 [05:17<10:00, 1.57s/it]
[2024-06-07 00:18:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.4.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
32%|███▏ | 181/563 [05:18<10:00, 1.57s/it]
[2024-06-07 00:18:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.4.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
32%|███▏ | 181/563 [05:18<10:00, 1.57s/it]
33%|███▎ | 184/563 [05:18<05:44, 1.10it/s]
[2024-06-07 00:18:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.4.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
33%|███▎ | 184/563 [05:18<05:44, 1.10it/s]
[2024-06-07 00:18:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.4.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
33%|███▎ | 184/563 [05:18<05:44, 1.10it/s]
33%|███▎ | 185/563 [05:18<05:03, 1.24it/s]
[2024-06-07 00:18:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.5.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
33%|███▎ | 185/563 [05:18<05:03, 1.24it/s]
[2024-06-07 00:18:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.5.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
33%|███▎ | 185/563 [05:18<05:03, 1.24it/s]
[2024-06-07 00:18:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.5.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
33%|███▎ | 185/563 [05:19<05:03, 1.24it/s]
33%|███▎ | 187/563 [05:19<03:49, 1.63it/s]
[2024-06-07 00:18:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.5.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
33%|███▎ | 187/563 [05:19<03:49, 1.63it/s]
[2024-06-07 00:18:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.5.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
33%|███▎ | 187/563 [05:19<03:49, 1.63it/s]
33%|███▎ | 188/563 [05:19<03:32, 1.77it/s]
[2024-06-07 00:18:05] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00003-of-00037.safetensors |
|
33%|███▎ | 188/563 [05:19<03:32, 1.77it/s]
[2024-06-07 00:18:05] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00015-of-00037.safetensors |
|
33%|███▎ | 188/563 [05:19<03:32, 1.77it/s]
[2024-06-07 00:18:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.30.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
33%|███▎ | 188/563 [05:28<03:32, 1.77it/s]
34%|███▎ | 189/563 [05:28<14:47, 2.37s/it]
[2024-06-07 00:18:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.30.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
34%|███▎ | 189/563 [05:29<14:47, 2.37s/it]
[2024-06-07 00:18:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.30.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
34%|███▎ | 189/563 [05:30<14:47, 2.37s/it]
34%|███▎ | 190/563 [05:30<13:56, 2.24s/it]
[2024-06-07 00:18:21] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.30.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
34%|███▎ | 190/563 [05:35<13:56, 2.24s/it]
[2024-06-07 00:18:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.30.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
34%|███▎ | 190/563 [05:36<13:56, 2.24s/it]
34%|███▍ | 191/563 [05:36<20:20, 3.28s/it]
[2024-06-07 00:18:22] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.30.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
34%|███▍ | 191/563 [05:36<20:20, 3.28s/it]
[2024-06-07 00:18:22] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.30.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
34%|███▍ | 191/563 [05:36<20:20, 3.28s/it]
[2024-06-07 00:18:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.30.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
34%|███▍ | 191/563 [05:36<20:20, 3.28s/it]
[2024-06-07 00:18:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.30.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
34%|███▍ | 191/563 [05:36<20:20, 3.28s/it]
34%|███▍ | 194/563 [05:36<10:08, 1.65s/it]
[2024-06-07 00:18:23] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.30.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
34%|███▍ | 194/563 [05:37<10:08, 1.65s/it]
[2024-06-07 00:18:23] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.30.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
34%|███▍ | 194/563 [05:37<10:08, 1.65s/it]
35%|███▍ | 195/563 [05:37<08:29, 1.39s/it]
[2024-06-07 00:18:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.31.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
35%|███▍ | 195/563 [05:37<08:29, 1.39s/it]
[2024-06-07 00:18:24] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.31.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
35%|███▍ | 195/563 [05:38<08:29, 1.39s/it]
[2024-06-07 00:18:24] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.31.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
35%|███▍ | 195/563 [05:38<08:29, 1.39s/it]
35%|███▍ | 197/563 [05:38<06:47, 1.11s/it]
[2024-06-07 00:18:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.31.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
35%|███▍ | 197/563 [05:42<06:47, 1.11s/it]
[2024-06-07 00:18:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.31.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
35%|███▍ | 197/563 [05:43<06:47, 1.11s/it]
35%|███▌ | 198/563 [05:43<11:14, 1.85s/it]
[2024-06-07 00:18:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.31.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
35%|███▌ | 198/563 [05:43<11:14, 1.85s/it]
[2024-06-07 00:18:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.31.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
35%|███▌ | 198/563 [05:43<11:14, 1.85s/it]
[2024-06-07 00:18:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.31.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
35%|███▌ | 198/563 [05:43<11:14, 1.85s/it]
[2024-06-07 00:18:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.31.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
35%|███▌ | 198/563 [05:43<11:14, 1.85s/it]
36%|███▌ | 201/563 [05:43<06:21, 1.05s/it]
[2024-06-07 00:18:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.31.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
36%|███▌ | 201/563 [05:43<06:21, 1.05s/it]
[2024-06-07 00:18:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.31.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
36%|███▌ | 201/563 [05:43<06:21, 1.05s/it]
36%|███▌ | 202/563 [05:43<05:32, 1.08it/s]
[2024-06-07 00:18:30] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.32.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
36%|███▌ | 202/563 [05:43<05:32, 1.08it/s]
[2024-06-07 00:18:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.32.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
36%|███▌ | 202/563 [05:44<05:32, 1.08it/s]
[2024-06-07 00:18:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.32.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
36%|███▌ | 202/563 [05:44<05:32, 1.08it/s]
36%|███▌ | 204/563 [05:44<04:05, 1.46it/s]
[2024-06-07 00:18:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.32.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
36%|███▌ | 204/563 [05:44<04:05, 1.46it/s]
[2024-06-07 00:18:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.32.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
36%|███▌ | 204/563 [05:44<04:05, 1.46it/s]
36%|███▋ | 205/563 [05:44<03:44, 1.60it/s]
[2024-06-07 00:18:30] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00015-of-00037.safetensors |
|
36%|███▋ | 205/563 [05:44<03:44, 1.60it/s]
[2024-06-07 00:18:31] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00016-of-00037.safetensors |
|
36%|███▋ | 205/563 [05:45<03:44, 1.60it/s]
[2024-06-07 00:18:40] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.32.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
36%|███▋ | 205/563 [05:54<03:44, 1.60it/s]
37%|███▋ | 206/563 [05:54<15:46, 2.65s/it]
[2024-06-07 00:18:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.32.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
37%|███▋ | 206/563 [05:56<15:46, 2.65s/it]
[2024-06-07 00:18:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.32.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
37%|███▋ | 206/563 [05:56<15:46, 2.65s/it]
37%|███▋ | 207/563 [05:56<14:34, 2.46s/it]
[2024-06-07 00:18:48] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.32.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
37%|███▋ | 207/563 [06:02<14:34, 2.46s/it]
[2024-06-07 00:18:48] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.32.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
37%|███▋ | 207/563 [06:02<14:34, 2.46s/it]
37%|███▋ | 208/563 [06:02<20:28, 3.46s/it]
[2024-06-07 00:18:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.32.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
37%|███▋ | 208/563 [06:02<20:28, 3.46s/it]
[2024-06-07 00:18:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.33.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
37%|███▋ | 208/563 [06:02<20:28, 3.46s/it]
[2024-06-07 00:18:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.33.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
37%|███▋ | 208/563 [06:03<20:28, 3.46s/it]
[2024-06-07 00:18:50] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.33.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
37%|███▋ | 208/563 [06:04<20:28, 3.46s/it]
37%|███▋ | 211/563 [06:04<11:02, 1.88s/it]
[2024-06-07 00:18:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.33.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
37%|███▋ | 211/563 [06:08<11:02, 1.88s/it]
[2024-06-07 00:18:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.33.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
37%|███▋ | 211/563 [06:09<11:02, 1.88s/it]
38%|███▊ | 212/563 [06:09<15:16, 2.61s/it]
[2024-06-07 00:18:55] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.33.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
38%|███▊ | 212/563 [06:09<15:16, 2.61s/it]
[2024-06-07 00:18:55] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.33.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
38%|███▊ | 212/563 [06:09<15:16, 2.61s/it]
[2024-06-07 00:18:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.33.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
38%|███▊ | 212/563 [06:10<15:16, 2.61s/it]
[2024-06-07 00:18:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.33.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
38%|███▊ | 212/563 [06:10<15:16, 2.61s/it]
38%|███▊ | 215/563 [06:10<08:34, 1.48s/it]
[2024-06-07 00:18:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.33.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
38%|███▊ | 215/563 [06:10<08:34, 1.48s/it]
[2024-06-07 00:18:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.33.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
38%|███▊ | 215/563 [06:10<08:34, 1.48s/it]
38%|███▊ | 216/563 [06:10<07:21, 1.27s/it]
[2024-06-07 00:18:56] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00017-of-00037.safetensors |
|
38%|███▊ | 216/563 [06:10<07:21, 1.27s/it]
[2024-06-07 00:19:10] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.34.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
38%|███▊ | 216/563 [06:24<07:21, 1.27s/it]
[2024-06-07 00:19:10] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.34.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
38%|███▊ | 216/563 [06:24<07:21, 1.27s/it]
39%|███▊ | 217/563 [06:24<22:48, 3.96s/it]
[2024-06-07 00:19:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.34.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
39%|███▊ | 217/563 [06:24<22:48, 3.96s/it]
[2024-06-07 00:19:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.34.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
39%|███▊ | 217/563 [06:25<22:48, 3.96s/it]
[2024-06-07 00:19:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.34.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
39%|███▊ | 217/563 [06:25<22:48, 3.96s/it]
39%|███▉ | 219/563 [06:25<15:01, 2.62s/it]
[2024-06-07 00:19:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.34.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
39%|███▉ | 219/563 [06:25<15:01, 2.62s/it]
[2024-06-07 00:19:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.34.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
39%|███▉ | 219/563 [06:25<15:01, 2.62s/it]
39%|███▉ | 220/563 [06:25<12:18, 2.15s/it]
[2024-06-07 00:19:12] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.34.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
39%|███▉ | 220/563 [06:25<12:18, 2.15s/it]
[2024-06-07 00:19:12] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.34.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
39%|███▉ | 220/563 [06:26<12:18, 2.15s/it]
[2024-06-07 00:19:13] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.34.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
39%|███▉ | 220/563 [06:27<12:18, 2.15s/it]
39%|███▉ | 222/563 [06:27<08:56, 1.57s/it]
[2024-06-07 00:19:13] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.34.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
39%|███▉ | 222/563 [06:27<08:56, 1.57s/it]
[2024-06-07 00:19:13] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.35.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
39%|███▉ | 222/563 [06:27<08:56, 1.57s/it]
[2024-06-07 00:19:14] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.35.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
39%|███▉ | 222/563 [06:27<08:56, 1.57s/it]
[2024-06-07 00:19:14] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.35.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
39%|███▉ | 222/563 [06:28<08:56, 1.57s/it]
40%|███▉ | 225/563 [06:28<05:57, 1.06s/it]
[2024-06-07 00:19:19] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.35.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
40%|███▉ | 225/563 [06:33<05:57, 1.06s/it]
[2024-06-07 00:19:20] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.35.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
40%|███▉ | 225/563 [06:33<05:57, 1.06s/it]
40%|████ | 226/563 [06:33<10:26, 1.86s/it]
[2024-06-07 00:19:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.35.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
40%|████ | 226/563 [06:33<10:26, 1.86s/it]
[2024-06-07 00:19:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.35.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
40%|████ | 226/563 [06:33<10:26, 1.86s/it]
[2024-06-07 00:19:20] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.35.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
40%|████ | 226/563 [06:34<10:26, 1.86s/it]
[2024-06-07 00:19:20] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.35.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
40%|████ | 226/563 [06:34<10:26, 1.86s/it]
41%|████ | 229/563 [06:34<06:16, 1.13s/it]
[2024-06-07 00:19:20] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.35.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
41%|████ | 229/563 [06:34<06:16, 1.13s/it]
[2024-06-07 00:19:20] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.35.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
41%|████ | 229/563 [06:34<06:16, 1.13s/it]
41%|████ | 230/563 [06:34<05:30, 1.01it/s]
[2024-06-07 00:19:25] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.36.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
41%|████ | 230/563 [06:39<05:30, 1.01it/s]
[2024-06-07 00:19:26] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.36.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
41%|████ | 230/563 [06:40<05:30, 1.01it/s]
41%|████ | 231/563 [06:40<10:10, 1.84s/it]
[2024-06-07 00:19:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.36.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
41%|████ | 231/563 [06:40<10:10, 1.84s/it]
[2024-06-07 00:19:26] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.36.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
41%|████ | 231/563 [06:40<10:10, 1.84s/it]
[2024-06-07 00:19:26] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.36.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
41%|████ | 231/563 [06:40<10:10, 1.84s/it]
41%|████▏ | 233/563 [06:40<06:56, 1.26s/it]
[2024-06-07 00:19:26] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.36.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
41%|████▏ | 233/563 [06:40<06:56, 1.26s/it]
[2024-06-07 00:19:27] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.36.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
41%|████▏ | 233/563 [06:40<06:56, 1.26s/it]
42%|████▏ | 234/563 [06:40<05:56, 1.08s/it]
[2024-06-07 00:19:27] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00017-of-00037.safetensors |
|
42%|████▏ | 234/563 [06:40<05:56, 1.08s/it]
[2024-06-07 00:19:27] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00016-of-00037.safetensors |
|
42%|████▏ | 234/563 [06:41<05:56, 1.08s/it]
[2024-06-07 00:19:27] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00018-of-00037.safetensors |
|
42%|████▏ | 234/563 [06:41<05:56, 1.08s/it]
[2024-06-07 00:19:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.36.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
42%|████▏ | 234/563 [06:50<05:56, 1.08s/it]
42%|████▏ | 235/563 [06:50<15:59, 2.93s/it]
[2024-06-07 00:19:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.36.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
42%|████▏ | 235/563 [06:51<15:59, 2.93s/it]
[2024-06-07 00:19:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.36.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
42%|████▏ | 235/563 [06:51<15:59, 2.93s/it]
42%|████▏ | 236/563 [06:51<13:38, 2.50s/it]
[2024-06-07 00:19:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.36.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
42%|████▏ | 236/563 [06:51<13:38, 2.50s/it]
[2024-06-07 00:19:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.37.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
42%|████▏ | 236/563 [06:51<13:38, 2.50s/it]
[2024-06-07 00:19:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.37.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
42%|████▏ | 236/563 [06:52<13:38, 2.50s/it]
[2024-06-07 00:19:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.37.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
42%|████▏ | 236/563 [06:52<13:38, 2.50s/it]
42%|████▏ | 239/563 [06:52<07:44, 1.43s/it]
[2024-06-07 00:19:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.37.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
42%|████▏ | 239/563 [06:56<07:44, 1.43s/it]
[2024-06-07 00:19:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.37.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
42%|████▏ | 239/563 [06:57<07:44, 1.43s/it]
43%|████▎ | 240/563 [06:57<11:36, 2.16s/it]
[2024-06-07 00:19:43] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.37.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
43%|████▎ | 240/563 [06:57<11:36, 2.16s/it]
[2024-06-07 00:19:43] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.37.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
43%|████▎ | 240/563 [06:57<11:36, 2.16s/it]
[2024-06-07 00:19:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.37.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
43%|████▎ | 240/563 [06:58<11:36, 2.16s/it]
[2024-06-07 00:19:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.37.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
43%|████▎ | 240/563 [06:58<11:36, 2.16s/it]
43%|████▎ | 243/563 [06:58<06:37, 1.24s/it]
[2024-06-07 00:19:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.37.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
43%|████▎ | 243/563 [06:58<06:37, 1.24s/it]
[2024-06-07 00:19:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.37.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
43%|████▎ | 243/563 [06:58<06:37, 1.24s/it]
43%|████▎ | 244/563 [06:58<05:43, 1.08s/it]
[2024-06-07 00:19:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.38.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
43%|████▎ | 244/563 [06:58<05:43, 1.08s/it]
[2024-06-07 00:19:45] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.38.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
43%|████▎ | 244/563 [06:59<05:43, 1.08s/it]
[2024-06-07 00:19:45] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.38.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
43%|████▎ | 244/563 [06:59<05:43, 1.08s/it]
44%|████▎ | 246/563 [06:59<04:52, 1.08it/s]
[2024-06-07 00:19:48] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.38.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
44%|████▎ | 246/563 [07:01<04:52, 1.08it/s]
[2024-06-07 00:19:48] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.38.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
44%|████▎ | 246/563 [07:02<04:52, 1.08it/s]
44%|████▍ | 247/563 [07:02<06:47, 1.29s/it]
[2024-06-07 00:19:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.38.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
44%|████▍ | 247/563 [07:02<06:47, 1.29s/it]
[2024-06-07 00:19:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.38.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
44%|████▍ | 247/563 [07:02<06:47, 1.29s/it]
[2024-06-07 00:19:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.38.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
44%|████▍ | 247/563 [07:02<06:47, 1.29s/it]
[2024-06-07 00:19:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.38.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
44%|████▍ | 247/563 [07:03<06:47, 1.29s/it]
44%|████▍ | 250/563 [07:03<04:02, 1.29it/s]
[2024-06-07 00:19:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.38.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
44%|████▍ | 250/563 [07:03<04:02, 1.29it/s]
[2024-06-07 00:19:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.38.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
44%|████▍ | 250/563 [07:03<04:02, 1.29it/s]
45%|████▍ | 251/563 [07:03<03:37, 1.43it/s]
[2024-06-07 00:19:49] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00018-of-00037.safetensors |
|
45%|████▍ | 251/563 [07:03<03:37, 1.43it/s]
[2024-06-07 00:19:49] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00019-of-00037.safetensors |
|
45%|████▍ | 251/563 [07:03<03:37, 1.43it/s]
[2024-06-07 00:19:57] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.39.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
45%|████▍ | 251/563 [07:11<03:37, 1.43it/s]
45%|████▍ | 252/563 [07:11<10:57, 2.12s/it]
[2024-06-07 00:19:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.39.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
45%|████▍ | 252/563 [07:12<10:57, 2.12s/it]
[2024-06-07 00:19:59] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.39.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
45%|████▍ | 252/563 [07:13<10:57, 2.12s/it]
45%|████▍ | 253/563 [07:13<10:58, 2.12s/it]
[2024-06-07 00:20:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.39.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
45%|████▍ | 253/563 [07:18<10:58, 2.12s/it]
[2024-06-07 00:20:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.39.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
45%|████▍ | 253/563 [07:19<10:58, 2.12s/it]
45%|████▌ | 254/563 [07:19<16:19, 3.17s/it]
[2024-06-07 00:20:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.39.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
45%|████▌ | 254/563 [07:19<16:19, 3.17s/it]
[2024-06-07 00:20:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.39.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
45%|████▌ | 254/563 [07:19<16:19, 3.17s/it]
[2024-06-07 00:20:06] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.39.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
45%|████▌ | 254/563 [07:20<16:19, 3.17s/it]
[2024-06-07 00:20:06] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.39.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
45%|████▌ | 254/563 [07:20<16:19, 3.17s/it]
46%|████▌ | 257/563 [07:20<08:25, 1.65s/it]
[2024-06-07 00:20:06] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.39.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
46%|████▌ | 257/563 [07:20<08:25, 1.65s/it]
[2024-06-07 00:20:06] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.39.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
46%|████▌ | 257/563 [07:20<08:25, 1.65s/it]
46%|████▌ | 258/563 [07:20<07:04, 1.39s/it]
[2024-06-07 00:20:06] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.40.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
46%|████▌ | 258/563 [07:20<07:04, 1.39s/it]
[2024-06-07 00:20:07] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.40.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
46%|████▌ | 258/563 [07:21<07:04, 1.39s/it]
[2024-06-07 00:20:07] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.40.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
46%|████▌ | 258/563 [07:21<07:04, 1.39s/it]
46%|████▌ | 260/563 [07:21<05:44, 1.14s/it]
[2024-06-07 00:20:13] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.40.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
46%|████▌ | 260/563 [07:27<05:44, 1.14s/it]
[2024-06-07 00:20:13] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.40.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
46%|████▌ | 260/563 [07:27<05:44, 1.14s/it]
46%|████▋ | 261/563 [07:27<10:42, 2.13s/it]
[2024-06-07 00:20:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.40.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
46%|████▋ | 261/563 [07:27<10:42, 2.13s/it]
[2024-06-07 00:20:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.40.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
46%|████▋ | 261/563 [07:27<10:42, 2.13s/it]
[2024-06-07 00:20:14] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.40.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
46%|████▋ | 261/563 [07:28<10:42, 2.13s/it]
[2024-06-07 00:20:14] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.40.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
46%|████▋ | 261/563 [07:28<10:42, 2.13s/it]
47%|████▋ | 264/563 [07:28<06:01, 1.21s/it]
[2024-06-07 00:20:14] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.40.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
47%|████▋ | 264/563 [07:28<06:01, 1.21s/it]
[2024-06-07 00:20:14] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.40.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
47%|████▋ | 264/563 [07:28<06:01, 1.21s/it]
47%|████▋ | 265/563 [07:28<05:12, 1.05s/it]
[2024-06-07 00:20:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.41.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
47%|████▋ | 265/563 [07:28<05:12, 1.05s/it]
[2024-06-07 00:20:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.41.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
47%|████▋ | 265/563 [07:29<05:12, 1.05s/it]
[2024-06-07 00:20:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.41.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
47%|████▋ | 265/563 [07:29<05:12, 1.05s/it]
47%|████▋ | 267/563 [07:29<03:48, 1.29it/s]
[2024-06-07 00:20:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.41.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
47%|████▋ | 267/563 [07:29<03:48, 1.29it/s]
[2024-06-07 00:20:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.41.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
47%|████▋ | 267/563 [07:29<03:48, 1.29it/s]
48%|████▊ | 268/563 [07:29<03:25, 1.44it/s]
[2024-06-07 00:20:15] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00019-of-00037.safetensors |
|
48%|████▊ | 268/563 [07:29<03:25, 1.44it/s]
[2024-06-07 00:20:16] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00020-of-00037.safetensors |
|
48%|████▊ | 268/563 [07:29<03:25, 1.44it/s]
[2024-06-07 00:20:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.41.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
48%|████▊ | 268/563 [07:37<03:25, 1.44it/s]
48%|████▊ | 269/563 [07:37<11:01, 2.25s/it]
[2024-06-07 00:20:25] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.41.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
48%|████▊ | 269/563 [07:38<11:01, 2.25s/it]
[2024-06-07 00:20:25] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.41.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
48%|████▊ | 269/563 [07:39<11:01, 2.25s/it]
48%|████▊ | 270/563 [07:39<10:35, 2.17s/it]
[2024-06-07 00:20:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.41.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
48%|████▊ | 270/563 [07:44<10:35, 2.17s/it]
[2024-06-07 00:20:31] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.41.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
48%|████▊ | 270/563 [07:45<10:35, 2.17s/it]
48%|████▊ | 271/563 [07:45<15:09, 3.11s/it]
[2024-06-07 00:20:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.41.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
48%|████▊ | 271/563 [07:45<15:09, 3.11s/it]
[2024-06-07 00:20:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.42.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
48%|████▊ | 271/563 [07:45<15:09, 3.11s/it]
[2024-06-07 00:20:32] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.42.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
48%|████▊ | 271/563 [07:46<15:09, 3.11s/it]
[2024-06-07 00:20:32] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.42.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
48%|████▊ | 271/563 [07:46<15:09, 3.11s/it]
49%|████▊ | 274/563 [07:46<08:16, 1.72s/it]
[2024-06-07 00:20:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.42.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
49%|████▊ | 274/563 [07:50<08:16, 1.72s/it]
[2024-06-07 00:20:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.42.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
49%|████▊ | 274/563 [07:51<08:16, 1.72s/it]
49%|████▉ | 275/563 [07:51<11:19, 2.36s/it]
[2024-06-07 00:20:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.42.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
49%|████▉ | 275/563 [07:51<11:19, 2.36s/it]
[2024-06-07 00:20:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.42.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
49%|████▉ | 275/563 [07:51<11:19, 2.36s/it]
[2024-06-07 00:20:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.42.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
49%|████▉ | 275/563 [07:51<11:19, 2.36s/it]
[2024-06-07 00:20:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.42.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
49%|████▉ | 275/563 [07:51<11:19, 2.36s/it]
49%|████▉ | 278/563 [07:51<06:22, 1.34s/it]
[2024-06-07 00:20:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.42.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
49%|████▉ | 278/563 [07:52<06:22, 1.34s/it]
[2024-06-07 00:20:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.42.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
49%|████▉ | 278/563 [07:52<06:22, 1.34s/it]
50%|████▉ | 279/563 [07:52<05:29, 1.16s/it]
[2024-06-07 00:20:38] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00021-of-00037.safetensors |
|
50%|████▉ | 279/563 [07:52<05:29, 1.16s/it]
[2024-06-07 00:20:52] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.43.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
50%|████▉ | 279/563 [08:06<05:29, 1.16s/it]
[2024-06-07 00:20:52] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.43.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
50%|████▉ | 279/563 [08:06<05:29, 1.16s/it]
50%|████▉ | 280/563 [08:06<18:34, 3.94s/it]
[2024-06-07 00:20:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.43.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
50%|████▉ | 280/563 [08:06<18:34, 3.94s/it]
[2024-06-07 00:20:53] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.43.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
50%|████▉ | 280/563 [08:07<18:34, 3.94s/it]
[2024-06-07 00:20:53] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.43.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
50%|████▉ | 280/563 [08:07<18:34, 3.94s/it]
50%|█████ | 282/563 [08:07<12:11, 2.60s/it]
[2024-06-07 00:20:53] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.43.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
50%|█████ | 282/563 [08:07<12:11, 2.60s/it]
[2024-06-07 00:20:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.43.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
50%|█████ | 282/563 [08:07<12:11, 2.60s/it]
50%|█████ | 283/563 [08:07<09:59, 2.14s/it]
[2024-06-07 00:20:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.43.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
50%|█████ | 283/563 [08:07<09:59, 2.14s/it]
[2024-06-07 00:20:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.43.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
50%|█████ | 283/563 [08:08<09:59, 2.14s/it]
[2024-06-07 00:20:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.43.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
50%|█████ | 283/563 [08:09<09:59, 2.14s/it]
51%|█████ | 285/563 [08:09<07:17, 1.57s/it]
[2024-06-07 00:20:55] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.43.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
51%|█████ | 285/563 [08:09<07:17, 1.57s/it]
[2024-06-07 00:20:55] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.44.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
51%|█████ | 285/563 [08:09<07:17, 1.57s/it]
[2024-06-07 00:20:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.44.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
51%|█████ | 285/563 [08:10<07:17, 1.57s/it]
[2024-06-07 00:20:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.44.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
51%|█████ | 285/563 [08:10<07:17, 1.57s/it]
51%|█████ | 288/563 [08:10<04:51, 1.06s/it]
[2024-06-07 00:21:00] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.44.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
51%|█████ | 288/563 [08:13<04:51, 1.06s/it]
[2024-06-07 00:21:00] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.44.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
51%|█████ | 288/563 [08:14<04:51, 1.06s/it]
51%|█████▏ | 289/563 [08:14<07:16, 1.59s/it]
[2024-06-07 00:21:00] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.44.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
51%|█████▏ | 289/563 [08:14<07:16, 1.59s/it]
[2024-06-07 00:21:00] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.44.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
51%|█████▏ | 289/563 [08:14<07:16, 1.59s/it]
[2024-06-07 00:21:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.44.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
51%|█████▏ | 289/563 [08:14<07:16, 1.59s/it]
[2024-06-07 00:21:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.44.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
51%|█████▏ | 289/563 [08:15<07:16, 1.59s/it]
52%|█████▏ | 292/563 [08:15<04:24, 1.03it/s]
[2024-06-07 00:21:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.44.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
52%|█████▏ | 292/563 [08:15<04:24, 1.03it/s]
[2024-06-07 00:21:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.44.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
52%|█████▏ | 292/563 [08:15<04:24, 1.03it/s]
52%|█████▏ | 293/563 [08:15<03:53, 1.16it/s]
[2024-06-07 00:21:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.45.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
52%|█████▏ | 293/563 [08:18<03:53, 1.16it/s]
[2024-06-07 00:21:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.45.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
52%|█████▏ | 293/563 [08:18<03:53, 1.16it/s]
52%|█████▏ | 294/563 [08:18<06:11, 1.38s/it]
[2024-06-07 00:21:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.45.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
52%|█████▏ | 294/563 [08:18<06:11, 1.38s/it]
[2024-06-07 00:21:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.45.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
52%|█████▏ | 294/563 [08:19<06:11, 1.38s/it]
[2024-06-07 00:21:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.45.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
52%|█████▏ | 294/563 [08:19<06:11, 1.38s/it]
53%|█████▎ | 296/563 [08:19<04:18, 1.03it/s]
[2024-06-07 00:21:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.45.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
53%|█████▎ | 296/563 [08:19<04:18, 1.03it/s]
[2024-06-07 00:21:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.45.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
53%|█████▎ | 296/563 [08:19<04:18, 1.03it/s]
53%|█████▎ | 297/563 [08:19<03:45, 1.18it/s]
[2024-06-07 00:21:05] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00021-of-00037.safetensors |
|
53%|█████▎ | 297/563 [08:19<03:45, 1.18it/s]
[2024-06-07 00:21:06] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00020-of-00037.safetensors |
|
53%|█████▎ | 297/563 [08:20<03:45, 1.18it/s]
[2024-06-07 00:21:06] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00022-of-00037.safetensors |
|
53%|█████▎ | 297/563 [08:20<03:45, 1.18it/s]
[2024-06-07 00:21:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.45.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
53%|█████▎ | 297/563 [08:28<03:45, 1.18it/s]
53%|█████▎ | 298/563 [08:28<11:29, 2.60s/it]
[2024-06-07 00:21:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.45.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
53%|█████▎ | 298/563 [08:29<11:29, 2.60s/it]
[2024-06-07 00:21:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.45.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
53%|█████▎ | 298/563 [08:29<11:29, 2.60s/it]
53%|█████▎ | 299/563 [08:29<09:58, 2.27s/it]
[2024-06-07 00:21:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.45.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
53%|█████▎ | 299/563 [08:29<09:58, 2.27s/it]
[2024-06-07 00:21:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.46.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
53%|█████▎ | 299/563 [08:29<09:58, 2.27s/it]
[2024-06-07 00:21:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.46.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
53%|█████▎ | 299/563 [08:30<09:58, 2.27s/it]
[2024-06-07 00:21:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.46.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
53%|█████▎ | 299/563 [08:30<09:58, 2.27s/it]
54%|█████▎ | 302/563 [08:30<05:43, 1.32s/it]
[2024-06-07 00:21:19] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.46.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
54%|█████▎ | 302/563 [08:33<05:43, 1.32s/it]
[2024-06-07 00:21:20] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.46.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
54%|█████▎ | 302/563 [08:34<05:43, 1.32s/it]
54%|█████▍ | 303/563 [08:34<07:25, 1.71s/it]
[2024-06-07 00:21:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.46.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
54%|█████▍ | 303/563 [08:34<07:25, 1.71s/it]
[2024-06-07 00:21:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.46.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
54%|█████▍ | 303/563 [08:34<07:25, 1.71s/it]
[2024-06-07 00:21:20] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.46.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
54%|█████▍ | 303/563 [08:34<07:25, 1.71s/it]
[2024-06-07 00:21:20] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.46.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
54%|█████▍ | 303/563 [08:34<07:25, 1.71s/it]
54%|█████▍ | 306/563 [08:34<04:17, 1.00s/it]
[2024-06-07 00:21:20] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.46.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
54%|█████▍ | 306/563 [08:34<04:17, 1.00s/it]
[2024-06-07 00:21:21] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.46.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
54%|█████▍ | 306/563 [08:34<04:17, 1.00s/it]
55%|█████▍ | 307/563 [08:34<03:45, 1.13it/s]
[2024-06-07 00:21:21] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.47.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
55%|█████▍ | 307/563 [08:34<03:45, 1.13it/s]
[2024-06-07 00:21:21] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.47.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
55%|█████▍ | 307/563 [08:35<03:45, 1.13it/s]
[2024-06-07 00:21:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.47.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
55%|█████▍ | 307/563 [08:36<03:45, 1.13it/s]
55%|█████▍ | 309/563 [08:36<03:20, 1.27it/s]
[2024-06-07 00:21:24] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.47.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
55%|█████▍ | 309/563 [08:38<03:20, 1.27it/s]
[2024-06-07 00:21:24] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.47.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
55%|█████▍ | 309/563 [08:38<03:20, 1.27it/s]
55%|█████▌ | 310/563 [08:38<04:50, 1.15s/it]
[2024-06-07 00:21:24] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.47.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
55%|█████▌ | 310/563 [08:38<04:50, 1.15s/it]
[2024-06-07 00:21:24] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.47.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
55%|█████▌ | 310/563 [08:38<04:50, 1.15s/it]
[2024-06-07 00:21:25] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.47.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
55%|█████▌ | 310/563 [08:39<04:50, 1.15s/it]
[2024-06-07 00:21:25] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.47.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
55%|█████▌ | 310/563 [08:39<04:50, 1.15s/it]
56%|█████▌ | 313/563 [08:39<02:53, 1.44it/s]
[2024-06-07 00:21:25] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.47.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
56%|█████▌ | 313/563 [08:39<02:53, 1.44it/s]
[2024-06-07 00:21:25] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.47.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
56%|█████▌ | 313/563 [08:39<02:53, 1.44it/s]
56%|█████▌ | 314/563 [08:39<02:37, 1.59it/s]
[2024-06-07 00:21:25] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00022-of-00037.safetensors |
|
56%|█████▌ | 314/563 [08:39<02:37, 1.59it/s]
[2024-06-07 00:21:26] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00023-of-00037.safetensors |
|
56%|█████▌ | 314/563 [08:39<02:37, 1.59it/s]
[2024-06-07 00:21:33] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.48.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
56%|█████▌ | 314/563 [08:47<02:37, 1.59it/s]
56%|█████▌ | 315/563 [08:47<08:23, 2.03s/it]
[2024-06-07 00:21:34] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.48.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
56%|█████▌ | 315/563 [08:47<08:23, 2.03s/it]
[2024-06-07 00:21:34] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.48.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
56%|█████▌ | 315/563 [08:48<08:23, 2.03s/it]
56%|█████▌ | 316/563 [08:48<07:36, 1.85s/it]
[2024-06-07 00:21:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.48.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
56%|█████▌ | 316/563 [08:52<07:36, 1.85s/it]
[2024-06-07 00:21:39] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.48.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
56%|█████▌ | 316/563 [08:53<07:36, 1.85s/it]
56%|█████▋ | 317/563 [08:53<10:54, 2.66s/it]
[2024-06-07 00:21:39] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.48.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
56%|█████▋ | 317/563 [08:53<10:54, 2.66s/it]
[2024-06-07 00:21:39] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.48.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
56%|█████▋ | 317/563 [08:53<10:54, 2.66s/it]
[2024-06-07 00:21:39] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.48.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
56%|█████▋ | 317/563 [08:53<10:54, 2.66s/it]
[2024-06-07 00:21:40] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.48.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
56%|█████▋ | 317/563 [08:53<10:54, 2.66s/it]
57%|█████▋ | 320/563 [08:53<05:36, 1.39s/it]
[2024-06-07 00:21:40] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.48.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
57%|█████▋ | 320/563 [08:54<05:36, 1.39s/it]
[2024-06-07 00:21:40] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.48.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
57%|█████▋ | 320/563 [08:54<05:36, 1.39s/it]
57%|█████▋ | 321/563 [08:54<04:46, 1.18s/it]
[2024-06-07 00:21:40] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.49.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
57%|█████▋ | 321/563 [08:54<04:46, 1.18s/it]
[2024-06-07 00:21:41] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.49.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
57%|█████▋ | 321/563 [08:55<04:46, 1.18s/it]
[2024-06-07 00:21:41] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.49.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
57%|█████▋ | 321/563 [08:55<04:46, 1.18s/it]
57%|█████▋ | 323/563 [08:55<03:58, 1.01it/s]
[2024-06-07 00:21:45] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.49.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
57%|█████▋ | 323/563 [08:59<03:58, 1.01it/s]
[2024-06-07 00:21:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.49.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
57%|█████▋ | 323/563 [08:59<03:58, 1.01it/s]
58%|█████▊ | 324/563 [08:59<06:40, 1.68s/it]
[2024-06-07 00:21:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.49.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
58%|█████▊ | 324/563 [08:59<06:40, 1.68s/it]
[2024-06-07 00:21:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.49.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
58%|█████▊ | 324/563 [08:59<06:40, 1.68s/it]
[2024-06-07 00:21:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.49.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
58%|█████▊ | 324/563 [09:00<06:40, 1.68s/it]
[2024-06-07 00:21:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.49.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
58%|█████▊ | 324/563 [09:00<06:40, 1.68s/it]
58%|█████▊ | 327/563 [09:00<03:48, 1.03it/s]
[2024-06-07 00:21:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.49.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
58%|█████▊ | 327/563 [09:00<03:48, 1.03it/s]
[2024-06-07 00:21:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.49.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
58%|█████▊ | 327/563 [09:00<03:48, 1.03it/s]
58%|█████▊ | 328/563 [09:00<03:20, 1.17it/s]
[2024-06-07 00:21:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.50.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
58%|█████▊ | 328/563 [09:00<03:20, 1.17it/s]
[2024-06-07 00:21:47] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.50.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
58%|█████▊ | 328/563 [09:01<03:20, 1.17it/s]
[2024-06-07 00:21:47] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.50.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
58%|█████▊ | 328/563 [09:01<03:20, 1.17it/s]
59%|█████▊ | 330/563 [09:01<02:29, 1.56it/s]
[2024-06-07 00:21:47] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.50.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
59%|█████▊ | 330/563 [09:01<02:29, 1.56it/s]
[2024-06-07 00:21:47] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.50.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
59%|█████▊ | 330/563 [09:01<02:29, 1.56it/s]
59%|█████▉ | 331/563 [09:01<02:16, 1.69it/s]
[2024-06-07 00:21:47] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00023-of-00037.safetensors |
|
59%|█████▉ | 331/563 [09:01<02:16, 1.69it/s]
[2024-06-07 00:21:48] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00004-of-00037.safetensors |
|
59%|█████▉ | 331/563 [09:02<02:16, 1.69it/s]
[2024-06-07 00:21:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.5.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
59%|█████▉ | 331/563 [09:10<02:16, 1.69it/s]
59%|█████▉ | 332/563 [09:10<09:19, 2.42s/it]
[2024-06-07 00:21:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.5.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
59%|█████▉ | 332/563 [09:11<09:19, 2.42s/it]
[2024-06-07 00:21:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.5.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
59%|█████▉ | 332/563 [09:12<09:19, 2.42s/it]
59%|█████▉ | 333/563 [09:12<08:46, 2.29s/it]
[2024-06-07 00:22:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.5.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
59%|█████▉ | 333/563 [09:18<08:46, 2.29s/it]
[2024-06-07 00:22:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.5.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
59%|█████▉ | 333/563 [09:18<08:46, 2.29s/it]
59%|█████▉ | 334/563 [09:18<12:51, 3.37s/it]
[2024-06-07 00:22:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.5.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
59%|█████▉ | 334/563 [09:18<12:51, 3.37s/it]
[2024-06-07 00:22:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.6.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
59%|█████▉ | 334/563 [09:18<12:51, 3.37s/it]
[2024-06-07 00:22:06] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.6.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
59%|█████▉ | 334/563 [09:19<12:51, 3.37s/it]
[2024-06-07 00:22:06] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.6.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
59%|█████▉ | 334/563 [09:20<12:51, 3.37s/it]
60%|█████▉ | 337/563 [09:20<06:58, 1.85s/it]
[2024-06-07 00:22:10] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.6.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
60%|█████▉ | 337/563 [09:24<06:58, 1.85s/it]
[2024-06-07 00:22:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.6.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
60%|█████▉ | 337/563 [09:25<06:58, 1.85s/it]
60%|██████ | 338/563 [09:25<09:21, 2.49s/it]
[2024-06-07 00:22:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.6.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
60%|██████ | 338/563 [09:25<09:21, 2.49s/it]
[2024-06-07 00:22:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.6.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
60%|██████ | 338/563 [09:25<09:21, 2.49s/it]
[2024-06-07 00:22:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.6.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
60%|██████ | 338/563 [09:25<09:21, 2.49s/it]
[2024-06-07 00:22:12] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.6.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
60%|██████ | 338/563 [09:25<09:21, 2.49s/it]
61%|██████ | 341/563 [09:25<05:14, 1.42s/it]
[2024-06-07 00:22:12] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.6.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
61%|██████ | 341/563 [09:26<05:14, 1.42s/it]
[2024-06-07 00:22:12] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.6.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
61%|██████ | 341/563 [09:26<05:14, 1.42s/it]
61%|██████ | 342/563 [09:26<04:30, 1.22s/it]
[2024-06-07 00:22:12] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00005-of-00037.safetensors |
|
61%|██████ | 342/563 [09:26<04:30, 1.22s/it]
[2024-06-07 00:22:27] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.7.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
61%|██████ | 342/563 [09:40<04:30, 1.22s/it]
[2024-06-07 00:22:27] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.7.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
61%|██████ | 342/563 [09:41<04:30, 1.22s/it]
61%|██████ | 343/563 [09:41<15:11, 4.15s/it]
[2024-06-07 00:22:27] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.7.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
61%|██████ | 343/563 [09:41<15:11, 4.15s/it]
[2024-06-07 00:22:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.7.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
61%|██████ | 343/563 [09:42<15:11, 4.15s/it]
[2024-06-07 00:22:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.7.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
61%|██████ | 343/563 [09:42<15:11, 4.15s/it]
61%|██████▏ | 345/563 [09:42<09:49, 2.70s/it]
[2024-06-07 00:22:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.7.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
61%|██████▏ | 345/563 [09:42<09:49, 2.70s/it]
[2024-06-07 00:22:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.7.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
61%|██████▏ | 345/563 [09:42<09:49, 2.70s/it]
61%|██████▏ | 346/563 [09:42<08:02, 2.22s/it]
[2024-06-07 00:22:28] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00005-of-00037.safetensors |
|
61%|██████▏ | 346/563 [09:42<08:02, 2.22s/it]
[2024-06-07 00:22:29] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00004-of-00037.safetensors |
|
61%|██████▏ | 346/563 [09:43<08:02, 2.22s/it]
[2024-06-07 00:22:29] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00024-of-00037.safetensors |
|
61%|██████▏ | 346/563 [09:43<08:02, 2.22s/it]
[2024-06-07 00:22:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.50.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
61%|██████▏ | 346/563 [09:50<08:02, 2.22s/it]
62%|██████▏ | 347/563 [09:50<13:00, 3.61s/it]
[2024-06-07 00:22:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.50.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
62%|██████▏ | 347/563 [09:51<13:00, 3.61s/it]
[2024-06-07 00:22:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.50.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
62%|██████▏ | 347/563 [09:52<13:00, 3.61s/it]
62%|██████▏ | 348/563 [09:52<10:49, 3.02s/it]
[2024-06-07 00:22:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.50.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
62%|██████▏ | 348/563 [09:55<10:49, 3.02s/it]
[2024-06-07 00:22:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.50.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
62%|██████▏ | 348/563 [09:56<10:49, 3.02s/it]
62%|██████▏ | 349/563 [09:56<12:13, 3.43s/it]
[2024-06-07 00:22:42] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.50.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
62%|██████▏ | 349/563 [09:56<12:13, 3.43s/it]
[2024-06-07 00:22:42] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.51.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
62%|██████▏ | 349/563 [09:56<12:13, 3.43s/it]
[2024-06-07 00:22:43] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.51.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
62%|██████▏ | 349/563 [09:57<12:13, 3.43s/it]
[2024-06-07 00:22:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.51.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
62%|██████▏ | 349/563 [09:57<12:13, 3.43s/it]
63%|██████▎ | 352/563 [09:57<06:27, 1.83s/it]
[2024-06-07 00:22:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.51.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
63%|██████▎ | 352/563 [10:00<06:27, 1.83s/it]
[2024-06-07 00:22:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.51.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
63%|██████▎ | 352/563 [10:00<06:27, 1.83s/it]
63%|██████▎ | 353/563 [10:00<07:08, 2.04s/it]
[2024-06-07 00:22:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.51.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
63%|██████▎ | 353/563 [10:00<07:08, 2.04s/it]
[2024-06-07 00:22:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.51.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
63%|██████▎ | 353/563 [10:00<07:08, 2.04s/it]
[2024-06-07 00:22:47] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.51.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
63%|██████▎ | 353/563 [10:01<07:08, 2.04s/it]
[2024-06-07 00:22:47] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.51.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
63%|██████▎ | 353/563 [10:01<07:08, 2.04s/it]
63%|██████▎ | 356/563 [10:01<04:02, 1.17s/it]
[2024-06-07 00:22:47] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.51.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
63%|██████▎ | 356/563 [10:01<04:02, 1.17s/it]
[2024-06-07 00:22:47] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.51.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
63%|██████▎ | 356/563 [10:01<04:02, 1.17s/it]
63%|██████▎ | 357/563 [10:01<03:29, 1.02s/it]
[2024-06-07 00:22:47] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00025-of-00037.safetensors |
|
63%|██████▎ | 357/563 [10:01<03:29, 1.02s/it]
[2024-06-07 00:23:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.52.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
63%|██████▎ | 357/563 [10:15<03:29, 1.02s/it]
[2024-06-07 00:23:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.52.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
63%|██████▎ | 357/563 [10:16<03:29, 1.02s/it]
64%|██████▎ | 358/563 [10:16<12:55, 3.78s/it]
[2024-06-07 00:23:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.52.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
64%|██████▎ | 358/563 [10:16<12:55, 3.78s/it]
[2024-06-07 00:23:03] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.52.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
64%|██████▎ | 358/563 [10:16<12:55, 3.78s/it]
[2024-06-07 00:23:03] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.52.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
64%|██████▎ | 358/563 [10:16<12:55, 3.78s/it]
64%|██████▍ | 360/563 [10:16<08:37, 2.55s/it]
[2024-06-07 00:23:03] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.52.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
64%|██████▍ | 360/563 [10:17<08:37, 2.55s/it]
[2024-06-07 00:23:03] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.52.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
64%|██████▍ | 360/563 [10:17<08:37, 2.55s/it]
64%|██████▍ | 361/563 [10:17<07:03, 2.10s/it]
[2024-06-07 00:23:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.52.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
64%|██████▍ | 361/563 [10:17<07:03, 2.10s/it]
[2024-06-07 00:23:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.52.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
64%|██████▍ | 361/563 [10:18<07:03, 2.10s/it]
[2024-06-07 00:23:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.52.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
64%|██████▍ | 361/563 [10:18<07:03, 2.10s/it]
64%|██████▍ | 363/563 [10:18<05:08, 1.54s/it]
[2024-06-07 00:23:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.52.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
64%|██████▍ | 363/563 [10:18<05:08, 1.54s/it]
[2024-06-07 00:23:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.53.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
64%|██████▍ | 363/563 [10:18<05:08, 1.54s/it]
[2024-06-07 00:23:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.53.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
64%|██████▍ | 363/563 [10:19<05:08, 1.54s/it]
[2024-06-07 00:23:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.53.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
64%|██████▍ | 363/563 [10:19<05:08, 1.54s/it]
65%|██████▌ | 366/563 [10:19<03:26, 1.05s/it]
[2024-06-07 00:23:10] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.53.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
65%|██████▌ | 366/563 [10:24<03:26, 1.05s/it]
[2024-06-07 00:23:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.53.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
65%|██████▌ | 366/563 [10:24<03:26, 1.05s/it]
65%|██████▌ | 367/563 [10:24<05:44, 1.76s/it]
[2024-06-07 00:23:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.53.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
65%|██████▌ | 367/563 [10:24<05:44, 1.76s/it]
[2024-06-07 00:23:11] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.53.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
65%|██████▌ | 367/563 [10:24<05:44, 1.76s/it]
[2024-06-07 00:23:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.53.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
65%|██████▌ | 367/563 [10:25<05:44, 1.76s/it]
[2024-06-07 00:23:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.53.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
65%|██████▌ | 367/563 [10:25<05:44, 1.76s/it]
66%|██████▌ | 370/563 [10:25<03:26, 1.07s/it]
[2024-06-07 00:23:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.53.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
66%|██████▌ | 370/563 [10:25<03:26, 1.07s/it]
[2024-06-07 00:23:11] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.53.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
66%|██████▌ | 370/563 [10:25<03:26, 1.07s/it]
66%|██████▌ | 371/563 [10:25<03:02, 1.05it/s]
[2024-06-07 00:23:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.54.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
66%|██████▌ | 371/563 [10:29<03:02, 1.05it/s]
[2024-06-07 00:23:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.54.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
66%|██████▌ | 371/563 [10:30<03:02, 1.05it/s]
66%|██████▌ | 372/563 [10:30<05:12, 1.63s/it]
[2024-06-07 00:23:16] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.54.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
66%|██████▌ | 372/563 [10:30<05:12, 1.63s/it]
[2024-06-07 00:23:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.54.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
66%|██████▌ | 372/563 [10:30<05:12, 1.63s/it]
[2024-06-07 00:23:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.54.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
66%|██████▌ | 372/563 [10:30<05:12, 1.63s/it]
66%|██████▋ | 374/563 [10:30<03:34, 1.14s/it]
[2024-06-07 00:23:17] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.54.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
66%|██████▋ | 374/563 [10:31<03:34, 1.14s/it]
[2024-06-07 00:23:17] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.54.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
66%|██████▋ | 374/563 [10:31<03:34, 1.14s/it]
67%|██████▋ | 375/563 [10:31<03:05, 1.01it/s]
[2024-06-07 00:23:17] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00025-of-00037.safetensors |
|
67%|██████▋ | 375/563 [10:31<03:05, 1.01it/s]
[2024-06-07 00:23:17] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00024-of-00037.safetensors |
|
67%|██████▋ | 375/563 [10:31<03:05, 1.01it/s]
[2024-06-07 00:23:18] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00026-of-00037.safetensors |
|
67%|██████▋ | 375/563 [10:32<03:05, 1.01it/s]
[2024-06-07 00:23:25] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.54.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
67%|██████▋ | 375/563 [10:39<03:05, 1.01it/s]
67%|██████▋ | 376/563 [10:39<08:16, 2.65s/it]
[2024-06-07 00:23:26] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.54.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
67%|██████▋ | 376/563 [10:40<08:16, 2.65s/it]
[2024-06-07 00:23:26] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.54.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
67%|██████▋ | 376/563 [10:40<08:16, 2.65s/it]
67%|██████▋ | 377/563 [10:40<07:08, 2.30s/it]
[2024-06-07 00:23:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.54.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
67%|██████▋ | 377/563 [10:40<07:08, 2.30s/it]
[2024-06-07 00:23:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.55.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
67%|██████▋ | 377/563 [10:40<07:08, 2.30s/it]
[2024-06-07 00:23:27] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.55.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
67%|██████▋ | 377/563 [10:41<07:08, 2.30s/it]
[2024-06-07 00:23:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.55.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
67%|██████▋ | 377/563 [10:41<07:08, 2.30s/it]
67%|██████▋ | 380/563 [10:42<04:04, 1.34s/it]
[2024-06-07 00:23:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.55.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
67%|██████▋ | 380/563 [10:44<04:04, 1.34s/it]
[2024-06-07 00:23:31] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.55.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
67%|██████▋ | 380/563 [10:45<04:04, 1.34s/it]
68%|██████▊ | 381/563 [10:45<05:06, 1.69s/it]
[2024-06-07 00:23:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.55.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
68%|██████▊ | 381/563 [10:45<05:06, 1.69s/it]
[2024-06-07 00:23:31] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.55.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
68%|██████▊ | 381/563 [10:45<05:06, 1.69s/it]
[2024-06-07 00:23:31] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.55.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
68%|██████▊ | 381/563 [10:45<05:06, 1.69s/it]
[2024-06-07 00:23:31] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.55.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
68%|██████▊ | 381/563 [10:45<05:06, 1.69s/it]
68%|██████▊ | 384/563 [10:45<02:56, 1.01it/s]
[2024-06-07 00:23:31] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.55.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
68%|██████▊ | 384/563 [10:45<02:56, 1.01it/s]
[2024-06-07 00:23:32] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.55.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
68%|██████▊ | 384/563 [10:45<02:56, 1.01it/s]
68%|██████▊ | 385/563 [10:45<02:35, 1.15it/s]
[2024-06-07 00:23:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.56.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
68%|██████▊ | 385/563 [10:45<02:35, 1.15it/s]
[2024-06-07 00:23:32] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.56.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
68%|██████▊ | 385/563 [10:46<02:35, 1.15it/s]
[2024-06-07 00:23:33] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.56.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
68%|██████▊ | 385/563 [10:47<02:35, 1.15it/s]
69%|██████▊ | 387/563 [10:47<02:18, 1.27it/s]
[2024-06-07 00:23:35] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.56.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
69%|██████▊ | 387/563 [10:49<02:18, 1.27it/s]
[2024-06-07 00:23:35] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.56.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
69%|██████▊ | 387/563 [10:49<02:18, 1.27it/s]
69%|██████▉ | 388/563 [10:49<03:22, 1.15s/it]
[2024-06-07 00:23:35] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.56.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
69%|██████▉ | 388/563 [10:49<03:22, 1.15s/it]
[2024-06-07 00:23:35] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.56.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
69%|██████▉ | 388/563 [10:49<03:22, 1.15s/it]
[2024-06-07 00:23:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.56.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
69%|██████▉ | 388/563 [10:50<03:22, 1.15s/it]
[2024-06-07 00:23:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.56.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
69%|██████▉ | 388/563 [10:50<03:22, 1.15s/it]
69%|██████▉ | 391/563 [10:50<01:59, 1.43it/s]
[2024-06-07 00:23:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.56.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
69%|██████▉ | 391/563 [10:50<01:59, 1.43it/s]
[2024-06-07 00:23:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.56.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
69%|██████▉ | 391/563 [10:50<01:59, 1.43it/s]
70%|██████▉ | 392/563 [10:50<01:47, 1.58it/s]
[2024-06-07 00:23:36] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00026-of-00037.safetensors |
|
70%|██████▉ | 392/563 [10:50<01:47, 1.58it/s]
[2024-06-07 00:23:37] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00027-of-00037.safetensors |
|
70%|██████▉ | 392/563 [10:51<01:47, 1.58it/s]
[2024-06-07 00:23:45] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.57.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
70%|██████▉ | 392/563 [10:58<01:47, 1.58it/s]
70%|██████▉ | 393/563 [10:58<06:13, 2.20s/it]
[2024-06-07 00:23:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.57.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
70%|██████▉ | 393/563 [10:59<06:13, 2.20s/it]
[2024-06-07 00:23:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.57.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
70%|██████▉ | 393/563 [11:00<06:13, 2.20s/it]
70%|██████▉ | 394/563 [11:00<05:34, 1.98s/it]
[2024-06-07 00:23:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.57.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
70%|██████▉ | 394/563 [11:03<05:34, 1.98s/it]
[2024-06-07 00:23:50] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.57.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
70%|██████▉ | 394/563 [11:04<05:34, 1.98s/it]
70%|███████ | 395/563 [11:04<07:07, 2.55s/it]
[2024-06-07 00:23:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.57.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
70%|███████ | 395/563 [11:04<07:07, 2.55s/it]
[2024-06-07 00:23:50] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.57.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
70%|███████ | 395/563 [11:04<07:07, 2.55s/it]
[2024-06-07 00:23:51] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.57.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
70%|███████ | 395/563 [11:04<07:07, 2.55s/it]
[2024-06-07 00:23:51] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.57.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
70%|███████ | 395/563 [11:04<07:07, 2.55s/it]
71%|███████ | 398/563 [11:04<03:38, 1.32s/it]
[2024-06-07 00:23:51] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.57.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
71%|███████ | 398/563 [11:05<03:38, 1.32s/it]
[2024-06-07 00:23:51] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.57.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
71%|███████ | 398/563 [11:05<03:38, 1.32s/it]
71%|███████ | 399/563 [11:05<03:05, 1.13s/it]
[2024-06-07 00:23:51] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.58.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
71%|███████ | 399/563 [11:05<03:05, 1.13s/it]
[2024-06-07 00:23:52] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.58.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
71%|███████ | 399/563 [11:06<03:05, 1.13s/it]
[2024-06-07 00:23:52] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.58.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
71%|███████ | 399/563 [11:06<03:05, 1.13s/it]
71%|███████ | 401/563 [11:06<02:33, 1.05it/s]
[2024-06-07 00:23:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.58.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
71%|███████ | 401/563 [11:09<02:33, 1.05it/s]
[2024-06-07 00:23:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.58.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
71%|███████ | 401/563 [11:10<02:33, 1.05it/s]
71%|███████▏ | 402/563 [11:10<04:12, 1.57s/it]
[2024-06-07 00:23:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.58.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
71%|███████▏ | 402/563 [11:10<04:12, 1.57s/it]
[2024-06-07 00:23:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.58.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
71%|███████▏ | 402/563 [11:10<04:12, 1.57s/it]
[2024-06-07 00:23:57] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.58.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
71%|███████▏ | 402/563 [11:10<04:12, 1.57s/it]
[2024-06-07 00:23:57] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.58.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
71%|███████▏ | 402/563 [11:10<04:12, 1.57s/it]
72%|███████▏ | 405/563 [11:11<02:23, 1.10it/s]
[2024-06-07 00:23:57] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.58.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
72%|███████▏ | 405/563 [11:11<02:23, 1.10it/s]
[2024-06-07 00:23:57] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.58.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
72%|███████▏ | 405/563 [11:11<02:23, 1.10it/s]
72%|███████▏ | 406/563 [11:11<02:06, 1.24it/s]
[2024-06-07 00:23:57] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.59.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
72%|███████▏ | 406/563 [11:11<02:06, 1.24it/s]
[2024-06-07 00:23:57] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.59.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
72%|███████▏ | 406/563 [11:11<02:06, 1.24it/s]
[2024-06-07 00:23:57] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.59.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
72%|███████▏ | 406/563 [11:11<02:06, 1.24it/s]
72%|███████▏ | 408/563 [11:11<01:34, 1.64it/s]
[2024-06-07 00:23:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.59.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
72%|███████▏ | 408/563 [11:12<01:34, 1.64it/s]
[2024-06-07 00:23:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.59.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
72%|███████▏ | 408/563 [11:12<01:34, 1.64it/s]
73%|███████▎ | 409/563 [11:12<01:26, 1.77it/s]
[2024-06-07 00:23:58] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00027-of-00037.safetensors |
|
73%|███████▎ | 409/563 [11:12<01:26, 1.77it/s]
[2024-06-07 00:23:58] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00028-of-00037.safetensors |
|
73%|███████▎ | 409/563 [11:12<01:26, 1.77it/s]
[2024-06-07 00:24:06] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.59.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
73%|███████▎ | 409/563 [11:19<01:26, 1.77it/s]
73%|███████▎ | 410/563 [11:19<05:28, 2.15s/it]
[2024-06-07 00:24:07] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.59.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
73%|███████▎ | 410/563 [11:21<05:28, 2.15s/it]
[2024-06-07 00:24:08] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.59.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
73%|███████▎ | 410/563 [11:21<05:28, 2.15s/it]
73%|███████▎ | 411/563 [11:21<05:20, 2.11s/it]
[2024-06-07 00:24:13] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.59.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
73%|███████▎ | 411/563 [11:27<05:20, 2.11s/it]
[2024-06-07 00:24:14] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.59.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
73%|███████▎ | 411/563 [11:28<05:20, 2.11s/it]
73%|███████▎ | 412/563 [11:28<07:58, 3.17s/it]
[2024-06-07 00:24:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.59.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
73%|███████▎ | 412/563 [11:28<07:58, 3.17s/it]
[2024-06-07 00:24:14] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.60.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
73%|███████▎ | 412/563 [11:28<07:58, 3.17s/it]
[2024-06-07 00:24:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.60.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
73%|███████▎ | 412/563 [11:29<07:58, 3.17s/it]
[2024-06-07 00:24:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.60.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
73%|███████▎ | 412/563 [11:29<07:58, 3.17s/it]
74%|███████▎ | 415/563 [11:29<04:23, 1.78s/it]
[2024-06-07 00:24:19] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.60.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
74%|███████▎ | 415/563 [11:33<04:23, 1.78s/it]
[2024-06-07 00:24:20] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.60.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
74%|███████▎ | 415/563 [11:34<04:23, 1.78s/it]
74%|███████▍ | 416/563 [11:34<05:50, 2.39s/it]
[2024-06-07 00:24:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.60.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
74%|███████▍ | 416/563 [11:34<05:50, 2.39s/it]
[2024-06-07 00:24:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.60.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
74%|███████▍ | 416/563 [11:34<05:50, 2.39s/it]
[2024-06-07 00:24:21] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.60.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
74%|███████▍ | 416/563 [11:34<05:50, 2.39s/it]
[2024-06-07 00:24:21] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.60.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
74%|███████▍ | 416/563 [11:35<05:50, 2.39s/it]
74%|███████▍ | 419/563 [11:35<03:16, 1.36s/it]
[2024-06-07 00:24:21] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.60.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
74%|███████▍ | 419/563 [11:35<03:16, 1.36s/it]
[2024-06-07 00:24:21] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.60.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
74%|███████▍ | 419/563 [11:35<03:16, 1.36s/it]
75%|███████▍ | 420/563 [11:35<02:48, 1.18s/it]
[2024-06-07 00:24:21] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00029-of-00037.safetensors |
|
75%|███████▍ | 420/563 [11:35<02:48, 1.18s/it]
[2024-06-07 00:24:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.61.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
75%|███████▍ | 420/563 [11:49<02:48, 1.18s/it]
[2024-06-07 00:24:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.61.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
75%|███████▍ | 420/563 [11:50<02:48, 1.18s/it]
75%|███████▍ | 421/563 [11:50<09:36, 4.06s/it]
[2024-06-07 00:24:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.61.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
75%|███████▍ | 421/563 [11:50<09:36, 4.06s/it]
[2024-06-07 00:24:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.61.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
75%|███████▍ | 421/563 [11:51<09:36, 4.06s/it]
[2024-06-07 00:24:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.61.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
75%|███████▍ | 421/563 [11:51<09:36, 4.06s/it]
75%|███████▌ | 423/563 [11:51<06:14, 2.68s/it]
[2024-06-07 00:24:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.61.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
75%|███████▌ | 423/563 [11:51<06:14, 2.68s/it]
[2024-06-07 00:24:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.61.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
75%|███████▌ | 423/563 [11:51<06:14, 2.68s/it]
75%|███████▌ | 424/563 [11:51<05:05, 2.20s/it]
[2024-06-07 00:24:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.61.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
75%|███████▌ | 424/563 [11:51<05:05, 2.20s/it]
[2024-06-07 00:24:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.61.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
75%|███████▌ | 424/563 [11:52<05:05, 2.20s/it]
[2024-06-07 00:24:39] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.61.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
75%|███████▌ | 424/563 [11:52<05:05, 2.20s/it]
76%|███████▌ | 426/563 [11:52<03:40, 1.61s/it]
[2024-06-07 00:24:39] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.61.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
76%|███████▌ | 426/563 [11:52<03:40, 1.61s/it]
[2024-06-07 00:24:39] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.62.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
76%|███████▌ | 426/563 [11:52<03:40, 1.61s/it]
[2024-06-07 00:24:39] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.62.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
76%|███████▌ | 426/563 [11:53<03:40, 1.61s/it]
[2024-06-07 00:24:40] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.62.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
76%|███████▌ | 426/563 [11:54<03:40, 1.61s/it]
76%|███████▌ | 429/563 [11:54<02:24, 1.08s/it]
[2024-06-07 00:24:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.62.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
76%|███████▌ | 429/563 [11:57<02:24, 1.08s/it]
[2024-06-07 00:24:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.62.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
76%|███████▌ | 429/563 [11:58<02:24, 1.08s/it]
76%|███████▋ | 430/563 [11:58<03:42, 1.68s/it]
[2024-06-07 00:24:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.62.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
76%|███████▋ | 430/563 [11:58<03:42, 1.68s/it]
[2024-06-07 00:24:44] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.62.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
76%|███████▋ | 430/563 [11:58<03:42, 1.68s/it]
[2024-06-07 00:24:45] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.62.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
76%|███████▋ | 430/563 [11:59<03:42, 1.68s/it]
[2024-06-07 00:24:45] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.62.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
76%|███████▋ | 430/563 [11:59<03:42, 1.68s/it]
77%|███████▋ | 433/563 [11:59<02:12, 1.02s/it]
[2024-06-07 00:24:45] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.62.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
77%|███████▋ | 433/563 [11:59<02:12, 1.02s/it]
[2024-06-07 00:24:45] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.62.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
77%|███████▋ | 433/563 [11:59<02:12, 1.02s/it]
77%|███████▋ | 434/563 [11:59<01:57, 1.10it/s]
[2024-06-07 00:24:48] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.63.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
77%|███████▋ | 434/563 [12:01<01:57, 1.10it/s]
[2024-06-07 00:24:48] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.63.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
77%|███████▋ | 434/563 [12:02<01:57, 1.10it/s]
77%|███████▋ | 435/563 [12:02<02:55, 1.37s/it]
[2024-06-07 00:24:48] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.63.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
77%|███████▋ | 435/563 [12:02<02:55, 1.37s/it]
[2024-06-07 00:24:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.63.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
77%|███████▋ | 435/563 [12:03<02:55, 1.37s/it]
[2024-06-07 00:24:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.63.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
77%|███████▋ | 435/563 [12:03<02:55, 1.37s/it]
78%|███████▊ | 437/563 [12:03<02:01, 1.04it/s]
[2024-06-07 00:24:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.63.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
78%|███████▊ | 437/563 [12:03<02:01, 1.04it/s]
[2024-06-07 00:24:49] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.63.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
78%|███████▊ | 437/563 [12:03<02:01, 1.04it/s]
78%|███████▊ | 438/563 [12:03<01:46, 1.18it/s]
[2024-06-07 00:24:49] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00028-of-00037.safetensors |
|
78%|███████▊ | 438/563 [12:03<01:46, 1.18it/s]
[2024-06-07 00:24:50] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00029-of-00037.safetensors |
|
78%|███████▊ | 438/563 [12:04<01:46, 1.18it/s]
[2024-06-07 00:24:50] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00030-of-00037.safetensors |
|
78%|███████▊ | 438/563 [12:04<01:46, 1.18it/s]
[2024-06-07 00:24:58] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.63.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
78%|███████▊ | 438/563 [12:12<01:46, 1.18it/s]
78%|███████▊ | 439/563 [12:12<05:37, 2.73s/it]
[2024-06-07 00:24:59] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.63.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
78%|███████▊ | 439/563 [12:13<05:37, 2.73s/it]
[2024-06-07 00:25:00] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.63.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
78%|███████▊ | 439/563 [12:13<05:37, 2.73s/it]
78%|███████▊ | 440/563 [12:13<04:49, 2.35s/it]
[2024-06-07 00:25:00] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.63.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
78%|███████▊ | 440/563 [12:14<04:49, 2.35s/it]
[2024-06-07 00:25:00] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.64.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
78%|███████▊ | 440/563 [12:14<04:49, 2.35s/it]
[2024-06-07 00:25:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.64.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
78%|███████▊ | 440/563 [12:14<04:49, 2.35s/it]
[2024-06-07 00:25:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.64.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
78%|███████▊ | 440/563 [12:15<04:49, 2.35s/it]
79%|███████▊ | 443/563 [12:15<02:43, 1.36s/it]
[2024-06-07 00:25:03] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.64.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
79%|███████▊ | 443/563 [12:17<02:43, 1.36s/it]
[2024-06-07 00:25:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.64.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
79%|███████▊ | 443/563 [12:18<02:43, 1.36s/it]
79%|███████▉ | 444/563 [12:18<03:17, 1.66s/it]
[2024-06-07 00:25:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.64.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
79%|███████▉ | 444/563 [12:18<03:17, 1.66s/it]
[2024-06-07 00:25:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.64.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
79%|███████▉ | 444/563 [12:18<03:17, 1.66s/it]
[2024-06-07 00:25:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.64.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
79%|███████▉ | 444/563 [12:18<03:17, 1.66s/it]
[2024-06-07 00:25:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.64.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
79%|███████▉ | 444/563 [12:18<03:17, 1.66s/it]
79%|███████▉ | 447/563 [12:18<01:52, 1.03it/s]
[2024-06-07 00:25:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.64.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
79%|███████▉ | 447/563 [12:18<01:52, 1.03it/s]
[2024-06-07 00:25:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.64.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
79%|███████▉ | 447/563 [12:18<01:52, 1.03it/s]
80%|███████▉ | 448/563 [12:18<01:38, 1.17it/s]
[2024-06-07 00:25:05] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.65.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
80%|███████▉ | 448/563 [12:18<01:38, 1.17it/s]
[2024-06-07 00:25:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.65.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
80%|███████▉ | 448/563 [12:19<01:38, 1.17it/s]
[2024-06-07 00:25:06] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.65.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
80%|███████▉ | 448/563 [12:20<01:38, 1.17it/s]
80%|███████▉ | 450/563 [12:20<01:27, 1.29it/s]
[2024-06-07 00:25:08] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.65.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
80%|███████▉ | 450/563 [12:22<01:27, 1.29it/s]
[2024-06-07 00:25:08] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.65.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
80%|███████▉ | 450/563 [12:22<01:27, 1.29it/s]
80%|████████ | 451/563 [12:22<02:08, 1.15s/it]
[2024-06-07 00:25:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.65.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
80%|████████ | 451/563 [12:22<02:08, 1.15s/it]
[2024-06-07 00:25:08] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.65.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
80%|████████ | 451/563 [12:22<02:08, 1.15s/it]
[2024-06-07 00:25:09] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.65.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
80%|████████ | 451/563 [12:23<02:08, 1.15s/it]
[2024-06-07 00:25:09] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.65.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
80%|████████ | 451/563 [12:23<02:08, 1.15s/it]
81%|████████ | 454/563 [12:23<01:15, 1.44it/s]
[2024-06-07 00:25:09] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.65.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
81%|████████ | 454/563 [12:23<01:15, 1.44it/s]
[2024-06-07 00:25:09] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.65.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
81%|████████ | 454/563 [12:23<01:15, 1.44it/s]
81%|████████ | 455/563 [12:23<01:07, 1.59it/s]
[2024-06-07 00:25:09] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00030-of-00037.safetensors |
|
81%|████████ | 455/563 [12:23<01:07, 1.59it/s]
[2024-06-07 00:25:10] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00031-of-00037.safetensors |
|
81%|████████ | 455/563 [12:24<01:07, 1.59it/s]
[2024-06-07 00:25:18] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.66.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
81%|████████ | 455/563 [12:32<01:07, 1.59it/s]
81%|████████ | 456/563 [12:32<04:02, 2.27s/it]
[2024-06-07 00:25:19] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.66.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
81%|████████ | 456/563 [12:33<04:02, 2.27s/it]
[2024-06-07 00:25:19] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.66.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
81%|████████ | 456/563 [12:33<04:02, 2.27s/it]
81%|████████ | 457/563 [12:33<03:35, 2.04s/it]
[2024-06-07 00:25:25] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.66.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
81%|████████ | 457/563 [12:39<03:35, 2.04s/it]
[2024-06-07 00:25:25] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.66.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
81%|████████ | 457/563 [12:39<03:35, 2.04s/it]
81%|████████▏ | 458/563 [12:39<05:24, 3.09s/it]
[2024-06-07 00:25:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.66.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
81%|████████▏ | 458/563 [12:39<05:24, 3.09s/it]
[2024-06-07 00:25:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.66.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
81%|████████▏ | 458/563 [12:39<05:24, 3.09s/it]
[2024-06-07 00:25:26] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.66.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
81%|████████▏ | 458/563 [12:40<05:24, 3.09s/it]
[2024-06-07 00:25:26] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.66.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
81%|████████▏ | 458/563 [12:40<05:24, 3.09s/it]
82%|████████▏ | 461/563 [12:40<02:43, 1.60s/it]
[2024-06-07 00:25:26] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.66.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
82%|████████▏ | 461/563 [12:40<02:43, 1.60s/it]
[2024-06-07 00:25:26] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.66.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
82%|████████▏ | 461/563 [12:40<02:43, 1.60s/it]
82%|████████▏ | 462/563 [12:40<02:16, 1.35s/it]
[2024-06-07 00:25:26] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.67.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
82%|████████▏ | 462/563 [12:40<02:16, 1.35s/it]
[2024-06-07 00:25:27] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.67.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
82%|████████▏ | 462/563 [12:41<02:16, 1.35s/it]
[2024-06-07 00:25:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.67.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
82%|████████▏ | 462/563 [12:41<02:16, 1.35s/it]
82%|████████▏ | 464/563 [12:42<01:48, 1.10s/it]
[2024-06-07 00:25:33] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.67.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
82%|████████▏ | 464/563 [12:46<01:48, 1.10s/it]
[2024-06-07 00:25:33] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.67.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
82%|████████▏ | 464/563 [12:47<01:48, 1.10s/it]
83%|████████▎ | 465/563 [12:47<03:18, 2.03s/it]
[2024-06-07 00:25:33] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.67.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
83%|████████▎ | 465/563 [12:47<03:18, 2.03s/it]
[2024-06-07 00:25:33] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.67.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
83%|████████▎ | 465/563 [12:47<03:18, 2.03s/it]
[2024-06-07 00:25:34] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.67.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
83%|████████▎ | 465/563 [12:48<03:18, 2.03s/it]
[2024-06-07 00:25:34] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.67.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
83%|████████▎ | 465/563 [12:48<03:18, 2.03s/it]
83%|████████▎ | 468/563 [12:48<01:49, 1.16s/it]
[2024-06-07 00:25:34] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.67.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
83%|████████▎ | 468/563 [12:48<01:49, 1.16s/it]
[2024-06-07 00:25:34] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.67.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
83%|████████▎ | 468/563 [12:48<01:49, 1.16s/it]
83%|████████▎ | 469/563 [12:48<01:34, 1.01s/it]
[2024-06-07 00:25:34] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.68.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
83%|████████▎ | 469/563 [12:48<01:34, 1.01s/it]
[2024-06-07 00:25:35] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.68.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
83%|████████▎ | 469/563 [12:48<01:34, 1.01s/it]
[2024-06-07 00:25:35] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.68.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
83%|████████▎ | 469/563 [12:48<01:34, 1.01s/it]
84%|████████▎ | 471/563 [12:48<01:08, 1.34it/s]
[2024-06-07 00:25:35] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.68.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
84%|████████▎ | 471/563 [12:49<01:08, 1.34it/s]
[2024-06-07 00:25:35] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.68.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
84%|████████▎ | 471/563 [12:49<01:08, 1.34it/s]
84%|████████▍ | 472/563 [12:49<01:01, 1.48it/s]
[2024-06-07 00:25:35] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00031-of-00037.safetensors |
|
84%|████████▍ | 472/563 [12:49<01:01, 1.48it/s]
[2024-06-07 00:25:35] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00032-of-00037.safetensors |
|
84%|████████▍ | 472/563 [12:49<01:01, 1.48it/s]
[2024-06-07 00:25:45] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.68.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
84%|████████▍ | 472/563 [12:59<01:01, 1.48it/s]
84%|████████▍ | 473/563 [12:59<04:12, 2.80s/it]
[2024-06-07 00:25:47] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.68.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
84%|████████▍ | 473/563 [13:01<04:12, 2.80s/it]
[2024-06-07 00:25:47] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.68.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
84%|████████▍ | 473/563 [13:01<04:12, 2.80s/it]
84%|████████▍ | 474/563 [13:01<03:52, 2.61s/it]
[2024-06-07 00:25:52] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.68.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
84%|████████▍ | 474/563 [13:06<03:52, 2.61s/it]
[2024-06-07 00:25:53] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.68.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
84%|████████▍ | 474/563 [13:07<03:52, 2.61s/it]
84%|████████▍ | 475/563 [13:07<04:58, 3.39s/it]
[2024-06-07 00:25:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.68.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
84%|████████▍ | 475/563 [13:07<04:58, 3.39s/it]
[2024-06-07 00:25:53] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.69.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
84%|████████▍ | 475/563 [13:07<04:58, 3.39s/it]
[2024-06-07 00:25:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.69.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
84%|████████▍ | 475/563 [13:08<04:58, 3.39s/it]
[2024-06-07 00:25:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.69.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
84%|████████▍ | 475/563 [13:08<04:58, 3.39s/it]
85%|████████▍ | 478/563 [13:08<02:36, 1.84s/it]
[2024-06-07 00:25:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.69.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
85%|████████▍ | 478/563 [13:12<02:36, 1.84s/it]
[2024-06-07 00:25:59] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.69.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
85%|████████▍ | 478/563 [13:13<02:36, 1.84s/it]
85%|████████▌ | 479/563 [13:13<03:17, 2.35s/it]
[2024-06-07 00:25:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.69.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
85%|████████▌ | 479/563 [13:13<03:17, 2.35s/it]
[2024-06-07 00:25:59] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.69.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
85%|████████▌ | 479/563 [13:13<03:17, 2.35s/it]
[2024-06-07 00:25:59] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.69.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
85%|████████▌ | 479/563 [13:13<03:17, 2.35s/it]
[2024-06-07 00:25:59] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.69.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
85%|████████▌ | 479/563 [13:13<03:17, 2.35s/it]
86%|████████▌ | 482/563 [13:13<01:49, 1.35s/it]
[2024-06-07 00:26:00] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.69.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
86%|████████▌ | 482/563 [13:13<01:49, 1.35s/it]
[2024-06-07 00:26:00] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.69.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
86%|████████▌ | 482/563 [13:13<01:49, 1.35s/it]
86%|████████▌ | 483/563 [13:13<01:33, 1.17s/it]
[2024-06-07 00:26:00] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00033-of-00037.safetensors |
|
86%|████████▌ | 483/563 [13:13<01:33, 1.17s/it]
[2024-06-07 00:26:14] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.70.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
86%|████████▌ | 483/563 [13:28<01:33, 1.17s/it]
[2024-06-07 00:26:15] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.70.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
86%|████████▌ | 483/563 [13:29<01:33, 1.17s/it]
86%|████████▌ | 484/563 [13:29<05:21, 4.07s/it]
[2024-06-07 00:26:15] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.70.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
86%|████████▌ | 484/563 [13:29<05:21, 4.07s/it]
[2024-06-07 00:26:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.70.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
86%|████████▌ | 484/563 [13:29<05:21, 4.07s/it]
[2024-06-07 00:26:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.70.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
86%|████████▌ | 484/563 [13:30<05:21, 4.07s/it]
86%|████████▋ | 486/563 [13:30<03:28, 2.70s/it]
[2024-06-07 00:26:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.70.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
86%|████████▋ | 486/563 [13:30<03:28, 2.70s/it]
[2024-06-07 00:26:16] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.70.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
86%|████████▋ | 486/563 [13:30<03:28, 2.70s/it]
87%|████████▋ | 487/563 [13:30<02:49, 2.23s/it]
[2024-06-07 00:26:16] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00032-of-00037.safetensors |
|
87%|████████▋ | 487/563 [13:30<02:49, 2.23s/it]
[2024-06-07 00:26:17] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00033-of-00037.safetensors |
|
87%|████████▋ | 487/563 [13:30<02:49, 2.23s/it]
[2024-06-07 00:26:17] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00005-of-00037.safetensors |
|
87%|████████▋ | 487/563 [13:31<02:49, 2.23s/it]
[2024-06-07 00:26:22] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.7.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
87%|████████▋ | 487/563 [13:35<02:49, 2.23s/it]
87%|████████▋ | 488/563 [13:35<03:42, 2.97s/it]
[2024-06-07 00:26:23] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.7.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
87%|████████▋ | 488/563 [13:36<03:42, 2.97s/it]
[2024-06-07 00:26:23] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.7.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
87%|████████▋ | 488/563 [13:37<03:42, 2.97s/it]
87%|████████▋ | 489/563 [13:37<03:09, 2.56s/it]
[2024-06-07 00:26:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.7.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
87%|████████▋ | 489/563 [13:37<03:09, 2.56s/it]
[2024-06-07 00:26:23] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.8.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
87%|████████▋ | 489/563 [13:37<03:09, 2.56s/it]
[2024-06-07 00:26:24] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.8.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
87%|████████▋ | 489/563 [13:38<03:09, 2.56s/it]
[2024-06-07 00:26:24] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.8.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
87%|████████▋ | 489/563 [13:38<03:09, 2.56s/it]
87%|████████▋ | 492/563 [13:38<01:43, 1.46s/it]
[2024-06-07 00:26:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.8.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
87%|████████▋ | 492/563 [13:42<01:43, 1.46s/it]
[2024-06-07 00:26:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.8.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
87%|████████▋ | 492/563 [13:42<01:43, 1.46s/it]
88%|████████▊ | 493/563 [13:42<02:22, 2.04s/it]
[2024-06-07 00:26:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.8.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
88%|████████▊ | 493/563 [13:42<02:22, 2.04s/it]
[2024-06-07 00:26:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.8.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
88%|████████▊ | 493/563 [13:42<02:22, 2.04s/it]
[2024-06-07 00:26:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.8.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
88%|████████▊ | 493/563 [13:43<02:22, 2.04s/it]
[2024-06-07 00:26:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.8.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
88%|████████▊ | 493/563 [13:43<02:22, 2.04s/it]
88%|████████▊ | 496/563 [13:43<01:19, 1.18s/it]
[2024-06-07 00:26:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.8.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
88%|████████▊ | 496/563 [13:43<01:19, 1.18s/it]
[2024-06-07 00:26:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.8.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
88%|████████▊ | 496/563 [13:43<01:19, 1.18s/it]
88%|████████▊ | 497/563 [13:43<01:08, 1.03s/it]
[2024-06-07 00:26:31] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.9.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
88%|████████▊ | 497/563 [13:45<01:08, 1.03s/it]
[2024-06-07 00:26:32] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.9.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
88%|████████▊ | 497/563 [13:46<01:08, 1.03s/it]
88%|████████▊ | 498/563 [13:46<01:29, 1.38s/it]
[2024-06-07 00:26:32] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.9.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
88%|████████▊ | 498/563 [13:46<01:29, 1.38s/it]
[2024-06-07 00:26:32] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.9.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
88%|████████▊ | 498/563 [13:46<01:29, 1.38s/it]
[2024-06-07 00:26:33] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.9.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
88%|████████▊ | 498/563 [13:46<01:29, 1.38s/it]
89%|████████▉ | 500/563 [13:46<01:00, 1.05it/s]
[2024-06-07 00:26:33] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.9.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
89%|████████▉ | 500/563 [13:47<01:00, 1.05it/s]
[2024-06-07 00:26:33] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.9.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
89%|████████▉ | 500/563 [13:47<01:00, 1.05it/s]
89%|████████▉ | 501/563 [13:47<00:51, 1.20it/s]
[2024-06-07 00:26:33] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00005-of-00037.safetensors |
|
89%|████████▉ | 501/563 [13:47<00:51, 1.20it/s]
[2024-06-07 00:26:33] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00033-of-00037.safetensors |
|
89%|████████▉ | 501/563 [13:47<00:51, 1.20it/s]
[2024-06-07 00:26:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.70.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
89%|████████▉ | 501/563 [13:50<00:51, 1.20it/s]
89%|████████▉ | 502/563 [13:50<01:25, 1.41s/it]
[2024-06-07 00:26:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.70.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
89%|████████▉ | 502/563 [13:51<01:25, 1.41s/it]
[2024-06-07 00:26:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.70.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
89%|████████▉ | 502/563 [13:51<01:25, 1.41s/it]
89%|████████▉ | 503/563 [13:51<01:21, 1.36s/it]
[2024-06-07 00:26:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.70.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
89%|████████▉ | 503/563 [13:51<01:21, 1.36s/it]
[2024-06-07 00:26:38] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.71.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
89%|████████▉ | 503/563 [13:51<01:21, 1.36s/it]
[2024-06-07 00:26:38] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.71.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
89%|████████▉ | 503/563 [13:52<01:21, 1.36s/it]
[2024-06-07 00:26:39] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.71.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
89%|████████▉ | 503/563 [13:53<01:21, 1.36s/it]
90%|████████▉ | 506/563 [13:53<00:50, 1.14it/s]
[2024-06-07 00:26:41] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.71.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
90%|████████▉ | 506/563 [13:55<00:50, 1.14it/s]
[2024-06-07 00:26:41] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.71.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
90%|████████▉ | 506/563 [13:55<00:50, 1.14it/s]
90%|█████████ | 507/563 [13:55<01:09, 1.24s/it]
[2024-06-07 00:26:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.71.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
90%|█████████ | 507/563 [13:55<01:09, 1.24s/it]
[2024-06-07 00:26:41] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.71.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
90%|█████████ | 507/563 [13:55<01:09, 1.24s/it]
[2024-06-07 00:26:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.71.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
90%|█████████ | 507/563 [13:56<01:09, 1.24s/it]
[2024-06-07 00:26:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.71.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
90%|█████████ | 507/563 [13:56<01:09, 1.24s/it]
91%|█████████ | 510/563 [13:56<00:39, 1.35it/s]
[2024-06-07 00:26:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.71.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
91%|█████████ | 510/563 [13:56<00:39, 1.35it/s]
[2024-06-07 00:26:42] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.71.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
91%|█████████ | 510/563 [13:56<00:39, 1.35it/s]
91%|█████████ | 511/563 [13:56<00:34, 1.50it/s]
[2024-06-07 00:26:44] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.72.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
91%|█████████ | 511/563 [13:58<00:34, 1.50it/s]
[2024-06-07 00:26:45] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.72.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
91%|█████████ | 511/563 [13:59<00:34, 1.50it/s]
91%|█████████ | 512/563 [13:59<00:54, 1.07s/it]
[2024-06-07 00:26:45] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.72.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
91%|█████████ | 512/563 [13:59<00:54, 1.07s/it]
[2024-06-07 00:26:45] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.72.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
91%|█████████ | 512/563 [13:59<00:54, 1.07s/it]
[2024-06-07 00:26:45] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.72.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
91%|█████████ | 512/563 [13:59<00:54, 1.07s/it]
91%|█████████▏| 514/563 [13:59<00:37, 1.32it/s]
[2024-06-07 00:26:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.72.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
91%|█████████▏| 514/563 [13:59<00:37, 1.32it/s]
[2024-06-07 00:26:46] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.72.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
91%|█████████▏| 514/563 [14:00<00:37, 1.32it/s]
91%|█████████▏| 515/563 [14:00<00:32, 1.47it/s]
[2024-06-07 00:26:46] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00033-of-00037.safetensors |
|
91%|█████████▏| 515/563 [14:00<00:32, 1.47it/s]
[2024-06-07 00:26:46] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00034-of-00037.safetensors |
|
91%|█████████▏| 515/563 [14:00<00:32, 1.47it/s]
[2024-06-07 00:26:55] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.72.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
91%|█████████▏| 515/563 [14:09<00:32, 1.47it/s]
92%|█████████▏| 516/563 [14:09<02:05, 2.68s/it]
[2024-06-07 00:26:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.72.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
92%|█████████▏| 516/563 [14:10<02:05, 2.68s/it]
[2024-06-07 00:26:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.72.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
92%|█████████▏| 516/563 [14:10<02:05, 2.68s/it]
92%|█████████▏| 517/563 [14:10<01:46, 2.31s/it]
[2024-06-07 00:26:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.72.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
92%|█████████▏| 517/563 [14:10<01:46, 2.31s/it]
[2024-06-07 00:26:56] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.73.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
92%|█████████▏| 517/563 [14:10<01:46, 2.31s/it]
[2024-06-07 00:26:57] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.73.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
92%|█████████▏| 517/563 [14:11<01:46, 2.31s/it]
[2024-06-07 00:26:58] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.73.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
92%|█████████▏| 517/563 [14:11<01:46, 2.31s/it]
92%|█████████▏| 520/563 [14:11<00:57, 1.34s/it]
[2024-06-07 00:27:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.73.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
92%|█████████▏| 520/563 [14:16<00:57, 1.34s/it]
[2024-06-07 00:27:03] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.73.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
92%|█████████▏| 520/563 [14:17<00:57, 1.34s/it]
93%|█████████▎| 521/563 [14:17<01:27, 2.09s/it]
[2024-06-07 00:27:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.73.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
93%|█████████▎| 521/563 [14:17<01:27, 2.09s/it]
[2024-06-07 00:27:03] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.73.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
93%|█████████▎| 521/563 [14:17<01:27, 2.09s/it]
[2024-06-07 00:27:03] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.73.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
93%|█████████▎| 521/563 [14:17<01:27, 2.09s/it]
[2024-06-07 00:27:03] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.73.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
93%|█████████▎| 521/563 [14:17<01:27, 2.09s/it]
93%|█████████▎| 524/563 [14:17<00:46, 1.20s/it]
[2024-06-07 00:27:03] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.73.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
93%|█████████▎| 524/563 [14:17<00:46, 1.20s/it]
[2024-06-07 00:27:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.73.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
93%|█████████▎| 524/563 [14:17<00:46, 1.20s/it]
93%|█████████▎| 525/563 [14:17<00:39, 1.04s/it]
[2024-06-07 00:27:04] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.74.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
93%|█████████▎| 525/563 [14:17<00:39, 1.04s/it]
[2024-06-07 00:27:04] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.74.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
93%|█████████▎| 525/563 [14:18<00:39, 1.04s/it]
[2024-06-07 00:27:05] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.74.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
93%|█████████▎| 525/563 [14:19<00:39, 1.04s/it]
94%|█████████▎| 527/563 [14:19<00:32, 1.11it/s]
[2024-06-07 00:27:09] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.74.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
94%|█████████▎| 527/563 [14:23<00:32, 1.11it/s]
[2024-06-07 00:27:09] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.74.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
94%|█████████▎| 527/563 [14:23<00:32, 1.11it/s]
94%|█████████▍| 528/563 [14:23<00:57, 1.65s/it]
[2024-06-07 00:27:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.74.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
94%|█████████▍| 528/563 [14:23<00:57, 1.65s/it]
[2024-06-07 00:27:09] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.74.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
94%|█████████▍| 528/563 [14:23<00:57, 1.65s/it]
[2024-06-07 00:27:10] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.74.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
94%|█████████▍| 528/563 [14:24<00:57, 1.65s/it]
[2024-06-07 00:27:10] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.74.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
94%|█████████▍| 528/563 [14:24<00:57, 1.65s/it]
94%|█████████▍| 531/563 [14:24<00:30, 1.04it/s]
[2024-06-07 00:27:10] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.74.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
94%|█████████▍| 531/563 [14:24<00:30, 1.04it/s]
[2024-06-07 00:27:10] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.74.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
94%|█████████▍| 531/563 [14:24<00:30, 1.04it/s]
94%|█████████▍| 532/563 [14:24<00:26, 1.17it/s]
[2024-06-07 00:27:10] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00034-of-00037.safetensors |
|
94%|█████████▍| 532/563 [14:24<00:26, 1.17it/s]
[2024-06-07 00:27:11] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00035-of-00037.safetensors |
|
94%|█████████▍| 532/563 [14:25<00:26, 1.17it/s]
[2024-06-07 00:27:20] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.75.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
94%|█████████▍| 532/563 [14:34<00:26, 1.17it/s]
95%|█████████▍| 533/563 [14:34<01:20, 2.67s/it]
[2024-06-07 00:27:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.75.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
95%|█████████▍| 533/563 [14:35<01:20, 2.67s/it]
[2024-06-07 00:27:22] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.75.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
95%|█████████▍| 533/563 [14:36<01:20, 2.67s/it]
95%|█████████▍| 534/563 [14:36<01:13, 2.52s/it]
[2024-06-07 00:27:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.75.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
95%|█████████▍| 534/563 [14:42<01:13, 2.52s/it]
[2024-06-07 00:27:28] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.75.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
95%|█████████▍| 534/563 [14:42<01:13, 2.52s/it]
95%|█████████▌| 535/563 [14:42<01:37, 3.48s/it]
[2024-06-07 00:27:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.75.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
95%|█████████▌| 535/563 [14:42<01:37, 3.48s/it]
[2024-06-07 00:27:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.75.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
95%|█████████▌| 535/563 [14:42<01:37, 3.48s/it]
[2024-06-07 00:27:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.75.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
95%|█████████▌| 535/563 [14:43<01:37, 3.48s/it]
[2024-06-07 00:27:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.75.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
95%|█████████▌| 535/563 [14:43<01:37, 3.48s/it]
96%|█████████▌| 538/563 [14:43<00:44, 1.80s/it]
[2024-06-07 00:27:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.75.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
96%|█████████▌| 538/563 [14:43<00:44, 1.80s/it]
[2024-06-07 00:27:29] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.75.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
96%|█████████▌| 538/563 [14:43<00:44, 1.80s/it]
96%|█████████▌| 539/563 [14:43<00:36, 1.51s/it]
[2024-06-07 00:27:29] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.76.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
96%|█████████▌| 539/563 [14:43<00:36, 1.51s/it]
[2024-06-07 00:27:30] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.76.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
96%|█████████▌| 539/563 [14:44<00:36, 1.51s/it]
[2024-06-07 00:27:31] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.76.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
96%|█████████▌| 539/563 [14:45<00:36, 1.51s/it]
96%|█████████▌| 541/563 [14:45<00:26, 1.20s/it]
[2024-06-07 00:27:35] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.76.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
96%|█████████▌| 541/563 [14:49<00:26, 1.20s/it]
[2024-06-07 00:27:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.76.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
96%|█████████▌| 541/563 [14:50<00:26, 1.20s/it]
96%|█████████▋| 542/563 [14:50<00:42, 2.02s/it]
[2024-06-07 00:27:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.76.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
96%|█████████▋| 542/563 [14:50<00:42, 2.02s/it]
[2024-06-07 00:27:36] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.76.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
96%|█████████▋| 542/563 [14:50<00:42, 2.02s/it]
[2024-06-07 00:27:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.76.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
96%|█████████▋| 542/563 [14:50<00:42, 2.02s/it]
[2024-06-07 00:27:36] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.76.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
96%|█████████▋| 542/563 [14:50<00:42, 2.02s/it]
97%|█████████▋| 545/563 [14:50<00:20, 1.15s/it]
[2024-06-07 00:27:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.76.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
97%|█████████▋| 545/563 [14:50<00:20, 1.15s/it]
[2024-06-07 00:27:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.76.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
97%|█████████▋| 545/563 [14:51<00:20, 1.15s/it]
97%|█████████▋| 546/563 [14:51<00:16, 1.00it/s]
[2024-06-07 00:27:37] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.77.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
97%|█████████▋| 546/563 [14:51<00:16, 1.00it/s]
[2024-06-07 00:27:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.77.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
97%|█████████▋| 546/563 [14:51<00:16, 1.00it/s]
[2024-06-07 00:27:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.77.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
97%|█████████▋| 546/563 [14:51<00:16, 1.00it/s]
97%|█████████▋| 548/563 [14:51<00:11, 1.36it/s]
[2024-06-07 00:27:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.77.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
97%|█████████▋| 548/563 [14:51<00:11, 1.36it/s]
[2024-06-07 00:27:37] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.77.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
97%|█████████▋| 548/563 [14:51<00:11, 1.36it/s]
98%|█████████▊| 549/563 [14:51<00:09, 1.50it/s]
[2024-06-07 00:27:38] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00035-of-00037.safetensors |
|
98%|█████████▊| 549/563 [14:51<00:09, 1.50it/s]
[2024-06-07 00:27:38] INFO huggingface_loader.py:185: Loading HF parameters from: /models/Qwen2-72B-Instruct/model-00036-of-00037.safetensors |
|
98%|█████████▊| 549/563 [14:52<00:09, 1.50it/s]
[2024-06-07 00:27:46] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.77.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
98%|█████████▊| 549/563 [15:00<00:09, 1.50it/s]
98%|█████████▊| 550/563 [15:00<00:31, 2.41s/it]
[2024-06-07 00:27:48] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.77.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
98%|█████████▊| 550/563 [15:02<00:31, 2.41s/it]
[2024-06-07 00:27:48] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.77.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
98%|█████████▊| 550/563 [15:02<00:31, 2.41s/it]
98%|█████████▊| 551/563 [15:02<00:27, 2.32s/it]
[2024-06-07 00:27:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.77.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
98%|█████████▊| 551/563 [15:08<00:27, 2.32s/it]
[2024-06-07 00:27:54] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.77.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
98%|█████████▊| 551/563 [15:08<00:27, 2.32s/it]
98%|█████████▊| 552/563 [15:08<00:36, 3.31s/it]
[2024-06-07 00:27:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.77.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
98%|█████████▊| 552/563 [15:08<00:36, 3.31s/it]
[2024-06-07 00:27:54] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.78.input_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
98%|█████████▊| 552/563 [15:08<00:36, 3.31s/it]
[2024-06-07 00:27:55] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.78.mlp.down_proj.q_weight[0m", shape: (8192, 3696), dtype: uint32 |
|
98%|█████████▊| 552/563 [15:09<00:36, 3.31s/it]
[2024-06-07 00:27:56] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.78.mlp.down_proj.q_scale[0m", shape: (8192, 924), dtype: float16 |
|
98%|█████████▊| 552/563 [15:10<00:36, 3.31s/it]
99%|█████████▊| 555/563 [15:10<00:14, 1.82s/it]
[2024-06-07 00:28:00] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.78.mlp.gate_up_proj.q_weight[0m", shape: (59136, 1024), dtype: uint32 |
|
99%|█████████▊| 555/563 [15:14<00:14, 1.82s/it]
[2024-06-07 00:28:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.78.mlp.gate_up_proj.q_scale[0m", shape: (59136, 256), dtype: float16 |
|
99%|█████████▊| 555/563 [15:15<00:14, 1.82s/it]
99%|█████████▉| 556/563 [15:15<00:17, 2.45s/it]
[2024-06-07 00:28:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.78.post_attention_layernorm.weight[0m", shape: (8192,), dtype: float16 |
|
99%|█████████▉| 556/563 [15:15<00:17, 2.45s/it]
[2024-06-07 00:28:01] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.78.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
99%|█████████▉| 556/563 [15:15<00:17, 2.45s/it]
[2024-06-07 00:28:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.78.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
99%|█████████▉| 556/563 [15:15<00:17, 2.45s/it]
[2024-06-07 00:28:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.78.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
99%|█████████▉| 556/563 [15:15<00:17, 2.45s/it]
99%|█████████▉| 559/563 [15:15<00:05, 1.39s/it]
[2024-06-07 00:28:01] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.78.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
99%|█████████▉| 559/563 [15:15<00:05, 1.39s/it]
[2024-06-07 00:28:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.78.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
99%|█████████▉| 559/563 [15:15<00:05, 1.39s/it]
99%|█████████▉| 560/563 [15:15<00:03, 1.20s/it]
[2024-06-07 00:28:02] INFO huggingface_loader.py:175: [Not quantized] Parameter: "[1mmodel.layers.79.self_attn.c_attn.bias[0m", shape: (10240,), dtype: float16 |
|
99%|█████████▉| 560/563 [15:15<00:03, 1.20s/it]
[2024-06-07 00:28:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.79.self_attn.c_attn.q_weight[0m", shape: (10240, 1024), dtype: uint32 |
|
99%|█████████▉| 560/563 [15:16<00:03, 1.20s/it]
[2024-06-07 00:28:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.79.self_attn.c_attn.q_scale[0m", shape: (10240, 256), dtype: float16 |
|
99%|█████████▉| 560/563 [15:16<00:03, 1.20s/it]
100%|█████████▉| 562/563 [15:16<00:00, 1.14it/s]
[2024-06-07 00:28:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.79.self_attn.o_proj.q_weight[0m", shape: (8192, 1024), dtype: uint32 |
|
100%|█████████▉| 562/563 [15:16<00:00, 1.14it/s]
[2024-06-07 00:28:02] INFO huggingface_loader.py:167: [Quantized] Parameter: "[1mmodel.layers.79.self_attn.o_proj.q_scale[0m", shape: (8192, 256), dtype: float16 |
|
100%|█████████▉| 562/563 [15:16<00:00, 1.14it/s]
100%|██████████| 563/563 [15:16<00:00, 1.28it/s]
100%|██████████| 563/563 [15:16<00:00, 1.63s/it] |
| [2024-06-07 00:28:02] INFO huggingface_loader.py:197: Unloading HF weight file: /models/Qwen2-72B-Instruct/model-00036-of-00037.safetensors |
| [2024-06-07 00:28:03] INFO stats.py:77: [92mTime usage[0m: HF loading: 325.663 sec; Pre-quantization mapping: 460.972 sec; Quantization: 5.259 sec |
| [2024-06-07 00:28:03] INFO stats.py:91: [92mRAM usage[0m: Peak RAM: 14.883 GB. Total bytes loaded from disk: 293.177 GB |
| [2024-06-07 00:28:03] INFO convert_weight.py:155: [92mParameter size[0m after quantization: 38.091 GB |
| [2024-06-07 00:28:03] INFO convert_weight.py:160: [92mTotal parameters[0m: 78,698,975,232 |
| [2024-06-07 00:28:03] INFO convert_weight.py:161: [92mBits per parameter[0m: 4.158 |
| [2024-06-07 00:28:03] INFO convert_weight.py:166: Saved to directory: [1m/models/mlc-delivery/hf/mlc-ai/Qwen2-72B-Instruct-q4f16_1-MLC[0m |
|
|
| All finished, 483 total shards committed, record saved to /models/mlc-delivery/hf/mlc-ai/Qwen2-72B-Instruct-q4f16_1-MLC/ndarray-cache.json |
|
|
