Text Generation
Transformers
Safetensors
hy_v3
hunyuan
hy3
Mixture of Experts
fp8
quantized
conversational
Instructions to use tencent/Hy3-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tencent/Hy3-FP8 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="tencent/Hy3-FP8") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("tencent/Hy3-FP8") model = AutoModelForCausalLM.from_pretrained("tencent/Hy3-FP8") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use tencent/Hy3-FP8 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tencent/Hy3-FP8" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy3-FP8", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/tencent/Hy3-FP8
- SGLang
How to use tencent/Hy3-FP8 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "tencent/Hy3-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy3-FP8", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "tencent/Hy3-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy3-FP8", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use tencent/Hy3-FP8 with Docker Model Runner:
docker model run hf.co/tencent/Hy3-FP8
Add files using upload-large-folder tool
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- .gitattributes +1 -0
- chat_template.jinja +204 -0
- config.json +55 -0
- generation_config.json +10 -0
- model-00002-of-00101.safetensors +3 -0
- model-00005-of-00101.safetensors +3 -0
- model-00007-of-00101.safetensors +3 -0
- model-00011-of-00101.safetensors +3 -0
- model-00012-of-00101.safetensors +3 -0
- model-00013-of-00101.safetensors +3 -0
- model-00015-of-00101.safetensors +3 -0
- model-00016-of-00101.safetensors +3 -0
- model-00019-of-00101.safetensors +3 -0
- model-00020-of-00101.safetensors +3 -0
- model-00024-of-00101.safetensors +3 -0
- model-00026-of-00101.safetensors +3 -0
- model-00030-of-00101.safetensors +3 -0
- model-00031-of-00101.safetensors +3 -0
- model-00032-of-00101.safetensors +3 -0
- model-00033-of-00101.safetensors +3 -0
- model-00040-of-00101.safetensors +3 -0
- model-00041-of-00101.safetensors +3 -0
- model-00042-of-00101.safetensors +3 -0
- model-00047-of-00101.safetensors +3 -0
- model-00049-of-00101.safetensors +3 -0
- model-00056-of-00101.safetensors +3 -0
- model-00058-of-00101.safetensors +3 -0
- model-00059-of-00101.safetensors +3 -0
- model-00060-of-00101.safetensors +3 -0
- model-00061-of-00101.safetensors +3 -0
- model-00065-of-00101.safetensors +3 -0
- model-00071-of-00101.safetensors +3 -0
- model-00073-of-00101.safetensors +3 -0
- model-00074-of-00101.safetensors +3 -0
- model-00075-of-00101.safetensors +3 -0
- model-00077-of-00101.safetensors +3 -0
- model-00078-of-00101.safetensors +3 -0
- model-00079-of-00101.safetensors +3 -0
- model-00080-of-00101.safetensors +3 -0
- model-00082-of-00101.safetensors +3 -0
- model-00084-of-00101.safetensors +3 -0
- model-00085-of-00101.safetensors +3 -0
- model-00088-of-00101.safetensors +3 -0
- model-00090-of-00101.safetensors +3 -0
- model-00093-of-00101.safetensors +3 -0
- model-00096-of-00101.safetensors +3 -0
- model-00097-of-00101.safetensors +3 -0
- model-00101-of-00101.safetensors +3 -0
- model.safetensors.index.json +3 -0
- tokenizer.json +0 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
|
chat_template.jinja
ADDED
|
@@ -0,0 +1,204 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{#- ----------‑‑‑ special token variables ‑‑‑---------- -#}
|
| 2 |
+
{%- set HYTK = ':opensource' %}
|
| 3 |
+
{%- set eos_token = '<|hy_eos{}|>'.format(HYTK) %}
|
| 4 |
+
{%- set bos_token = '<|hy_begin_of_sentence{}|>'.format(HYTK) %}
|
| 5 |
+
{%- set pad_token = '<|hy_pad{}|>'.format(HYTK) %}
|
| 6 |
+
{%- set user_token = '<|hy_User{}|>'.format(HYTK) %}
|
| 7 |
+
{%- set assistant_token = '<|hy_Assistant{}|>'.format(HYTK) %}
|
| 8 |
+
{%- set think_begin_token = '<think{}>'.format(HYTK) %}
|
| 9 |
+
{%- set think_end_token = '</think{}>'.format(HYTK) %}
|
| 10 |
+
{%- set toolcalls_begin_token = '<tool_calls{}>'.format(HYTK) %}
|
| 11 |
+
{%- set toolcalls_end_token = '</tool_calls{}>'.format(HYTK) %}
|
| 12 |
+
{%- set toolcall_begin_token = '<tool_call{}>'.format(HYTK) %}
|
| 13 |
+
{%- set toolcall_end_token = '</tool_call{}>'.format(HYTK) %}
|
| 14 |
+
{%- set toolsep_token = '<tool_sep{}>'.format(HYTK) %}
|
| 15 |
+
{%- set argkey_begin_token = '<arg_key{}>'.format(HYTK) %}
|
| 16 |
+
{%- set argkey_end_token = '</arg_key{}>'.format(HYTK) %}
|
| 17 |
+
{%- set argvalue_begin_token = '<arg_value{}>'.format(HYTK) %}
|
| 18 |
+
{%- set argvalue_end_token = '</arg_value{}>'.format(HYTK) %}
|
| 19 |
+
{%- set toolresponses_begin_token = '<tool_responses{}>'.format(HYTK) %}
|
| 20 |
+
{%- set toolresponses_end_token = '</tool_responses{}>'.format(HYTK) %}
|
| 21 |
+
{%- set toolresponse_begin_token = '<tool_response{}>'.format(HYTK) %}
|
| 22 |
+
{%- set toolresponse_end_token = '</tool_response{}>'.format(HYTK) %}
|
| 23 |
+
{%- set reasoning_mode_token = '<|reasoning_mode{}|>'.format(HYTK) %}
|
| 24 |
+
|
| 25 |
+
{#- ----------‑‑‑ hyperparameters variables ‑‑‑---------- -#}
|
| 26 |
+
{%- if not add_generation_prompt is defined %}
|
| 27 |
+
{%- set add_generation_prompt = false %}
|
| 28 |
+
{%- endif %}
|
| 29 |
+
{%- if not preserved_thinking is defined %}
|
| 30 |
+
{%- if not tools %}
|
| 31 |
+
{%- set preserved_thinking = false %}
|
| 32 |
+
{%- else %}
|
| 33 |
+
{%- set preserved_thinking = true %}
|
| 34 |
+
{%- endif %}
|
| 35 |
+
{%- endif %}
|
| 36 |
+
{%- if not is_training is defined %}
|
| 37 |
+
{%- set is_training = false %}
|
| 38 |
+
{%- endif %}
|
| 39 |
+
|
| 40 |
+
{%- if not reasoning_effort is defined %}
|
| 41 |
+
{%- set reasoning_effort = 'no_think' %}
|
| 42 |
+
{%- elif reasoning_effort not in ['high', 'low', 'no_think'] %}
|
| 43 |
+
{%- if reasoning_effort is none %}
|
| 44 |
+
{{- raise_exception('reasoning_effort error : None, should be no_think/low/high') }}
|
| 45 |
+
{%- else %}
|
| 46 |
+
{{- raise_exception('reasoning_effort error : ' + reasoning_effort + ', should be no_think/low/high') }}
|
| 47 |
+
{%- endif %}
|
| 48 |
+
{%- endif %}
|
| 49 |
+
|
| 50 |
+
{%- if fallback_strategy is defined and fallback_strategy == 'reasoning_toolcall_retry' %}
|
| 51 |
+
{%- set reasoning_effort = 'high' %}
|
| 52 |
+
{%- set add_generation_prompt = false %}
|
| 53 |
+
{%- endif %}
|
| 54 |
+
{%- if not raw_last_assistant is defined %}
|
| 55 |
+
{%- set raw_last_assistant = false %}
|
| 56 |
+
{%- endif %}
|
| 57 |
+
|
| 58 |
+
{%- macro visible_text(content) -%}
|
| 59 |
+
{%- if content is string -%}
|
| 60 |
+
{{- content }}
|
| 61 |
+
{%- elif content is iterable and content is not mapping -%}
|
| 62 |
+
{%- for item in content -%}
|
| 63 |
+
{%- if item is mapping and item.type == 'text' -%}
|
| 64 |
+
{{- item.text }}
|
| 65 |
+
{%- elif item is string -%}
|
| 66 |
+
{{- item }}
|
| 67 |
+
{%- endif -%}
|
| 68 |
+
{%- endfor -%}
|
| 69 |
+
{%- elif content is none -%}
|
| 70 |
+
{{- '' }}
|
| 71 |
+
{%- else -%}
|
| 72 |
+
{{- content }}
|
| 73 |
+
{%- endif -%}
|
| 74 |
+
{%- endmacro -%}
|
| 75 |
+
|
| 76 |
+
{%- set ns = namespace(last_user_index=-1) %}
|
| 77 |
+
{%- set sp_ns = namespace(system_prompt='', is_first_sp=true) %}
|
| 78 |
+
{%- for message in messages %}
|
| 79 |
+
{%- if message['role'] == 'system' %}
|
| 80 |
+
{%- set sp_ns.system_prompt = sp_ns.system_prompt + visible_text(message['content']) %}
|
| 81 |
+
{%- endif %}
|
| 82 |
+
{%- if message['role'] == 'user' %}
|
| 83 |
+
{%- set ns.last_user_index = loop.index0 %}
|
| 84 |
+
{%- endif %}
|
| 85 |
+
{%- endfor %}
|
| 86 |
+
{%- if reasoning_effort is defined and reasoning_effort is string and reasoning_effort != '' and not tools %}
|
| 87 |
+
{%- set sp_ns.system_prompt = sp_ns.system_prompt + reasoning_mode_token + 'reasoning_effort:' + reasoning_effort %}
|
| 88 |
+
{%- endif %}
|
| 89 |
+
{{- bos_token }}
|
| 90 |
+
{{- sp_ns.system_prompt }}
|
| 91 |
+
{%- if tools %}
|
| 92 |
+
{%- if sp_ns.system_prompt != '' %}
|
| 93 |
+
{{- '\n\n# Tools\n\nYou may call one or more functions to assist with the user query.' }}
|
| 94 |
+
{%- else %}
|
| 95 |
+
{{- '# Tools\n\nYou may call one or more functions to assist with the user query.' }}
|
| 96 |
+
{%- endif %}
|
| 97 |
+
{{- '\n\nYou are provided with function signatures within <tools></tools> XML tags:' }}
|
| 98 |
+
{{- '\n<tools>\n' }}
|
| 99 |
+
{%- for tool in tools %}
|
| 100 |
+
{%- if loop.index0 > 0 %}
|
| 101 |
+
{{- '\n' }}
|
| 102 |
+
{%- endif %}
|
| 103 |
+
{{- tool | tojson }}
|
| 104 |
+
{%- endfor %}
|
| 105 |
+
{{- '\n</tools>\n\n' }}
|
| 106 |
+
{{- 'For function call returns, you should first print ' + toolcalls_begin_token + '\n' }}
|
| 107 |
+
{{- 'For each function call, you should return object like:\n' }}
|
| 108 |
+
{{- toolcall_begin_token + '{function-name}' + toolsep_token + '\n' }}
|
| 109 |
+
{{- argkey_begin_token + '{arg-key-1}' + argkey_end_token + '\n' }}
|
| 110 |
+
{{- argvalue_begin_token + '{arg-value-1}' + argvalue_end_token + '\n' }}
|
| 111 |
+
{{- argkey_begin_token + '{arg-key-2}' + argkey_end_token + '\n' }}
|
| 112 |
+
{{- argvalue_begin_token + '{arg-value-2}' + argvalue_end_token + '\n' }}
|
| 113 |
+
{{- '...\n' }}
|
| 114 |
+
{{- toolcall_end_token + '\n' }}
|
| 115 |
+
{%- if reasoning_effort is defined and reasoning_effort is string and reasoning_effort != '' %}
|
| 116 |
+
{{- 'At the end of function call returns, you should print ' + toolcalls_end_token + reasoning_mode_token + 'reasoning_effort:' + reasoning_effort }}
|
| 117 |
+
{%- else %}
|
| 118 |
+
{{- 'At the end of function call returns, you should print ' + toolcalls_end_token }}
|
| 119 |
+
{%- endif %}
|
| 120 |
+
{%- endif %}
|
| 121 |
+
|
| 122 |
+
{%- set prev_ns = namespace(is_tool=false, is_tool_first=true) %}
|
| 123 |
+
{%- set last_ns = namespace(last_is_assistant=false) %}
|
| 124 |
+
{%- for message in messages %}
|
| 125 |
+
{%- if message['role'] == 'user' %}
|
| 126 |
+
{%- if prev_ns.is_tool %}
|
| 127 |
+
{{- toolresponses_end_token }}
|
| 128 |
+
{%- endif %}
|
| 129 |
+
{{- user_token + visible_text(message['content']) }}
|
| 130 |
+
{%- set prev_ns.is_tool = false %}
|
| 131 |
+
{%- endif %}
|
| 132 |
+
{%- if message['role'] == 'assistant' %}
|
| 133 |
+
{%- if is_training %}
|
| 134 |
+
{%- if 'reasoning_content' in message and message['reasoning_content'] is string %}
|
| 135 |
+
{%- set content = think_begin_token + message['reasoning_content'] + think_end_token + visible_text(message['content']) %}
|
| 136 |
+
{%- else %}
|
| 137 |
+
{%- set content = think_begin_token + think_end_token + visible_text(message['content']) %}
|
| 138 |
+
{%- endif %}
|
| 139 |
+
{%- else %}
|
| 140 |
+
{%- if ((preserved_thinking is defined and preserved_thinking) or loop.index0 > ns.last_user_index) and 'reasoning_content' in message and message['reasoning_content'] is string %}
|
| 141 |
+
{%- set content = think_begin_token + message['reasoning_content'] + think_end_token + visible_text(message['content']) %}
|
| 142 |
+
{%- else %}
|
| 143 |
+
{%- set content = think_begin_token + think_end_token + visible_text(message['content']) %}
|
| 144 |
+
{%- endif %}
|
| 145 |
+
{%- endif %}
|
| 146 |
+
{%- if prev_ns.is_tool %}
|
| 147 |
+
{{- toolresponses_end_token }}
|
| 148 |
+
{%- endif %}
|
| 149 |
+
{{- assistant_token }}
|
| 150 |
+
{%- if message['tool_calls'] is defined and message['tool_calls'] %}
|
| 151 |
+
{%- set prev_ns.is_tool_first = true %}
|
| 152 |
+
{{- content }}
|
| 153 |
+
{{- toolcalls_begin_token + '\n' }}
|
| 154 |
+
{%- for tool in message['tool_calls'] %}
|
| 155 |
+
{%- set arguments = tool['function']['arguments'] %}
|
| 156 |
+
{{- toolcall_begin_token + tool['function']['name'] + toolsep_token + '\n' }}
|
| 157 |
+
{%- for key, value in arguments.items() %}
|
| 158 |
+
{{- argkey_begin_token + key + argkey_end_token + '\n' }}
|
| 159 |
+
{%- if value is not string %}
|
| 160 |
+
{%- set value = value | tojson(ensure_ascii=False) %}
|
| 161 |
+
{%- endif %}
|
| 162 |
+
{{- argvalue_begin_token + value + argvalue_end_token + '\n' }}
|
| 163 |
+
{%- endfor %}
|
| 164 |
+
{{- toolcall_end_token + '\n' }}
|
| 165 |
+
{%- endfor %}
|
| 166 |
+
{{- toolcalls_end_token + eos_token }}
|
| 167 |
+
{%- else %}
|
| 168 |
+
{%- if loop.last and raw_last_assistant %}
|
| 169 |
+
{{- visible_text(message['content']) }}
|
| 170 |
+
{%- elif not loop.last or is_training %}
|
| 171 |
+
{{- content + eos_token }}
|
| 172 |
+
{%- else %}
|
| 173 |
+
{{- content }}
|
| 174 |
+
{%- endif %}
|
| 175 |
+
{%- endif %}
|
| 176 |
+
{%- set prev_ns.is_tool = false %}
|
| 177 |
+
{%- endif %}
|
| 178 |
+
{%- if message['role'] == 'tool' %}
|
| 179 |
+
{%- set prev_ns.is_tool = true %}
|
| 180 |
+
{%- if prev_ns.is_tool_first %}
|
| 181 |
+
{{- toolresponses_begin_token + '\n' }}
|
| 182 |
+
{%- set prev_ns.is_tool_first = false %}
|
| 183 |
+
{%- endif %}
|
| 184 |
+
{{- toolresponse_begin_token + '\n' + visible_text(message['content']) + '\n' + toolresponse_end_token + '\n' }}
|
| 185 |
+
{%- endif %}
|
| 186 |
+
{%- if loop.last and message['role'] == 'assistant' %}
|
| 187 |
+
{%- set last_ns.last_is_assistant = true %}
|
| 188 |
+
{%- endif %}
|
| 189 |
+
|
| 190 |
+
{%- endfor %}
|
| 191 |
+
{%- if prev_ns.is_tool %}
|
| 192 |
+
{{- toolresponses_end_token }}
|
| 193 |
+
{%- endif %}
|
| 194 |
+
{%- if add_generation_prompt %}
|
| 195 |
+
{%- if not last_ns.last_is_assistant %}
|
| 196 |
+
{%- if reasoning_effort is defined and reasoning_effort in ['low', 'high'] %}
|
| 197 |
+
{{- assistant_token + think_begin_token }}
|
| 198 |
+
{%- elif reasoning_effort is defined and reasoning_effort == 'no_think' %}
|
| 199 |
+
{{- assistant_token + think_begin_token + think_end_token }}
|
| 200 |
+
{%- else %}
|
| 201 |
+
{{- assistant_token }}
|
| 202 |
+
{%- endif %}
|
| 203 |
+
{%- endif %}
|
| 204 |
+
{%- endif %}
|
config.json
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"HYV3ForCausalLM"
|
| 4 |
+
],
|
| 5 |
+
"bos_token_id": 120000,
|
| 6 |
+
"enable_attention_fp32_softmax": false,
|
| 7 |
+
"enable_lm_head_fp32": true,
|
| 8 |
+
"enable_moe_fp32_combine": false,
|
| 9 |
+
"eod_token_id": 120026,
|
| 10 |
+
"eos_token_id": 120025,
|
| 11 |
+
"expert_hidden_dim": 1536,
|
| 12 |
+
"moe_intermediate_size": 1536,
|
| 13 |
+
"first_k_dense_replace": 1,
|
| 14 |
+
"head_dim": 128,
|
| 15 |
+
"hidden_act": "silu",
|
| 16 |
+
"hidden_size": 4096,
|
| 17 |
+
"initializer_range": 0.006,
|
| 18 |
+
"intermediate_size": 13312,
|
| 19 |
+
"max_position_embeddings": 262144,
|
| 20 |
+
"model_type": "hy_v3",
|
| 21 |
+
"moe_router_enable_expert_bias": true,
|
| 22 |
+
"moe_router_use_sigmoid": true,
|
| 23 |
+
"num_attention_heads": 64,
|
| 24 |
+
"num_experts": 192,
|
| 25 |
+
"num_experts_per_tok": 8,
|
| 26 |
+
"num_hidden_layers": 80,
|
| 27 |
+
"num_key_value_heads": 8,
|
| 28 |
+
"num_shared_experts": 1,
|
| 29 |
+
"output_router_logits": true,
|
| 30 |
+
"pad_token_id": 120002,
|
| 31 |
+
"qk_norm": true,
|
| 32 |
+
"rms_norm_eps": 1e-05,
|
| 33 |
+
"rope_parameters": {
|
| 34 |
+
"rope_theta": 11158840.0,
|
| 35 |
+
"rope_type": "default"
|
| 36 |
+
},
|
| 37 |
+
"route_norm": true,
|
| 38 |
+
"router_scaling_factor": 2.826,
|
| 39 |
+
"sep_token_id": 120007,
|
| 40 |
+
"tie_word_embeddings": false,
|
| 41 |
+
"transformers_version": "5.6.0",
|
| 42 |
+
"use_cache": true,
|
| 43 |
+
"use_grouped_mm": false,
|
| 44 |
+
"vocab_size": 120832,
|
| 45 |
+
"num_nextn_predict_layers": 1,
|
| 46 |
+
"quantization_config": {
|
| 47 |
+
"activation_scheme": "static",
|
| 48 |
+
"ignored_layers": [
|
| 49 |
+
"lm_head",
|
| 50 |
+
"model.embed_tokens"
|
| 51 |
+
],
|
| 52 |
+
"quant_method": "fp8",
|
| 53 |
+
"kv_cache_scheme": "static"
|
| 54 |
+
}
|
| 55 |
+
}
|
generation_config.json
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token_id": 120000,
|
| 3 |
+
"do_sample": true,
|
| 4 |
+
"eos_token_id": 120025,
|
| 5 |
+
"pad_token_id": 120002,
|
| 6 |
+
"temperature": 0.9,
|
| 7 |
+
"top_k": -1,
|
| 8 |
+
"top_p": 1,
|
| 9 |
+
"transformers_version": "5.6.0"
|
| 10 |
+
}
|
model-00002-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4be0522a52d42f348a896d44ddd79b8c4b7d8eddf40182f2439360e63c5a6600
|
| 3 |
+
size 3624016320
|
model-00005-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:23cd016378faa2e360a7bae1890206adf8fac6afe3dab70a9bac9318201299a2
|
| 3 |
+
size 3624016320
|
model-00007-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:70f38d53b6ed824494d8c773f44003869f7659c6ee5dca300b37d06422cbb7db
|
| 3 |
+
size 3624015168
|
model-00011-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:13d3f155987648fdcea2d0d1a6111df8fdaf76cb8a40b2c46a4c0a412543e065
|
| 3 |
+
size 3624016320
|
model-00012-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bd73c45cf3cabe4c8c64a603189ac27a7ae8e3b30d8cd7e182861eaee097a612
|
| 3 |
+
size 1031799590
|
model-00013-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:afd5a6ff98b17d40e9a0ad17c03170307482c9979732a10d8711afe0dc858c42
|
| 3 |
+
size 3624015168
|
model-00015-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d823434369efc5bffb46ea43b098e0261e6e0b9d0a849b952bdd9dcc7e8cf466
|
| 3 |
+
size 3624016320
|
model-00016-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fcae250c7b467ef854e24540c1b60e9830057c9ebf4de6caa2b8e7f51d74ec3b
|
| 3 |
+
size 3624016320
|
model-00019-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1c122e49a584241b37348138b8a2fa33557f230269e5e1362fdc38e861a9031c
|
| 3 |
+
size 3624015168
|
model-00020-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d892f3bf7b7f28f5d588ae101e1f6f1e361d48162e7ee072665d7ca178c63741
|
| 3 |
+
size 3624016320
|
model-00024-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c8aec27e294739d096c9081bdd42a909c7f1b4b2847da35e1421ae9648ee40cb
|
| 3 |
+
size 543176570
|
model-00026-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c43619a1630381b2a66e14ad5a0dadde5a1bd1b1907b7168aa41e0fd9693ba1c
|
| 3 |
+
size 3624016320
|
model-00030-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cfa7cd2ca6b6ab694a41626b5148bdfc92920f7e999bf88fb3a0a6391efc0465
|
| 3 |
+
size 549328424
|
model-00031-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6bd9aee94a8454d50646071656ea8dfc7daeb23351bd0a86b792715d7ce248cf
|
| 3 |
+
size 3624015168
|
model-00032-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:86c30ee8d5bcc25e170c4c808b177e32b1329fc9d4c0bf535f86cb718b90fddc
|
| 3 |
+
size 3624016320
|
model-00033-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4e55b8965cde45387d1d390c7f5ac9a532dfcc918e5173e3387b8be629c7c57c
|
| 3 |
+
size 3624016320
|
model-00040-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4ced5a625ed7f6ef557028a8f78e5b32a41155cdbb7a5deedb3b9aa9a44b41c4
|
| 3 |
+
size 3624016320
|
model-00041-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ffb7ea35af8ddaf129ff82e8edd1168bfdc204f2156e28ae2696a3ba1b866d71
|
| 3 |
+
size 3624016320
|
model-00042-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a0333b1c4f85669f4325143407ba04b0676c35b8832f35592bab42507bc1a55d
|
| 3 |
+
size 528488456
|
model-00047-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5de8b7ef7dfb59d9962ae3a6ffd5d82f73107364dd624767840fc3ef366307ca
|
| 3 |
+
size 3624016320
|
model-00049-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:56429fcf5f87e7c2f06bf0449bbdcf044d49be640a3a7d6bb4c7c3c5afbac118
|
| 3 |
+
size 3624015168
|
model-00056-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:177bdda2146e0a09626dfa424dd6d3998d672b77b6cfc4c9809c3ce296ebc171
|
| 3 |
+
size 3624016320
|
model-00058-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:889f9c1d1a1fed76d07d33e57dc0bf078358e5da20b0358393ccd3a39a6c41a7
|
| 3 |
+
size 3624016320
|
model-00059-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:faa96414575b45bc76959aac09209c1ba308815d5233d9e2db2ca5f261ebb0ef
|
| 3 |
+
size 3624016320
|
model-00060-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3af9d726b77d07b0e666dfe9a6d4c2ca35d51365398321db84934879142335f8
|
| 3 |
+
size 511708810
|
model-00061-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6f6dc55402d9b67ed94baafbe5bb50d1075471f33e0b2da6096f2841aba4e96b
|
| 3 |
+
size 3624016320
|
model-00065-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:384c2872e04daf41ada905f668e582d39e4a2b6583f04fe615ac0f40a792242e
|
| 3 |
+
size 3624016320
|
model-00071-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:575657ea40051397746b5fabb7ec3727939c94fbb352f2a3baf111e9e486fc2b
|
| 3 |
+
size 3624016320
|
model-00073-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:894ef334e46857558e8f92127fc5d3b0fd1bab02de7e05b66799c4a72582f85e
|
| 3 |
+
size 3624016320
|
model-00074-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3fd01b03a205a29f26f7240d438a4ca9fca1f6ac118f7b922a8070789ed96b1b
|
| 3 |
+
size 3624016320
|
model-00075-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0212036df39f9308147d897d0d763ad549c5f9b7808f2597fe05da508c7e8b74
|
| 3 |
+
size 3624016320
|
model-00077-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d386176ea72fc700c79e43cb6d06d358e22e8048f34058519a331c1d830eaa41
|
| 3 |
+
size 3624016320
|
model-00078-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bf12543bc9331c9e3bf4fd36ac504dc05cf86de398dad3b2c955f946b63c5501
|
| 3 |
+
size 503322842
|
model-00079-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4265258bce76165caed22aeccc42badad66816b6b680abaeadc28778ad18e374
|
| 3 |
+
size 3624016320
|
model-00080-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:166115748546ff6699a3c031308b8afbe6568a7db41809e483dff68816267bea
|
| 3 |
+
size 3624016320
|
model-00082-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ff8f68b6dfa26e4f656c3bd0a733fcc4185a70a7fd36e8a8040f514e320b0090
|
| 3 |
+
size 3624016320
|
model-00084-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5a6d5ae7a518868a91db77091ddda02edba5da1ad372d90ca5374afdd4a32301
|
| 3 |
+
size 503324384
|
model-00085-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:45d8a2caea51fd88535968b23598a0313afd150bf654ed595172361d4e82ddd5
|
| 3 |
+
size 3624016320
|
model-00088-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4cb3f873529555cff124554d67ece8ebf97d9222de2a4588e82d601e3b89115d
|
| 3 |
+
size 3624016320
|
model-00090-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1446b1e24c106d0370bf5e8def440fa3d604a0ce35315485926709f93dc02230
|
| 3 |
+
size 499128062
|
model-00093-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:98b575f27c1cb0b349f5e3fbdbbf9f061859d9f22600a82b110b940ad3534f99
|
| 3 |
+
size 3624016320
|
model-00096-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6f52e74fe2ac9f5145f77b1a52e42432431c6cdf1c55c46241af0a0763a385d0
|
| 3 |
+
size 2416010376
|
model-00097-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a12d3065171fbc4e28757c9f14b046b7f9fde80c87b47b30b5d753bd74612646
|
| 3 |
+
size 1208005344
|
model-00101-of-00101.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a3560418718a16323073ff35a4b47e38fc3288307b4d5e47c025558405059a67
|
| 3 |
+
size 5361104
|
model.safetensors.index.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:50156f496352ea45d328b9995edef44eab9aece876ec67afbb6d0db5fb481d54
|
| 3 |
+
size 13230997
|
tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|