Hugging Face's logo

meituan-longcat
/
LongCat-Flash-Lite

Text Generation
Safetensors
Transformers
LongCat-Flash-Lite
conversational
custom_code
Eval Results
Model card Files
xet
 Community
8

Update model files by removing redundant n-gram embededding weight duplication

#3
by LongCat0830 - opened
base: refs/heads/main
←
from: refs/pr/3
Discussion Files changed
+121
-113
This view is limited to 50 files because it contains too many changes. See the raw diff here.
Files changed (50) hide show
  1. .gitignore +1 -0
  2. README.md +3 -28
  3. generation_config.json +1 -5
  4. model-00000-of-00026.safetensors → model-00000-of-00038.safetensors +2 -2
  5. model-00004-of-00026.safetensors → model-00001-of-00038.safetensors +2 -2
  6. model-00001-of-00026.safetensors → model-00002-of-00038.safetensors +2 -2
  7. model-00002-of-00026.safetensors → model-00003-of-00038.safetensors +2 -2
  8. model-00003-of-00026.safetensors → model-00004-of-00038.safetensors +2 -2
  9. model-00005-of-00026.safetensors +0 -3
  10. model-00005-of-00038.safetensors +3 -0
  11. model-00006-of-00026.safetensors +0 -3
  12. model-00006-of-00038.safetensors +3 -0
  13. model-00007-of-00026.safetensors +0 -3
  14. model-00007-of-00038.safetensors +3 -0
  15. model-00008-of-00026.safetensors +0 -3
  16. model-00008-of-00038.safetensors +3 -0
  17. model-00009-of-00026.safetensors +0 -3
  18. model-00009-of-00038.safetensors +3 -0
  19. model-00010-of-00026.safetensors +0 -3
  20. model-00010-of-00038.safetensors +3 -0
  21. model-00011-of-00026.safetensors +0 -3
  22. model-00011-of-00038.safetensors +3 -0
  23. model-00012-of-00026.safetensors +0 -3
  24. model-00012-of-00038.safetensors +3 -0
  25. model-00013-of-00026.safetensors +0 -3
  26. model-00013-of-00038.safetensors +3 -0
  27. model-00014-of-00026.safetensors +0 -3
  28. model-00014-of-00038.safetensors +3 -0
  29. model-00015-of-00026.safetensors +0 -3
  30. model-00015-of-00038.safetensors +3 -0
  31. model-00016-of-00026.safetensors +0 -3
  32. model-00016-of-00038.safetensors +3 -0
  33. model-00017-of-00026.safetensors +0 -3
  34. model-00017-of-00038.safetensors +3 -0
  35. model-00018-of-00026.safetensors +0 -3
  36. model-00018-of-00038.safetensors +3 -0
  37. model-00019-of-00026.safetensors +0 -3
  38. model-00019-of-00038.safetensors +3 -0
  39. model-00020-of-00026.safetensors +0 -3
  40. model-00020-of-00038.safetensors +3 -0
  41. model-00021-of-00026.safetensors +0 -3
  42. model-00021-of-00038.safetensors +3 -0
  43. model-00022-of-00026.safetensors +0 -3
  44. model-00022-of-00038.safetensors +3 -0
  45. model-00023-of-00026.safetensors +0 -3
  46. model-00023-of-00038.safetensors +3 -0
  47. model-00024-of-00026.safetensors +0 -3
  48. model-00024-of-00038.safetensors +3 -0
  49. model-00025-of-00026.safetensors +0 -3
  50. model-00025-of-00038.safetensors +3 -0
.gitignore CHANGED
@@ -0,0 +1 @@
 
 
1
+ *.safetensors
README.md CHANGED
@@ -39,9 +39,8 @@ tags:
39
  </a>
40
  </div>
41
 
42
- <p align="center">
43
- <a href="https://arxiv.org/abs/2601.21204"><b>Tech Report</b>&nbsp;📄</a>
44
- </p>
45
 
46
  ## Model Introduction
47
  We introduce LongCat-Flash-Lite, a non-thinking 68.5B parameter Mixture-of-Experts (MoE) model with approximately 3B activated parameters, supporting a 256k context length through the YaRN method. Building upon the LongCat-Flash architecture, LongCat-Flash-Lite distinguishes itself through the integration of an **N-gram embedding** table designed to enhance both model performance and inference speed. Despite allocating over 30B parameters to embeddings, LongCat-Flash-Lite not only outperforms parameter-equivalent MoE baselines but also demonstrates exceptional competitiveness against existing models of comparable scale, particularly in the agentic and coding domains.
@@ -59,7 +58,7 @@ In contrast to FFN-based experts, the N-gram embedding table inherently mitigate
59
  #### 🌟 Strong Agentic and Coding Performance
60
  LongCat-Flash-Lite demonstrates robust capabilities in agentic tool use and coding proficiency that are highly competitive relative to its model scale.
61
 
62
- Please refer to our [technical report](https://arxiv.org/abs/2601.21204) for details!
63
 
64
  ## Evaluation Results
65
 
@@ -181,14 +180,6 @@ response = tokenizer.decode(output_ids, skip_special_tokens=True).strip("\n")
181
  parsed_message = parse_model_response(response, tools)
182
  ```
183
  See [`parse_model_response.py`](./parse_model_response.py) for detailed implementation and examples.
184
-
185
- <br>
186
-
187
- Recommended Sampling Setting:
188
- ```shell
189
- { "repetition_penalty": 1.06, "temperature": 0.7, "top_p": 0.95, "top_k": 4 }
190
- ```
191
-
192
  ## Deployment
193
 
194
  We have implemented basic adaptations in SGLang ([PR](https://github.com/sgl-project/sglang/pull/17838)) to support the deployment of LongCat-Flash-Lite.
@@ -238,22 +229,6 @@ It is the responsibility of developers and downstream users to understand and co
238
  Nothing in this Model Card should be interpreted as altering or restricting the terms of the MIT License under which the model is released.
239
 
240
 
241
- ## Citation
242
-
243
- We kindly encourage citation of our work if you find it useful.
244
-
245
- ```
246
- @misc{liu2026scalingembeddingsoutperformsscaling,
247
- title={Scaling Embeddings Outperforms Scaling Experts in Language Models},
248
- author={Hong Liu and Jiaqi Zhang and Chao Wang and Xing Hu and Linkun Lyu and Jiaqi Sun and Xurui Yang and Bo Wang and Fengcun Li and Yulei Qian and Lingtong Si and Yerui Sun and Rumei Li and Peng Pei and Yuchen Xie and Xunliang Cai},
249
- year={2026},
250
- eprint={2601.21204},
251
- archivePrefix={arXiv},
252
- primaryClass={cs.CL},
253
- url={https://arxiv.org/abs/2601.21204},
254
- }
255
- ```
256
-
257
 
258
  ## Contact
259
  Please contact us at <a href="mailto:longcat-team@meituan.com">longcat-team@meituan.com</a> or open an issue if you have any questions.
 
39
  </a>
40
  </div>
41
 
42
+
43
+
 
44
 
45
  ## Model Introduction
46
  We introduce LongCat-Flash-Lite, a non-thinking 68.5B parameter Mixture-of-Experts (MoE) model with approximately 3B activated parameters, supporting a 256k context length through the YaRN method. Building upon the LongCat-Flash architecture, LongCat-Flash-Lite distinguishes itself through the integration of an **N-gram embedding** table designed to enhance both model performance and inference speed. Despite allocating over 30B parameters to embeddings, LongCat-Flash-Lite not only outperforms parameter-equivalent MoE baselines but also demonstrates exceptional competitiveness against existing models of comparable scale, particularly in the agentic and coding domains.
 
58
  #### 🌟 Strong Agentic and Coding Performance
59
  LongCat-Flash-Lite demonstrates robust capabilities in agentic tool use and coding proficiency that are highly competitive relative to its model scale.
60
 
61
+ Please refer to our [technical report](./tech_report.pdf) for details!
62
 
63
  ## Evaluation Results
64
 
 
180
  parsed_message = parse_model_response(response, tools)
181
  ```
182
  See [`parse_model_response.py`](./parse_model_response.py) for detailed implementation and examples.
 
 
 
 
 
 
 
 
183
  ## Deployment
184
 
185
  We have implemented basic adaptations in SGLang ([PR](https://github.com/sgl-project/sglang/pull/17838)) to support the deployment of LongCat-Flash-Lite.
 
229
  Nothing in this Model Card should be interpreted as altering or restricting the terms of the MIT License under which the model is released.
230
 
231
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
232
 
233
  ## Contact
234
  Please contact us at <a href="mailto:longcat-team@meituan.com">longcat-team@meituan.com</a> or open an issue if you have any questions.
generation_config.json CHANGED
@@ -3,9 +3,5 @@
3
  "bos_token_id": 1,
4
  "eos_token_id": 2,
5
  "pad_token_id": 3,
6
- "transformers_version": "4.57.6",
7
- "repetition_penalty": 1.06,
8
- "temperature": 0.7,
9
- "top_p": 0.95,
10
- "top_k": 4
11
  }
 
3
  "bos_token_id": 1,
4
  "eos_token_id": 2,
5
  "pad_token_id": 3,
6
+ "transformers_version": "4.55.0"
 
 
 
 
7
  }
model-00000-of-00026.safetensors → model-00000-of-00038.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:19412c9358bc484c8c9af94bb2e135a6939a39441488c6c042b16261bfe746ad
3
- size 5313604160
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58fa5258626248582925148ffaec79f752045d76b3073eff9035f78244174e27
3
+ size 5289947960
model-00004-of-00026.safetensors → model-00001-of-00038.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bd0ca68e274a10a89944da1984cf583adb7b51a9f54782f4e72ba6e6305aa7c8
3
- size 5315503392
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4862c4cc80ac36701781ed02a84822a152e7a285b57cf1260f8fd23209d99cce
3
+ size 5289947968
model-00001-of-00026.safetensors → model-00002-of-00038.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bff23594192d36756bdac96c7ee67dd641b8969f7d37c12d2a68a5e3ed9b8d87
3
- size 5313604768
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:244ba1cc3f7da33f1b3bfce6eec2a38c1b1410eb1aea22c7796bd0e2bec53dd5
3
+ size 5289946944
model-00002-of-00026.safetensors → model-00003-of-00038.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3dcfc95fd273d4a8a57674916db168d977c10cddb87e0cc5b1fd5fc682ad121f
3
- size 5313604016
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73aebcb63519a519befe68cb1c26b05bcc310ee14dfac44ca49b1f18d82c69bf
3
+ size 5289946936
model-00003-of-00026.safetensors → model-00004-of-00038.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b465510bc231ab86c9f7403bd82e03a12b72b323cbe319547453b786577791d1
3
- size 5313604136
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ed6f0578b36b47680355b736a48ab732380b77d29d29f8c0e18386397523ca59
3
+ size 5289945920
model-00005-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:edfcfed05027a48b2cab645e44b7dc8ec0544c1a64a0277bb0b76de703e85913
3
- size 5313604136
 
 
 
 
model-00005-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:411ac9baf753252971f9b7072ac08c950ff66c2acd410207a5c48480d898f532
3
+ size 5289945912
model-00006-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:29959ee6a47ed392e05966031f46a589d9c41cfce369ee0be0f32a83246bc3ba
3
- size 5313604136
 
 
 
 
model-00006-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:367a91bc240583a8a701b41ded917297cebd04113cdd13c3b1703be396a1fdab
3
+ size 5289944880
model-00007-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:ba85ee575bdafb34270bbab3f7256c87178c3a2c05a5e637bb9ee83e565fdf2b
3
- size 5313604760
 
 
 
 
model-00007-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a66b7d59d3bb94d1874dfa7fc7b91afa2632af4d2dd19b14ee503596b38f4e2
3
+ size 5289944888
model-00008-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:0533404a0b7ee44b0ade17bf9b9c00969e37cf864ff5d5f7818b030c702abb06
3
- size 5313604216
 
 
 
 
model-00008-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e10c54ec49ab87bb7abf1b9df07b2767a37a26ddf1c62014d464144c01ff8ea9
3
+ size 5289943856
model-00009-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:b65ca6876f65b690af110232c4d460c912beab21c195c4e0170e9f01ca1d8bb2
3
- size 5315105048
 
 
 
 
model-00009-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99d1e0f997b7f6e4cc7c2245649f0facbcef377bf88f883a31f2faa557cd8ac9
3
+ size 5289943880
model-00010-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:8d1032fedf9f1029544ac164405278774fd2f9e3b8399d8466b797d08f756fc6
3
- size 5315104024
 
 
 
 
model-00010-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f964d48cf6d02f34fc4ba242bbe1a11d83c4df5fa660695f9fee53b53b84eab
3
+ size 5289942840
model-00011-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:1e13ef0f626c36a424cbed2e4181b853384de4cb8ad0660b65084c7408cfc729
3
- size 5315103008
 
 
 
 
model-00011-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:15b59cf9ecdb92b277d5b7a10f25f35db7373daa5050d55a65f8be2a1f283624
3
+ size 5289942848
model-00012-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d1c91f60ac661af3c195337d4ca771e274947ef8e7f258607b79e0e19a02c30d
3
- size 5315190600
 
 
 
 
model-00012-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:70511491f956b62c2418781e3d9db56df42510e807735ec6d8f395ccb3340e82
3
+ size 5289941824
model-00013-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:2b6994f58b462df2107a899881309116d8b668e78588786e33fdcda5deee7308
3
- size 5315190584
 
 
 
 
model-00013-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e614af5d55350bda182d21e2e39a69c4436f416f301bc536a8280d6e5fbbc465
3
+ size 5289941816
model-00014-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:a646727dce5f710bb23f38b01faae5a97e22966c0152a7f538ba711ec730ea9e
3
- size 5315190384
 
 
 
 
model-00014-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ecba8f4358c5fa7a84c4e2a38b32e1c579e0601d661822288cd4ac629b58f71
3
+ size 5289940800
model-00015-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:2fe62f582173fe86bac2a206b5b0d358a745355136c968401203305456b3e220
3
- size 5315199112
 
 
 
 
model-00015-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c549541ab83b5013190c191de4db8ff7dcabc5fa78cb68796970dabda4a14636
3
+ size 5289940792
model-00016-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:97d63ab634d0d6ee498b3e97db332c283b7d4c2c4aa65f2d5d949d7451a6c75c
3
- size 5315199120
 
 
 
 
model-00016-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3647fdc2158e2201da25a6e6b93ebc77d52e085f28995920c94346d79f5da845
3
+ size 5289939768
model-00017-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d3cc16384a3b881669a1616091877297f08c6e6fd1a212b79a991f1adc7d3192
3
- size 5315199120
 
 
 
 
model-00017-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:978483a120263813f2f0d1f0a4f741390202d0364e5c449b79e8f0fc05819a1c
3
+ size 5289939776
model-00018-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6130194e56cd991ffc0bcaab6f2ca2198e2a38db30b9cf3eef1833da86b97c80
3
- size 5315199168
 
 
 
 
model-00018-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9372820581b918bfa6866b6628013f331d91946846c65062374750cd5ca16fe9
3
+ size 5289938744
model-00019-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:19b89d889c49b935ba4c6579c2b5f9fd3a5005b97cd719059ac9ee3237f99e7c
3
- size 5315199192
 
 
 
 
model-00019-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c5828059c2f5e3fb067e07232856190a018863fae38a1e1858fc2dd1fb86ca1
3
+ size 5289938752
model-00020-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:2d274ec0a1f29bd9fa060c39de83e8dafd62e5b3882d6f907bf6f7fc40535a4b
3
- size 5314674968
 
 
 
 
model-00020-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:94f4f1b53f863bf2e4fb7f817bfd722c4e3f712180a47b4c173efe86f9f708ff
3
+ size 5289544376
model-00021-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:701ed226f316cd6d608234d2f77714b54fc071445beba1fd9da9375d2115b831
3
- size 5314674952
 
 
 
 
model-00021-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae5f05ffeeeceaa0d09a4703340fd7e7834d87d23d8a7b5f928a3a767a4d29cf
3
+ size 5289937728
model-00022-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6c194582d2c6af1bcf698927aeff47128401c0302047c8465b5ab5882e7e697d
3
- size 5315068040
 
 
 
 
model-00022-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ebae9b3f3ff94b0b36b352ec72a0cc228fbe611dc59b3026a44d932961654791
3
+ size 5289543352
model-00023-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:4a084318a109e2776edd5a0fe71f87db3a513843c1eda6c374eed9469ad2c965
3
- size 5315068048
 
 
 
 
model-00023-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8fee6ecd8a561d353cd135a681252f923dc247d27c624d083b45c21354d4080
3
+ size 5289543352
model-00024-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:b2b2af8fa51ccc188f5c55afac3e3e0b59fb4c713dda44f9ec7820749cd851d6
3
- size 5315199128
 
 
 
 
model-00024-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad229f37d38cd12675367219d16f803f2a07d5fb71bc84c9d9c62598692a4100
3
+ size 5289630936
model-00025-of-00026.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:c7643ca1f768712f319b01e552071b67f8550fb42c633bcee98b5fc40ab5501a
3
- size 5315199232
 
 
 
 
model-00025-of-00038.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:139cc01fc2445267a046da2939d589367d17cc146be8259f69bfabc9e23eca63
3
+ size 5289630912