Asilarkness commited on 28 days ago

Commit

7fcc98b

verified ·

1 Parent(s): e2032a6

Fix short-conv layout: fuse q/k/v conv1d, rename forget_gate keys

Browse files

Files changed (37) hide show

README.md +4 -0
model.safetensors.index.json +0 -0
shard-00001.safetensors +2 -2
shard-00002.safetensors +2 -2
shard-00003.safetensors +2 -2
shard-00006.safetensors +2 -2
shard-00007.safetensors +2 -2
shard-00009.safetensors +2 -2
shard-00011.safetensors +2 -2
shard-00013.safetensors +2 -2
shard-00014.safetensors +2 -2
shard-00016.safetensors +2 -2
shard-00017.safetensors +2 -2
shard-00018.safetensors +2 -2
shard-00020.safetensors +2 -2
shard-00023.safetensors +2 -2
shard-00024.safetensors +2 -2
shard-00025.safetensors +2 -2
shard-00028.safetensors +2 -2
shard-00030.safetensors +2 -2
shard-00032.safetensors +2 -2
shard-00035.safetensors +2 -2
shard-00037.safetensors +2 -2
shard-00038.safetensors +2 -2
shard-00041.safetensors +2 -2
shard-00042.safetensors +2 -2
shard-00044.safetensors +2 -2
shard-00046.safetensors +2 -2
shard-00048.safetensors +2 -2
shard-00049.safetensors +2 -2
shard-00051.safetensors +2 -2
shard-00053.safetensors +2 -2
shard-00055.safetensors +2 -2
shard-00056.safetensors +2 -2
shard-00059.safetensors +2 -2
shard-00060.safetensors +2 -2
shard-00061.safetensors +2 -2

README.md CHANGED Viewed

@@ -81,3 +81,7 @@ and `max_memory` with an offload folder.
   were prioritized during expert selection.
 - Only smoke-tested after pruning (short greedy generations); run your own evals
   (HumanEval, GSM8K, MMLU) before production use.

   were prioritized during expert selection.
 - Only smoke-tested after pruning (short greedy generations); run your own evals
   (HumanEval, GSM8K, MMLU) before production use.
+## Fix 2026-09-03: short-conv remap
+Linear-attention short convolutions were stored as split `q/k/v_conv1d` + flat `A_log`/`dt_bias`; remapped losslessly to the fused `conv1d` + `forget_gate.*` layout expected by `transformers>=5.16` modeling (`fused = cat([q,k,v], dim=0)`). No values changed.

model.safetensors.index.json CHANGED Viewed

The diff for this file is too large to render. See raw diff

shard-00001.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7f5aecf0ec1a7d4028a9c1624eb0ae358179938a2085fba8b3f37638c9446ae2
-size 956668904

 version https://git-lfs.github.com/spec/v1
+oid sha256:95c2e02ce248850e32c7574f895cc7f6f64ca32793b8ddeb4de7e8974cf752fd
+size 956603072

shard-00002.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d6805f2ae09e3bde357b386addedb7797e8db5872362debff2b50d018886aa3e
-size 953363820

 version https://git-lfs.github.com/spec/v1
+oid sha256:d32e306b1dd347ab2dd8bcc45f4c5ea417280d418a726144334a78d34a286002
+size 953429220

shard-00003.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:58428665928b911aa27524d3814ad7de62a41665072fb1d253344d9c9b8409b3
-size 1205798720

 version https://git-lfs.github.com/spec/v1
+oid sha256:c7c61a12e1b4af3b699a87e36709a265fe5932a0d57b1708db7135995e8e02a0
+size 1205798504

shard-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dc5bf8e6c4ea052e8d43e768e867583c5bf02b693fa96144c4e63150aed2de8d
-size 1205798872

 version https://git-lfs.github.com/spec/v1
+oid sha256:093a21bac21fd19a1b3b058aca0f6361a6240f843f8a78999709740581aea23f
+size 1205798656

shard-00007.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:58a18da5f3b338b26bb995128fa0346c3956eb51920ae92750a72c81f44b388c
-size 979285608

 version https://git-lfs.github.com/spec/v1
+oid sha256:1bcbce8051caa2d029c329d6fc92cab3174f1d3695f8ba85e9f78db885e701c1
+size 979285392

shard-00009.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:89cee9385c45a9696c929ee6cc7c53281c2ba3d9690b54f88b97141651d7674e
-size 1077729596

 version https://git-lfs.github.com/spec/v1
+oid sha256:c26f42bfa9bbe5decc931c649a6a0923fb82f5d5d491af656aaa3e3988a2773c
+size 1077729380

shard-00011.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:59a11fe7cc414c1d7558ea8086d89eb27addb99dbfdbf4f0b7818897be05b336
-size 1177484352

 version https://git-lfs.github.com/spec/v1
+oid sha256:8018e8bc7e63c5accaa6e1ee6903b0ce31793f40abe11a5a9658eb41e616f65a
+size 1177484136

shard-00013.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f2877dee40f5000aa17a3ed5bcfbd98a3ed4499f5a665c1cc18d343af3680100
-size 1191641796

 version https://git-lfs.github.com/spec/v1
+oid sha256:c049bda942d106c2818f7584fc18c2620ebac459dfeb9fbbc38c8ffecd7ccf64
+size 1191641572

shard-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3b8b692d045c93dfece60bfe3b2016b4c6387c7339c2c53ee9a1ba15eda308aa
-size 1078385020

 version https://git-lfs.github.com/spec/v1
+oid sha256:5e9fbfd9131eff70c1cb9de998011d2b35abd7d734350efa70e4ec618cbf1bd0
+size 1078384804

shard-00016.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a129b015c873ce036c3fd1b1cd3cf3844d155153f6e5a336046d641bb7c96151
-size 1546854684

 version https://git-lfs.github.com/spec/v1
+oid sha256:afc5444a5f71f6682f80298e5421f2217dbc3e177d7a384a7662d787d7e7e990
+size 1546854468

shard-00017.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4b43f8e6b2b721d333ac2e4174c0df04e4f2f7a5dcce1710fd00abd8ecc7a576
-size 1205798624

 version https://git-lfs.github.com/spec/v1
+oid sha256:c23dc0a100acd77790a2dc971807d163fe3c4f5d9d28d9c562c9c71e7bd8d8e0
+size 1205798408

shard-00018.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:32891999c8b0a40f7fedf4b15fd4d3f4fde8332ff7c769058c18870597ea9665
-size 1149170136

 version https://git-lfs.github.com/spec/v1
+oid sha256:69e4d0402bab32b8dcf2faf550857ced83a70647025819b9efdb0c2bdd8facb5
+size 1149169920

shard-00020.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a04f9353b81047678127f8287e29a97de369b30da12f089b03252d23008f69ef
-size 1135013548

 version https://git-lfs.github.com/spec/v1
+oid sha256:b744ca9addd73006aa20c0a4aaf1653f0d1e737ee352bd895a0f01082c294b9d
+size 1135013332

shard-00023.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fc3e1ee32ea731dfcf922a7bdd8a22cee2013bf30fa6fe068e003d909b4a7c99
-size 1120856632

 version https://git-lfs.github.com/spec/v1
+oid sha256:fc3953387c63af44b2235bfe2b0edcc364a81b64be865d30325a4cc8e2a494d5
+size 1120856416

shard-00024.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3a91487b7e4b6ea8c11679bdbf7f8dfbb201c1fbe7528ae015ab8595e00a7704
-size 1149170472

 version https://git-lfs.github.com/spec/v1
+oid sha256:2887110ba399b2be683f19d39eac903381b32e257c4faaa2841386668c0f6501
+size 1149170256

shard-00025.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9be9cc451ab910f296cc3c9063de734f95f2b42ef563383939c929087f2f9a03
-size 1078384844

 version https://git-lfs.github.com/spec/v1
+oid sha256:32da9bf46986dd50779a6e98ed17d1c5c01ac4846714f7f5980e4c85a58f5ac0
+size 1078384628

shard-00028.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ea03b6edf04d2b00d543d7b4cc35caa01638b8ecfdb0d74c20b41d0527521710
-size 1149170296

 version https://git-lfs.github.com/spec/v1
+oid sha256:5adcae3354b2c7f805e336fba784d635e3064217f00fbb93d90b4e7f4337a161
+size 1149170080

shard-00030.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fa4f75cd4354d757bbc0db2b1795cb00137b2d82da65991b5a0a6f3139fd8a57
-size 1050070860

 version https://git-lfs.github.com/spec/v1
+oid sha256:577ca362804884d54707c14c7b78b509e29025518be41e704bbe876fb0aaa119
+size 1050070644

shard-00032.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5da0b87997d06d1ebdcfbb7fafed307fa39d1ebf943853c8c94b216b54083a00
-size 1050070740

 version https://git-lfs.github.com/spec/v1
+oid sha256:1f76a62658018c75b25ddae384ff48b89c7e59fdc8db7368d99bd3d8a3456f45
+size 1050070524

shard-00035.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf7a737c4c1999edc3c06f1161fd8788fcfd1803182b7447c3492785b7585d90
-size 1078384924

 version https://git-lfs.github.com/spec/v1
+oid sha256:ccb99df46c189342a0e35223f4727c50aba02d2ae084ddd45a592065949f22c4
+size 1078384708

shard-00037.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c9fd118be4e4c5d3dde9471b7c280498d4b6aeeea511a3aecd97ac0d33d93d4c
-size 1120856648

 version https://git-lfs.github.com/spec/v1
+oid sha256:e8a07e7cc564567f2494867edd67d50c37bb80941d49bd22d13e9097da42cab4
+size 1120856424

shard-00038.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9292cb66c3879843a2eb129704b5f0592b5c029c1b9efa56d02ce8ea992a691c
-size 1106699252

 version https://git-lfs.github.com/spec/v1
+oid sha256:9e2d858a982ced8bd42b1d6272c79d926193d79cb6e43bc3644fa420a42456a8
+size 1106699028

shard-00041.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8c4ff88274b4260eef7df3892159b86702e6b021250c1cca2b3099ddbb2db834
-size 950971712

 version https://git-lfs.github.com/spec/v1
+oid sha256:c6634d005cd6b12bab935d5a53702a9c65fca21e54c67f330c4fd1433e1e945e
+size 950971496

shard-00042.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:de997da1aa0f4d7168142ba9d40a3db0e312c27f9901d89dec20161c8d9b2e70
-size 1234112560

 version https://git-lfs.github.com/spec/v1
+oid sha256:85436bc6d99029b29a4fc9a17dbb052fc9b43690583ba6f0572a25a4b24ed8a4
+size 1234112336

shard-00044.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0392f3a550e687c57d2faf91267b5ebdcdc351fb8532e098f07b02fecc6df88b
-size 1149170752

 version https://git-lfs.github.com/spec/v1
+oid sha256:db36ad9e4efc185c4cd24ef26d2036d8f51aaa1e1a55898094f756448ee4dba2
+size 1149170536

shard-00046.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b140e6f36635d2d6cca39427e3ebba994fefd005c968608f97835728c503c751
-size 1149169528

 version https://git-lfs.github.com/spec/v1
+oid sha256:f9ac746aabc36fdd18ef41bb199286e6fd4127fe97e9c39129a14078d5475bd1
+size 1149169320

shard-00048.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1fd6c6a02512321d533a9a4d1fb041c90c0d897fb7208844eb21e5c5582aa21a
-size 1135013524

 version https://git-lfs.github.com/spec/v1
+oid sha256:04152089aca517b3fdb23013efcaa1bba0117d89e57828d348948ea142b66faa
+size 1135013308

shard-00049.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:92dc29e3f48c547b3183422be37e8d40ba7055b48d88ba2bfbacbd9ceae7f6d0
-size 1035913784

 version https://git-lfs.github.com/spec/v1
+oid sha256:e9eb859db00911ab5bd8d1ad9f69ab101550fc398b3b67302034aa4a22d07af6
+size 1035913568

shard-00051.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ef4bc8e1c8e69ac63f4fec208f1c8cab1a9c898af9a80b0c48e94f89179501ed
-size 1120856552

 version https://git-lfs.github.com/spec/v1
+oid sha256:9be0f14e48825deb42536d6e3537fcf031bd0597b621c3e694ac2de8c8eac49d
+size 1120856336

shard-00053.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e1e772d149df5feac1d21155616aa9f0d79c250542812690e944641a2edcfc61
-size 1106698908

 version https://git-lfs.github.com/spec/v1
+oid sha256:d2d275aba35e1b29723f51a68dc539cd4800acad5c58217de23672c859df6716
+size 1106698692

shard-00055.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:080dee5bd0bc6f215f810d2f0cb0ff702ca86dbbb33079799412c6e781cd1d0e
-size 1276583444

 version https://git-lfs.github.com/spec/v1
+oid sha256:fee4e58eb2be387171f80a3ba4c0e5916c37aa0f31888873949e243c33a933d6
+size 1276583228

shard-00056.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8c2c76f7654801cd28dff1a6c78c4f2ea990c042c146f7aeed21efab233cfb57
-size 1007599176

 version https://git-lfs.github.com/spec/v1
+oid sha256:de6c9efcd8e09ba7e02a3156c7288104c7dfc501057051abf64231e77a5f3497
+size 1007598960

shard-00059.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0f3e2d14277be8b4160be5a563d430b2f80b03e358bf75a9001e8fffb504dcea
-size 1135012756

 version https://git-lfs.github.com/spec/v1
+oid sha256:7bcafa78445771b98b24e95aaff5c02121a72932076c74493f0e9ef34de14357
+size 1135012540

shard-00060.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c938e35fd251094a06011611a8ae6434b0e321d7905d31be609a3ad416f4ee85
-size 1027460812

 version https://git-lfs.github.com/spec/v1
+oid sha256:64e03f543fd593728dcce38b77502f6b18020226a11008d1355a54e61e69672a
+size 1027526268

shard-00061.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2063c29497c70faf736b6c630ab5d5cb256c726172de0455e4df5dd1232c5bda
-size 1261584968

 version https://git-lfs.github.com/spec/v1
+oid sha256:4645742d97d08f851be106944685eef300c6033f3b549a343067b700c4ce53db
+size 1261519304