Instructions to use brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw")
model = AutoModelForCausalLM.from_pretrained("brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Trellis

How to use brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw with Trellis:

# No code snippets available yet for this library.

# To use this model, check the repository files and the library's documentation.

# Want to help? PRs adding snippets are welcome at:
# https://github.com/huggingface/huggingface.js

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw

SGLang

How to use brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw with Docker Model Runner:
```
docker model run hf.co/brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw
```

GLM-5.2-EXL3-TR3-3.0bpw / config.json

brandonmusic

MTP layer-78 -> EXL3-TR3 rank-sliced (malaiwah 3bpw-keep0, LDLQ-calibrated same corpus); config moe_layers [3,78], total_size -15.66GB

41a1c8a verified 2 months ago

Raw History Blame Contribute Delete

20.2 kB

	{
	"architectures": [
	"GlmMoeDsaForCausalLM"
	],
	"attention_bias": false,
	"attention_dropout": 0.0,
	"dtype": "bfloat16",
	"eos_token_id": [
	154820,
	154827,
	154829
	],
	"ep_size": 1,
	"first_k_dense_replace": 3,
	"head_dim": 192,
	"hidden_act": "silu",
	"hidden_size": 6144,
	"hybrid_tr3_tail": {
	"bits": 3.0,
	"calibration": {
	"axis_rows": {
	"axis1_general": 3057,
	"axis2_legal": 3057,
	"axis3_code_agentic": 3057,
	"axis4_reasoning_termination": 3057
	},
	"capture_fingerprint": "2efd10279b8c953e3e46a469d9ec9970593795859c7a2cebc98c9ea707115b51",
	"corpus_rows": 12228,
	"corpus_sha256": "cf247acc7c5da9f0600c7d6ab3b7c2fcfc54ec30b794e3b6047559285fa44df4",
	"layer_h_fallback_experts_total": 0,
	"min_routed_floor": 1024,
	"passes": [
	{
	"axis": "axis1_general",
	"name": "axis1_general",
	"samples": 247,
	"tokens": 294790
	},
	{
	"axis": "axis2_legal",
	"name": "axis2_legal",
	"samples": 832,
	"tokens": 294681
	},
	{
	"axis": "axis3_code_agentic",
	"name": "axis3_code_agentic",
	"samples": 361,
	"tokens": 295687
	},
	{
	"axis": "axis4_reasoning_termination",
	"name": "axis4_reasoning_termination",
	"samples": 3057,
	"tokens": 165310
	}
	],
	"per_layer_ids_sha256": {
	"10": "944172cb95fd0d5e04f6530c576fdbd3655017dd2f84df4a6026327bb3bd8b29",
	"11": "96c038deadb3a6246e43c3721baec2e38c74da892c4cc4bf4eafc752924b3a62",
	"12": "b534b4fabfdf4106f29fb2c6da9ba02f922c5e0de8b540214093b6d9a4ab83d4",
	"13": "9281ba7da8317e3698d99b3d8a0bb49451381ed307b6bca7aa1728976cea048f",
	"14": "e30587c05ac7a057ed3baee76c8d52137351bc60ad310cc7a936b504b5c1b6a2",
	"15": "bb0a47a86f174c2c2ea1850ac5b306e603f0c2c4d073f51229e4b08577ded702",
	"16": "297126f807808607f099f8d22bd42ee568a370219c23a3a32bd8f8b938c5ce9b",
	"17": "7fe9c08f797dfe88e7be5a93f9f367c5dd6dbc2b0dad6ba10962c9df0803d51f",
	"18": "b4bd0335dba1bc416210505a2e51c42d2f395b1d14e7136bce854367cbe3de47",
	"19": "f3013f105748cef65cf9635515e6fe126f9dc67640ea28b4512d4030f5c40a87",
	"20": "6def65ec244e56203184ec5d941c96fc16a642c917d21c24d18f649f4d073a23",
	"21": "f183d6273accde7f8e1d3818d584121b155c70dc697c27fb1eead713f09bf1ea",
	"22": "313bdcc88cee8693f86d74c7179720374d0f1dae6ce8499533bc1761f4acb7e1",
	"23": "f7005e9ce1ba7fd0ec2b35eba18611156ac9104f86bc94574b02ba5766e6e36e",
	"24": "d60b184183f7ed719a634f64dbf80e40a7e055537da33ae0ddaa4810b7592e43",
	"25": "1937356771d8de5d112e0bf047261e820294199f30b477a20f00fa161a7dcc12",
	"26": "2233b8fe22d5b26e570b349d56608300bd23774a2813d1570d5768221d57187c",
	"27": "6fecbaa85d3903b3c6ff754fa4fc87c60c1015acdf20223f0be472ed207ac56f",
	"28": "252eb337da398832349416b6a6bbc0035b6459326890ed7cbb206cc95c45a450",
	"29": "cd19bdf2ccd19d5cbbd2c122f0ebbaba027730fc37a5025c6458fea4159382ba",
	"3": "160c5353b4ce34cb74fa06e31e5bb67d3ab483b2720c676aec2130624fdad26f",
	"30": "cbf67bf960f554666cd79b150db925f5b084c0febafcd5506162586daaeb1ab6",
	"31": "9547132cf221eac0cb626ada87b51de8a89530c22dee8d2af79397e2d8d27a02",
	"32": "9b8535a00a5fc4751b90975cd61ebcf9c01cf029d9184ac2e46ec54df96dbc35",
	"33": "02c9f4c12120ad5dfdd94c8016b8071094e57b850494f4c5a0de96e83fdbc150",
	"34": "6ad1e3337ea2b7028174cbf8efd11ee7fb7561c188ec77ce5f0875582ec0b523",
	"35": "f7333c478ca20eabbbd1a83eb0d31bdce456ce4f996611cc0171aada6cb5a9a4",
	"36": "e4d3a0a2f39816f28c12c13a09c9ef53d3309e67e9ccbca0853a53806e5487f5",
	"37": "802c36092a22bd973c8d03f15491ef8fc2b2a076bd45f94426ff21a76b46c747",
	"38": "f96d105a5d266870b7d789dac882c9c6d9e7f9097165ed11a66bc1823e434caf",
	"39": "5badf1a5c79952b3e0f664f8cc3c041d58637a1825bbcce3fd248cabb6a0f276",
	"4": "d461d0a2725c9e38d8016531d9a0433f932c7934fc8b41fbab5d7a2d8fbcf982",
	"40": "52fe99f4b029272b57c443b33dfdbf684ea94ed190146d6bce0563f924fd48a9",
	"41": "6eb3a5c47cfa1a4aa1b20c7ce75f964bb8086238ca24ad527e08af3f5521dcee",
	"42": "cfbc83865cd9e7e9c751447469a2ae288a0d9010d6f6c8ed1ad2fdd283c3bc57",
	"43": "c5779321bfdbcd6e672682907cd894bc893c2d9d4983e8050afd4f2f17978016",
	"44": "9cd7861ebf2d971c39f67ab00f7bea361b7f64f861c61bd2e6639ac995a71d62",
	"45": "7d2cd2026f6a23517a9cb88c803bc2792dea4060b16d09ef340a9eb9f302c2ab",
	"46": "e9a6a0248c7cb6f1be4d59c077c8741e6264d20f4003b2f050ad0dfc9c1152d8",
	"47": "2c6512673c31c775c5d603581db34d4bd62634e712f6ddaca24cc3f40bbf8b42",
	"48": "c37925833ae9e0f87a92aeb89237908d21048d0fe812072952eaeaa9937dec27",
	"49": "f682b58165c1ad8c225e1f28677127442baa2e25b3cb906d5df010e236428d38",
	"5": "59515e224e0ca8a066f168c207f8917782f752dff8255c0e8b40007608a4e363",
	"50": "a382c0b7c24d8437b44314b5f1020c75401461e73f0d036d10f0df37b47a01fa",
	"51": "e7368340cc69dd39c1a9d7618ed5624e366c938dcc144ab8830aa72491af5dae",
	"52": "186dca03622be49f93de41a32da2e4557e96c1d6274c64f49e5e931ffef53810",
	"53": "4c14e704cc39f92a35ec5ebd50ae4467595546a02029f5a56a42a3cfa8056247",
	"54": "99810633dd399fae9bda0db56f35d2e30e86e2a6bf92c41d24b82dc951578f8f",
	"55": "be1b9ff06a176ff0d84e9879426ec207933ffb1d6d61819afff76afe4d77d640",
	"56": "458fb45785a2d8cce6f781b2dc8f1995f98ef0cb9ea6714d0a16fecbd047913e",
	"57": "7cf63b0ff0a929c1996e1b00ec958b4682ae635ef551a87c25305ae068a0e4ce",
	"58": "e09af62bd0f15531b9d7c911c255544f90b9808904ca461a4639379a96f44760",
	"59": "3a2edee2590ad7cee5c051c52ecae5a89ee14a5c252efabf7997e338d230d834",
	"6": "c7ad2fbfacd9a5c442634bf4ddd24188f8662271d30be979a01ce301da0860a3",
	"60": "62a5b98c3805af186ad8296b82b46be09de73adea2cb6c54df67ecb954fc4dd3",
	"61": "5bf327d957ed3f9b7ee2df33b883073d054599fae1c2844036143f57f6b6e27f",
	"62": "4f9dcd6e578a281dc9ad2a7038c183dd4b724b87b5d9570139de4e860a83a1ed",
	"63": "78bc32a22b0f3237aa3a2e3f4487f8678f4d5b735441e4a4893738247fdb4af5",
	"64": "41247f7196e88c95462bcc8484210b742b13faeec674a70e9a6867704f9ffeb4",
	"65": "dde15bb42ee0b54f41e7cffdef083a325220f5ce74877ebebd7c8b64844df5e2",
	"66": "e5460abb6dd4de3699c2d29d9de833dd52129e9f10d778e892baad655a5ab5f4",
	"67": "e06e0cd4d64492853a0c25085e5d796af0e388b9545187d06d11c8a70cf3d045",
	"68": "5965fc8d1eb4adda502449a6483c0f3dfca084f7f75e9ad09146d0093c207ded",
	"69": "45fc01960dbc7f7d36b2f05a17a3bf21a96b2589385a9d8dda646cd33a353e3a",
	"7": "0b3387dc099d979559dfa043190202198d1743aef89d7f0d059fb635fc75e674",
	"70": "7118c4343021dc4432fc0165ee055262b9792574a6cd94937da1151a0ac3d8dd",
	"71": "0c37b81467e03903d414029bea5b6f2bcee196f802c0bac19ce9419ada0d0e24",
	"72": "1fd11098af6ca9232edfc93538e53939d0e2f00adcc3846a782b7dd0aaaf6230",
	"73": "413c33b7509fbc9e0a6b2e1ac4cff3d47ccd66fd77580289920197c96e526612",
	"74": "e2a92bcf38b4a8fbe4188375c87793a3a03cba9db5f3f97bdc81cd55a68268be",
	"75": "0ab6352069bccc138ed7a70aa755187308ba538747590136c8a4b436a207ec45",
	"76": "666662199d3778752d69bff59c13ee163023d857a9d1d0cbc3f35d059922728f",
	"77": "f4d604947ce189ae14e6d6ac54ab188f2d9339fc37ffdb6c32117e08d93fd4f8",
	"8": "75469d18e1cb717a712e6c13a1f11e927eac38e4ea1a585e07d1e9b820045861",
	"9": "10adf8399353453e4f8ec61345f958d2ffb3fc61532d5245f46729eba1889861"
	},
	"per_layer_x_sha256": {
	"10": "630795bba68861abf750e67f5594b8312917ae716c421d8e14dccae26ae3dcdb",
	"11": "2b888bfb5fdd4a4de1a1e1039ef8ab8bad7c603d3b14f7e17d8fb15e956bee81",
	"12": "5712b76ef3252481e9218f5faaa06809ad824d7ba40e9bf824f1b19f3d047f38",
	"13": "fad146e05a179f166ab98320e6c04b700b1899f9e10c7eff9bd3fc541243e1b0",
	"14": "eecf2af2b26627a305930a869712f7eb9056e2b1b6c9b069c4054b7c2948c8e5",
	"15": "8cc3d0efa9092a2b5890e8d5b65a2432152110ad1b083527cdb01d095c6e028e",
	"16": "836579943de48a3c7ba9bb868f7d506ffa608ecdac8f5200ec43db094c20607f",
	"17": "302c72af994b580b4b102323963331856d61bd18d8d9cdbd7ec4a617a5c4eed5",
	"18": "fe598ec588ca35ab05ee3c40752a23d9f3a3f7a59486696971103d125792569e",
	"19": "157255508ae292a65f8cd8876ec93b9641efcd038fff4913ff5422b1b18cd78b",
	"20": "df15c4fbd286647dce38818b821cddb60c2baf744138dfc0ab4b13c8d06eaf8d",
	"21": "a3dba41861914213faecb88b761be90f7f5240c9d6ef939719c72c667962fa7a",
	"22": "749d9923d1ce4e8d4c9d48808f90dbf71ea399eea9e6a308e58bca7be0c3aa25",
	"23": "001c6acf96a90dc26297b5bf9d23862e8c15a08f1dfe57dfdd4368d07b70a383",
	"24": "52001d2a5d1b209cd05ed26750a410db774a0880588f15bdb225828febf7e104",
	"25": "7290fd2d3e78a3ef509246dbcc3b4516fa6cc0d717859db5fa74dea7c8806b80",
	"26": "5cbe9137b601650881126e7075dd32cc11134fa89734b8e096fbdd70e1ee2103",
	"27": "136f8234f84234cbef5e5f35d4aba2d9ef9db15f9886d5c76998e4b2a8755c2f",
	"28": "29e04c8c651296725b95ced18e119ab84f0f7bf107336af9b6f3e520550a7e09",
	"29": "943412684ba07bdbb16e01bc232d8771b51366ecbce01b57e25e2c9ea4e017e7",
	"3": "0d355318242b05301c317ec77657c65a51263c5de9a84a6a4ad5130328ae70f3",
	"30": "52951f087a1508cf3aa474cc735009be7085c3eae9a88499c8c0d6c3b6736f4a",
	"31": "dc7be9c6a09af6aeaba79ef884d831ffd0113e7e663d1ec2e49d997840224e9e",
	"32": "163c140020e201e288921e0a1617384eba8975567422b201dcabfcb7e792427b",
	"33": "c182636e040e3b28b08f3f34b0b4802ba4d0b31c46e6df2a58b2bd96c030cde3",
	"34": "e4c74876aa00085294c523229dc4ef6b44dcbc2a8d4d25d189b973126f0d7f66",
	"35": "214643ceb1d9a17139c4827fa5b03454fa4e4c62510a889d2649822f65d79a37",
	"36": "88ce87b5b5a782cadbce1326cc9b31c8f22edf1d5bd09e25fc0711ddc6386b3a",
	"37": "77b8130e6a17621f6d1246e8a1c403bc1b7e0b897a0643dda857b8ae3f130924",
	"38": "018ac2a9b0fcaeb359a5da57babf8b0ca82d9aef13380693b002d3c790bf0d7e",
	"39": "27f291b8f64e6ae1584f84a3c8adb641f37128152eb4be7f2c5b2a236c09b254",
	"4": "37baf4eefc878b6a24a36591dc7e8940c6b4c024ec65c656fc57f55fe3da41e6",
	"40": "cd2981874a400416187547a16091c9eee09a36d33d61d46ef69f1b4b6cd5800f",
	"41": "92b2acd32f78d95527fc43c97e9aa9e83731796cebbec296b0ff0f9a19ccc90c",
	"42": "1a7565a32dc1fe7dda07756c865779bdaa64b22a1d110933d2ea2e44c290f0ca",
	"43": "68b2a4db375e8f15865674630c700dea7d1d606d47693fb0ea65593df88c6a65",
	"44": "3f254dc62e45155fddf83b640aa50f59c5ef9b2e8d80d25dd593c708effdba5f",
	"45": "9a1ec43bd81d4a3a2dca73661307defc55d5ec06265d4c704129e36a3e512528",
	"46": "9dad9939c7dd222cce110b08b57026724b7b26767c08fbdbbb27ed44303bbe84",
	"47": "d40758d70e4f2e7647a3809cd2f10a2f58f87e4003da9bc98f07c6d1ecd261a0",
	"48": "acbe3f79ecec5d42266f4581ec799d0ff7ddfdde30df31b447036d162eb4022e",
	"49": "df6668d6934376d6eea3386e640b8fd479767baa4d235595a9d3296a1b4a56b7",
	"5": "71d5355a424386c8da2e68f5f003e03476610f5affc6fe4d90b9bfd155e43766",
	"50": "829c06ea3e13f1d304d2c48bb5021fabd501bd5b723976a95eb21b0496103e85",
	"51": "ef4762f7a9fd2a76ee0898083283d598da6cf5bd1d753a7c54cfe3fcf180e3fc",
	"52": "84974f752e28ece7c3300dddc0edd3f977ef0068d291f6ca2292fb417b10d8d5",
	"53": "a75256eb1541825ec96ffb2af2fa51647c62f97fd2922a648e368e64f011e797",
	"54": "f15bd7e990817708cb9672f270da69a6c749044a75636223a6be9a8e2d3e1b36",
	"55": "ee91530bcde2ba90c23574b1b7c9fa57aed6a39724156c77304a1dcba80ae983",
	"56": "32509d298e8136f131b6dedcfd894e47bc40fa1dcebeff3672ce0cb4b97a31af",
	"57": "8581236d84a23c08bebadbfb5f311e752c34ab583c5f9e2c45a146282ff7dbd6",
	"58": "d747b80f367c74c19cdf85b16d30dfb78852094421a60e36b13685ff34a56a71",
	"59": "897f8ba2318aad388a02d007703c033ac1658f8fc554d8d60ebbfbe569037461",
	"6": "62dab46cbe4a8b99701343421be579eadfdbaef9099758393b0aa98f3393c024",
	"60": "3187b62f18530096342f2faa72bf36b967e4768aba50bc9115c9902229fff0f6",
	"61": "27bd21e3d10c1764238341e89ee63fa0a3068a27a9f1c2084ff29114a4b6cf96",
	"62": "5e6489ce16a50af7fc3be6f8ffda9414aadc50673492dd9cfdceddce7094e6b1",
	"63": "ccd2173bab0009596732462726a950e89dc387370a5054b138af9c6368caab83",
	"64": "4e6864cee05926986fda8d8715af580b0343b21236a9999a2c544f433e4eabb1",
	"65": "6948dd47c19f10cf02f996c07fbe9cacf8a2919fc4fc94ff05a707ed2d6df4bb",
	"66": "db3b5a5a42cfdc9c60d8e0307e6d46a87fc5987a534fca070890b5758e39a59d",
	"67": "fe52111f18cc6b9ea412698ab544037cfc9c6db462ec82d94c9eaf5f392c493c",
	"68": "a7c85dcea4160b582fd0f320542ec5ec7a2c4680e4d1fb02f6efa149245979c2",
	"69": "5edb0b2f3eb115616e97e99eb8c08c27efd683321445d9c440d5ca279a1e3091",
	"7": "8f5a093166f9a319cec7125a3ee0e93d37c870a336973b882124a665d5b7873d",
	"70": "0f4304523d6d706f18223769f192d051185498d77da38763cc1df8352da8d122",
	"71": "e3c7ca04ff5a93926e02678e05f3de52d376a879bfa5ed562dff569ca4a9f8d6",
	"72": "a6af9992479ebd42005c0698e836068e8a6d17a52427e348992a94947162a32c",
	"73": "315f8eade152cb1fc45504cbd4b32f9b40bc4417163d548dbc50c4b64709ae45",
	"74": "98398c3cfb5d1ce79dd7473fc5b4ccbda4eca571f2ea5b401410f74d3e773888",
	"75": "b9d8903d95d798959dc2aaa2fb711c5e58e3622166a75b150f4085c08195090c",
	"76": "f0b300212791b3b2be25b504cd69bd52d68464b3906c822a4956305077a5bd50",
	"77": "82fe852e9e12ecec67b8e3385cf4e21b8ae82c6e4b9a26e9adebaf72540da039",
	"8": "06184a67d58fe6e3cda7494c01a51cbcbe02559834f151f107dba7609b4c9169",
	"9": "bba3bbbdcdfb72b1c37305068c8399b3de5a7e27399703aaedb798a2a3041010"
	},
	"q_fallback_slices_total": 0,
	"selection_policy": "owner-corpus-axis-separated-luke-multipass-no-repeat-v1",
	"sigma_reg": 0.025,
	"tokens_per_layer": 1050468
	},
	"codebook": "mcg",
	"exllamav3_version": "0.0.43",
	"experts_per_layer": 256,
	"format": "exl3-trellis",
	"hessian": "ldlq-calibrated",
	"mcg_multiplier": 3417055213,
	"modelopt_dispatch_note": "quantization_config selects the b12x ModelOpt interception only; this artifact contains zero NVFP4 expert payloads. Layer 78 is ignored and remains BF16.",
	"moe_layers": [
	3,
	78
	],
	"nvfp4_keep_per_layer": 0,
	"producer": "encode_b300.py",
	"producer_version": "1",
	"recipe_fingerprint": "db7e6a6d4191e93de86833261e693658abd6b0acc161ef8e6a1f91ed84179a7f",
	"scope": {
	"bf16_byte_exact": [
	"attention",
	"dense MLPs layers 0..2",
	"shared experts",
	"router/gates",
	"MTP layer 78 non-expert (attention, router, shared experts, eh_proj, norms)",
	"embeddings",
	"lm_head"
	],
	"quantized": "routed MoE expert gate/up/down projections, all 256 experts, layers 3..78"
	},
	"slicing": {
	"down_proj": "TP4 K-slice: rank r owns input columns [512r,512r+512)",
	"gate_proj": "TP4 N-slice: rank r owns output rows [512r,512r+512)",
	"up_proj": "TP4 N-slice: rank r owns output rows [512r,512r+512)"
	},
	"source_config_sha256": "185f93ee6d12548e16a847e279dc0c3c90b1524c970b0866b42fb545747d859a",
	"source_format": "BF16",
	"source_index_sha256": "5fd47a926aefce0f2c917f42523e5e0f3c87e23e389e767c3681536a62f5cf5e",
	"tensor_schema": "model.layers.{L}.mlp.experts.{E}.{proj}.rank{r}.{trellis\|suh\|svh\|mcg}",
	"tier_bitmap": "tier_bitmap.json",
	"tp": 4,
	"tr3_tail_per_layer": 256,
	"mtp78_overlay": {
	"source": "malaiwah/GLM-5.2-EXL3-TR3-MTP78/3bpw-keep0",
	"file_sha256": "a63c64a8356d326b6078475071cec359a6c07e161aa22759a216305986e4b7ff",
	"merged_layer078_sha256": "5448b63a32e394e8cbff5a4737fb50b40fc53c6c3c41305f1a7ee540c4d9a6e3"
	}
	},
	"index_head_dim": 128,
	"index_n_heads": 32,
	"index_share_for_mtp_iteration": true,
	"index_skip_topk_offset": 3,
	"index_topk": 2048,
	"index_topk_freq": 4,
	"index_topk_pattern": null,
	"indexer_rope_interleave": true,
	"indexer_types": [
	"full",
	"full",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared",
	"full",
	"shared",
	"shared",
	"shared"
	],
	"initializer_range": 0.02,
	"intermediate_size": 12288,
	"kv_lora_rank": 512,
	"max_position_embeddings": 1048576,
	"mlp_layer_types": [
	"dense",
	"dense",
	"dense",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse",
	"sparse"
	],
	"model_type": "glm_moe_dsa",
	"moe_intermediate_size": 2048,
	"moe_layer_freq": 1,
	"moe_router_dtype": "float32",
	"n_group": 1,
	"n_routed_experts": 256,
	"n_shared_experts": 1,
	"norm_topk_prob": true,
	"num_attention_heads": 64,
	"num_experts_per_tok": 8,
	"num_hidden_layers": 78,
	"num_key_value_heads": 64,
	"num_nextn_predict_layers": 1,
	"pad_token_id": 154820,
	"pretraining_tp": 1,
	"q_lora_rank": 2048,
	"qk_head_dim": 256,
	"qk_nope_head_dim": 192,
	"qk_rope_head_dim": 64,
	"quantization_config": {
	"config_groups": {
	"group_0": {
	"input_activations": {
	"dynamic": false,
	"group_size": 16,
	"num_bits": 4,
	"type": "float"
	},
	"targets": [
	"Linear"
	],
	"weights": {
	"dynamic": false,
	"group_size": 16,
	"num_bits": 4,
	"type": "float"
	}
	}
	},
	"ignore": [
	"lm_head",
	"embed_tokens",
	"model.norm*",
	"self_attn",
	"model.layers.0.mlp*",
	"model.layers.1.mlp*",
	"model.layers.2.mlp*",
	"shared_experts",
	"mlp.gate",
	"model.layers.78.eh_proj*"
	],
	"producer": {
	"name": "b300-exl3-modelopt-dispatch-shim",
	"version": "1"
	},
	"quant_algo": "NVFP4",
	"quant_method": "modelopt"
	},
	"rms_norm_eps": 1e-05,
	"rope_interleave": true,
	"rope_parameters": {
	"rope_theta": 8000000,
	"rope_type": "default"
	},
	"routed_scaling_factor": 2.5,
	"scoring_func": "sigmoid",
	"tie_word_embeddings": false,
	"topk_group": 1,
	"topk_method": "noaux_tc",
	"transformers_version": "5.12.0",
	"use_cache": true,
	"v_head_dim": 256,
	"vocab_size": 154880
	}