Instructions to use lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated")
model = AutoModelForMultimodalLM.from_pretrained("lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated

SGLang

How to use lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated with Docker Model Runner:
```
docker model run hf.co/lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated
```

GLM-5.3-Flash-tr3-4bpw-Abliterated / results /tp2-runtime-qualified.json

lovesenko

Add files using upload-large-folder tool

ca44eaa verified 2 days ago

Raw

History Blame Contribute Delete

2.5 kB

{"bits":4,"decoded_reference":{"max_abs_error":10.546875,"max_abs_tolerance":0.25,"mean_abs_error":0.2410409301519394,"mean_abs_tolerance":0.01,"raw_parity_passed":false,"receipt_sha256":"c15ce9f45b764c0b282628921f6f4467104c99e0233dd7ec2b422abb6fe81f17","sha256":"7717a60b6ede38010aec60dc3ca40d40ec38f6252d460f15815707e74d55a651"},"example_only":false,"exllamav3_commit":"c5d9c657966ffeeaa9353f0cc899f18629da4a13","failure_reasons":[],"generation_verified":true,"materialization_receipt_sha256":"092be1ffa8db66bf02d4c370d0433a57aa48d4a6e5ce89723ef6a3bb7ca32643","qualification_basis":"actual_runtime_teacher_kld_plus_cross_rank_identity_and_generation","qualified":true,"rank_output_identical":true,"rank_output_sha256":["8483685110c131f1bd8bcdcb93687b8d6485a2a72df74e4d1abb4316ba9c97d7","8483685110c131f1bd8bcdcb93687b8d6485a2a72df74e4d1abb4316ba9c97d7"],"rank_reports":[{"bf16_routed_weight_parameter_count":0,"bits":4,"device":"NVIDIA B200","generated_token_count":4,"packed_matrix_count":36288,"peak_cuda_bytes":96425032704,"rank":0,"steady_cuda_bytes":89381623296,"world_size":2},{"bf16_routed_weight_parameter_count":0,"bits":4,"device":"NVIDIA B200","generated_token_count":4,"packed_matrix_count":36288,"peak_cuda_bytes":96425032704,"rank":1,"steady_cuda_bytes":89381623296,"world_size":2}],"raw_decoded_parity_is_not_the_qualification_basis":true,"raw_runtime_receipt_sha256":"cbd13544b47f7a0fa9df62af43da94e30ad107c6ebac5eea40443a2a485e3d3a","receipt_sha256":"a4631352b8fe705e9c6106ea51bb08325be6019c186e7d148306c1df7aab3951","runtime_logits_artifact":{"bytes":1268157784,"path":"/workspace/artifacts/qualification/glm53-packed-k4-tp2-runtime-v14-logits.safetensors","raw_tensor_sha256":"8483685110c131f1bd8bcdcb93687b8d6485a2a72df74e4d1abb4316ba9c97d7","sha256":"280c12452f429a79724cd1715a5bed2462de98a547746a80f04e7b8cdcd34746"},"runtime_module_sha256":"11617266954541f4c7337bf90501ec6f540ae3a346c486d90bdbe3380d625e67","schema":"quant-pipeline.glm53-custom-transformers-exl3-mcg-tp2-runtime-kld-qualified.v1","stock_exllamav3_model_compatible":false,"stock_vllm_compatible":false,"teacher_runtime_kld":{"max":1.018581137984496,"mean":0.022750847877671544,"p95":0.07140553100728228,"passed":true,"prediction_positions":2047,"report_sha256":"2cfd1b90a65162784de5b67e1b05ef7a15f98979b77156092d5f7500c607ba67","teacher_receipt_sha256":"2ae08117c3d4247f747b2a9a889b68e1a06387b788d56a0bf23bb950c77bc5a5","threshold":0.06,"top1_agreement":0.9384465070835368,"window_id":"final-0000"},"tp_size":2}