Instructions to use lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated")
model = AutoModelForMultimodalLM.from_pretrained("lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated

SGLang

How to use lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated with Docker Model Runner:
```
docker model run hf.co/lovesenko/GLM-5.3-Flash-tr3-4bpw-Abliterated
```

GLM-5.3-Flash-tr3-4bpw-Abliterated / results /five-cold-run-kld.json

lovesenko

Add files using upload-large-folder tool

ca44eaa verified 2 days ago

Raw

History Blame Contribute Delete

3.94 kB

{"all_individual_quality_gates_passed":true,"cold_execution_count":5,"compute_dtype":"float64","kld_direction":"teacher_to_student","maximum_run_mean":0.024554564249958208,"mean_of_run_means":0.024554564249958208,"minimum_run_mean":0.024554564249958208,"population_stddev_of_run_means":0.0,"profile":"k4-tp2","qualified":true,"quality_gate":{"metric":"mean_of_five_run_mean_tokenwise_kld","threshold_lt":0.06},"quality_gate_passed":true,"receipt_sha256":"df3d2388967f3ff578bd5c92e11875c4198d0337c25aa98eeb328271c0034ba3","run_count":5,"runs":[{"mean_kld":0.024554564249958208,"prediction_positions":51175,"quality_gate_passed":true,"report_file_sha256":"bc5c1e241c7e515ad1b8d4a734a587259b6dd0aa9ce58baa0a8eab4db2970e31","report_sha256":"ef6a8dedc20f11e582658f94923da3e66c2b6cea4ff62d936abb790e376e2461","run":1,"student_backend_identity_sha256":"966c47c49d154102aa49c0832f8e8b1eb0f6fb50c1f451f750efa29b2ce6f3e6","student_capture_receipt_sha256":"013759025d8414f8811fa140250e2c79097c1082926edd4ae2cfc6751722fc8d","tokenwise_kld_sha256":"2a596810dcdd52fc654eb94fffe1cf394b826ea6b25d8f411049d8354e52f562"},{"mean_kld":0.024554564249958208,"prediction_positions":51175,"quality_gate_passed":true,"report_file_sha256":"fd221d512e0130b5d96522fef6cca25c0c05e2c434ec1706fa940bc8eeec135c","report_sha256":"b7d1cac829f6b21471da4ea724aac479f9db250d4286edd412e099fa747f8257","run":2,"student_backend_identity_sha256":"7cd000dfb8a35798b3ec0384cb5535f0d9da30c91273281973cf647e63798df4","student_capture_receipt_sha256":"eae08903737bde9f31bf6f8632d2de7b6539f4b1efd8113c5f81461d92aaf671","tokenwise_kld_sha256":"2a596810dcdd52fc654eb94fffe1cf394b826ea6b25d8f411049d8354e52f562"},{"mean_kld":0.024554564249958208,"prediction_positions":51175,"quality_gate_passed":true,"report_file_sha256":"958a72fac966fec607a32caaf298fa83c22ed3b182583e11fa86cbc2c91d0b05","report_sha256":"663629ccd2bda08a4c299d767b7e6e6d622a81ad6830ad1acf08d0eb8ca1a196","run":3,"student_backend_identity_sha256":"ac80965b615e5682c7367ebb0d539a28eb0b32760eb1cfea769c38bb2cc417b7","student_capture_receipt_sha256":"000896721ea7116322eb31d8e75718985d29240fabd6a921627bb02c03516bec","tokenwise_kld_sha256":"2a596810dcdd52fc654eb94fffe1cf394b826ea6b25d8f411049d8354e52f562"},{"mean_kld":0.024554564249958208,"prediction_positions":51175,"quality_gate_passed":true,"report_file_sha256":"c6f574a8e6c4cda43bfefd9edbd067215256ba0bb16bc6730fa1d1d132b80a42","report_sha256":"cdb2d8ee4ce795f695f335f0bb3ce7bd135dcf6df4f48c6e3862b40cd1340586","run":4,"student_backend_identity_sha256":"142731457be7d04e11d332da38ba872a4004058c457af2ba298afa17fa920e28","student_capture_receipt_sha256":"7ece4defa651c3693bffd624ad7d07ff85c0dceb7674a7752ec136dea6370c3f","tokenwise_kld_sha256":"2a596810dcdd52fc654eb94fffe1cf394b826ea6b25d8f411049d8354e52f562"},{"mean_kld":0.024554564249958208,"prediction_positions":51175,"quality_gate_passed":true,"report_file_sha256":"a75a9d33892cfe31950d17246968925cd5e3bfac9dcbbd3b4e57b4b077a36263","report_sha256":"ac4d6d94aef27b09ca9b2dd513516e793cf5f4afe3d1f2b008a3fb4ed64ae243","run":5,"student_backend_identity_sha256":"9fd6be2589da5ee83780ad8b8a80f57f3f35d5cc45fd2d9155d042eb8ef1ff74","student_capture_receipt_sha256":"5b59145332206b4c0fb82f791e2c09be8fadb16d18e6e58818b78e919294cb65","tokenwise_kld_sha256":"2a596810dcdd52fc654eb94fffe1cf394b826ea6b25d8f411049d8354e52f562"}],"runtime_reader_sha256":"1fb3be878e0a9445b640565558fc34715891bfd60a63e976002181c620a41a69","schema":"quant-pipeline.glm53-packed-student-kld-five-cold-run.v1","student_checkpoint_identity_sha256":"598ce08d20714903c393b523bee3cebb9eb498e46109bec4193037af401f6332","student_label":"uniform-k4","target_bits":4,"teacher_backend_identity_sha256":"85b11599c6b36a83fa8099a09a298a386a0c603d1f18d3702e7fb1c470962ce4","teacher_receipt_sha256":"2ae08117c3d4247f747b2a9a889b68e1a06387b788d56a0bf23bb950c77bc5a5","token_panel_receipt_sha256":"0beec5770e5107547731b084f1bc5f9fb8ba79d67af56ddb70d919da367737d5"}