Make Baseten deployment work from a clean account

README.md

@@ -36,12 +36,13 @@ token space.
 
 ## Quickstart
 
-SGLang needs a small out-of-tree plugin, because `Glm5vForConditionalGeneration` is not yet an
+SGLang needs a small out-of-tree plugin because `Glm5vForConditionalGeneration` is not yet an
 upstream architecture. It ships inside this repo, so there is nothing else to clone:
 
 ```bash
-hf download baseten/GLM-5.2-Vision-NVFP4 --include 'plugins/*' --local-dir ./glm5v
-pip install ./glm5v/plugins
+uvx --from huggingface-hub hf download baseten/GLM-5.2-Vision-NVFP4 \
+  --include 'plugins/*' --local-dir ./glm5v
+uv pip install ./glm5v/plugins
 ```
 
 ### SGLang
@@ -50,7 +51,7 @@ pip install ./glm5v/plugins
 export SGLANG_EXTERNAL_MODEL_PACKAGE=sglang_glm5v
 export SGLANG_EXTERNAL_MM_PROCESSOR_PACKAGE=sglang_glm5v
 export SGLANG_EXTERNAL_MM_MODEL_ARCH=Glm5vForConditionalGeneration
-python -m sglang_glm5v.patch          # registers the arch on SGLang's DSA/MLA paths
+python -m sglang_glm5v.patch
 ```
 
 #### 8×B200 — full 1M context
@@ -91,7 +92,7 @@ python -m sglang.launch_server \
 
 ### Query it
 
-Standard OpenAI multimodal messages — the image is delivered as `image_url`:
+Standard OpenAI multimodal messages deliver the image as `image_url`:
 
 ```python
 from openai import OpenAI
@@ -108,28 +109,64 @@ r = client.chat.completions.create(
 print(r.choices[0].message.content)
 ```
 
-GLM-5.2 is a reasoning model: with `--reasoning-parser glm45` the chain of thought arrives in
+GLM-5.2 is a reasoning model: with `--reasoning-parser glm45`, the chain of thought arrives in
 `message.reasoning_content` and the answer in `message.content`.
 
 ## Deploy on Baseten
 
-Ready-to-push [Truss](https://truss.baseten.co) configs ship in this repo under `truss/`:
+The repository includes ready-to-push [Truss](https://truss.baseten.co) configs. The only
+credential you need is an API key for your own Baseten account; no Hugging Face token or
+pre-created Baseten secret is required.
+
+1. Install [`uv`](https://docs.astral.sh/uv/getting-started/installation/) and create a Baseten API key.
+2. Export the key, download the small Truss directory, and deploy one of the two configurations:
 
 ```bash
-hf download baseten/GLM-5.2-Vision-NVFP4 --include 'truss/*' --local-dir ./glm5v
+export BASETEN_API_KEY="your-baseten-api-key"
+uvx truss login --api-key "$BASETEN_API_KEY" --remote baseten --non-interactive
+
+uvx --from huggingface-hub hf download baseten/GLM-5.2-Vision-NVFP4 \
+  --include 'truss/*' --local-dir ./glm5v
 cd glm5v/truss
+
+# Recommended starting point: 4×B200 and 256k context.
+uvx truss push --remote baseten --config config_nvfp4_4gpu.yaml --wait --output json
+
+# Or use 8×B200 for the full 1M-token context.
+# uvx truss push --remote baseten --config config_nvfp4.yaml --wait --output json
 ```
 
-| Config | GPUs | Context |
-|---|---|---|
-| `config_nvfp4.yaml` | 8×B200 | 1M |
-| `config_nvfp4_4gpu.yaml` | 4×B200 | 256k |
+The command creates a new model and published deployment in your Baseten account and prints
+JSON containing `model_id`, `model_version_id`, `predict_url`, and `logs_url`. It does not
+promote the deployment to production.
+
+Set `PREDICT_URL` to the returned `predict_url`, then run a multimodal smoke test:
 
 ```bash
-truss push --config config_nvfp4.yaml        # 8 GPU
-truss push --config config_nvfp4_4gpu.yaml   # 4 GPU
+export PREDICT_URL="https://model-...api.baseten.co/deployment/.../predict"
+
+curl -fsS "$PREDICT_URL" \
+  -H "Authorization: Api-Key $BASETEN_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "glm-5.2-vision",
+    "messages": [{
+      "role": "user",
+      "content": [
+        {"type": "image_url", "image_url": {"url": "https://ultralytics.com/images/bus.jpg"}},
+        {"type": "text", "text": "Describe this image in detail."}
+      ]
+    }],
+    "max_tokens": 512,
+    "temperature": 1.0,
+    "top_p": 0.95
+  }'
 ```
 
+The first deployment downloads about 466 GB of weights and initializes SGLang, so startup can
+take several minutes. If Baseten reports a transient infrastructure or download error, retry the
+same push; no configuration change or additional credential should be necessary.
+
 ## License
 
 MIT, following both parents: GLM-5.2 (MIT) and Kimi-K2.6 (Modified MIT). The projector weights

truss/config_nvfp4.yaml

@@ -1,7 +1,10 @@
+# yaml-language-server: $schema=https://raw.githubusercontent.com/basetenlabs/truss/main/truss/config.schema.json
+
 model_name: glm-5.2-vision-nvfp4
+description: GLM-5.2-Vision NVFP4 with 1M context on 8 B200 GPUs.
 model_metadata:
   example_model_input:
-    model: glm-5.2-vision-nvfp4
+    model: glm-5.2-vision
     messages:
       - role: user
         content:
@@ -17,14 +20,13 @@ model_metadata:
     - openai-compatible
 
 # Deploy GLM-5.2-Vision (NVFP4 build) on Baseten with Truss.
-#   truss push --config config_nvfp4.yaml --remote <your-remote>
+#   uvx truss push --config config_nvfp4.yaml
 #
-# Unlike the internal deployments this was derived from, there is NO checkpoint
-# assembly here: the model_cache repo below is already the fully assembled
-# checkpoint (GLM-5.2 text + MoonViT vision tower + trained projector + processor
-# remote code), so start_server.sh only installs the plugin and launches SGLang.
+# There is no checkpoint assembly here: the public Hugging Face repo below is
+# already the fully assembled checkpoint (GLM-5.2 text + MoonViT vision tower +
+# trained projector + processor remote code), so start_server.sh only installs
+# the plugin and launches SGLang.
 #
-# To deploy:  truss push --remote <your-remote>
 base_image:
   # SGLang 0.5.13, CUDA 13, Blackwell (sm_100). Ships the DSA sparse-attention and
   # trtllm-gen fp4-MoE kernels this checkpoint needs.
@@ -39,7 +41,7 @@ docker_server:
   server_port: 8000
 
 model_cache:
-  # The whole model, one repo — no assembly at serve time.
+  # Public Hugging Face repo: no Hugging Face token or Baseten secret is needed.
   - repo_id: baseten/GLM-5.2-Vision-NVFP4
     revision: main
     volume_folder: glm5v_nvfp4
@@ -51,9 +53,6 @@ environment_variables:
   GLM5V_CKPT: /app/model_cache/glm5v_nvfp4
   GLM5V_QUANTIZATION: modelopt_fp4
 
-secrets:
-  hf_access_token: null
-
 resources:
   accelerator: B200:8
   use_gpu: true

truss/config_nvfp4_4gpu.yaml

@@ -1,7 +1,10 @@
+# yaml-language-server: $schema=https://raw.githubusercontent.com/basetenlabs/truss/main/truss/config.schema.json
+
 model_name: glm-5.2-vision-nvfp4-4gpu
+description: GLM-5.2-Vision NVFP4 with 256k context on 4 B200 GPUs.
 model_metadata:
   example_model_input:
-    model: glm-5.2-vision-nvfp4-4gpu
+    model: glm-5.2-vision
     messages:
       - role: user
         content:
@@ -16,19 +19,17 @@ model_metadata:
   tags:
     - openai-compatible
 
-# Deploy GLM-5.2-Vision (NVFP4 build) on FOUR B200s instead of eight.
-#   truss push --config config_nvfp4_4gpu.yaml --remote <your-remote>
+# Deploy GLM-5.2-Vision (NVFP4 build) on four B200s instead of eight.
+#   uvx truss push --config config_nvfp4_4gpu.yaml
 #
-# 465GB of NVFP4 weights across 4x183GB leaves ~11GB/GPU of headroom, so this runs
+# 466GB of NVFP4 weights across 4x183GB leaves ~11GB/GPU of headroom, so this runs
 # at mem-fraction 0.90 and a 256k context rather than the full 1M. Measured on
 # 4xB200: ~172GB/GPU resident, KV budget ~973k tokens — comfortably above 256k.
 #
-# Unlike the internal deployments this was derived from, there is NO checkpoint
-# assembly here: the model_cache repo below is already the fully assembled
-# checkpoint (GLM-5.2 text + MoonViT vision tower + trained projector + processor
-# remote code), so start_server.sh only installs the plugin and launches SGLang.
-#
-# To deploy:  truss push --remote <your-remote>
+# There is no checkpoint assembly here: the public Hugging Face repo below is
+# already the fully assembled checkpoint (GLM-5.2 text + MoonViT vision tower +
+# trained projector + processor remote code), so start_server.sh only installs
+# the plugin and launches SGLang.
 base_image:
   # SGLang 0.5.13, CUDA 13, Blackwell (sm_100). Ships the DSA sparse-attention and
   # trtllm-gen fp4-MoE kernels this checkpoint needs.
@@ -43,7 +44,7 @@ docker_server:
   server_port: 8000
 
 model_cache:
-  # The whole model, one repo — no assembly at serve time.
+  # Public Hugging Face repo: no Hugging Face token or Baseten secret is needed.
   - repo_id: baseten/GLM-5.2-Vision-NVFP4
     revision: main
     volume_folder: glm5v_nvfp4
@@ -58,9 +59,6 @@ environment_variables:
   GLM5V_MAX_MODEL_LEN: "262144"
   GLM5V_MEM_FRACTION: "0.90"
 
-secrets:
-  hf_access_token: null
-
 resources:
   accelerator: B200:4
   use_gpu: true