Kairo Experimental
Collection
More interesting models, don't expect all these models to work as intended. • 3 items • Updated
Text Generation
Transformers
Safetensors
English
gemma4_text
causal-lm
gemma4
gemma4-text
base
tiny-language-model
How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "veyra-ai/Kairo-5M-Gemma4-Base" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "veyra-ai/Kairo-5M-Gemma4-Base",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
Configuration Parsing Warning:In config.json: "num_experts" must be a number
Kairo-5M-Gemma4-Base
Kairo-5M-Gemma4-Base is a tiny Gemma4-style causal language model trained as an architecture and pipeline smoke test.
This repository is a native Hugging Face Transformers compatibility export using gemma4_text / Gemma4ForCausalLM.
Important fidelity note
The faithful checkpoint artifact is the ONNX export in veyra-ai/Kairo-5M-Gemma4-Base-ONNX-Int8. This Transformers repo uses HF's native Gemma4 implementation, which has stricter architecture expectations than the training smoke-test model. The exporter copied every compatible trained tensor into native Gemma4ForCausalLM and kept HF-only tensors initialized by Transformers. See conversion_report.json for details.
Model metadata
- Repo:
veyra-ai/Kairo-5M-Gemma4-Base - Native architecture:
Gemma4ForCausalLM - Native model type:
gemma4_text - Faithful local parameter count before native conversion:
4,377,856 - Checkpoint tokens seen:
350486528 - Best validation loss from checkpoint:
4.382341027259827 - Context length:
4096 - Sliding window:
512 - Vocab size:
4096 - HF layer types:
['sliding_attention', 'sliding_attention', 'sliding_attention', 'full_attention', 'sliding_attention', 'full_attention']
Conversion summary
- copied_exact: 54
- copied_partial_overlap: 1
- hf_only_keep_native_init: 30
Load
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained('veyra-ai/Kairo-5M-Gemma4-Base', trust_remote_code=False)
model = AutoModelForCausalLM.from_pretrained('veyra-ai/Kairo-5M-Gemma4-Base', trust_remote_code=False)
- Downloads last month
- 811
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "veyra-ai/Kairo-5M-Gemma4-Base" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "veyra-ai/Kairo-5M-Gemma4-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'