Text Generation
PEFT
Safetensors
Transformers
lora
turn-taking
multi-party-dialogue
ami
text-classification
Instructions to use ishiki-labs/qwen3-8b-ami with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use ishiki-labs/qwen3-8b-ami with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B") model = PeftModel.from_pretrained(base_model, "ishiki-labs/qwen3-8b-ami") - Transformers
How to use ishiki-labs/qwen3-8b-ami with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ishiki-labs/qwen3-8b-ami")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ishiki-labs/qwen3-8b-ami", dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ishiki-labs/qwen3-8b-ami with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ishiki-labs/qwen3-8b-ami" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ishiki-labs/qwen3-8b-ami", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/ishiki-labs/qwen3-8b-ami
- SGLang
How to use ishiki-labs/qwen3-8b-ami with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ishiki-labs/qwen3-8b-ami" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ishiki-labs/qwen3-8b-ami", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ishiki-labs/qwen3-8b-ami" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ishiki-labs/qwen3-8b-ami", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use ishiki-labs/qwen3-8b-ami with Docker Model Runner:
docker model run hf.co/ishiki-labs/qwen3-8b-ami
| { | |
| "best_global_step": 80, | |
| "best_metric": 0.9958399986721193, | |
| "best_model_checkpoint": "/home/ubuntu/context_aware_modeling/fine_tuning/checkpoints/qwen3-8b_r32/checkpoint-80", | |
| "epoch": 1.8181818181818183, | |
| "eval_steps": 20, | |
| "global_step": 100, | |
| "is_hyper_param_search": false, | |
| "is_local_process_zero": true, | |
| "is_world_process_zero": true, | |
| "log_history": [ | |
| { | |
| "epoch": 0.36363636363636365, | |
| "eval_accuracy": 0.9044056525353283, | |
| "eval_balanced_accuracy": 0.901877133105802, | |
| "eval_loss": 0.08732637017965317, | |
| "eval_macro_f1": 0.90297659081011, | |
| "eval_runtime": 39.9812, | |
| "eval_samples_per_second": 30.089, | |
| "eval_steps_per_second": 0.475, | |
| "step": 20 | |
| }, | |
| { | |
| "epoch": 0.7272727272727273, | |
| "eval_accuracy": 0.742310889443059, | |
| "eval_balanced_accuracy": 0.735494880546075, | |
| "eval_loss": 0.08577165752649307, | |
| "eval_macro_f1": 0.7197970138129186, | |
| "eval_runtime": 39.9877, | |
| "eval_samples_per_second": 30.084, | |
| "eval_steps_per_second": 0.475, | |
| "step": 40 | |
| }, | |
| { | |
| "epoch": 0.9090909090909091, | |
| "grad_norm": 0.41501519083976746, | |
| "learning_rate": 8.517573744099741e-05, | |
| "loss": 0.26, | |
| "step": 50 | |
| }, | |
| { | |
| "epoch": 1.0909090909090908, | |
| "eval_accuracy": 0.9858686616791354, | |
| "eval_balanced_accuracy": 0.985494880546075, | |
| "eval_loss": 0.08195541054010391, | |
| "eval_macro_f1": 0.9858461282921714, | |
| "eval_runtime": 39.8666, | |
| "eval_samples_per_second": 30.176, | |
| "eval_steps_per_second": 0.477, | |
| "step": 60 | |
| }, | |
| { | |
| "epoch": 1.4545454545454546, | |
| "eval_accuracy": 0.9958437240232751, | |
| "eval_balanced_accuracy": 0.9957337883959044, | |
| "eval_loss": 0.07636160403490067, | |
| "eval_macro_f1": 0.9958399986721193, | |
| "eval_runtime": 39.884, | |
| "eval_samples_per_second": 30.162, | |
| "eval_steps_per_second": 0.476, | |
| "step": 80 | |
| }, | |
| { | |
| "epoch": 1.8181818181818183, | |
| "grad_norm": 0.09980760514736176, | |
| "learning_rate": 3.845094616073783e-05, | |
| "loss": 0.0789, | |
| "step": 100 | |
| }, | |
| { | |
| "epoch": 1.8181818181818183, | |
| "eval_accuracy": 0.9958437240232751, | |
| "eval_balanced_accuracy": 0.9957337883959044, | |
| "eval_loss": 0.07304411381483078, | |
| "eval_macro_f1": 0.9958399986721193, | |
| "eval_runtime": 39.9125, | |
| "eval_samples_per_second": 30.141, | |
| "eval_steps_per_second": 0.476, | |
| "step": 100 | |
| } | |
| ], | |
| "logging_steps": 50, | |
| "max_steps": 165, | |
| "num_input_tokens_seen": 0, | |
| "num_train_epochs": 3, | |
| "save_steps": 20, | |
| "stateful_callbacks": { | |
| "EarlyStoppingCallback": { | |
| "args": { | |
| "early_stopping_patience": 5, | |
| "early_stopping_threshold": 0.0 | |
| }, | |
| "attributes": { | |
| "early_stopping_patience_counter": 1 | |
| } | |
| }, | |
| "TrainerControl": { | |
| "args": { | |
| "should_epoch_stop": false, | |
| "should_evaluate": false, | |
| "should_log": false, | |
| "should_save": true, | |
| "should_training_stop": false | |
| }, | |
| "attributes": {} | |
| } | |
| }, | |
| "total_flos": 3.010343436759859e+17, | |
| "train_batch_size": 16, | |
| "trial_name": null, | |
| "trial_params": null | |
| } | |