Instructions to use p-e-w/Qwen3-0.6B-heretic-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use p-e-w/Qwen3-0.6B-heretic-lora with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="p-e-w/Qwen3-0.6B-heretic-lora")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("p-e-w/Qwen3-0.6B-heretic-lora", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use p-e-w/Qwen3-0.6B-heretic-lora with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "p-e-w/Qwen3-0.6B-heretic-lora" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "p-e-w/Qwen3-0.6B-heretic-lora", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/p-e-w/Qwen3-0.6B-heretic-lora
- SGLang
How to use p-e-w/Qwen3-0.6B-heretic-lora with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "p-e-w/Qwen3-0.6B-heretic-lora" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "p-e-w/Qwen3-0.6B-heretic-lora", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "p-e-w/Qwen3-0.6B-heretic-lora" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "p-e-w/Qwen3-0.6B-heretic-lora", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use p-e-w/Qwen3-0.6B-heretic-lora with Docker Model Runner:
docker model run hf.co/p-e-w/Qwen3-0.6B-heretic-lora
| absl-py==2.4.0 | |
| accelerate==1.14.0 | |
| alembic==1.18.4 | |
| annotated-doc==0.0.4 | |
| annotated-types==0.7.0 | |
| anyio==4.6.0 | |
| bitsandbytes==0.49.2 | |
| certifi==2024.8.30 | |
| chardet==6.0.0.post1 | |
| charset-normalizer==3.3.2 | |
| click==8.4.1 | |
| colorama==0.4.6 | |
| colorlog==6.10.1 | |
| dataproperty==1.1.1 | |
| datasets==4.8.5 | |
| dill==0.4.1 | |
| evaluate==0.4.6 | |
| filelock==3.13.1 | |
| fsspec==2024.2.0 | |
| greenlet==3.5.1 | |
| h11==0.14.0 | |
| heretic-llm==1.4.0 | |
| hf-xet==1.5.1 | |
| httpcore==1.0.5 | |
| httpx==0.27.2 | |
| huggingface-hub==1.19.0 | |
| idna==3.10 | |
| immutabledict==4.3.1 | |
| importlib-metadata==4.6.4 | |
| jinja2==3.1.3 | |
| joblib==1.5.3 | |
| langdetect==1.0.9 | |
| lm-eval==0.4.12 | |
| lxml==5.3.0 | |
| mako==1.3.12 | |
| markdown-it-py==4.2.0 | |
| markupsafe==2.1.5 | |
| mbstrdecoder==1.1.5 | |
| mdurl==0.1.2 | |
| more-itertools==8.10.0 | |
| mpmath==1.3.0 | |
| multiprocess==0.70.19 | |
| narwhals==2.22.1 | |
| networkx==3.2.1 | |
| nltk==3.9.4 | |
| numpy==2.4.6 | |
| nvidia-cublas-cu12==12.4.2.65 | |
| nvidia-cuda-cupti-cu12==12.4.99 | |
| nvidia-cuda-nvrtc-cu12==12.4.99 | |
| nvidia-cuda-runtime-cu12==12.4.99 | |
| nvidia-cudnn-cu12==9.1.0.70 | |
| nvidia-cufft-cu12==11.2.0.44 | |
| nvidia-curand-cu12==10.3.5.119 | |
| nvidia-cusolver-cu12==11.6.0.99 | |
| nvidia-cusparse-cu12==12.3.0.142 | |
| nvidia-nccl-cu12==2.20.5 | |
| nvidia-nvjitlink-cu12==12.4.99 | |
| nvidia-nvtx-cu12==12.4.99 | |
| optuna==4.9.0 | |
| packaging==24.1 | |
| pandas==3.0.3 | |
| pathvalidate==3.3.1 | |
| peft==0.19.1 | |
| pillow==10.2.0 | |
| portalocker==3.2.0 | |
| prompt-toolkit==3.0.47 | |
| psutil==7.2.2 | |
| py-cpuinfo==9.0.0 | |
| pyarrow==24.0.0 | |
| pydantic==2.13.4 | |
| pydantic-core==2.46.4 | |
| pydantic-settings==2.14.1 | |
| pygments==2.18.0 | |
| pytablewriter==1.2.1 | |
| python-dateutil==2.9.0.post0 | |
| python-dotenv==1.2.2 | |
| pyyaml==6.0.2 | |
| questionary==2.1.1 | |
| regex==2026.5.9 | |
| requests==2.32.3 | |
| rich==14.3.4 | |
| rouge-score==0.1.2 | |
| sacrebleu==2.6.0 | |
| safetensors==0.8.0 | |
| scikit-learn==1.9.0 | |
| scipy==1.17.1 | |
| setuptools==75.1.0 | |
| shellingham==1.5.4 | |
| six==1.16.0 | |
| sniffio==1.3.1 | |
| sqlalchemy==2.0.50 | |
| sqlitedict==2.1.0 | |
| sympy==1.12 | |
| tabledata==1.3.5 | |
| tabulate==0.10.0 | |
| tcolorpy==0.1.7 | |
| threadpoolctl==3.6.0 | |
| tokenizers==0.22.2 | |
| tomli==2.0.1 | |
| tomli-w==1.2.0 | |
| torch==2.4.1 | |
| torchaudio==2.4.1 | |
| torchvision==0.19.1 | |
| tqdm==4.68.2 | |
| transformers==5.12.0 | |
| triton==3.0.0 | |
| typepy==1.3.5 | |
| typer==0.25.1 | |
| typing-extensions==4.15.0 | |
| typing-inspection==0.4.2 | |
| urllib3==2.2.3 | |
| wcwidth==0.2.13 | |
| word2number==1.1 | |
| xxhash==3.7.0 | |