Instructions to use Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k")
model = AutoModelForCausalLM.from_pretrained("Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k

SGLang

How to use Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Unsloth Desktop
Docker Model Runner
How to use Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k with Docker Model Runner:
```
docker model run hf.co/Ramikan-BR/tinyllama_PY-CODER-bnb-4bit-lora_model-4k
```

Ramikan-BR commited on Apr 25, 2024

Commit

07dde94

verified ·

1 Parent(s): ce792c2

Step and Loss

Browse files

==((====))== Unsloth - 2x faster free finetuning | Num GPUs = 1
\\ /| Num examples = 967 | Num Epochs = 1
O^O/ \_/ \ Batch size per device = 2 | Gradient Accumulation steps = 16
\ / Total batch size = 32 | Total steps = 30
"-____-" Number of trainable parameters = 100,925,440
[30/30 26:26, Epoch 0/1]
Step Training Loss
1 1.737000
2 1.738000
3 1.384700
4 1.086400
5 1.009600
6 0.921000
7 0.830400
8 0.808900
9 0.774500
10 0.759900
11 0.736100
12 0.721200
13 0.733200
14 0.701000
15 0.711700
16 0.701400
17 0.689500
18 0.678800
19 0.675200
20 0.680500
21 0.685800
22 0.681200
23 0.672000
24 0.679900
25 0.675500
26 0.666600
27 0.687900
28 0.653600
29 0.672500
30 0.660900

Files changed (1) hide show

README.md +0 -38

README.md CHANGED Viewed

@@ -14,44 +14,6 @@ tags:
 base_model: unsloth/tinyllama-bnb-4bit
 ---
---- >-
-==((====))== Unsloth - 2x faster free finetuning | Num GPUs = 1 \\ /| Num
-examples = 967 | Num Epochs = 1 O^O/ \_/ \ Batch size per device = 2 | Gradient
-Accumulation steps = 16 \ / Total batch size = 32 | Total steps = 30 "-____-"
-Number of trainable parameters = 100,925,440 [30/30 26:26, Epoch 0/1] Step
-Training Loss
-1 1.737000
-2 1.738000
-3 1.384700
-4 1.086400
-5 1.009600
-6 0.921000
-7 0.830400
-8 0.808900
-9 0.774500
-10 0.759900
-11 0.736100
-12 0.721200
-13 0.733200
-14 0.701000
-15 0.711700
-16 0.701400
-17 0.689500
-18 0.678800
-19 0.675200
-20 0.680500
-21 0.685800
-22 0.681200
-23 0.672000
-24 0.679900
-25 0.675500
-26 0.666600
-27 0.687900
-28 0.653600
-29 0.672500
-30 0.660900
 ---
 null

 base_model: unsloth/tinyllama-bnb-4bit
 ---
 ---
 null