Instructions to use typer-org/booper-story-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use typer-org/booper-story-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="typer-org/booper-story-v2")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("typer-org/booper-story-v2")
model = AutoModelForCausalLM.from_pretrained("typer-org/booper-story-v2", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use typer-org/booper-story-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "typer-org/booper-story-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "typer-org/booper-story-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/typer-org/booper-story-v2

SGLang

How to use typer-org/booper-story-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "typer-org/booper-story-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "typer-org/booper-story-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "typer-org/booper-story-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "typer-org/booper-story-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use typer-org/booper-story-v2 with Docker Model Runner:
```
docker model run hf.co/typer-org/booper-story-v2
```

booper-story-v2

Long-form SFT of ProCreations/Booper-Big-Chat-INT8 (Mixtral MoE, 150M total / ~50M active, vocab 16384) so booper answers story/long-answer requests instead of one-liners. Trained with sft/sft_longform.py from https://github.com/frgmt0/babble in the pair layout <bos> prompt <sep> response <eos> (loss on the response).

Data mix

tinystories: 32000 examples
writingprompts: 40000 examples
no_robots: 16000 examples
smoltalk: 16000 examples
discord: 56000 examples

Training

steps: 1220, tokens: 37,022,555
val loss: 2.0340 -> 1.7995
device: mps, lr 4e-05, seq len 1024

Samples (temperature 0.8)

write me a short story about a dragon who is afraid of fire

Once upon a time, there was a dragon who lived in the forest. He lived in a small and strong forest, where he had to be careful of trees. One day, the dragon heard a noise from his neighbors. He told the dragon that he wanted to play with his friends, but it was too late.

write a short story about a detective in a city that never sleeps

Once upon a time there was a very strong detective. It would be a very good night until it was time to go back to sleep. One day, the detective had been up for the night. He was getting ready to go. As he was looking around, he saw a big room in his right hand. The detective

hey booper whats up

i gotta go on a walk

Loading

Same INT8 layout as the base: load_int8.py in this repo, or babble.hfserve with BABBLE_HF_MODEL_DIR pointed at a snapshot.

Downloads last month: 268

Model tree for typer-org/booper-story-v2

Base model

ProCreations/Booper-Big

Finetuned

ProCreations/Booper-Big-Chat

Finetuned

ProCreations/Booper-Big-Chat-INT8

Finetuned

(2)

this model