Instructions to use yuanxin112/babylm-deu-bpe16k-gpt2-100m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use yuanxin112/babylm-deu-bpe16k-gpt2-100m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="yuanxin112/babylm-deu-bpe16k-gpt2-100m")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("yuanxin112/babylm-deu-bpe16k-gpt2-100m")
model = AutoModelForCausalLM.from_pretrained("yuanxin112/babylm-deu-bpe16k-gpt2-100m", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use yuanxin112/babylm-deu-bpe16k-gpt2-100m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "yuanxin112/babylm-deu-bpe16k-gpt2-100m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yuanxin112/babylm-deu-bpe16k-gpt2-100m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/yuanxin112/babylm-deu-bpe16k-gpt2-100m

SGLang

How to use yuanxin112/babylm-deu-bpe16k-gpt2-100m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "yuanxin112/babylm-deu-bpe16k-gpt2-100m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yuanxin112/babylm-deu-bpe16k-gpt2-100m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "yuanxin112/babylm-deu-bpe16k-gpt2-100m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yuanxin112/babylm-deu-bpe16k-gpt2-100m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use yuanxin112/babylm-deu-bpe16k-gpt2-100m with Docker Model Runner:
```
docker model run hf.co/yuanxin112/babylm-deu-bpe16k-gpt2-100m
```

GPT-2 (BabyLM deu-baseline-small recipe), German, vocab 16384 — ~108M words

German GPT-2 trained from scratch under the exact recipe of BabyLM-community/deu-baseline-small. The ONLY deviation is the vocabulary: 16384 instead of 8192 (a locally trained byte-level BPE, same construction as that corpus's baseline series). Everything else — architecture and every training hyperparameter — is reproduced field-for-field; see recipe.json in this repo for the machine-readable record.

Architecture (= official card, except vocab)


architecture	GPT2LMHeadModel
n_layer / n_head	4 / 8
n_embd / n_inner	512 / 2048
n_positions = n_ctx	512
activation	gelu
dropouts (attn/embd/resid)	0.1 / 0.1 / 0.1
initializer_range / ln eps	0.02 / 1e-5
vocab_size	16384 (official: 8192)
params	21.3M (official: 17.1M — the difference is the embedding table)

Training (= official card, verbatim)

learning_rate 1e-4, linear decay (no warmup), 5 epochs, train batch 64, eval batch 8, seed 42, AdamW betas (0.9, 0.999) eps 1e-8, weight_decay 0, fp32, block 512, glosses of unspecified card fields follow HF defaults. Steps/epoch 4,093 (official card: 4,944 at vocab 8192 on the 100M-word corpus — the smaller count here reflects the larger vocabulary's lower fertility, not less data).

Data: German BabyLM corpus, ~108M words (local reconstruction babylm_deu.txt — EN/DE-matched genre shares; NOT the official hub download). 1% held out for eval: final eval_loss 3.56. Note the official deu-baseline-small itself was trained on the 100M-word corpus (its "small" names the model, not the data), so the -100m repo of this pair is the direct-scale counterpart; its 3.56 vs the official 3.0813 is not an apples-to-apples number (different vocab size changes per-token entropy — smaller vocab → lower per-token loss; different corpus reconstruction and eval split).

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("yuanxin112/babylm-deu-bpe16k-gpt2-100m")
model = AutoModelForCausalLM.from_pretrained("yuanxin112/babylm-deu-bpe16k-gpt2-100m")
ids = tok("Die Frauen hätten mit ihrem Bruder", return_tensors="pt")
print(tok.decode(model.generate(**ids, max_new_tokens=20, do_sample=False,
                                pad_token_id=tok.pad_token_id)[0],
                 skip_special_tokens=True))

Tokenizer: byte-level BPE, specials [PAD]=0 [UNK]=1 [BOS]=2 [EOS]=3 (unlike the official baseline's config, whose bos/eos fields carry the stale GPT-2 default 50256 that is not even in its 8192 vocab).

Downloads last month: 147

Safetensors

Model size

21.3M params

Tensor type

F32