---
language:
- en
base_model:
- talkie-lm/talkie-1930-13b-it
license: apache-2.0
pipeline_tag: text-generation
tags:
- talkie
- vllm
---

# talkie-1930-13b-it (vLLM-servable repackage)

This is a HuggingFace + vLLM-ready repackage of [`talkie-lm/talkie-1930-13b-it`](https://huggingface.co/talkie-lm/talkie-1930-13b-it). The original release ships as a raw torch state-dict (`rl-refined.pt`) plus a tiktoken vocab, with no `config.json`, `tokenizer.json`, or HF modeling code, so it can't be loaded by `transformers` or served by vLLM out of the box.

`talkie-1930-13b-it` is a 13B parameter instruction-tuned model from the [talkie-lm](https://talkie-lm.com/) project. The base model was pretrained on ~260B tokens of pre-1931 English text; the IT variant was instruction-tuned on a dataset built from pre-1931 reference works (etiquette manuals, encyclopedias, letter-writing guides) and refined with online DPO.

## What this repo adds

| File | What it is |
|---|---|
| `model.safetensors` | bf16 weights, ~25 GB. `lm_head_gain` (a learned scalar) is pre-multiplied into `lm_head.weight` so vLLM's transformers backend doesn't need to know about it. |
| `config.json` | `TalkieConfig` (vocab=65540, hidden=5120, 40 layers × 40 heads, head_dim=128, ctx=2048, RoPE θ=1e6) plus `auto_map` for `AutoConfig`/`AutoModel`/`AutoModelForCausalLM`. |
| `tokenizer.json`, `tokenizer_config.json` | HF fast BPE built from the original `vocab.txt`, with the 5 chat specials at fixed ids 65535..65539. EOS = `<\|end\|>`, pad = `<\|endoftext\|>`. |
| `chat_template.jinja` | Renders to `<\|system\|>…<\|end\|><\|user\|>…<\|end\|><\|assistant\|>…<\|end\|><\|assistant\|>`, byte-matching `format_chat` from the official inference repo. |
| `generation_config.json` | `eos_token_id=[65536, 65535]`, `pad_token_id=65535`. |
| `modeling_talkie.py`, `configuration_talkie.py` | HF `PreTrainedModel` implementation with `ALL_ATTENTION_FUNCTIONS` dispatch (vLLM transformers-backend compatible). Adapted from [ricdomolm/1930-coder](https://huggingface.co/collections/ricdomolm/1930-coder); `TalkieForCausalLM.lm_head` is an `nn.Linear` so the same `lm_head.weight` blob loads for both HF and vLLM. |

## Serving with vLLM

Tested with vLLM 0.19, transformers backend, on a single H100 (80 GB). bf16 only — fp8 is broken on this architecture.

```bash
vllm serve awilliamson/talkie-1930-13b-it-vllm \
    --model-impl transformers \
    --trust-remote-code \
    --dtype bfloat16 \
    --max-model-len 2048
```

Then hit it like any OpenAI-style chat endpoint:

```bash
curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "awilliamson/talkie-1930-13b-it-vllm",
        "messages": [{"role":"user","content":"Write one sentence about the year 1925."}],
        "temperature": 0.7,
        "max_tokens": 80
    }'
```

### Sampling notes

- **Use `temperature ≥ 0.5`** — greedy decoding (`temperature=0`) can collapse into single-token loops on this model.
- `top_p` / `top_k` and `repetition_penalty` don't reliably help with that failure mode; temperature does.
- Default `max_position_embeddings=2048` matches the original IT training. The talkie-coder SWE recipe extends to 64K with NTK `rope_theta=4e7`, but loses ~14% on short evals (GSM8K) — only worth it for long-context agentic use.

## Plain HuggingFace usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tok = AutoTokenizer.from_pretrained("awilliamson/talkie-1930-13b-it-vllm", trust_remote_code=True)
m = AutoModelForCausalLM.from_pretrained(
    "awilliamson/talkie-1930-13b-it-vllm",
    trust_remote_code=True,
    dtype=torch.bfloat16,
).cuda().eval()

chat = tok.apply_chat_template(
    [{"role": "user", "content": "Write one sentence about the year 1925."}],
    tokenize=False, add_generation_prompt=True,
)
ids = tok([chat], return_tensors="pt").to("cuda")
out = m.generate(
    **ids, max_new_tokens=80, do_sample=True, temperature=0.7, top_p=0.9,
    pad_token_id=tok.pad_token_id,
    eos_token_id=[tok.convert_tokens_to_ids("<|end|>"), tok.eos_token_id],
)
print(tok.decode(out[0, ids.input_ids.shape[1]:], skip_special_tokens=True))
```

## Provenance

- Weights: `rl-refined.pt` from `talkie-lm/talkie-1930-13b-it` (bf16, vocab=65540, `lm_head_gain.w_g=3.890625` baked in).
- Tokenizer: built from `vocab.txt` from the same release (truncated to ranks < 65535, then the 5 chat specials appended at fixed ids).
- Modeling code: adapted from [ricdomolm/1930-coder/sft/modeling_talkie.py](https://github.com/ricardodominguez/1930-coder), with `TalkieForCausalLM.lm_head` switched from `nn.Parameter` to `nn.Linear` so the baked-in `lm_head.weight` loads cleanly for both HF and vLLM.

For the full inference reference and CLI, see the official [talkie-lm/talkie](https://github.com/talkie-lm/talkie) repo.

## License

Apache 2.0, matching the upstream `talkie-lm/talkie-1930-13b-it` release.