Instructions to use xlr8harder/talkie-1930-13b-it-tf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use xlr8harder/talkie-1930-13b-it-tf with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="xlr8harder/talkie-1930-13b-it-tf", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("xlr8harder/talkie-1930-13b-it-tf", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use xlr8harder/talkie-1930-13b-it-tf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "xlr8harder/talkie-1930-13b-it-tf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xlr8harder/talkie-1930-13b-it-tf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/xlr8harder/talkie-1930-13b-it-tf

SGLang

How to use xlr8harder/talkie-1930-13b-it-tf with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "xlr8harder/talkie-1930-13b-it-tf" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xlr8harder/talkie-1930-13b-it-tf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "xlr8harder/talkie-1930-13b-it-tf" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xlr8harder/talkie-1930-13b-it-tf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use xlr8harder/talkie-1930-13b-it-tf with Docker Model Runner:
```
docker model run hf.co/xlr8harder/talkie-1930-13b-it-tf
```

talkie-1930-13b-it-tf (Transformers + safetensors conversion)

This repository is a Transformers-compatible conversion of talkie-lm/talkie-1930-13b-it, the original Talkie instruction-tuned chat model.

The upstream model is an instruction-tuned post-train of talkie-lm/talkie-1930-13b-base, fine-tuned from instruction-response pairs extracted from pre-1931 reference works and then refined with online DPO, according to the original model card.

The upstream instruction-tuned checkpoint is already BF16. This repository adds Transformers AutoModelForCausalLM / AutoTokenizer support, a chat template matching the Talkie reference code, and BF16 sharded safetensors.

This is not an official Talkie release; refer to the upstream model card for the author-provided provenance and usage notes.

Source Model

Original model: talkie-lm/talkie-1930-13b-it
Talkie report: talkie-lm.com
Reference code: github.com/talkie-lm/talkie

Conversion Details

Weight dtype: BF16
Weight format: sharded safetensors
Context length: 4096 tokens
Architecture: custom Talkie code loaded with trust_remote_code=True
Tokenizer: Talkie tiktoken-compatible tokenizer exposed through AutoTokenizer

The public reference configuration originally advertised 2,048 positions, but the Talkie team later clarified that the model was trained with a 4,096-token context. This conversion uses the corrected 4,096-token limit.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

path = "xlr8harder/talkie-1930-13b-it-tf"
tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    path,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map={"": "cuda"},
    use_safetensors=True,
)

For chat-style prompts:

messages = [{"role": "user", "content": "Write an essay predicting life in 1960."}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
).to("cuda")
output = model.generate(**inputs, max_new_tokens=128)
reply = output[0, inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(reply, skip_special_tokens=True))

vLLM

The included remote-code model implements the Transformers attention-interface hooks expected by vLLM's Transformers modeling backend. For compatibility with that backend, the original single-scalar lm_head_gain is folded into lm_head.weight during conversion; the other Talkie gain parameters remain explicit model parameters. Using vLLM's logit_scale-style approach was not used because it applies scaling after the output matmul, while Talkie applies the gain to the head weight before the matmul. In BF16 this can introduce small rounding differences and, in smoke tests, changed one near-tied top-token ordering.

vllm serve xlr8harder/talkie-1930-13b-it-tf \
  --task generate \
  --model-impl transformers \
  --trust-remote-code \
  --dtype bfloat16 \
  --max-model-len 4096

Validation

The Transformers safetensors model was compared against the original Talkie IT checkpoint on a forward-pass smoke test. The top-10 next-token ordering matched exactly; observed max absolute logit difference was 0.25.