Sankar-2910/genz-translator

🚀 GenZ Translator

A compact, 29.65M-parameter language model designed to translate Gen Z slang into clear, natural English.

GenZ-Translator is a lightweight language model built for a focused task: understanding informal internet language and rewriting it in standard conversational English while preserving the original meaning, emotion, attitude, and context.

Version 2 continues training from the original GenZ Translator checkpoint, with additional cleaned translation data and curated examples targeting abbreviations, slang, and meaning-related questions.

• ⚡ Compact model for local inference
• 🧠 Specialized slang and abbreviation translation
• 🔤 Custom tokenizer
• 🤗 Compatible with the Transformers library
• 🪶 Approximately 30 million parameters

---

✨ What changed in v2?

Version 2 introduces a continued fine-tuning stage designed to improve translation quality, especially for common abbreviations and contextual slang.

Improvements observed in the current evaluation include:

• Better handling of common abbreviations such as WYD, tbh, omw, ty, and np
• Improved responses to prompts asking what slang expressions mean
• Better performance on the handcrafted evaluation benchmark
• Reduced loss on the curated validation examples

The model still has limitations involving unusual spellings, repeated characters, mixed-language text, URLs, and preservation of every detail in the input.

Important: Improvements vary by evaluation set. The current handcrafted benchmark shows a clear gain, but the held-out dataset benchmark does not show a corresponding improvement in exact-match accuracy.

---

📊 Model overview

Property	Value
Model name	GenZ Translator
Repository	Sankar-2910/genz-translator
Architecture	Llama-style decoder-only Transformer
Parameters	29,653,568
Layers	7
Hidden size	448
Attention heads	7
KV heads	7
Intermediate size	1,792
Context length	384 tokens
Vocabulary	8,000
Tokenizer	Custom Byte-Level BPE
Framework	PyTorch / Transformers
Weight format	Safetensors
Intended task	Single-turn slang translation

The architecture uses separate input embeddings and output-head weights (tie_word_embeddings=False).

---

🎯 Intended use

Suitable for

• Translating common internet abbreviations
• Rewriting informal messages into standard English
• Interpreting common Gen Z expressions
• Processing short social media captions and messages
• Lightweight experimentation with compact language models

Not designed for

• General-purpose conversation
• Coding or mathematical reasoning
• Knowledge retrieval
• Long-context conversations
• Reliable translation of every regional or rapidly evolving slang expression

This model is a specialized translator, not a general-purpose assistant.

---

🧪 Evaluation

Handcrafted benchmark

Category	Examples	v1 pass rate	v2 pass rate
Abbreviations	39	82%	100%
Contextual abbreviations	16	88%	94%
Meaning questions	11	27%	100%
Overflow	1	100%	100%
Paragraphs	5	100%	100%
Passthrough	8	75%	75%
Polarity	10	90%	90%
Meaning preservation	7	57%	71%
Robustness	37	86%	92%
Slang	34	91%	88%
Overall scored categories	—	80.2%	92.4%

The handcrafted benchmark measures whether outputs satisfy category-specific evaluation rules. Its results should not be interpreted as a universal translation accuracy percentage.

Held-out dataset benchmark

Metric	v1	v2
Exact match	15.2%	14.5%
Token F1	0.701	0.699
EOS-terminated outputs	100%	100%
Slang remaining in output	0.0%	0.1%
Definition-style outputs	0.1%	0.0%
Latency, batch size 32	4.1 ms/prompt	4.7 ms/prompt

Exact-match evaluation is strict: a valid translation can be marked incorrect if its wording differs from the reference. Token F1 offers a measure of lexical overlap but does not fully measure semantic correctness.

Validation loss during the v2 fine-tuning run

Validation metric	Before fine-tuning	Selected checkpoint
General validation loss	0.5268	0.5241
Curated validation loss	1.3334	0.2168
Combined validation loss	0.9301	0.3704

The combined loss is the arithmetic mean of general and curated validation losses. The curated validation set contained 82 examples, so its result should be interpreted cautiously.

These loss figures come from the current fine-tuning run. They should not be compared directly with validation losses from a different dataset split or evaluation pipeline.

Interpretation

The results suggest that v2 has learned useful mappings for common abbreviations and some meaning-related prompts. However, generalization remains a work in progress.

Known failure cases include:

• Losing negation or other important details
• Mistranslating phrase-level expressions such as pull up and stan
• Rewriting ordinary English that does not need translation
• Producing poor results on repeated characters and adversarially malformed inputs
• Mishandling URLs, usernames, and some mixed-language messages

Further evaluation on an independently held-out test set is needed before making stronger claims about general translation quality.

---

🏗️ Training approach

Original model

The original GenZ Translator was trained from scratch using a custom tokenizer and a specialized Gen Z-to-English dataset.

It did not start from pretrained Meta Llama, Qwen, or Mistral model weights.

Version 2

Version 2 continues from the original Sankar-2910/genz-translator checkpoint rather than starting from random initialization.

The current run used:

Setting	Value
Starting checkpoint	Sankar-2910/genz-translator
Training examples after preparation	333,993 tokenized examples
General validation examples	1,995 tokenized examples
Curated training pairs	437 before repetition
Curated validation pairs	82
Epochs	1
Batch size	32
Peak learning rate	5 × 10⁻⁵
Optimizer	AdamW
Learning-rate schedule	Cosine decay with warmup
Gradient clipping	1.0
Maximum sequence length	384 tokens
Hardware	Kaggle T4x2 GPU
Checkpoint selection	Combined general and curated validation loss

The dataset pipeline loaded a source dataset of approximately one million rows and filtered it before training. The figures above describe the prepared data for this run, not the full raw dataset.

Training examples included curated examples repeated during training. The held-out curated examples were kept separate from those repeated training examples.

Loss was computed over response tokens rather than prompt tokens.

---

🔤 Tokenizer and prompt format

The model uses a custom tokenizer trained for the original translation task.

Use the prompt structure expected by the model:

<s><|instruction|>Translate the following Gen Z slang sentence into clear, standard English.<|input|>YOUR TEXT<|response|>

The model is optimized for single-turn translation. It is not designed to maintain conversational history.

---

⚡ Quick start

Install the required packages:

pip install -U transformers torch safetensors

Load v2 with Transformers:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "Sankar-2910/genz-translator"

tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    tie_word_embeddings=False
).eval()

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)

def translate(text, max_new_tokens=96):
    prompt = (
        "<s><|instruction|>"
        "Translate the following Gen Z slang sentence into clear, standard English."
        "<|input|>" + text +
        "<|response|>"
    )

    inputs = tokenizer(
        prompt,
        add_special_tokens=False,
        return_tensors="pt"
    ).to(device)

    with torch.inference_mode():
        output = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=False,
            eos_token_id=tokenizer.eos_token_id,
            pad_token_id=tokenizer.eos_token_id
        )

    generated = output[0, inputs["input_ids"].shape[1]:]

    return tokenizer.decode(
        generated,
        skip_special_tokens=True
    ).strip()

print(translate("WYD"))
print(translate("tbh, that movie was mid"))
print(translate("omw, traffic is crazy"))

Greedy decoding (do_sample=False) provides repeatable outputs under the same model and inference conditions. It does not guarantee that every output will be accurate.

For longer paragraphs, increase max_new_tokens as needed.

---

⚠️ Limitations

GenZ Translator v2 is a compact, specialized model. It can still:

• Misinterpret ambiguous slang
• Lose negation, names, or contextual details
• Over-rewrite already-natural English
• Produce malformed outputs for adversarial or repetitive inputs
• Struggle with regional slang, multilingual text, and recent expressions
• Generate wording different from the reference despite retaining some meaning

Do not assume a translation is correct solely because it is fluent. For important text, verify the output.

---

📦 Repository files

The Hugging Face repository includes the model configuration, weights, and tokenizer files required for loading the model with Transformers.

The repository is intended to distribute the standard safetensors checkpoint. Quantized or GGUF variants are not included unless published separately.

---

📖 Citation

If you use this model in research, please cite:

@misc{genztranslator2026v2,
  title={GenZ Translator v2},
  author={Sankar Narayanan},
  year={2026},
  note={A 29.65M-parameter decoder-only Transformer for Gen Z slang translation, continued from an earlier custom-trained checkpoint},
  url={https://huggingface.co/Sankar-2910/genz-translator-v2}
}

For the underlying Llama architecture, see:

@article{touvron2023llama,
  title={LLaMA: Open and Efficient Foundation Language Models},
  author={Touvron, Hugo and others},
  journal={arXiv preprint arXiv:2302.13971},
  year={2023}
}

---

❤️ Built by a student

GenZ Translator is an exploration of compact language models, custom tokenization, dataset preparation, and specialized natural-language generation.

Created by Sankara Narayanan.

Project: Sankar-2910/genz-translator-v2