Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,129 @@
+---
+license: mit
+language:
+  - en
+library_name: pytorch
+pipeline_tag: text-generation
+tags:
+  - gpt
+  - from-scratch
+  - pytorch
+  - resume
+  - product-management
+  - career
+  - text-generation
+  - custom-tokenizer
+datasets:
+  - InferencePrince555/Resume-Dataset
+---
+
+# Sid-GPT: a 92M-parameter GPT built from scratch to write stronger PM resume bullets
+
+**Sid-GPT** takes a weak, responsibility-style resume bullet and rewrites it the way a resume that got selected for a product role would: strong action verb, clear scope, and a quantified result.
+
+> "Responsible for managing the product roadmap for the mobile app."
+> → "Managed the product roadmap for the mobile app, increasing the user engagement by 20%."
+
+Everything here was built end to end by one person on a single consumer GPU (RTX 3060, 12 GB): the tokenizer, the transformer, the pretraining loop, the fine-tuning pipeline, the evaluation, and the demo. No pretrained weights, no `AutoModel`, no borrowed checkpoints.
+
+## Highlights
+
+| | |
+|---|---|
+| Parameters | **91.6M** (12 layers, 12 heads, 768 hidden, 512 context) |
+| Tokenizer | Custom byte-level BPE, 8,000 vocab, trained on the corpus |
+| Pretraining | 8,000 steps over a ~47M-token resume corpus (36.6M public tokens plus consented real resumes), from random initialisation (≈77 minutes on one RTX 3060) |
+| Fine-tuning | 1,104 instruction pairs: 904 from consented candidate resumes + 200 hand-written coverage pairs |
+| Quantified rewrites | **0% → 93%** of unseen weak bullets come back with a metric-bearing result |
+| Held-out resume perplexity | **21.1** (down from the 14M-parameter baseline's 33) |
+| Scale-up | 14M → 92M parameters cut pretraining validation loss from 3.94 to 3.57 |
+
+## Quick examples
+
+Settings: `temperature=0.5`, `top_k=20`.
+
+| Weak bullet | Sid-GPT rewrite |
+|---|---|
+| Responsible for managing the product roadmap for the mobile app. | Managed the product roadmap for the mobile app, increasing the user engagement by 20%. |
+| Responsible for writing user stories and working with developers. | Created user stories and worked with developers and designers to improve user experience by 25%. |
+| Responsible for analyzing user behavior data with SQL. | Analyzed user behavior patterns in SQL to identify a drop-off point affecting 30% of trial users via funnel analysis in Excel. |
+| Responsible for building dashboards for the business team. | Built dashboards for the business team, improving the visibility of the business by 30%. |
+| Worked on an automation project to reduce manual work. | Rebuilt a self-serve workflow that eliminated manual review time by 40% and improved team productivity. |
+
+A larger, unfiltered batch of 30 generations is in [`samples.json`](samples.json).
+
+## Try it
+
+**In the browser:** open the companion Space, *Sid-GPT: PM resume bullet rewriter*.
+
+**In Python:**
+
+```bash
+pip install torch tokenizers huggingface_hub
+huggingface-cli download Sidhartha-Rajput/sid-gpt-resume-llm --local-dir sid-gpt
+cd sid-gpt
+python generate.py --prompt "Responsible for running A/B tests on the checkout page."
+```
+
+The prompt template used in fine-tuning is
+`Turn this into a strong, quantified PM resume bullet:\n<your bullet>`, which `generate.py` applies for you. Sampling at `--temperature 0.4–0.6 --top_k 20` gives the cleanest phrasing.
+
+## How it was built
+
+1. **Tokenizer.** Byte-level BPE (8k vocab) trained directly on resume text, so common resume vocabulary like "stakeholders", "KPIs", "A/B" and "cross-functional" becomes compact tokens.
+2. **Architecture.** A decoder-only GPT written in plain PyTorch: token and position embeddings, causal multi-head self-attention, GELU MLP blocks, pre-LayerNorm, weight-tied output head. Source in [`model.py`](model.py).
+3. **Pretraining.** Next-token prediction from random weights on a public resume corpus ([InferencePrince555/Resume-Dataset](https://huggingface.co/datasets/InferencePrince555/Resume-Dataset)) plus a set of consented real resumes. Mixed precision, AdamW, validation split held out from the public data.
+4. **Supervised fine-tuning.** Instruction/response pairs with loss computed on the response tokens only. Weak-to-strong pairs come from bullets in resumes of candidates who were selected for APM, PM, and BA roles (anonymised, with names and contact details removed), plus 200 hand-written pairs spanning growth, analytics, B2B, fintech, e-commerce, edtech, AI, and operations ([`data/synthetic_pairs.json`](data/synthetic_pairs.json)).
+5. **Evaluation.** A held-out set of 99 pairs from unseen candidates, a held-out resume perplexity measurement, a result-orientation check on 30 unseen weak bullets, and a generation scan confirming no verbatim reproduction of training text and no contact details in outputs.
+
+Training curves: [`pretrain_loss.png`](pretrain_loss.png). Numbers: [`training_summary.json`](training_summary.json) and [`results.json`](results.json).
+
+## Data and privacy
+
+- Candidates contributed their resumes with permission. Names, emails, phone numbers, and profile links were removed before any training text was produced, and candidates are identified only by a one-way hash.
+- The raw resumes and the real fine-tuning pairs are **not** included in this repository. Only the model, tokenizer, code, and the 200 hand-written pairs are published.
+- A scan of 30 generations found no verbatim 10-word span from the training resumes and no email, phone, or URL patterns.
+
+## What it is best at
+
+- Turning "Responsible for X" into an action-led, outcome-oriented line.
+- Giving you a structure to react to: verb → scope → measurable result.
+- Producing quick drafts that you then correct with your real context.
+
+Use it as a drafting partner. The figures it writes are illustrative placeholders that show *where* a metric belongs and *what kind* of metric fits; swap in your own real numbers before using any line on a resume. Prompts closest to its fine-tuning distribution (product, analytics, growth, operations bullets of one to two sentences) work best.
+
+## Roadmap
+
+- Scale to a ~350M-parameter model with a larger consented corpus.
+- Metric placeholders (`[X%]`) mode so users fill in their own numbers.
+- Role-aware rewriting (APM vs PM vs BA vs data analyst).
+- Preference tuning on rewrites that real hiring managers rank higher.
+- `safetensors` export and a `transformers`-compatible wrapper (`export_safetensors.py` is included).
+
+## Files
+
+| File | Purpose |
+|---|---|
+| `sft.pt` | Fine-tuned weights and config |
+| `model.py` | The GPT architecture |
+| `generate.py` | Standalone inference script |
+| `tokenizer/` | BPE vocab, merges, and `tokenizer.json` |
+| `config.json` | Model hyperparameters |
+| `data/synthetic_pairs.json` | The 200 hand-written weak→strong pairs |
+| `samples.json` | 30 generations at `temperature=0.5, top_k=20` |
+| `results.json`, `training_summary.json` | Metrics |
+| `pretrain_loss.png` | Pretraining curve |
+| `export_safetensors.py` | Optional weights export |
+
+## Citation
+
+```
+@misc{sidgpt2026,
+  title  = {Sid-GPT: a from-scratch GPT for product-management resume bullets},
+  author = {Sidhartha},
+  year   = {2026},
+  url    = {https://huggingface.co/Sidhartha-Rajput/sid-gpt-resume-llm}
+}
+```
+
+Released under the MIT license.