agoel7029 commited on
Commit
39e4d9e
·
verified ·
1 Parent(s): 24201af

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +61 -0
README.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ tags:
4
+ - gpt
5
+ - weather
6
+ - character-level
7
+ - pytorch
8
+ - nano
9
+ license: mit
10
+ ---
11
+
12
+ # nanoWeatherGPT
13
+
14
+ A character-level GPT (~813K parameters) trained from scratch on synthetic weather forecast text.
15
+
16
+ ## Download & Use
17
+
18
+ ```python
19
+ from huggingface_hub import snapshot_download
20
+ import torch, pickle, sys
21
+
22
+ # Download all files
23
+ path = snapshot_download("agoel7029/nano-weather-gpt-model")
24
+ sys.path.insert(0, path)
25
+
26
+ from model import GPT
27
+
28
+ ckpt = torch.load(f"{path}/model.pt", map_location="cpu", weights_only=False)
29
+ meta = pickle.load(open(f"{path}/meta.pkl", "rb"))
30
+ stoi, itos = meta["stoi"], meta["itos"]
31
+
32
+ model = GPT(ckpt["config"])
33
+ model.load_state_dict(ckpt["model_state_dict"])
34
+ model.eval()
35
+
36
+ prompt = "Today the weather is"
37
+ ctx = torch.tensor([[stoi[c] for c in prompt]], dtype=torch.long)
38
+ with torch.no_grad():
39
+ out = model.generate(ctx, max_new_tokens=200)
40
+ print("".join(itos[i] for i in out[0].tolist()))
41
+ ```
42
+
43
+ ## Fine-tune on your own data
44
+
45
+ Edit `train.py` — replace `generate_weather_corpus()` with your own text, then run:
46
+
47
+ ```bash
48
+ python train.py
49
+ ```
50
+
51
+ ## Model details
52
+
53
+ | Property | Value |
54
+ |---|---|
55
+ | Architecture | Character-level GPT (Transformer) |
56
+ | Parameters | ~813K |
57
+ | Layers | 4 |
58
+ | Attention heads | 4 |
59
+ | Embedding dim | 128 |
60
+ | Context length | 128 characters |
61
+ | Trained on | Synthetic Swiss city weather reports |