Export weights and config from checkpoint latest_checkpoint.pt

README.md

@@ -1,3 +1,63 @@
 ---
+language:
+- en
+- akk
 license: mit
+base_model:
+- AlgoDriveAI/Sanskrit_Akkadian_LLM
+tags:
+- dense-llm
+- akkadian
+- old-babylonian
+- ancient-languages
+- english
+- translation
+- causal-lm
+- custom-architecture
+- research
+pipeline_tag: text-generation
 ---
+
+# Akkadian-English DenseLLM 1B
+
+This is an experimental DenseLLM model focused on **English ↔ Akkadian / Old Babylonian translation and transliteration work**.
+
+This version is a continuation of the earlier `AlgoDriveAI/Sanskrit_Akkadian_LLM` model, but the project direction has changed. The previous public version mixed **English, Sanskrit, Akkadian, and some auxiliary training material**. For this release, the model has been expanded to approximately **1B parameters** and narrowed to focus on **English and Akkadian only**.
+
+## Architecture
+
+| Component | Details |
+|---|---|
+| Type | Custom Dense Transformer / DenseLLM |
+| Parameters | ~1B |
+| Context Length | 2048 tokens |
+| Positional Encoding | RoPE |
+| Activation | SwiGLU |
+| Normalization | RMSNorm |
+| Task Type | Causal Language Modeling |
+
+| Hyperparameter | Value |
+|---|---|
+| d_model | 1280 |
+| n_layers | 30 |
+| n_heads | 32 |
+| n_kv_heads | unknown |
+| ff_mult | unknown |
+| vocab_size | 32000 |
+
+## Known Limitations
+
+- **Not a scholarly authority:** Outputs should be checked against reliable Akkadian grammars, dictionaries, and corpora.
+- **Hallucinated forms:** The model may invent plausible-looking Akkadian words or endings.
+- **Translation uncertainty:** Akkadian is highly context-dependent, and short isolated sentences may have multiple possible readings.
+
+## Contact
+
+**Organization:** AlgoDriveAI  
+**Author:** Christopher Smith  
+**Base model / previous version:** [AlgoDriveAI/Sanskrit_Akkadian_LLM](https://huggingface.co/AlgoDriveAI/Sanskrit_Akkadian_LLM)  
+**Repository:** https://huggingface.co/AlgoDriveAI/Akkadian_English_DenseLLM_1B
+
+## License
+
+MIT

config.json

@@ -0,0 +1,10 @@
+{
+  "vocab_size": 32000,
+  "d_model": 1280,
+  "n_layers": 30,
+  "n_heads": 32,
+  "max_seq_len": 2048,
+  "architectures": [
+    "DenseLLM"
+  ]
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "max_new_tokens": 200,
+  "temperature": 0.55,
+  "top_k": 35,
+  "top_p": 0.88,
+  "do_sample": true
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd702d2a0503cdd46237e10df6f2a5be5ba5d24e48adebb550c3642b1f917214
+size 2337936635

requirements.txt

@@ -0,0 +1,4 @@
+torch
+transformers
+huggingface_hub
+einops