Add model card with usage instructions and license

README.md

@@ -0,0 +1,160 @@
+---
+license: apache-2.0
+base_model: tiiuae/Falcon-E-3B-Instruct
+tags:
+  - bitnet
+  - 1.58-bit
+  - code
+  - text-generation
+  - falcon
+  - ternary
+datasets:
+  - m-a-p/CodeFeedback-Filtered-Instruction
+  - ise-uiuc/Magicoder-OSS-Instruct-75K
+  - ise-uiuc/Magicoder-Evol-Instruct-110K
+language:
+  - en
+---
+
+# Falcon-Coder-3B (1.58-bit / TQ1_0)
+
+A fine-tuned 1.58-bit ternary quantization of [Falcon-E-3B-Instruct](https://huggingface.co/tiiuae/Falcon-E-3B-Instruct), optimized for **CPU inference** via vanilla [llama.cpp](https://github.com/ggerganov/llama.cpp).
+
+This model produces code (Python, TypeScript, etc.) at ~24 tokens/sec on a typical laptop CPU with a 710 MB on-disk footprint.
+
+## Model Details
+
+| Property | Value |
+|----------|-------|
+| Base model | tiiuae/Falcon-E-3B-Instruct |
+| Training method | 1.58-bit full fine-tune via [onebitllms](https://github.com/tiiuae/onebitllms) |
+| Training data | 365k coding instruction examples (Magicoder + CodeFeedback) |
+| Training duration | ~92 hours on RTX 4090 (24 GB) |
+| Final loss | 0.5008 (started at 0.91) |
+| Effective batch size | 32 (per_device=1 × grad_accum=32) |
+| Optimizer | paged_adamw_8bit |
+| Learning rate | 1e-4 (cosine schedule, 3% warmup) |
+| Sequence length | 1024 |
+| Epochs | 2 |
+| Stored precision | BF16 (5.7 GB) |
+| **Inference precision** | **TQ1_0 ternary (1.69 bpw, ~710 MB)** |
+| **Inference engine** | **vanilla llama.cpp** (TQ1_0 quant) |
+| Inference speed | ~24 tok/s on laptop CPU |
+
+## What's in the repo
+
+This is the **training-time BF16 checkpoint**. To use it on CPU, you must convert it to a 1.58-bit ternary GGUF. See "Usage" below.
+
+## Usage
+
+### Inference on CPU (recommended)
+
+This BF16 model is too large for fast CPU inference. **Convert to a 1.58-bit ternary GGUF first:**
+
+```powershell
+# 1. Download the BF16 model
+hf download anthonylee991/falcon-coder-3b --local-dir falcon-coder-3b-bf16
+
+# 2. Convert to F16 GGUF
+python llama.cpp/convert_hf_to_gguf.py falcon-coder-3b-bf16 `
+    --outfile falcon-coder-3b.gguf --outtype f16
+
+# 3. Quantize to TQ1_0 (1.58-bit ternary, ~710 MB)
+llama.cpp/build/bin/Release/llama-quantize.exe falcon-coder-3b.gguf `
+    falcon-coder-3b-tq1.gguf TQ1_0 8
+
+# 4. Run inference
+llama.cpp/build/bin/Release/llama-cli.exe `
+    -m falcon-coder-3b-tq1.gguf `
+    -p "def fibonacci(n):" -n 100 --threads 8
+```
+
+### Inference on GPU (BF16)
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model = AutoModelForCausalLM.from_pretrained(
+    "anthonylee991/falcon-coder-3b",
+    torch_dtype=torch.bfloat16,
+    device_map="cuda",
+)
+tokenizer = AutoTokenizer.from_pretrained("anthonylee991/falcon-coder-3b")
+
+prompt = "def quicksort(arr):"
+inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
+output = model.generate(**inputs, max_new_tokens=200, do_sample=False)
+print(tokenizer.decode(output[0], skip_special_tokens=True))
+```
+
+## Intended Use
+
+This model is a **code generation assistant**. Verified strong performance on:
+
+- ✅ Pure algorithms (binary search, sort, recursive functions)
+- ✅ Type definitions (TypeScript interfaces, Pydantic models)
+- ✅ Test scaffolding (pytest, Jest)
+- ✅ Mechanical refactors (if/elif → dict dispatch)
+- ✅ Docstrings (Google-style with examples)
+
+Verified **weak** performance on:
+
+- ⚠️ PowerShell (uses deprecated cmdlets like `Get-WmiObject`)
+- ⚠️ Complex business logic with multiple interacting rules
+- ⚠️ Anything requiring framework-specific knowledge not in training data
+
+For a detailed 10-test evaluation, see [the project repository](https://huggingface.co/anthonylee991/falcon-coder-3b) or the companion HOW-TO guide.
+
+## Training Data
+
+Combined and deduplicated from:
+
+| Dataset | Rows | Purpose |
+|---------|------|---------|
+| [m-a-p/CodeFeedback-Filtered-Instruction](https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction) | ~156k | High-quality code instructions with feedback |
+| [ise-uiuc/Magicoder-OSS-Instruct-75K](https://huggingface.co/datasets/ise-uiuc/Magicoder-OSS-Instruct-75K) | ~75k | Magicoder-style OSS examples |
+| [ise-uiuc/Magicoder-Evol-Instruct-110K](https://huggingface.co/datasets/ise-uiuc/Magicoder-Evol-Instruct-110K) | ~110k | Evolved instructions |
+| Various smaller PowerShell/TypeScript corpuses | ~30k | Multi-language coverage |
+
+After deduplication via MinHash LSH @ 0.85: **365,251 train rows + 2,000 eval rows**.
+
+The training data is generic Python code. **PowerShell, FastAPI, and TypeScript quality is limited** compared to Python. See the V2 plan in the project docs for how to address this.
+
+## Limitations
+
+- **PowerShell quality is poor** — the model defaults to deprecated cmdlets. Use a more recent code model for PowerShell or fine-tune on PS-specific data.
+- **Framework-specific code** (FastAPI deps, SQLAlchemy patterns, React state management) is hit-or-miss.
+- **No held-out domain eval** — the eval split was drawn from the same training distribution.
+- **Small model (3B)** — complex reasoning across multiple files is out of scope.
+- **Output may include explanatory prose** — extract code blocks from the response, don't paste the whole output into your code.
+
+## Training Infrastructure
+
+- Cloud GPU: Hivenet GPU-optimized container, single RTX 4090 (24 GB VRAM)
+- 92 hours wall time, $40 approximate cost
+- BF16 + 8-bit AdamW + gradient checkpointing to fit in 24 GB
+
+## License
+
+This model is released under the **Apache 2.0 license**, consistent with the base [Falcon-E-3B-Instruct](https://huggingface.co/tiiuae/Falcon-E-3B-Instruct) license.
+
+## Citation
+
+If you use this model, please cite the base model and the BitNet approach:
+
+```bibtex
+@misc{falcon-e-3b-instruct,
+  title={Falcon-E: A Family of Universal, Pre-trained 1.58-bit Models},
+  author={TII Falcon Team},
+  year={2025},
+  url={https://huggingface.co/tiiuae/Falcon-E-3B-Instruct}
+}
+
+@misc{bitnet2025,
+  title={bitnet.cpp: Efficient Edge Inference for Ternary LLMs},
+  author={Jinheng Wang and others},
+  year={2025},
+  url={https://github.com/microsoft/BitNet}
+}
+```