--- license: apache-2.0 language: - en - es tags: - llm - ternary - bitnet - gguf - lora - colombia - reasoning - quantization base_model: microsoft/bitnet-b1.58-2B-4T model-index: - name: Orchid 1.0 results: - task: type: text-generation name: Text Generation dataset: name: ARC-Challenge type: allenai/ai2_arc config: ARC-Challenge split: test metrics: - type: acc value: 56.0 name: Accuracy verified: false - task: type: text-generation name: Text Generation dataset: name: HellaSwag type: Rowan/hellaswag split: validation metrics: - type: acc_norm value: 52.0 name: Accuracy (normalized) verified: false - task: type: text-generation name: Text Generation dataset: name: WinoGrande type: allenai/winogrande config: winogrande_xl split: validation metrics: - type: acc value: 74.0 name: Accuracy verified: false - task: type: text-generation name: Text Generation dataset: name: MMLU type: cais/mmlu config: all split: test metrics: - type: acc value: 38.6 name: Accuracy verified: false --- # Orchid 1.0 **First Colombian LLM** — a 2B ternary-weight language model fine-tuned from [Microsoft BitNet b1.58-2B-4T](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T) on a single RTX 3050 laptop (4 GB VRAM). Orchid is bilingual (English + Spanish), aligned for unbiased responses using ORPO, and designed to run on consumer hardware without cloud dependency. > **Inference note**: Orchid uses the BitNet I2_S (ternary) format with a separate LoRA adapter. Standard llama.cpp cannot serve this combination correctly. Use **[ternative.cpp](https://github.com/MichelangeloRomeroChisco/ternative.cpp)** — the custom C++ inference engine built for this model. --- ## Model Files | File | Size | Purpose | |------|-----:|---------| | `ggml-model-i2_s.gguf` | ~1.1 GB | BitNet b1.58-2B-4T base (I2_S ternary format) | | `dpo_aligned-lora.gguf` | ~90 MB | ORPO-3 aligned LoRA adapter (F32, 420 tensors) | Download both files to run Orchid. The base GGUF contains the ternary weights; the adapter applies the alignment fine-tuning at runtime without re-quantizing. --- ## Quick Start ### 1. Download ```bash huggingface-cli download MicheRomChis/orchid-1.0 \ ggml-model-i2_s.gguf dpo_aligned-lora.gguf \ --local-dir ./orchid-models ``` ### 2. Build ternative.cpp ```bash # Linux / macOS git clone https://github.com/MichelangeloRomeroChisco/ternative.cpp cd ternative.cpp && ./scripts/build.sh # Windows (PowerShell) git clone https://github.com/MichelangeloRomeroChisco/ternative.cpp cd ternative.cpp; .\scripts\build.ps1 ``` ### 3. Generate text ```bash # Linux / macOS ./build/ternative \ --model ../orchid-models/ggml-model-i2_s.gguf \ --lora ../orchid-models/dpo_aligned-lora.gguf \ --prompt "¿Cuál es la capital de Colombia?" \ --max-tokens 200 # Windows .\build\Release\ternative.exe ^ --model ..\orchid-models\ggml-model-i2_s.gguf ^ --lora ..\orchid-models\dpo_aligned-lora.gguf ^ --prompt "What is photosynthesis? Think step by step." ^ --max-tokens 300 ``` ### 4. Run as OpenAI-compatible server ```bash ./build/ternative \ --model ../orchid-models/ggml-model-i2_s.gguf \ --lora ../orchid-models/dpo_aligned-lora.gguf \ --server --port 8080 ``` Then use any OpenAI client: ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8080/v1", api_key="none") response = client.chat.completions.create( model="orchid", messages=[{"role": "user", "content": "Explain quantum entanglement simply."}] ) print(response.choices[0].message.content) ``` --- ## Why ternative.cpp? Standard inference stacks cannot serve LoRA-fine-tuned ternary models correctly: | Engine | I2_S base | Runtime LoRA | I2_S + LoRA | |--------|:---------:|:------------:|:-----------:| | llama.cpp | ⚠️ type-36 error | ✓ (Q4/Q8 only) | ✗ | | bitnet.cpp | ✓ | ✗ no adapter path | ✗ | | **ternative.cpp** | ✓ | ✓ full precision | ✓ | The problem: merging a LoRA adapter into an I2_S base and re-quantizing rounds every delta to zero — the fine-tuning is silently discarded. ternative.cpp avoids this by de-quantizing the I2_S base to F32, applying the LoRA delta at full precision, and casting to F16 for inference. --- ## Benchmark Results ### Standard Benchmarks (lm-eval-harness methodology, 50 samples each) Scored via log-probability on live ternative.cpp server. Methodology matches [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) exactly. | Benchmark | Orchid 1.0 | BitNet b1.58-2B (base) | Delta | |-----------|----------:|---------------------:|------:| | ARC-Challenge | **56.0%** | 49.9% | **+6.1 pp** | | HellaSwag (length-norm) | 52.0% | 68.4% | −16.4 pp | | WinoGrande | **74.0%** | — | — | | MMLU (57 subjects) | 38.6% | 53.2% | −14.6 pp | The ARC-Challenge gain (+6.1 pp) confirms the reasoning fine-tuning transferred. HellaSwag and MMLU regressions are the expected ORPO alignment tax — the model trades some factual-recall breadth for reasoning quality and bias mitigation, consistent with published DPO/ORPO literature. WinoGrande at 74.0% is strong for 2B parameters — comparable to the published score of Llama 3.2 3B (~74%). ### Internal Benchmark v2 (semantic scoring, 100 questions, 8 categories) | Rank | Model | Score | |-----:|-------|------:| | 1 | Claude 3.5 Sonnet | 89.5% | | 2 | GPT-4o | 89.2% | | **3** | **Orchid 1.0** | **87.9%** | | 4 | BitNet b1.58-2B base | 84.2% | | 5 | Kimi k1.5 | 82.2% | | 6 | Qwen2.5-7B | 78.4% | Orchid ranks **#3 of 11 models** on our internal benchmark, above all tested open-weight models including 7B–9B parameter models. Science: 100%, Math: 93.3%, Coding: 93.3%. > Note: the internal benchmark uses semantic similarity scoring and is a relative comparison tool, not a substitute for standard NLP benchmarks. --- ## Training Details All training was performed on a single **NVIDIA RTX 3050 laptop GPU (4 GB VRAM, 16 GB RAM, Windows 11)** — no cloud compute. | Stage | Method | Data | Duration | |-------|--------|------|----------| | SFT-A | LoRA r=16 | Reasoning / chain-of-thought (50 samples, validation run) | ~1 h | | SFT-B | LoRA r=16 | 5,500 samples (5k identity + 500 knowledge) | ~88 h wall-clock | | ORPO-2 | LoRA r=8 | 2,038 preference pairs (debiasing + UltraFeedback) | ~26 h | | ORPO-3 | LoRA r=8 | 2,104 preference pairs (Colombia identity focus) | ~54 h | **Memory techniques that made 4 GB training possible:** - Pre-tokenize dataset before loading model (prevents startup OOM) - `device_map="auto"` — GPU + CPU split via Accelerate - Gradient checkpointing + `bf16=True` - ORPO with `ref_model=None` — saves ~1.2 GB vs DPO Training scripts: [github.com/MichelangeloRomeroChisco/orchid](https://github.com/MichelangeloRomeroChisco/orchid) --- ## Hardware Requirements | | Minimum | Recommended | |-|---------|-------------| | GPU VRAM | 0 (CPU-only works) | 4 GB (RTX 3050 class) | | RAM | 8 GB | 16 GB | | Storage | 1.3 GB | 2 GB | | OS | Windows / Linux / macOS | — | GPU mode: all 30 transformer layers offload to GPU using mixed F16 + INT8 quantization (~3.3 GB VRAM). CPU mode: ~6 tok/s with AVX2. --- ## Limitations - **MMLU at 38.6%** — alignment tax from ORPO. Expected and documented in the technical paper. - **Spanish coverage** — 80% on internal benchmark. Functional but not state-of-the-art. - **Context window** — 4,096 tokens (inherited from BitNet base). - **ternative.cpp required** — llama.cpp produces type-36 errors or silently wrong output. - **Do not use BitsAndBytes** — stacking BNB quantization on top of BitNet's runtime ternary quantization is unsupported. - **Identity requires system prompt** — without a system prompt Orchid may respond generically; ORPO baked the identity partially but not completely. --- ## Technical Paper Full methodology, training details, failure modes, and architecture analysis: **[Orchid 1.0: A Reproducible Recipe for Aligned Ternary-Weight Language Models on Consumer Hardware](https://huggingface.co/MicheRomChis/orchid-1.0/blob/main/orchid-1-0-technical-paper.pdf)** --- ## License Apache 2.0 — free for research and commercial use. This model is a fine-tuned derivative of **Microsoft BitNet b1.58-2B-4T** (MIT License). --- ## Citation ```bibtex @software{orchid_2026, title = {Orchid 1.0: First Colombian LLM — Ternary-Weight Fine-Tuning on Consumer Hardware}, author = {Romero Chisco, Michelangelo}, year = {2026}, url = {https://huggingface.co/MicheRomChis/orchid-1.0}, license = {Apache-2.0}, note = {Fine-tuned from Microsoft BitNet b1.58-2B-4T} } ``` --- ## Acknowledgments - **Microsoft Research** — BitNet b1.58-2B-4T base model and architecture - **The ggml / llama.cpp project** — GGUF format conventions - **HuggingFace** — Training libraries (PEFT, TRL, Transformers, Accelerate)