Upload 3 files

.gitattributes

@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
 ganda_gemma_ascending_chart.png filter=lfs diff=lfs merge=lfs -text
 ganda_gemma_ascending_clean.png filter=lfs diff=lfs merge=lfs -text
+luganda_comprehensive_chart.png filter=lfs diff=lfs merge=lfs -text

EVALUATION.md

@@ -0,0 +1,77 @@
+# Comprehensive FLORES Translation Evaluation Results
+
+## Overview
+This package contains comprehensive evaluation results for English→Luganda and English→Swahili translation using the FLORES+ dataset. The evaluation includes specialized fine-tuned models, commercial services, and baseline models.
+
+## Contents
+
+### 📊 Charts (`/charts/`)
+- `luganda_comprehensive_chart.png` - Complete Luganda translation performance comparison (17 models)
+- `swahili_comprehensive_chart.png` - Complete Swahili translation performance comparison (16 models)
+
+### 📈 Data (`/data/`)
+- `luganda_results.csv` - Detailed Luganda evaluation results with rankings
+- `swahili_results.csv` - Detailed Swahili evaluation results with rankings
+- `summary.csv` - Executive summary of our models' performance
+
+## Key Results
+
+### 🏆 Our Models Performance
+
+| Language | Model | Rank | BLEU | chrF++ | Percentile | Efficiency (BLEU/B) |
+|----------|-------|------|------|--------|------------|---------------------|
+| **Luganda** | Ganda Gemma 1B | 5/17 | 6.99 | 40.32 | 76.5% | 6.99 |
+| **Swahili** | Swahili Gemma 1B | 12/16 | 27.59 | 56.84 | 31.2% | 27.59 |
+
+### 🎯 Key Insights
+
+**Language Resource Impact:**
+- **Swahili** significantly outperforms **Luganda** (27.59 vs 6.99 BLEU)
+- Reflects the resource availability gap between the two languages
+- Demonstrates the challenge of low-resource language translation
+
+**Competitive Standing:**
+- **Luganda**: Ranks 5th out of 17 models (76.5th percentile)
+- **Swahili**: Ranks 12th out of 16 models (31.2nd percentile)
+- Both models show excellent parameter efficiency
+
+**Baseline Comparison:**
+- Our specialized models vastly outperform the general Gemma-3-1B baseline
+- **Luganda**: 6.99 vs 0.51 BLEU (13.8x improvement)
+- **Swahili**: 27.59 vs 2.78 BLEU (9.9x improvement)
+
+## Methodology
+
+**Dataset:** FLORES+ devtest split (1,012 sentence pairs per language)
+**Metrics:** BLEU and chrF++ scores
+**Evaluation:** Comprehensive comparison across 17 different models/services
+**Baseline:** vLLM-served Gemma-3-1B-IT for fair comparison
+
+## Models Evaluated
+
+### Commercial Services
+- Google Translate (top performer in both languages)
+
+### Specialized Models (Ours)
+- Ganda Gemma 1B (fine-tuned for Luganda)
+- Swahili Gemma 1B (fine-tuned for Swahili)
+
+### General Models
+- Claude Sonnet 4, GPT variants, Gemini models, Llama models
+- Gemma-3-1B baseline (vLLM)
+
+## Files Description
+
+### Data Files
+- **CSV Structure**: Rank, Model, Type, Parameters (B), BLEU, chrF++, BLEU per Billion Params, Our Model
+- **Rankings**: Sorted by BLEU score (descending)
+- **Efficiency**: BLEU score per billion parameters for fair comparison
+
+### Charts
+- **Visual comparison** of all models with our models highlighted
+- **Color coding**: Red (BLEU), Black (chrF++)
+- **Special marking**: Diagonal stripes for our models
+
+---
+
+*Evaluation Framework: FLORES+ English→African Languages*

luganda_results.csv

@@ -0,0 +1,18 @@
+Rank,Model,Type,BLEU,chrF++,Our Model
+1,Google Translate,Commercial Service,9.27,46.44,FALSE
+2,Claude Sonnet 4,Anthropic,8.07,43.54,FALSE
+3,Gemini 2.0 Flash 001,Google,7.94,43.38,FALSE
+4,Gemini 2.5 Pro,Google,7.46,44.74,FALSE
+5,Ganda Gemma 1B (Our Model),Specialized Fine-tuned,6.99,40.32,TRUE
+6,Gemini 2.5 Flash,Google,6.28,40.51,FALSE
+7,Chatgpt 4o Latest,OpenAI,6.19,40,FALSE
+8,Gpt Oss 120B,OpenAI,5.16,34.7,FALSE
+9,Gpt 5 Mini,OpenAI,5.14,36.55,FALSE
+10,Gpt 5 Nano,OpenAI,4.93,32.86,FALSE
+11,Llama 4 Maverick,Meta,4.52,33.75,FALSE
+12,Gemma 3 27B,Google,3.65,31.37,FALSE
+13,Llama 4 Scout,Meta,3.59,27.63,FALSE
+14,Gpt Oss 20B,OpenAI,3.39,27.98,FALSE
+15,Gemma 3 4B,Google,1.1,20.05,FALSE
+16,Gemma 3N E4B,Google,0.84,17.6,FALSE
+17,Gemma 3 1B (vLLM Baseline),General Model,0.51,9.79,FALSE