feat: fine-tuned model + GGUF export pipeline fixes

- Swap to fine-tuned model: build-small-hackathon/lfed-qwen2.5-coder-7b-sql-gguf
- Fix modal_app.py: add_local_dir for scripts, shutil.move for cross-fs
- Fix train.py: save_strategy=no + pickle error catch
- Fix export_gguf.py: FP16 merge via PeftModel, llama.cpp converter, force cleanup
- Fix model_inference.py: point to fine-tuned GGUF

docs/DATA_RESEARCH.md

@@ -0,0 +1,285 @@
+# Deep Research: Text-to-SQL Training Data for Education Domain
+
+> **Date:** 2026-06-08
+> **Goal:** Find existing datasets, proven approaches, and the best path for building a high-quality NL→SQL training set for K-12 school district analytics
+
+---
+
+## Part 1: Existing Datasets That Do This Work
+
+### Tier 1: Large-Scale Text-to-SQL Datasets (General Domain)
+
+These are the big ones. None are education-specific, but they contain transferable patterns.
+
+#### 1. gretelai/synthetic_text_to_sql (HuggingFace)
+- **Size:** 106K pairs (100K train / 5.8K test)
+- **Domains:** 100 domains including healthcare, government, education-related
+- **Complexity:** Basic SQL through window functions, CTEs, multi-joins
+- **Format:** Each row has: domain, sql_prompt (NL question), sql_context (CREATE TABLE + INSERT), sql (answer), sql_explanation
+- **License:** Apache 2.0
+- **Why it matters:** This is THE standard dataset people use to fine-tune text-to-SQL models. Rubrik/Predibase used it to fine-tune Llama-3-8B to outperform GPT-4 on SQL tasks.
+- **Relevance:** HIGH — contains the SQL patterns (aggregation, joins, GROUP BY, CASE WHEN, HAVING, window functions) you need. But the schemas are generic, not education-specific.
+- **How to use:** Filter for domains closest to education, then combine with your own education-specific pairs.
+
+#### 2. NumbersStation/NSText2SQL (HuggingFace)
+- **Size:** 289K pairs
+- **Sources:** 26 different public datasets merged (Spider, CoSQL, SparC, WikiSQL, etc.)
+- **Format:** instruction (schema + question) → output (SQL)
+- **License:** Various (curated from permissive sources)
+- **Why it matters:** Largest merged text-to-SQL corpus. Contains course/student/enroll schemas from Spider that are closest to education.
+- **Relevance:** MEDIUM — has some education-adjacent schemas (courses, enrollment, professors) but not K-12 specific.
+
+#### 3. OmniSQL / SynSQL-2.5M (HuggingFace + GitHub)
+- **Size:** 2.5 MILLION pairs across 16,575 schemas
+- **Approach:** LLM generates schemas, then synthesizes NL questions + SQL
+- **Paper:** VLDB 2025
+- **Why it matters:** First million-scale text-to-SQL dataset. Their synthesis pipeline is open-source — you could adapt it for education schemas.
+- **Relevance:** MEDIUM — the pipeline (schema → question → SQL → validation) is exactly what you need to replicate for education.
+
+#### 4. SQaLe (trl-lab/SQaLe-text-to-SQL-dataset)
+- **Size:** 517K validated triples from 135,875 schemas
+- **Approach:** Start with real schemas from SchemaPile, extend with LLM, generate questions from Spider/BIRD examples, validate via execution
+- **Why it matters:** Proves that schema-variety matters more than raw volume. 135K schemas beat datasets with 10x more pairs on fewer schemas.
+- **Relevance:** MEDIUM — methodology is spot-on, but no education schemas specifically.
+
+#### 5. Spider + BIRD Benchmarks
+- **Spider:** 10K questions, 200 databases, 138 domains. Used as the standard benchmark.
+- **BIRD:** 12K questions, 95 databases. Focuses on real-world noisy data.
+- **Why they matter:** These define what "good" looks like. Every serious text-to-SQL paper benchmarks against them.
+- **Relevance:** LOW for direct use (small, no education focus), but HIGH for understanding what evaluation metrics matter.
+
+### Tier 2: Education-Specific Raw Data (No NL→SQL Pairs)
+
+These have the DATA but not the training pairs. You'd need to generate NL→SQL from them.
+
+#### 6. Kaggle: Student Performance Data Set
+- **Source:** UCI ML Repository, 2 Portuguese schools
+- **Size:** 649 students, 33 columns
+- **Columns:** grades (G1, G2, G3), demographics (sex, age, address), parental education, study time, failures, absences, social factors
+- **License:** CC0 Public Domain
+- **Relevance:** HIGH — closest to your use case. Grades + demographics + attendance. But it's flat (one table), not normalized like a real SIS.
+
+#### 7. Kaggle: Sample Highschool Database
+- **Source:** Student project
+- **Size:** 1,000 students
+- **Format:** SQL database file
+- **Relevance:** MEDIUM — has normalized tables but limited scope.
+
+#### 8. California Department of Education (CDE) Downloadable Data
+- **URL:** https://www.cde.ca.gov/ds/ad/downloadabledata.asp
+- **Data:** Enrollment, demographics, test scores, graduation rates, discipline, absenteeism, staff, financials — all at school/district/county/state level
+- **Format:** CSV files, updated annually
+- **License:** Public (CA government data)
+- **Relevance:** VERY HIGH — this is REAL data from the exact domain you serve. You could build a realistic seed database from CDE data and generate NL→SQL pairs against it.
+
+#### 9. NCES (National Center for Education Statistics) DataLab
+- **URL:** https://nces.ed.gov/datalab/
+- **Data:** National education data, PowerStats tool
+- **Relevance:** HIGH — federal-level education data, complementary to CDE.
+
+#### 10. Ed-Data.org
+- **URL:** https://www.ed-data.org/
+- **Data:** CA school/district profiles, financial data, test scores
+- **Relevance:** HIGH — partnership of CDE + EdSource, already structured for queries.
+
+---
+
+## Part 2: Proven Approaches for Generating NL→SQL Training Data
+
+### Approach A: Template-Based (What You're Doing Now)
+
+**How it works:** Write SQL templates with placeholders, parameterize with real values, generate NL from templates.
+
+**Pros:**
+- 100% accurate SQL (you write it)
+- Cheap, fast, deterministic
+- Full control over coverage
+
+**Cons:**
+- Questions feel synthetic/stilted
+- Limited by your imagination
+- Hard to scale past ~2K pairs without exhaustion
+- Doesn't train for ambiguity, typos, or real-world messiness
+
+**Your current state:** 1,289 pairs from 32 templates. Good start, but ceiling is ~2K.
+
+### Approach B: LLM-Augmented Synthesis (Recommended Next Step)
+
+**How it works:** Use a powerful LLM (GPT-4, Claude, Qwen-72B) to generate NL→SQL pairs from your schema.
+
+**Proven by:**
+- OmniSQL (2.5M pairs via LLM synthesis)
+- SQaLe (517K pairs via LLM + validation)
+- SING-SQL (Bilkent University, 2025) — specifically designed for single-database in-domain training
+
+**The SING-SQL pipeline is the most relevant to your case:**
+1. Take your database schema
+2. Partition schema into sub-schemas (e.g., attendance-only, grades+demographics, cross-table)
+3. For each sub-schema, have LLM generate SQL queries at multiple complexity levels (basic SELECT → aggregation → joins → window functions → CTEs)
+4. For each SQL, have LLM generate the NL question
+5. Validate: run SQL against real data, check it executes
+6. LLM-as-judge: have another LLM verify Q↔SQL match
+7. Auto-repair broken queries
+8. Balance column coverage (ensure all columns get queried)
+
+**SING-SQL results:** Their 3B model (fine-tuned on synthetic data) hit 82.87% Soft F1 on BIRD — beating prior 3B baselines by +16 points.
+
+**Pros:**
+- Scales to 10K-100K+ pairs
+- Questions sound natural
+- Covers SQL patterns you wouldn't think of
+- Validated against real data
+
+**Cons:**
+- Needs API credits (or local LLM)
+- Some generated SQL will be wrong (need validation)
+- May generate SQL for impossible queries
+
+### Approach C: Hybrid (Best for Your Timeline)
+
+Combine templates for known patterns + LLM for natural variation + real data for seed realism.
+
+**Recommended pipeline:**
+1. **Foundation:** Your existing 1,289 template pairs (keep these — they're 100% accurate)
+2. **Schema expansion:** Add grades, discipline, demographics, assessments, programs tables
+3. **Seed with real data:** Download CDE data, build realistic seed database
+4. **LLM augmentation:** Use Qwen2.5-72B (free on Modal or via HF Inference) to generate 5,000 new NL→SQL pairs
+5. **Rephrasing:** For each of the 6,000+ pairs, generate 3-5 NL phrasings (formal, casual, abbreviated, typo-prone)
+6. **Validation:** Run every SQL against seed data, discard failures
+7. **Mix with Gretel:** Filter gretelai/synthetic_text_to_sql for relevant domains (government, healthcare, HR) and add 2,000 pairs as general SQL knowledge
+
+**Target:** 15,000-25,000 validated pairs
+
+---
+
+## Part 3: Specific Recommendations for Your Hackathon
+
+### Timeline Reality Check
+
+You have until June 15. That's 7 days. Here's what's realistic:
+
+| Approach | Pairs | Time | Quality | Recommended? |
+|---|---|---|---|---|
+| Templates only (current) | ~2K | Already done | High accuracy, low diversity | Keep as base |
+| + LLM synthesis (local Qwen-72B or Modal) | +5K | 2-3 days | Medium-high | YES — best ROI |
+| + Gretel dataset filtered | +2K | 1 day | Medium | YES — free, fast |
+| + Rephrasing augmentation | ×3-5 | 1 day | Medium | YES — cheap multiplier |
+| + CDE real data seed | N/A | 1 day | N/A | YES — makes everything more realistic |
+| Full SING-SQL pipeline | +50K | 2 weeks | High | Too ambitious for hackathon |
+
+### The Playbook (7-Day Plan)
+
+**Day 1-2: Schema + Seed Data**
+- Define 6 new tables (grades, discipline, demographics, assessments, programs, staff)
+- Download CA Department of Education data files
+- Build realistic seed database with 10K+ students
+- Update prompts.py with expanded schema
+
+**Day 3: LLM Synthesis**
+- Use Qwen2.5-Coder-7B (or GPT-4 if you have credits) to generate NL→SQL pairs
+- Prompt: "Given this DuckDB schema [schema], generate a natural language question and its corresponding SQL query. Complexity: [basic/aggregation/join/window]. Focus on: [table_name]."
+- Target: 5,000 pairs across all tables
+
+**Day 4: Validation + Filtering**
+- Run every generated SQL against the seed database
+- Discard any that fail to execute
+- Discard any that return 0 rows (unless that's the expected answer)
+- Verify column references match schema
+
+**Day 5: Rephrasing + Augmentation**
+- For each validated pair, generate 3-5 NL rephrasings
+- Add typo variants ("Whats the avg gpa" instead of "What is the average GPA")
+- Add informal variants ("How are our kids doing on tests?" for assessment queries)
+- Filter Gretel dataset for relevant domains, add 1,500-2,000 pairs
+
+**Day 6: Train v2**
+- Update training config: lower LR (1e-4), higher LoRA rank (32), longer seq (4096)
+- Train on expanded dataset (~15K-20K pairs)
+- Monitor for overfitting
+
+**Day 7: Evaluate + Deploy**
+- Test v1 vs v2 on holdout questions
+- Deploy to HF Space
+- Smoke test with real-world questions
+
+### Quick Wins You Can Do Today
+
+1. **Download Gretel dataset** — it's free, Apache 2.0, and ready to use:
+   ```python
+   from datasets import load_dataset
+   ds = load_dataset("gretelai/synthetic_text_to_sql")
+   # Filter for relevant domains
+   relevant = ds['train'].filter(lambda x: x['domain'] in [
+       'education', 'government', 'public health', 'human resources',
+       'insurance', 'social services', 'nonprofit'
+   ])
+   ```
+
+2. **Download CDE data** — real CA school data:
+   https://www.cde.ca.gov/ds/ad/downloadabledata.asp
+   Key files: enrollment, absenteeism, demographics, test scores, discipline
+
+3. **Look at OmniSQL's synthesis code** — open source, can adapt for your schema:
+   https://github.com/RUCKBReasoning/OmniSQL/tree/main/data_synthesis
+
+---
+
+## Part 4: What the Kaggle Education Datasets Are (and Why They're Different)
+
+You mentioned seeing datasets on Kaggle. Here's what's there and why they're not quite right:
+
+| Dataset | What It Is | Why It's Different |
+|---|---|---|
+| Student Performance (UCI) | 649 students, flat CSV, Portuguese schools | Raw data, no NL→SQL pairs, foreign schools |
+| Student Information | 200 students, 7 attributes | Tiny, toy dataset for SQL practice |
+| Student Exam Performance | Demographics + test scores | Flat analysis dataset, not text-to-SQL |
+| Sample Highschool Database | SQL file, 1K students | Closer but limited scope, no NL pairs |
+
+**The gap:** Kaggle has education DATA but not education TEXT-TO-SQL TRAINING DATA. Nobody has done the work of turning education data into NL→SQL training pairs at scale. That's your opportunity.
+
+---
+
+## Part 5: The Big Opportunity
+
+Nobody has built a production-quality text-to-SQL model specifically for K-12 school district analytics. This is a real gap:
+
+- **Gretel** covers 100 domains but education is generic
+- **Spider/BIRD** have no education schemas
+- **OmniSQL/SQaLe** generate across all domains but not deep on education
+- **Vanna AI** does RAG-based text-to-SQL but requires you to provide training pairs
+
+If you build this dataset well — with real CA education data, realistic schemas, validated NL→SQL pairs covering enrollment, attendance, grades, discipline, demographics, assessments, programs — you'd have something genuinely valuable beyond the hackathon:
+
+1. **Hackathon submission:** Fine-tuned model that does K-12 analytics
+2. **Open-source dataset:** First high-quality education text-to-SQL dataset
+3. **Product differentiator:** LTC can offer this to CA school districts
+4. **Community contribution:** Publish to HuggingFace, get visibility
+
+---
+
+## Appendix: Key Resources
+
+### Datasets
+| Name | URL | Size | License |
+|---|---|---|---|
+| gretelai/synthetic_text_to_sql | huggingface.co/datasets/gretelai/synthetic_text_to_sql | 106K | Apache 2.0 |
+| NumbersStation/NSText2SQL | huggingface.co/datasets/NumbersStation/NSText2SQL | 289K | Various |
+| OmniSQL/SynSQL-2.5M | github.com/RUCKBReasoning/OmniSQL | 2.5M | Check repo |
+| SQaLe | huggingface.co/datasets/trl-lab/SQaLe-text-to-SQL-dataset | 517K | Check paper |
+| Student Performance (UCI) | kaggle.com/datasets/larsen0966/student-performance-data-set | 649 | CC0 |
+| CDE Downloadable Data | cde.ca.gov/ds/ad/downloadabledata.asp | Real data | Public |
+
+### Papers/Frameworks
+| Name | What It Does | URL |
+|---|---|---|
+| SING-SQL | Best framework for single-database in-domain training | github.com/HasanAlpCaferoglu/SING-SQL |
+| OmniSQL | Million-scale synthesis pipeline | github.com/RUCKBReasoning/OmniSQL |
+| SQaLe | Schema-variety-driven dataset | huggingface.co/blog/cwolff/sqale |
+| Vanna AI | RAG-based text-to-SQL (alternative to fine-tuning) | github.com/vanna-ai/vanna |
+
+### Tutorials
+| Name | What It Covers | URL |
+|---|---|---|
+| Rubrik + Gretel + Predibase | Full fine-tune tutorial with Gretel data | rubrik.com/blog/ai/24/... |
+| Google Gemma QLoRA | Fine-tune Gemma on text-to-SQL with QLoRA | ai.google.dev/gemma/docs/core/huggingface_text_finetune_qlora |
+| Towards AI GRPO series | 60 training sessions, Qwen2.5-Coder experiments | pub.towardsai.net/fine-tuning-open-source-llms-for-text-to-sql |

docs/EXPANSION_PLAN.md

@@ -0,0 +1,316 @@
+# LFED Training Data Expansion Plan
+
+> **Last updated:** 2026-06-08
+> **Goal:** Expand from 1,289 pairs on 2 tables → 10,000+ pairs across 8+ analytics domains
+
+---
+
+## Current State
+
+### What We Have Now
+- **Tables:** 2 (enrollment, attendance)
+- **Training pairs:** 1,289 (template-generated)
+- **Templates:** 32
+- **Coverage:** Chronic absenteeism, enrollment counts, absence rates
+- **Quality:** Good for narrow domain, but synthetic-feeling
+
+### What's in local-data-stack (Untapped Analytics Domains)
+
+From `/Users/flucido/projects/local-data-stack/rill_project/data/`:
+
+| Domain | Table | Rows | Key Columns |
+|---|---|---|---|
+| Chronic Absenteeism Risk | chronic_absenteeism_risk | 1,700 | risk_score, risk_level, attendance_rate_30d/90d, discipline_incidents, demographics |
+| Student Wellbeing | wellbeing_risk_profiles | 1,700 | attendance/discipline/academic_risk_scores, wellbeing_risk_level, primary_concern |
+| Performance Correlations | performance_correlations | 3 | correlation_pair, coefficient, strength |
+| Class Effectiveness | class_effectiveness | 300 | avg_grade, pct_passed, effectiveness_rating, ELL/SpEd/FRL pass rates |
+| Equity Outcomes | equity_outcomes_by_demographics | 10 | race_ethnicity, ELL, SpEd, FRL, avg_gpa, pct_below_c |
+
+---
+
+## Phase 1: Schema Expansion (Add 6 New Tables)
+
+Add these tables to the DuckDB seed data and update prompts.py:
+
+### 1.1 grades
+```sql
+CREATE TABLE grades (
+    student_id INTEGER,
+    school_name VARCHAR,
+    school_year VARCHAR,
+    grade_level INTEGER,
+    course_name VARCHAR,
+    term VARCHAR,              -- 'Fall', 'Spring'
+    letter_grade VARCHAR,      -- 'A', 'B', 'C', 'D', 'F'
+    grade_numeric DOUBLE,      -- 4.0, 3.0, etc.
+    gpa DOUBLE,
+    credit_hours DOUBLE
+);
+```
+
+### 1.2 discipline
+```sql
+CREATE TABLE discipline (
+    incident_id INTEGER,
+    student_id INTEGER,
+    school_name VARCHAR,
+    school_year VARCHAR,
+    grade_level INTEGER,
+    incident_type VARCHAR,     -- 'Defiance', 'Fighting', 'Vandalism', 'Substance', 'Bullying'
+    incident_date DATE,
+    severity VARCHAR,          -- 'Minor', 'Major', 'Severe'
+    action_taken VARCHAR,      -- 'Warning', 'Detention', 'Suspension', 'Expulsion'
+    days_suspended INTEGER
+);
+```
+
+### 1.3 demographics
+```sql
+CREATE TABLE demographics (
+    student_id INTEGER,
+    school_name VARCHAR,
+    school_year VARCHAR,
+    grade_level INTEGER,
+    gender VARCHAR,
+    race_ethnicity VARCHAR,
+    english_learner BOOLEAN,
+    special_education BOOLEAN,
+    economically_disadvantaged BOOLEAN,
+    homeless_flag BOOLEAN,
+    migrant_flag BOOLEAN,
+    foster_youth BOOLEAN
+);
+```
+
+### 1.4 assessments
+```sql
+CREATE TABLE assessments (
+    student_id INTEGER,
+    school_name VARCHAR,
+    school_year VARCHAR,
+    grade_level INTEGER,
+    assessment_type VARCHAR,   -- 'SBAC', 'CAASPP', 'CELCAST', 'District Benchmark'
+    subject VARCHAR,           -- 'ELA', 'Math', 'Science'
+    score DOUBLE,
+    proficiency_level VARCHAR, -- 'Below Standard', 'Near Standard', 'At/Above Standard'
+    growth_percentile INTEGER
+);
+```
+
+### 1.5 programs
+```sql
+CREATE TABLE programs (
+    student_id INTEGER,
+    school_name VARCHAR,
+    school_year VARCHAR,
+    program_type VARCHAR,      -- 'Title I', 'ELL Support', 'SpEd IEP', '504 Plan', 'MTSS Tier 1/2/3'
+    start_date DATE,
+    end_date DATE,
+    status VARCHAR             -- 'Active', 'Exited', 'Transferred'
+);
+```
+
+### 1.6 staff
+```sql
+CREATE TABLE staff (
+    staff_id INTEGER,
+    school_name VARCHAR,
+    school_year VARCHAR,
+    role VARCHAR,              -- 'Teacher', 'Counselor', 'Admin', 'Aide'
+    subject_area VARCHAR,
+    years_experience INTEGER,
+    credential_type VARCHAR,
+    student_load INTEGER
+);
+```
+
+---
+
+## Phase 2: Training Data Expansion (1,289 → 10,000+ pairs)
+
+### 2.1 New Template Categories (add ~150 templates)
+
+#### Grades & GPA (~30 templates)
+- Average GPA by school/grade/demographic
+- Grade distribution (A/B/C/D/F counts and percentages)
+- GPA trends over time
+- Failing rate by course/teacher
+- GPA comparison between schools
+- Students below 2.0 GPA
+- Honor roll counts
+
+#### Discipline (~30 templates)
+- Incident counts by type, school, year
+- Suspension rates by demographic
+- Discipline trends over time
+- Most common incident types
+- Students with multiple incidents
+- Days lost to suspension by school
+- Discipline correlation with attendance
+
+#### Demographics (~20 templates)
+- Enrollment by race/ethnicity
+- ELL student counts and percentages
+- SpEd population by school
+- Economically disadvantaged rates
+- Foster/homeless student counts
+- Demographic breakdowns of outcomes
+
+#### Assessments (~30 templates)
+- Proficiency rates by subject and school
+- Growth percentiles by grade
+- Assessment score trends
+- Below-standard student counts
+- Demographic gaps in test scores
+- School performance rankings
+
+#### Programs (~20 templates)
+- Active program counts by type
+- MTSS tier distribution
+- IEP/504 plan counts
+- Program participation by school
+- Title I eligible counts
+
+#### Cross-Table Joins (~20 templates)
+- Attendance vs. grades correlation
+- Discipline incidents vs. GPA
+- ELL status vs. assessment scores
+- SpEd vs. chronic absenteeism
+- Program participation vs. outcomes
+
+### 2.2 Data Augmentation Strategies
+
+#### A. Rephrasing (3-5x multiplier)
+For each template, generate additional natural-language phrasings:
+- **Formal:** "What is the average GPA for 9th graders at Jefferson High?"
+- **Informal:** "What's the avg GPA for freshmen at Jefferson?"
+- **Abbreviated:** "9th grade GPA at Jefferson High?"
+- **Typo-prone:** "Whats the avg gpa for 9th graders at jefferson hiogh?"
+- **Context-rich:** "I'm preparing for the board meeting — need 9th grade GPA at Jefferson High for 2023-2024"
+
+Implementation: Use a small LLM (Qwen2.5-1.5B) to rephrase each template's question while keeping the SQL identical.
+
+#### B. Question Decomposition
+Train on multi-part questions:
+- Q: "Compare chronic absenteeism rates and average GPA between Lincoln Elementary and Jefferson High"
+- SQL: Two CTEs or UNION ALL
+
+#### C. Ambiguous Questions
+Train the model to ask clarifying questions or make reasonable assumptions:
+- Q: "How are our students doing?"
+- SQL: SELECT school_name, AVG(gpa) ... (with a reasonable default)
+
+#### D. Error Recovery
+Train on edge cases:
+- Questions referencing non-existent columns → model should generate closest valid query
+- Questions about data that doesn't exist → model should return empty result gracefully
+
+### 2.3 Seed Data Expansion
+
+Current: 2,900 students, 5 schools, 4 years
+
+Expand to:
+- **10,000 students** across 8 schools
+- **6 school years** (2019-2025)
+- **Realistic distributions:**
+  - Chronic absenteeism: 15% (current) — keep
+  - GPA distribution: normal around 2.8, std 0.8
+  - Discipline: 8% of students with 1+ incident
+  - ELL: 18%, SpEd: 12%, FRL: 45%
+  - Assessment proficiency: 55% at/above standard
+
+### 2.4 Quality Assurance Pipeline
+
+1. **SQL Validation:** Run every generated SQL against seed data — must return results (or empty for legitimate queries)
+2. **Schema Match:** Every column/table referenced must exist in schema
+3. **Dedup:** Exact-match dedup on questions, near-match dedup on SQL structure
+4. **Balance Check:** Ensure even coverage across all tables and query patterns
+5. **Human Review:** Sample 5% of pairs for manual review
+
+---
+
+## Phase 3: Training Improvements
+
+### 3.1 Data Quality
+| Current | Target |
+|---|---|
+| 1,289 pairs | 10,000+ pairs |
+| 2 tables | 8 tables |
+| 32 templates | 150+ templates |
+| Template-only questions | Template + LLM-rephrased + typo variants |
+| No join queries | 20% multi-table joins |
+| No ambiguous queries | 5% ambiguous/clarification-needed |
+
+### 3.2 Training Config Improvements
+
+| Param | Current | Proposed | Why |
+|---|---|---|---|
+| Epochs | 3 | 2 | More data needs fewer epochs to avoid overfitting |
+| Learning rate | 2e-4 | 1e-4 | More data = can use lower LR for better convergence |
+| LoRA rank | 16 | 32 | More data supports higher rank without overfitting |
+| Max seq length | 2048 | 4096 | Multi-table joins need longer sequences |
+| Batch size | 4×4=16 | 4×8=32 | Larger effective batch for larger dataset |
+
+### 3.3 Evaluation Metrics
+
+Add eval set (10% holdout):
+- **Exact match:** Generated SQL matches expected SQL
+- **Execution match:** Generated SQL returns same results as expected
+- **Schema validity:** All referenced columns/tables exist
+- **Safety:** No DDL/DML statements generated
+
+---
+
+## Phase 4: Implementation Order
+
+| Step | Task | Effort | Priority |
+|---|---|---|---|
+| 1 | Define new table schemas (above) | 1 hour | P0 |
+| 2 | Generate seed data for 6 new tables | 2 hours | P0 |
+| 3 | Add new templates for grades + discipline (60 templates) | 3 hours | P0 |
+| 4 | Run synthetic generation → 5,000+ pairs | 30 min | P0 |
+| 5 | SQL validation pass (run all pairs against seed data) | 1 hour | P0 |
+| 6 | Add demographic + assessment templates (50 templates) | 2 hours | P1 |
+| 7 | LLM rephrasing augmentation (3x multiplier) | 2 hours | P1 |
+| 8 | Add cross-table join templates (20 templates) | 1 hour | P1 |
+| 9 | Re-run generation → 10,000+ pairs | 30 min | P1 |
+| 10 | Train v2 model with expanded data | 2-3 hours | P1 |
+| 11 | Eval v1 vs v2 on holdout set | 1 hour | P2 |
+| 12 | Deploy v2 to HF Space | 30 min | P2 |
+
+**Total estimated effort:** ~15-18 hours of work
+
+---
+
+## Phase 5: Quick Wins (Do First)
+
+### 5.1 Expand Seed Data (immediate)
+Write a `generate_seed_v2.py` that creates all 6 new tables with realistic distributions. This unblocks everything else.
+
+### 5.2 Port local-data-stack Patterns
+The local-data-stack already has analytics models for:
+- Chronic absenteeism risk scoring
+- Wellbeing composite scores
+- Class effectiveness ratings
+- Equity outcome comparisons
+
+Port these SQL patterns into training templates. The queries are already validated — just need NL phrasings.
+
+### 5.3 Add Join Templates
+The biggest gap in current training is zero join queries. Even 20 join templates would dramatically improve the model's ability to answer cross-domain questions.
+
+---
+
+## Appendix: local-data-stack Column Reference
+
+### chronic_absenteeism_risk
+student_key, grade_level, school_id, gender, race_ethnicity, english_learner, special_education, economically_disadvantaged, homeless_flag, attendance_rate_30d, unexcused_absence_rate_30d, discipline_incidents_30d, absence_discipline_correlation_score, attendance_rate_90d, attendance_trend_90d, chronic_absence_flag, chronic_absenteeism_risk_score, risk_level
+
+### wellbeing_risk_profiles
+student_key, grade_level, school_id, attendance_risk_score, discipline_risk_score, academic_risk_score, high_risk_domain_count, wellbeing_risk_score, wellbeing_risk_level, primary_concern
+
+### class_effectiveness
+course_id, school_id, grade_level, enrollment_count, avg_grade_numeric, pct_passed, pct_a_b_grades, course_avg_grade, grade_diff_from_course_avg, pct_passed_ell, pct_passed_sped, pct_passed_frl, pass_rate_rank, grade_rank, effectiveness_rating, term
+
+### equity_outcomes_by_demographics
+race_ethnicity, english_learner, special_education, economically_disadvantaged, cohort_size, pct_good_attendance, pct_no_discipline, avg_gpa, pct_gpa_2_5_plus, pct_below_c

docs/HACKATHON_PLAN_V2.md

@@ -0,0 +1,257 @@
+# LFED Hackathon Plan v2 — Concrete 7-Day Plan
+
+> **Scope:** Single domain (attendance), local-first, M2 Mac 16GB
+> **Credits:** $250 Modal
+> **Deadline:** June 15, 2026
+> **Philosophy:** Hackathon = skill-honing, not production launch
+
+---
+
+## Decision: Model Selection
+
+### Your Constraints
+- M2 Mac, 16GB unified memory
+- Local-first, no API calls at inference
+- llama.cpp + GGUF format
+- ~10-11GB usable for model (5GB reserved for OS + Gradio + DuckDB)
+
+### Current Model: Qwen2.5-Coder-7B Q4_K_M (~4.4GB)
+Good. Works. Leaves plenty of headroom. But 7B is the floor for complex SQL.
+
+### Upgrade Options That Fit in 16GB
+
+| Model | Params | Q4_K_M Size | SQL Skill | Coding | Notes |
+|---|---|---|---|---|---|
+| **Qwen2.5-Coder-7B** (current) | 7B | ~4.4 GB | Good | Great | Safe choice, proven |
+| **Gemma 4 12B** | 12B | ~7.4 GB | Very good | Great | Brand new, Apache 2.0, multimodal, 256K ctx |
+| **Qwen2.5-14B** | 14B | ~8.5 GB | Great | Great | Best in 16GB bracket per apxml.com |
+| **Qwen2.5-Coder-14B** | 14B | ~8.5 GB | Great | Best | Coder variant, optimized for code/SQL |
+| Gemma 4 E4B | 4.5B | ~3 GB | OK | Good | Too small for reliable SQL |
+
+### Recommendation: Two-Model Strategy
+
+**Inference model (runs on your Mac):** Qwen2.5-Coder-14B Q4_K_M
+- 14B is the sweet spot for 16GB Mac — proven by multiple benchmarks
+- Coder variant specifically trained on code/SQL tasks
+- ~8.5GB leaves ~7.5GB for OS + app overhead
+- Unsloth has GGUF versions ready to download
+- Fine-tunable on Modal A10G (24GB fits 14B QLoRA)
+
+**Data generation model (runs on Modal):** Qwen2.5-72B-Instruct on A100 80GB
+- Much smarter — generates higher quality NL→SQL pairs
+- Runs on Modal, not your Mac
+- ~$2.50/hr, 3-4 hours = ~$8-10 to generate 10K+ pairs
+
+**Why not Gemma 4 12B?**
+- Excellent model, but Qwen2.5-Coder-14B is specifically trained for code/SQL
+- Gemma 4 is newer (fewer fine-tuned GGUFs, less community testing for SQL)
+- For a SQL-focused hackathon, the Coder variant wins
+- Gemma 4 would be a great choice for a general-purpose app
+
+**Why not stay with 7B?**
+- You can — it works. But 14B at Q4_K_M fits your Mac with room to spare
+- 14B handles complex queries (joins, subqueries, CASE WHEN) noticeably better
+- The jump from 7B → 14B is the biggest quality gain per parameter in this range
+
+---
+
+## The Plan: 7 Days to Submission
+
+### Day 1 (Today — June 8): Foundation
+
+**Goal:** Expanded schema + seed data for attendance domain
+
+1. Expand `data_engine.py` schema to include richer attendance tables:
+   - Keep existing: `enrollment`, `attendance`
+   - Add: `students` (demographics), `discipline` (incidents), `grades` (GPA)
+   - This gives the model more to work with while keeping scope tight
+
+2. Generate expanded seed data (10K students, 8 schools, 6 years)
+   - Realistic distributions: 15% chronic absenteeism, demographic mix
+   - Use CA Department of Education patterns as reference
+
+3. Update `prompts.py` with expanded schema docs
+
+**Time:** ~3 hours
+**Modal cost:** $0 (all local)
+
+### Day 2 (June 9): Training Data Generation (Big Model)
+
+**Goal:** 10,000+ NL→SQL pairs using Modal + 72B model
+
+1. Write `generate_synthetic_v2.py` that:
+   - Takes your expanded schema as input
+   - Calls Qwen2.5-72B on Modal to generate NL→SQL pairs
+   - Covers: basic counts, aggregations, GROUP BY, HAVING, CASE WHEN,
+     window functions, subqueries, multi-table joins
+   - Each pair includes: question, SQL, complexity level
+
+2. Run on Modal A100 80GB (~3-4 hours)
+   - Generate 10,000 pairs
+   - Validate each SQL against seed data (must execute + return results)
+   - Discard failures
+
+3. Combine with your existing 1,289 template pairs
+
+**Time:** ~4 hours (mostly waiting for Modal)
+**Modal cost:** ~$10
+
+### Day 3 (June 10): Data Augmentation + Quality
+
+**Goal:** 25,000+ validated pairs with diversity
+
+1. Rephrasing pass: For each validated pair, generate 3 NL variations
+   - Formal: "What is the average number of absences per school?"
+   - Casual: "What's the avg absences by school?"
+   - Abbreviated: "avg absences per school?"
+   - Typo-prone: "Whats the avg abscences per scool?"
+
+2. Pull 2,000 relevant pairs from Gretel dataset (free, already validated)
+   - Filter for: aggregation, GROUP BY, CASE WHEN, window functions
+   - These add general SQL knowledge to complement your domain-specific pairs
+
+3. Final validation: run all SQL against seed data
+4. Quality check: sample 100 pairs for manual review
+
+**Target:** 25,000-30,000 validated pairs
+**Time:** ~3 hours
+**Modal cost:** ~$2-3 (rephrasing on T4)
+
+### Day 4 (June 11): Model Training v2
+
+**Goal:** Fine-tune Qwen2.5-Coder-14B with expanded data
+
+1. Update `train.py`:
+   - Base model: `unsloth/Qwen2.5-Coder-14B-Instruct`
+   - LoRA rank: 32 (up from 16 — more data supports this)
+   - Learning rate: 1e-4 (down from 2e-4 — more data)
+   - Epochs: 2 (down from 3 — more data, avoid overfitting)
+   - Max seq length: 4096 (up from 2048 — for longer queries)
+   - Batch: 4 × 8 = 32 effective
+
+2. Run training on Modal A10G (~2-3 hours for 25K pairs)
+   - Monitor loss curve
+   - Save checkpoints
+
+3. Export GGUF Q4_K_M
+
+**Time:** ~3-4 hours
+**Modal cost:** ~$4-5
+
+### Day 5 (June 12): Evaluation + Model Selection
+
+**Goal:** Pick the best model, verify quality
+
+1. Create eval set: 50 attendance questions with expected SQL
+   - Mix of simple, medium, complex
+   - Include edge cases (empty results, ambiguous questions)
+
+2. Compare:
+   - v1 (Qwen2.5-Coder-7B, 1,289 pairs)
+   - v2 (Qwen2.5-Coder-14B, 25K pairs)
+   - Unfine-tuned Qwen2.5-Coder-14B (baseline)
+
+3. Metrics:
+   - Exact SQL match
+   - Execution match (same results)
+   - Schema validity (no hallucinated columns)
+   - Safety (no DDL/DML)
+
+4. Pick winner, download GGUF to local Mac
+
+**Time:** ~3 hours
+**Modal cost:** ~$2
+
+### Day 6 (June 13): Integration + Polish
+
+**Goal:** Working end-to-end demo
+
+1. Update `model_inference.py` to use new model
+2. Update `prompts.py` with expanded schema
+3. Test end-to-end on Mac M2
+4. Polish Gradio UI
+5. Update README with v2 results
+
+**Time:** ~4 hours
+**Modal cost:** $0 (all local)
+
+### Day 7 (June 14): Deploy + Submit
+
+**Goal:** Live on HuggingFace Spaces
+
+1. Push to HF Space
+2. Smoke test on Space (Zero GPU)
+3. Record demo if needed
+4. Submit to hackathon
+
+**Time:** ~2 hours
+**Modal cost:** $0
+
+---
+
+## Budget Summary
+
+| Item | Hours | Modal Cost |
+|---|---|---|
+| Day 1: Schema + seed data | 3 | $0 |
+| Day 2: 72B data generation | 4 | $10 |
+| Day 3: Augmentation + Gretel | 3 | $3 |
+| Day 4: 14B training | 4 | $5 |
+| Day 5: Evaluation | 3 | $2 |
+| Day 6: Integration | 4 | $0 |
+| Day 7: Deploy | 2 | $0 |
+| **Total** | **23** | **~$20** |
+| **Remaining credits** | | **~$230** |
+
+You'll use less than 10% of your credits. The rest is bank for future iterations or the expanded multi-domain project post-hackathon.
+
+---
+
+## What This Gets You
+
+**Hackathon submission:**
+- Gradio app on HF Spaces
+- Fine-tuned 14B model for attendance SQL
+- 25K+ training pairs (could open-source this)
+- Local-first: runs on M2 Mac with no API calls
+- Targets: Off the Grid, Well-Tuned, Llama Champion, Off-Brand badges
+
+**Skills honed:**
+- Synthetic data generation with large models
+- QLoRA fine-tuning pipeline on Modal
+- GGUF export + quantization
+- Schema-aware NL→SQL evaluation
+- End-to-end ML product deployment
+
+**Post-hackathon assets:**
+- Reusable training pipeline (swap schema → new domain)
+- $230 Modal credits for expanded project
+- Training data generation methodology
+- Foundation for LTC school district analytics product
+
+---
+
+## Key Risk: 14B on A10G
+
+Qwen2.5-Coder-14B in 4-bit QLoRA needs ~18-20GB VRAM. A10G has 24GB. This fits, but tighter than 7B. If it OOMs:
+
+**Fallback:** Use Qwen2.5-Coder-7B for training (proven), but evaluate 14B unfine-tuned as inference model. The 14B base model might be good enough without fine-tuning for a single-domain attendance task — test this on Day 5.
+
+---
+
+## Appendix: Where to Get the Models
+
+```bash
+# Qwen2.5-Coder-14B GGUF (for local inference)
+# Check: huggingface.co/unsloth/Qwen2.5-Coder-14B-Instruct-GGUF
+# Or: mradermacher/Qwen2.5-Coder-14B-Instruct-GGUF
+
+# Qwen2.5-Coder-14B (for Modal fine-tuning)
+# Unsloth: unsloth/Qwen2.5-Coder-14B-Instruct
+
+# Qwen2.5-72B (for data generation on Modal)
+# Qwen/Qwen2.5-72B-Instruct
+
+# Gretel dataset (for augmentation)
+# huggingface.co/datasets/gretelai/synthetic_text_to_sql
+```

docs/TRAINING_PLAYBOOK.md

@@ -0,0 +1,187 @@
+# LFED Training Playbook
+
+> **Last updated:** 2026-06-08
+> **Status:** Training actively running on Modal A10G
+
+---
+
+## 1. How the Training Pipeline Works
+
+### Architecture Overview
+
+```
+generate_synthetic.py  →  train.jsonl (1,289 pairs)
+         ↓
+     train.py           →  lora-adapter/ (QLoRA weights)
+         ↓
+   export_gguf.py       →  GGUF Q4_K_M → HF Hub
+```
+
+All three steps run on **Modal** (cloud GPU platform) orchestrated by `modal_app.py`.
+
+### Step-by-Step Commands
+
+```bash
+cd /Users/flucido/projects/build-small-hackathon/Kasualdad_LFED
+source .venv/bin/activate
+
+# Full pipeline (all 3 steps sequentially)
+modal run modal_train/modal_app.py
+
+# Or deploy as persistent app (survives client disconnect)
+modal deploy modal_train/modal_app.py
+
+# Monitor progress
+modal app logs <app-id>
+
+# Check running apps
+modal app list
+```
+
+### Prerequisites
+
+| Requirement | How | Status |
+|---|---|---|
+| Modal account | Sign up at modal.com | ✅ `flucido` |
+| Modal credits | Hackathon-provided | ✅ |
+| HF_TOKEN secret | `modal secret create huggingface HF_TOKEN=<token>` | ✅ Created |
+| HF Hub repo | Auto-created by export_gguf.py | ⏳ After training |
+
+### Training Hyperparameters (Current)
+
+| Param | Value | Notes |
+|---|---|---|
+| Base model | `unsloth/Qwen2.5-Coder-7B-Instruct` | 7B params, passed 7/7 sanity |
+| Quantization | 4-bit QLoRA (bitsandbytes) | Fits A10G 24GB |
+| LoRA rank (r) | 16 | |
+| LoRA alpha | 16 | Equal to r = stable |
+| Target modules | q, k, v, o, gate, up, down proj | All attention + MLP |
+| Max seq length | 2048 | |
+| Batch size | 4 | |
+| Grad accumulation | 4 | Effective batch = 16 |
+| Learning rate | 2e-4 | Standard for QLoRA |
+| Warmup ratio | 0.1 | |
+| Epochs | 3 | |
+| Optimizer | adamw_8bit | |
+| Total steps | 243 | (1,289 / 16) × 3 |
+| Training speed | ~1.7s/step | ~7 min total on A10G |
+| Loss at step 72 | 0.1008 | Good — converged fast |
+
+---
+
+## 2. What Went Right
+
+### Model Selection
+- Qwen2.5-Coder-7B passed **7/7 sanity checks** for DuckDB SQL generation
+- GGUF Q4_K_M quantization (4.4 GB) fits in HF Spaces free tier
+- Good at code/SQL tasks out of the box
+
+### Architecture
+- Modular codebase: app.py / data_engine.py / model_inference.py / prompts.py
+- Clean separation of concerns
+- 81 tests passing
+
+### Synthetic Data Generation
+- Template-based approach works well for narrow domains
+- 32 templates with weighted sampling
+- Questions are parameterized (school names, years, thresholds) for variety
+- Dedup prevents exact duplicate questions
+- Each template generates matched Q/S pairs — no hallucinated SQL
+
+### Training Execution
+- Unsloth provides 2x speedup + 50% VRAM reduction
+- Training converged fast (loss 0.1008 by step 72/243)
+- QLoRA at 0.53% trainable params = cheap, fast, effective
+
+### Key Fix: fn.spawn() Fire-and-Forget
+- **Problem:** 5 training runs crashed because `modal run` keeps a client connection open. When the local CLI times out or disconnects, Modal cancels the running function.
+- **Solution:** Switch to `modal deploy` + `fn.spawn()` which fires the function with no client connection to kill.
+- **Result:** Training survived past step 72 where all previous runs crashed.
+
+---
+
+## 3. What Went Wrong (and Fixes)
+
+### Issue 1: 5 Consecutive Training Crashes
+- **Symptom:** Training started, ran ~20-30 steps, then silently stopped
+- **Root cause:** `modal run` maintains a gRPC connection to Modal. When the local terminal session exits (timeout, sleep, Ctrl+C), Modal cancels the remote function.
+- **Fix:** Use `modal deploy` to create a persistent app, then call `fn.spawn()` from a Modal function that has no client connection. The deployed app stays alive independently.
+- **Lesson:** NEVER use `modal run` for long-running GPU work. Always deploy + spawn.
+
+### Issue 2: Flash Attention 2 Broken on Modal
+- **Symptom:** Warning "Your Flash Attention 2 installation seems to be broken. Using Xformers instead."
+- **Impact:** Minimal — Xformers works fine, no perf change observed
+- **Fix:** Not needed for hackathon. For production, pin specific flash-attn + CUDA versions.
+
+### Issue 3: Only 1,289 Pairs (vs 2,000 Target)
+- **Symptom:** Generator hit dedup ceiling before reaching 2,000
+- **Root cause:** 32 templates × limited parameter combinations (5 schools, 4 years) = ~1,300 unique questions max before repeats
+- **Fix needed:** Add more templates, more parameter variety, or augment with LLM-generated rephrasings (see Expansion Plan)
+
+### Issue 4: Narrow Schema Coverage
+- **Symptom:** Training data only covers 2 tables (enrollment, attendance)
+- **Impact:** Model can only answer attendance/enrollment questions — can't handle grades, discipline, demographics, wellbeing
+- **Fix needed:** Expand schema + training pairs (see Expansion Plan)
+
+### Issue 5: Memory Issue During Merge/Export
+- **Symptom:** User reported memory issues on Modal after training reached 30%
+- **Status:** Being investigated during current training run
+- **Likely cause:** Merging LoRA back into 16-bit base model requires loading full model (14GB+), which may exceed A10G 24GB when combined with training checkpoint memory
+- **Potential fix:** Free GPU memory between train and merge steps (already implemented in train.py lines 201-206), or use separate A10G container for merge step
+
+### Issue 6: Template-Based Questions Feel Synthetic
+- **Symptom:** Questions are grammatically correct but lack the messiness of real admin questions
+- **Impact:** Model may struggle with typos, abbreviations, informal phrasing
+- **Fix:** Add LLM-augmented rephrasings (see Expansion Plan)
+
+---
+
+## 4. Post-Training Checklist
+
+After training completes:
+
+```bash
+# 1. Verify GGUF pushed to Hub
+# Check https://huggingface.co/kasualdad/lfed-qwen2.5-coder-7b-sql-gguf
+
+# 2. Update model_inference.py to use fine-tuned model
+# Change REPO_ID and MODEL_FILE (lines ~103-107)
+
+# 3. Test locally
+python app.py
+# Ask: "How many students were chronically absent in 2023-2024?"
+
+# 4. Run tests
+pytest tests/ -v
+
+# 5. Deploy to HF Space
+git push space main
+```
+
+---
+
+## 5. Reproducing from Scratch
+
+```bash
+# 1. Clone
+cd /Users/flucido/projects/build-small-hackathon
+# (or clone from HF: git clone https://huggingface.co/spaces/build-small-hackathon/Kasualdad_LFED)
+
+# 2. Setup
+cd Kasualdad_LFED
+python3.12 -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
+modal token set  # authenticate
+
+# 3. Create HF secret for Modal
+modal secret create huggingface HF_TOKEN=$(cat ~/.huggingface/token)
+
+# 4. Run training
+modal deploy modal_train/modal_app.py
+# (use deploy, not run — see Issue 1 above)
+
+# 5. Monitor
+modal app list
+modal app logs <app-id>
+```

modal_train/export_gguf.py

@@ -19,43 +19,65 @@ from pathlib import Path
 # ── Configuration ──────────────────────────────────────────────────────
 
 BASE_MODEL = "unsloth/Qwen2.5-Coder-7B-Instruct"
-LORA_DIR = Path(__file__).parent / "lora-adapter"
-MERGED_DIR = Path(__file__).parent / "merged-model"
-GGUF_DIR = Path(__file__).parent / "gguf-output"
+LORA_DIR = Path("/data") / "lora-adapter"
+MERGED_DIR = Path("/data") / "merged-model"
+GGUF_DIR = Path("/data") / "gguf-output"
 GGUF_QUANT = "Q4_K_M"
 
 # HF repo — set via env or fall back to interactive
-HF_USERNAME = os.getenv("HF_USERNAME", "kasualdad")
+HF_USERNAME = os.getenv("HF_USERNAME", "build-small-hackathon")
 HF_REPO = f"{HF_USERNAME}/lfed-qwen2.5-coder-7b-sql-gguf"
 
 
 # ── Step 1: Merge LoRA → full model ────────────────────────────────────
 
-def merge_lora():
+def merge_lora(force: bool = False):
     """Merge the LoRA adapter weights into the base model."""
     print("🔀 Merging LoRA adapter into base model...")
 
-    if MERGED_DIR.exists():
-        print(f"   Merged model already exists at {MERGED_DIR}, skipping.")
-        return
-
-    from unsloth import FastLanguageModel
+    # Ensure GPU memory is clear
+    import gc
+    import torch
+    gc.collect()
+    torch.cuda.empty_cache()
+    print(f"   GPU: {torch.cuda.memory_allocated()/1e9:.1f}GB used, "
+          f"{torch.cuda.memory_reserved()/1e9:.1f}GB reserved")
 
-    model, tokenizer = FastLanguageModel.from_pretrained(
-        model_name=BASE_MODEL,
-        max_seq_length=2048,
-        dtype=None,
-        load_in_4bit=False,  # Load in 16-bit for merge
+    if MERGED_DIR.exists():
+        if force:
+            print(f"   Removing stale merged model at {MERGED_DIR}...")
+            shutil.rmtree(MERGED_DIR)
+        else:
+            print(f"   Merged model already exists at {MERGED_DIR}, skipping.")
+            return
+
+    # Use PEFT to load adapter onto base model, then merge
+    # Must load base in FP16 (not 4-bit) because merge_and_unload doesn't support quantized
+    print("   Loading base model in FP16 + adapter...")
+    from transformers import AutoModelForCausalLM, AutoTokenizer
+    from peft import PeftModel
+
+    base_model = AutoModelForCausalLM.from_pretrained(
+        BASE_MODEL,
+        torch_dtype=torch.float16,
+        device_map="auto",
+        low_cpu_mem_usage=True,
     )
+    model = PeftModel.from_pretrained(base_model, str(LORA_DIR))
+    tokenizer = AutoTokenizer.from_pretrained(str(LORA_DIR))
 
-    # Load and merge adapter
-    model.load_adapter(str(LORA_DIR))
+    print("   Merging adapter into base model...")
     model = model.merge_and_unload()
 
     MERGED_DIR.mkdir(parents=True, exist_ok=True)
     model.save_pretrained(str(MERGED_DIR))
     tokenizer.save_pretrained(str(MERGED_DIR))
 
+    # Free GPU
+    del model, tokenizer
+    gc.collect()
+    torch.cuda.empty_cache()
+
     print(f"✅ Merged model saved to {MERGED_DIR}")
 
 
@@ -67,53 +89,37 @@ def convert_to_gguf():
 
     GGUF_DIR.mkdir(parents=True, exist_ok=True)
 
-    # Check if llama.cpp conversion script is available (installed in Modal image)
-    # The script is typically at: llama.cpp/convert_hf_to_gguf.py
-    convert_script = shutil.which("convert_hf_to_gguf.py")
-    if not convert_script:
-        # Try common locations
-        candidates = [
-            "/app/llama.cpp/convert_hf_to_gguf.py",
-            "/llama.cpp/convert_hf_to_gguf.py",
-            "convert_hf_to_gguf.py",
-        ]
-        for c in candidates:
-            if Path(c).exists() or shutil.which(c):
-                convert_script = c
-                break
-
-    if not convert_script:
-        # Fallback: use llama-cpp-python's built-in converter
-        print("   Using llama-cpp-python converter...")
-        from llama_cpp import Llama
-        # llama-cpp-python >= 0.2.0 can convert directly
-        # We'll save as FP16 GGUF first, then quantize
-        f16_path = GGUF_DIR / "model-f16.gguf"
-
-        subprocess.run([
-            "python", "-m", "llama_cpp.convert",
-            str(MERGED_DIR),
-            "--outfile", str(f16_path),
-            "--outtype", "f16",
-        ], check=True)
-
-        # Quantize
-        out_path = GGUF_DIR / f"lfed-qwen2.5-coder-7b-sql-{GGUF_QUANT}.gguf"
-        subprocess.run([
-            "python", "-m", "llama_cpp.quantize",
-            str(f16_path),
-            str(out_path),
-            GGUF_QUANT,
-        ], check=True)
+    # Clone llama.cpp for the conversion script
+    llama_dir = Path("/tmp/llama.cpp")
+    if not llama_dir.exists():
+        print("   Cloning llama.cpp repo...")
+        subprocess.run(["git", "clone", "--depth=1", "https://github.com/ggml-org/llama.cpp", str(llama_dir)], check=True)
+
+    # Install gguf package
+    subprocess.run(["pip", "install", "-q", "gguf>=0.10.0"], check=True)
+
+    # Save as FP16 GGUF first, then quantize
+    f16_path = GGUF_DIR / "model-f16.gguf"
+    out_path = GGUF_DIR / f"lfed-qwen2.5-coder-7b-sql-{GGUF_QUANT}.gguf"
+
+    print(f"   Converting to FP16 GGUF...")
+    convert_script = llama_dir / "convert_hf_to_gguf.py"
+    subprocess.run([
+        "python", str(convert_script),
+        str(MERGED_DIR),
+        "--outfile", str(f16_path),
+        "--outtype", "f16",
+    ], check=True)
+
+    # Quantize using llama-quantize (built-in to llama-cpp-python)
+    print(f"   Quantizing to {GGUF_QUANT}...")
+    quantize_bin = shutil.which("llama-quantize") or shutil.which("quantize")
+    if quantize_bin:
+        subprocess.run([quantize_bin, str(f16_path), str(out_path), GGUF_QUANT], check=True)
     else:
-        # Use llama.cpp's built-in converter with --outtype for direct quant
-        out_path = GGUF_DIR / f"lfed-qwen2.5-coder-7b-sql-{GGUF_QUANT}.gguf"
-        subprocess.run([
-            "python", convert_script,
-            str(MERGED_DIR),
-            "--outfile", str(out_path),
-            "--outtype", GGUF_QUANT.lower(),
-        ], check=True)
+        # Fallback: use the FP16 GGUF directly
+        print("   ⚠️  llama-quantize not found, using FP16 GGUF")
+        out_path = f16_path
 
     print(f"✅ GGUF saved to {out_path}")
     return out_path
@@ -216,6 +222,12 @@ def main():
     print(f"  Repo: {HF_REPO}")
     print("=" * 60)
 
+    # Clean stale output dirs from failed previous runs
+    for d in [MERGED_DIR, GGUF_DIR]:
+        if d.exists():
+            print(f"   Cleaning stale {d}...")
+            shutil.rmtree(d)
+
     merge_lora()
     gguf_path = convert_to_gguf()
     push_to_hub(gguf_path)

modal_train/modal_app.py

@@ -36,8 +36,13 @@ train_image = (
         "peft>=0.13.0",
         "huggingface_hub>=0.26.0",
         "llama-cpp-python>=0.3.0",
+        "gguf>=0.10.0",
     )
     .env({"HF_HUB_ENABLE_HF_TRANSFER": "1"})
+    .add_local_dir(
+        Path(__file__).parent,
+        remote_path="/root",
+    )
 )
 
 # ── Volume for persistent storage (training data + checkpoints) ────────
@@ -70,9 +75,10 @@ def generate_synthetic():
     main()
 
     # Copy output back
+    import shutil
     train_jsonl = Path(__file__).parent / "train.jsonl"
     if train_jsonl.exists():
-        train_jsonl.rename(MOUNT_DIR / "train.jsonl")
+        shutil.move(str(train_jsonl), str(MOUNT_DIR / "train.jsonl"))
 
     print(f"✅ Synthetic data ready at {MOUNT_DIR / 'train.jsonl'}")
     volume.commit()
@@ -112,6 +118,13 @@ def train_model():
     from train import main
     main()
 
+    # Free GPU memory for the export step
+    import gc
+    import torch
+    del sys.modules['train']
+    gc.collect()
+    torch.cuda.empty_cache()
+
     volume.commit()
     print("✅ Training complete")
 
@@ -128,15 +141,21 @@ def train_model():
 def export_and_push():
     """Merge LoRA, export GGUF, push to HF Hub."""
     import sys
-    sys.path.insert(0, str(MOUNT_DIR))
+    import importlib
 
-    # Copy export script
-    export_script = Path(__file__).parent / "export_gguf.py"
-    dest = MOUNT_DIR / "export_gguf.py"
-    dest.write_text(export_script.read_text())
+    # Import directly from mount path (not volume copy)
+    mount_path = str(Path(__file__).parent)
+    if mount_path not in sys.path:
+        sys.path.insert(0, mount_path)
 
-    from export_gguf import main
-    main()
+    # Clear any cached module
+    for key in list(sys.modules.keys()):
+        if "export_gguf" in key:
+            del sys.modules[key]
+
+    import export_gguf
+    importlib.reload(export_gguf)
+    export_gguf.main()
 
 
 # ── Full pipeline (run all steps sequentially) ─────────────────────────
@@ -161,7 +180,7 @@ def run_full_pipeline():
     train_model.local()
 
     print("\n📦 Step 3/3: Merging → GGUF → HF Hub...")
-    export_and_push.local()
+    export_and_push.remote()  # remote() = fresh container, clean GPU
 
     print("\n🎉 Full pipeline complete!")
     print(f"   Model repo: check your HF Hub for 'lfed-qwen2.5-coder-7b-sql-gguf'")

modal_train/train.py

@@ -154,17 +154,25 @@ def train(model, tokenizer, dataset: Dataset):
             fp16=not is_bfloat16_supported(),
             bf16=is_bfloat16_supported(),
             logging_steps=LOGGING_STEPS,
-            save_steps=SAVE_STEPS,
-            save_total_limit=2,
+            save_strategy="no",  # manual save only — prevent pickle error
             optim="adamw_8bit",
             seed=42,
             report_to="none",  # No wandb on Modal
         ),
     )
 
-    trainer.train()
-
-    # Save final adapter
+    import traceback
+    try:
+        trainer.train()
+    except Exception as e:
+        if "Pickle" in str(e) or "pickle" in str(e):
+            print("\n⚠️  Trainer save failed (pickle error on args) — continuing with manual save...")
+            traceback.print_exc()
+        else:
+            raise
+
+    # Save final adapter (manual — bypasses trainer's pickle-prone save)
+    print("\n💾 Saving adapter weights...")
     OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
     model.save_pretrained(str(OUTPUT_DIR))
     tokenizer.save_pretrained(str(OUTPUT_DIR))
@@ -176,9 +184,28 @@ def train(model, tokenizer, dataset: Dataset):
 # ── Main ───────────────────────────────────────────────────────────────
 
 def main():
+    # Skip training if adapter already exists on volume
+    if (Path("/data") / "lora-adapter" / "adapter_config.json").exists():
+        print("✅ LoRA adapter already exists in volume — skipping training")
+        return
+    
+    if OUTPUT_DIR.is_dir() and (OUTPUT_DIR / "adapter_config.json").exists():
+        print("✅ LoRA adapter already exists — skipping training")
+        return
+    
     dataset = load_training_data()
     model, tokenizer = load_model_and_tokenizer()
     train(model, tokenizer, dataset)
+    
+    # Free GPU memory before merge step
+    import gc
+    del model
+    del tokenizer
+    gc.collect()
+    import torch
+    torch.cuda.empty_cache()
+    print("🧹 GPU memory cleared for merge step")
+    
     print("\n🎉 Training complete!")
 
 

model_inference.py

@@ -98,8 +98,8 @@ from prompts import build_prompt
 LOCAL_MODEL_PATH = "/tmp/lfed-models/qwen/Qwen2.5-Coder-7B-Instruct.Q4_K_M.gguf"
 
 # Fallback: download from HF Hub (base model, pre-fine-tune)
-HF_REPO_ID = "mradermacher/Qwen2.5-Coder-7B-Instruct-GGUF"
-HF_MODEL_FILE = "Qwen2.5-Coder-7B-Instruct.Q4_K_M.gguf"
+HF_REPO_ID = "build-small-hackathon/lfed-qwen2.5-coder-7b-sql-gguf"
+HF_MODEL_FILE = "model-f16.gguf"
 
 # Post-Phase-7: swap HF_REPO_ID / HF_MODEL_FILE to the fine-tuned GGUF
 # after Modal training completes and the model is pushed to HF Hub.