---
license: mit
base_model: google/gemma-4-e4b-it
library_name: transformers
tags:
  - gemma4
  - solar-energy
  - community-solar
  - function-calling
  - multimodal
  - lora
  - unsloth
  - ollama
  - edge-ai
  - energy
  - sustainability
  - hackathon
datasets:
  - Truthseeker87/solarhive-community-solar-multimodal
language:
  - en
pipeline_tag: image-text-to-text
model-index:
  - name: SolarHive-E4B-Merged
    results:
      - task:
          type: question-answering
          name: Domain Q&A
        metrics:
          - name: Accuracy
            type: accuracy
            value: 1.0
            verified: false
      - task:
          type: text-generation
          name: Tool Calling
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.8
            verified: false
---

![SolarHive](SolarHive_HeaderImage_1920x1080_HFModelCard.png)

# SolarHive E4B — BF16 Merged Safetensors

**LoRA fine-tuned Gemma 4 E4B (8B), merged to 16-bit safetensors.** Source artifact for **direct transformers inference** and **llama.cpp `convert_hf_to_gguf.py` → Q4_K_M GGUF conversion** (which powers Ollama + llama.cpp edge deployment via the [solarhive-e4b-gguf](https://huggingface.co/Truthseeker87/solarhive-e4b-gguf) companion repo).

> **For Ollama or llama.cpp edge deployment on a 16 GB CPU laptop, use the [solarhive-e4b-gguf](https://huggingface.co/Truthseeker87/solarhive-e4b-gguf) repo instead** — it ships the 5.34 GB Q4_K_M GGUF (standard Q6_K-PLE recipe) plus the 992 MB `mmproj-BF16.gguf` companion (vision + audio), with `Modelfile`s ready for `ollama create` and a 10/10 score on the single-pass SolarHive project-held-out 10-prompt parity check.
>
> The `--experimental` Ollama path documented previously OOMs `ollama create` on ≤16 GB RAM (the 16 GB BF16 safetensors blob does not fit during ingestion). On hardware ≥24 GB RAM the experimental import works, but the GGUF pipeline (built using **llama.cpp** `convert_hf_to_gguf.py` + `llama-quantize`) is the recommended edge deployment path for everyone else.

This repository now serves three roles:
1. **Source for GGUF conversion** via [llama.cpp](https://github.com/ggerganov/llama.cpp)'s `convert_hf_to_gguf.py` (text tower) and `convert_hf_to_gguf.py --mmproj` (vision + audio projector). See [solarhive-e4b-gguf](https://huggingface.co/Truthseeker87/solarhive-e4b-gguf) for the produced GGUF artifacts.
2. **Transformers-native multimodal use** — load with `AutoModelForCausalLM` for full image + audio + text in Python (requires ≥24 GB RAM or A100-class GPU).
3. **Reference for further fine-tuning** — extend the LoRA on additional data using Unsloth `FastVisionModel`.

Built for the [Gemma 4 Good Hackathon](https://kaggle.com/competitions/gemma-4-good-hackathon) (Google DeepMind x Kaggle).

| | |
|---|---|
| **Base Model** | [google/gemma-4-e4b-it](https://kaggle.com/models/google/gemma-4) |
| **Architecture** | Dense + PLE — 8B total, 4.5B effective |
| **Fine-Tuning** | LoRA via [Unsloth](https://unsloth.ai) (BF16) |
| **Training Data** | 1,727 examples ([solarhive-community-solar-multimodal](https://huggingface.co/datasets/Truthseeker87/solarhive-community-solar-multimodal)) — text-only fine-tune; VQA at inference uses the base [Gemma 4 vision encoder](https://ai.google.dev/gemma/docs/core/model_card_4) (~150M params), unmodified by our LoRA per the [Vertex AI SFT recipe](https://huggingface.co/blog/gemma4) |
| **Converged Loss** | **0.9218** |
| **Project-held-out check** | 9/10 (5/5 domain Q&A + 4/5 tool calling) — May 2026 final run, multi-call regression on TQ5 (see Multi-Variant Deployment Validation below) |
| **Training Time** | 420 seconds (~7 minutes) |
| **Compute** | Google Colab Pro |
| **License** | MIT (adapters) / Gemma Terms (base model) |

---

## Model Overview

SolarHive E4B is the **edge companion** to [SolarHive 26B A4B](https://huggingface.co/Truthseeker87/solarhive-26b-a4b-lora). While the 26B model powers cloud inference with full multimodal VQA, the E4B model is optimized for local deployment via Ollama on consumer hardware.

**Privacy-first:** Running Gemma 4 locally means community energy data never leaves the neighborhood. No cloud dependency, no internet requirement, no data privacy concerns. A village in rural India, a suburb in Michigan, and a coastal town recovering from a hurricane all get the same intelligence.

This repository contains the **fully merged model** (base + LoRA baked together) — no separate base model download needed.

---

## Training Details

| Parameter | Value |
|-----------|-------|
| Method | LoRA via Unsloth `FastVisionModel` (BF16, RTX PRO 6000 96 GB) |
| LoRA rank | 16 |
| LoRA alpha | 16 |
| LoRA dropout | 0 |
| Target modules | All linear layers |
| Learning rate | 2e-4 |
| Optimizer | AdamW 8-bit |
| Warmup steps | 5 |
| Epochs | 3 |
| Max sequence length | 2048 |
| Precision | BF16 |
| Seed | 3407 |
| Trainable parameters | 41.2M / 8.0B (0.51%) |

### Training Loss

| Metric | Value |
|--------|-------|
| Converged loss (last 20 steps) | **0.9218** |
| Final step loss | 0.0635 |
| Minimum loss | 0.0635 |
| Total steps | 324 |
| Training time | 420 seconds |

> **Canonical metric:** the bolded **Converged loss (last 20 steps)** is the only smoothed convergence indicator. *Final step* and *Minimum* are single-batch point statistics — mini-batch loss is noisy step-to-step, so one easy batch can drop a point estimate well below the rolling-average trend.

### Training Data

Same canonical training corpus as the 26B A4B model — [solarhive-community-solar-multimodal](https://huggingface.co/datasets/Truthseeker87/solarhive-community-solar-multimodal), 1,727 rows:

- 413 hand-crafted examples spanning 15+ US cities and 9 energy domains
- ~1,117 API-grounded examples from live Open-Meteo, PVWatts, OWM, and EIA data
- 183 tool-calling examples (positive, negative refusals, follow-up clarifications, failure-recovery)
- 14 image-grounded Q&A turns from 7 manually-labeled Ann Arbor sky photographs

### Hardware

- **GPU:** NVIDIA RTX PRO 6000 Blackwell Server Edition (102 GB GDDR7 total, 94.97 GB max usable per Unsloth)
- **Platform:** Google Colab Pro (G4 VM)

---

## Project-Held-Out Results

### Domain Q&A (5/5)

| Question | Result |
|----------|--------|
| Solar production when humidity exceeds 80%? | Correct |
| Battery SOC threshold for grid export? | Correct |
| Home #3 underperforming 22% — diagnostic checklist? | Correct |
| Winter snow on panels — prioritize actions? | Correct |
| Grid frequency 59.8 Hz — microgrid implications? | Correct |

> **Note on validation history:** the 5/5 Q&A above is from the initial 8-question validation harness used during fine-tune development. The canonical headline number is the May 2026 final-run multi-variant validation (single-pass project-held-out 10-question parity check) — see below.

### Tool inventory + inference-time When2Call validation

`solarhive_inference.py` exposes **5 tools** to the model — all three keyed APIs (`OWM_API_KEY`, `EIA_API_KEY`, `NREL_API_KEY`) actively wired:

| Tool | API | Returns |
|------|-----|---------|
| `get_weather(location)` | OpenWeatherMap (`OWM_API_KEY`) | Temperature, clouds %, wind, humidity, sunrise/sunset |
| `get_solar_production(clouds_pct, temp_f)` | Open-Meteo GHI (keyless) | Production kW, efficiency %, GHI W/m², temp derating |
| `get_battery_state()` | Community BMS (sim) | State of charge, capacity, charging status |
| `get_grid_status()` | EIA Open Data (`EIA_API_KEY`) | Pricing period, rate/kWh, renewable %, CO2 intensity |
| `get_nrel_pvwatts_baseline()` | NREL PVWatts v8 (`NREL_API_KEY`) | Annual + current-month typical kWh + avg kW for the 72 kW array |

Tool results feed back as a **2-message sequence** matching the training distribution: `{"role": "assistant", "tool_calls": [...]}` then `{"role": "tool", "name": "<fn>", "content": json.dumps(result)}` per call. Shared across the data-generation pipeline, the fine-tune SFT preprocessing layer, and the inference agentic loop — inference matches training distribution exactly.

**When2Call probes.** Three held-out probes validate 3 of the 4 failure-mode categories from [Ross, H., Mahabaleshwarkar, A. S., & Suhara, Y. (2025). *When2Call: When (not) to Call Tools.* arXiv:2504.18851](https://arxiv.org/abs/2504.18851) — the paper documents 9–67% tool-hallucination rates on (c)+(d) in untrained community models:

- **(b)** *"What's the current grid rate?"* → expect `get_grid_status` call (well-specified, in-scope)
- **(c)** *"How much will a 10 kW array produce today?"* → expect follow-up question (does NOT auto-fill location default)
- **(d)** *"What's the current air quality index in Ann Arbor?"* → expect refusal + redirect (does NOT hallucinate a tool)

Models trained without explicit *unable-to-answer* and *follow-up clarification* examples typically fail (c) + (d). The SolarHive corpus includes 16 such examples (10 *unable-to-answer* + 6 *follow-up clarification*) following the When2Call taxonomy.

### Multi-Variant Deployment Validation (Final Run, May 2026) — E4B regression on When2Call (c)

End-to-end inference run on Colab Pro G4. This E4B BF16 merged variant
loaded from a local cache (16.9 GB VRAM utilization, ~10 min runtime).

**Project-held-out parity check: 5/5 Q&A + 4/5 tool = 9/10** on the 10-question set —
matches the [A4B family](https://huggingface.co/Truthseeker87/solarhive-26b-a4b-lora)
on the 9 deterministic questions; the single FAIL is the lenient
multi-call probe (TQ5 — *"Compare today's irradiance forecast across
Ann Arbor, Phoenix, and Seattle"*, `min_calls=2`) where this variant
emitted only 1 `get_weather` call. Notably, the
[E4B LoRA + base variant](https://huggingface.co/Truthseeker87/solarhive-e4b-lora)
(same fine-tune, applied via Unsloth instead of merged) DOES chain
3 calls on the same probe and scores 10/10 — pattern reproducible
across runs.

**[When2Call](https://arxiv.org/abs/2504.18851) probes — measured 2/3 (final run May 2026):**

| Probe | E4B merged behavior | Score |
|-------|---------------------|:-:|
| **(b)** *"current grid rate?"* | ✅ Correctly calls `get_grid_status` | PASS |
| **(c)** *"How much will a 10 kW array produce today?"* | ❌ Auto-fills location and calls `get_solar_production` instead of asking back | FAIL |
| **(d)** *"current AQI in Ann Arbor?"* | ✅ Genuinely disclaims (no fabrication, no tool call) | PASS |

**Cross-variant pattern:** the [E4B LoRA + base variant](https://huggingface.co/Truthseeker87/solarhive-e4b-lora) is inferred to score **2/3** by mathematical lossless equivalence with this merged variant (the merge step is lossless on weights, so the When2Call decision boundary is identical). The +1/3 W2C delta vs the [A4B family](https://huggingface.co/Truthseeker87/solarhive-26b-a4b-lora) (3/3 directly measured on A4B LoRA, inferred-lossless on A4B merged + NF4) is the empirical signature of size-vs-refusal scaling.

**Honest finding — size-vs-refusal scaling is real, and was the pre-stated hypothesis.**
This E4B fine-tune regresses on (c) compared to the
[A4B LoRA baseline](https://huggingface.co/Truthseeker87/solarhive-26b-a4b-lora)
(which scores 3/3). The smaller model with less reasoning depth more readily
auto-fills missing parameters when it should ask back — exactly the failure
mode Ross et al. 2025 document at 9-67% rates in untrained community models.
The fine-tune closes (b)+(d) at this size but doesn't fully close (c).

**This was the expected outcome going in**, per the [official Google Gemma 4 Core docs](https://ai.google.dev/gemma/docs/core) *"Parameter sizes and quantization"*: *"Models with higher parameters and bit counts (higher precision) are generally more capable, but are more expensive to run."* E4B's 8B total / 4.5B effective parameters / ~150M vision encoder vs A4B's 25.2B total / 3.8B active (MoE) / ~550M vision encoder reflect a deliberate ~3× capacity gap on the dimension that drives reasoning-heavy refusal/follow-up behavior. The validation confirms the documented scaling — not a defect, but architecture-aware deployment design.

**Quantitative reinforcement** from [Unsloth's published Gemma 4 benchmarks](https://unsloth.ai/docs/models/gemma-4): E4B scores **69.4% on MMLU Pro** (vs 26B A4B's 82.6% — a 13.2 pp gap), **52.6% on MMMU Pro** (vs 73.8% — 21.2 pp gap), and **42.5% on AIME 2026** (vs 88.3% — a 45.8 pp gap). The AIME math-reasoning gap and MMMU Pro multimodal-reasoning gap directly **predict** the (c)/(d) [When2Call](https://arxiv.org/abs/2504.18851) regression we observe here — the smaller model's published reasoning-benchmark deltas scale cleanly into the 2-of-3 behavioral regression vs the [A4B baseline](https://huggingface.co/Truthseeker87/solarhive-26b-a4b-lora). E4B is the right choice for the volume of well-specified, in-scope queries that dominate everyday community-energy interactions; A4B handles the harder reasoning edge cases.

**Deployment recommendation:** Use this E4B variant for the volume of
**well-specified, in-scope queries** (production estimates, grid pricing,
maintenance guidance) where (b)-category routing dominates. Route
**under-specified or out-of-scope queries** to the A4B cloud variant for
correct refusal + follow-up behavior. A future fine-tune could increase
the E4B *follow-up clarification* example count (currently 6) and
*unable-to-answer* count (currently 10) to close the gap.

---

## How to Use

### Loading with transformers

```python
from transformers import AutoModelForCausalLM, AutoProcessor
import torch

model = AutoModelForCausalLM.from_pretrained(
    "Truthseeker87/solarhive-e4b-ollama",  # This repo (merged safetensors)
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
    "Truthseeker87/solarhive-e4b-ollama",
    trust_remote_code=True,
)
```

### Edge Deployment — use the GGUF repo

For Ollama or llama.cpp on a 16 GB CPU laptop, **download the GGUF artifacts** instead of trying to import these safetensors:

```bash
hf download Truthseeker87/solarhive-e4b-gguf \
  solarhive-e4b-q4_k_m.gguf Modelfile \
  --local-dir ./solarhive-gguf
cd ./solarhive-gguf
ollama create solarhive -f Modelfile
ollama run solarhive "What's the best time to run my dishwasher today?"
```

The [solarhive-e4b-gguf](https://huggingface.co/Truthseeker87/solarhive-e4b-gguf) repo ships the Q4_K_M GGUF (standard Q6_K-PLE recipe, 5.34 GB) plus a 992 MB `mmproj-BF16.gguf` for full multimodal via `llama-server --mmproj`.

### Edge Deployment via Ollama `--experimental` (≥24 GB RAM only)

If you have ≥24 GB system RAM, you can experimentally import these safetensors directly via Ollama:

```bash
git clone https://huggingface.co/Truthseeker87/solarhive-e4b-ollama
cd solarhive-e4b-ollama
cat > Modelfile << 'EOF'
FROM .
SYSTEM "You are SolarHive, an AI energy advisor for a community of 12 homes with rooftop solar and shared battery storage in Ann Arbor, Michigan. Use the available tools to get real-time data before answering. Be specific, reference actual data, and keep responses concise (3-5 sentences)."
PARAMETER temperature 1.0
PARAMETER top_p 0.95
PARAMETER top_k 64
PARAMETER num_ctx 4096
EOF
ollama create solarhive --experimental -f Modelfile
ollama run solarhive "What's the best time to run my dishwasher today?"
```

> **OOM warning:** on 16 GB RAM, `ollama create --experimental` crashes around 44% blob processing as Ollama tries to materialize the full 16 GB BF16 model in memory during ingestion. Use the GGUF path above instead.
>
> The official base (non-fine-tuned) E4B is also available as a pre-built GGUF on [ollama.com/library/gemma4:e4b](https://ollama.com/library/gemma4:e4b) (9.6 GB, Q4_K_M). Our [solarhive-e4b-gguf](https://huggingface.co/Truthseeker87/solarhive-e4b-gguf) adds 1,727 examples of community solar domain expertise on top.

### GGUF Conversion via llama.cpp (reproducibility recipe)

These safetensors are the source artifact for the GGUF deployment. Reproducible via [llama.cpp](https://github.com/ggerganov/llama.cpp) tooling:

```bash
# Text tower → BF16 GGUF (~14 GB intermediate)
python convert_hf_to_gguf.py --outtype bf16 \
  --outfile solarhive-e4b-bf16.gguf \
  Truthseeker87/solarhive-e4b-ollama/

# Quantize text → Q4_K_M (shipped recipe, 5.34 GB; needs ≥30 GB RAM)
llama-quantize \
  solarhive-e4b-bf16.gguf solarhive-e4b-q4_k_m.gguf Q4_K_M

# Multimodal projector (vision SigLIP + audio Conformer, ~992 MB)
python convert_hf_to_gguf.py --mmproj --outtype bf16 \
  --outfile mmproj-solarhive-e4b-BF16.gguf \
  Truthseeker87/solarhive-e4b-ollama/
```

The shipped 5.34 GB Q4_K_M quant requires ≥30 GB RAM at quantization time (the Q6_K-PLE tensor needs a ~10.7 GB float32 buffer). To quantize on a 16 GB laptop instead, add `--tensor-type per_layer_token_embd.weight=q4_0` — it bypasses that buffer and yields a smaller (~4.6 GB) GGUF, validated quality-safe in development. See the [`solarhive_quantize_e4b.ipynb`](https://github.com/youshen-lim/the-gemma4-good-hackathon-solarhive) notebook for the high-RAM recipe.

---

## Core Capabilities

### 1. Multimodal Visual Question Answering (3 Modes)

Available because the base Gemma 4 E4B vision encoder (~150M params) is preserved unmodified in these merged weights:

| Mode | Input | Output |
|------|-------|--------|
| Sky Analysis | Sky photograph | Cloud coverage %, production forecast, storage recommendation |
| Panel Inspection | Panel photograph | Dirt/damage/shading detection, efficiency impact estimate |
| Neighborhood Assessment | Aerial/satellite image | Panel inventory, expansion priorities, shading analysis |

### 2. Native Function Calling (5 Tools — all 3 keyed APIs wired)

| Tool | API | Returns |
|------|-----|---------|
| `get_weather(location)` | OpenWeatherMap (`OWM_API_KEY`) | Temperature, clouds %, wind, humidity, sunrise/sunset |
| `get_solar_production(clouds_pct, temp_f)` | Open-Meteo GHI (keyless) | Production kW, efficiency %, GHI W/m², temp derating |
| `get_battery_state()` | Community BMS (sim) | State of charge, capacity, charging status |
| `get_grid_status()` | EIA Open Data (`EIA_API_KEY`) | Pricing period, rate/kWh, renewable %, CO2 intensity |
| `get_nrel_pvwatts_baseline()` | NREL PVWatts v8 (`NREL_API_KEY`) | Annual + current-month typical kWh + avg kW for the 72 kW array |

### 3. Selective Tool Reasoning

The model decides when to call tools — it does not blindly invoke all of them:

```
"What time does peak pricing start?"
→ Calls: get_grid_status() only

"Should I run my pool heater now?"
→ Calls: get_weather() + get_solar_production() + get_battery_state() + get_grid_status()

"What are general maintenance tips for panels?"
→ Calls: none (answers from training knowledge)
```

---

## Community Model

| Parameter | Value |
|-----------|-------|
| Location | Ann Arbor, Michigan (42.2808°N, 83.7430°W) |
| Community size | 12 homes |
| Total panel capacity | 72 kW |
| Shared battery storage | 100 kWh |
| Grid region | MISO (Midcontinent Independent System Operator) |

---

## Technical Notes

- **Merged BF16 safetensors.** Base + LoRA fused via Unsloth `save_pretrained_merged("merged_16bit")`. Loads with plain `transformers.AutoModelForCausalLM.from_pretrained(...)` — no PEFT or Unsloth dependency at inference time.
- **Vision tower frozen during fine-tune.** VQA at inference uses the base model's pretrained vision encoder unmodified, matching the [Vertex AI SFT recipe](https://huggingface.co/blog/gemma4) which freezes both vision and audio towers during text-focused fine-tuning.
- **Two-step tokenization at inference.** Single-step `tokenize=True` crashes in transformers 5.5.x on messages without a `content` key (e.g., `tool_calls` messages). Always render text first (`tokenize=False`) then tokenize separately.
- **Sampling defaults.** `temperature=1.0, top_p=0.95, top_k=64` (Kaggle-recommended Gemma 4 defaults).
- **Chat template.** `gemma-4` (per Unsloth Tip #1 for E2B/E4B). The `gemma-4-thinking` template is reserved for 26B/31B reasoning-class variants. The simpler template is more robust across downstream Ollama / llama.cpp runtimes that don't expose `enable_thinking=False` at the runtime layer.

---

## Limitations

- **Prototype scope.** Tested on a single community model (12 homes, Ann Arbor, MI). Real-world deployment requires validation across diverse geographies and community sizes.
- **Smaller model, weaker refusal/follow-up.** When2Call (c) regression vs the A4B baseline (2/3 vs 3/3 — see Multi-Variant Deployment Validation above). Route under-specified or out-of-scope queries to the A4B cloud variant for correct refusal + follow-up behavior.
- **Occasional capacity hallucination.** The base model's prior occasionally surfaces "60 kW" instead of the correct 72 kW community capacity in direct (no-tool) responses. The tool-calling path (which queries actual capacity from `get_nrel_pvwatts_baseline`) avoids this.
- **External API dependence.** Tool responses depend on Open-Meteo, OWM, EIA, and PVWatts availability with their respective rate limits.
- **Battery state is simulated.** `get_battery_state()` is a deterministic in-memory simulator for demonstrations — real deployment requires integration with actual battery management systems.
- **Single-trial multi-variant validation.** The May 2026 final-run project-held-out numbers are from one inference pass; a multi-trial bootstrap would strengthen the multi-call regression claim against temperature-1.0 stochasticity.
- **Memory.** ~16 GB BF16 safetensors require ≥24 GB system RAM at load time — does not fit on consumer 16 GB laptops in this format. For 16 GB laptops, use the [solarhive-e4b-gguf](https://huggingface.co/Truthseeker87/solarhive-e4b-gguf) Q4_K_M variant.

---

## Future Iteration — Multi-Token Prediction (MTP) Drafters

> **Not in the measured numbers above.** Google announced Gemma 4 MTP drafters on **May 5, 2026** ([blog](https://blog.google/innovation-and-ai/technology/developers-tools/multi-token-prediction-gemma-4/), [overview](https://ai.google.dev/gemma/docs/mtp/overview), [HF collection](https://huggingface.co/collections/google/gemma-4), [Kaggle](https://www.kaggle.com/models/google/gemma-4), [@GoogleGemma](https://x.com/googlegemma/status/2051694045869879749)) — after this artifact's final project-held-out check was captured. The numbers above reflect standard autoregressive decoding only. **MTP integration is documented here as future iteration; no measured speedup is claimed in this release.**

**Theoretical foundation.** Speculative decoding (Leviathan, Kalman & Matias, *Fast Inference from Transformers via Speculative Decoding*, ICML 2023, [arXiv:2211.17192](https://arxiv.org/abs/2211.17192)) accelerates generation **without changing the output distribution under argmax decoding**: a smaller drafter proposes γ candidate tokens, the target verifies all γ in a single parallel forward pass, accepted tokens are kept, and any rejection is resampled from a corrected distribution. The output distribution is preserved exactly regardless of drafter quality; only **acceptance rate α**, and therefore walltime speedup, varies.

**What Google released on May 5, 2026.** Paired drafter checkpoints for all four IT-tuned Gemma 4 variants — `gemma-4-E2B-it-assistant`, `gemma-4-E4B-it-assistant`, `gemma-4-26B-A4B-it-assistant`, `gemma-4-31B-it-assistant` — discoverable via the [`google/gemma-4` Hugging Face collection](https://huggingface.co/collections/google/gemma-4) and on [Kaggle Models](https://www.kaggle.com/models/google/gemma-4). The drafters share the input embedding table with their paired target and consume the target's last-layer activations (architecture per the [MTP overview](https://ai.google.dev/gemma/docs/mtp/overview)). For the E4B target family the paired drafter is [`google/gemma-4-E4B-it-assistant`](https://huggingface.co/google/gemma-4-E4B-it-assistant) (~78.8 M params). Google reports **up to 3× decode speedup with no quality degradation** on the headline 26B-A4B configuration and **~2.2×** on Apple Silicon at batch sizes 4–8; per-variant E4B numbers were not enumerated in the announcement. Tested runtimes named in the blog: LiteRT-LM, MLX, Hugging Face Transformers, vLLM, SGLang, Ollama.

**Integration via Hugging Face Transformers** is a plain-`AutoModelForCausalLM` two-line load plus one extra kwarg:

```python
target    = AutoModelForCausalLM.from_pretrained("Truthseeker87/solarhive-e4b-ollama",        dtype=torch.bfloat16, ...)
assistant = AutoModelForCausalLM.from_pretrained("google/gemma-4-E4B-it-assistant",          dtype=torch.bfloat16, ...)
target.generate(**inputs, assistant_model=assistant)  # MTP enabled
```

The merged-safetensors load path on this repo is the cleanest E4B integration surface — no PEFT/Unsloth wrapping; the Hugging Face Transformers `assistant_model=` kwarg works directly.

**Open question specific to this LoRA-merged BF16 target.** Per the 2023 speculative-sampling guarantee, correctness is invariant to drafter quality — the target's verification step preserves the exact output distribution regardless of what the drafter proposes. What varies is **acceptance rate α**, since Google's released drafter was trained against the base `gemma-4-E4B-it`, not against this LoRA-merged target. Measured α at the edge BF16 tier is the planned post-hackathon contribution; a cloud-tier measurement against the A4B merged target is captured by the gated future-iteration cell in [`solarhive_inference.py`](https://github.com/youshen-lim/the-gemma4-good-hackathon-solarhive/blob/main/solarhive_inference.py) §14.

---

## Companion Repositories

| Model | Repository | Purpose |
|-------|-----------|---------|
| **SolarHive 26B A4B LoRA** | [solarhive-26b-a4b-lora](https://huggingface.co/Truthseeker87/solarhive-26b-a4b-lora) | Cloud inference with full multimodal + function calling (LoRA adapters) |
| **SolarHive 26B A4B Merged** | [solarhive-26b-a4b-merged](https://huggingface.co/Truthseeker87/solarhive-26b-a4b-merged) | Full BF16 cloud model (~48 GB) — production inference, no PEFT/Unsloth dep |
| **SolarHive 26B A4B NF4** | [solarhive-26b-a4b-nf4](https://huggingface.co/Truthseeker87/solarhive-26b-a4b-nf4) | Pre-quantized 4-bit cloud model for HF Spaces / 24 GB+ GPUs |
| **SolarHive E4B LoRA** | [solarhive-e4b-lora](https://huggingface.co/Truthseeker87/solarhive-e4b-lora) | E4B adapter weights (~200 MB) — apply over base via Unsloth |
| **SolarHive E4B safetensors** | This repo | Source safetensors for transformers research / GGUF conversion via llama.cpp |
| **SolarHive E4B GGUF** | [solarhive-e4b-gguf](https://huggingface.co/Truthseeker87/solarhive-e4b-gguf) | **Edge deployment** — Q4_K_M GGUF + mmproj for Ollama / llama.cpp on 16 GB CPU laptop. **10/10 project-held-out check.** |
| **SolarHive Dataset** | [solarhive-community-solar-multimodal](https://huggingface.co/datasets/Truthseeker87/solarhive-community-solar-multimodal) | 1,727 training examples (1,713 text + 14 image-grounded) |
| **LiteRT-LM Python edge runtime** | [`solarhive_e4b_litert_v3.1.ipynb`](https://github.com/youshen-lim/the-gemma4-good-hackathon-solarhive/blob/main/solarhive_e4b_litert_v3.1.ipynb) | LiteRT Special Tech Track entry — runs upstream base [`litert-community/gemma-4-E4B-it-litert-lm`](https://huggingface.co/litert-community/gemma-4-E4B-it-litert-lm) `.litertlm` (3.66 GB) + SolarHive UX layer + on-device agentic loop with native Gemma 4 function calling. **Q&A 8/8** on Colab Pro CPU + High-RAM. Fine-tuned LiteRT-LM bundle is a planned next iteration once upstream `gemma4` example module lands in `ai_edge_torch.generative.examples/`. |
| **GitHub** | [the-gemma4-good-hackathon-solarhive](https://github.com/youshen-lim/the-gemma4-good-hackathon-solarhive) | Full source code, training & quantization notebooks, data principles |

---

## Fine-Tuning Architecture — Text-Only on the Multimodal-Capable Corpus

**The shipped fine-tune is text-only** on the canonical
[`solarhive-community-solar-multimodal`](https://huggingface.co/datasets/Truthseeker87/solarhive-community-solar-multimodal)
corpus (1,727 rows = 1,713 text + 14 image-grounded). Image rows are
skipped at the data-prep layer; the training pipeline pre-renders only
text rows for TRL's default text collator. Multimodal fine-tuning is
deferred post-hackathon — a real image corpus and a held-out VQA
benchmark would be prerequisites; the dataset's image schema is
preserved so a future multimodal fine-tune can re-enable image rows
without changing the corpus.

**VQA at inference time uses the base Gemma 4 E4B model's pretrained
vision encoder (~150M params per the [official model card](https://ai.google.dev/gemma/docs/core/model_card_4)).**
Our LoRA targets only the language-model linear layers
(`target=all-linear`); the vision tower is not modified. This matches
the Vertex AI Gemma 4 SFT recipe documented in the
[Hugging Face blog](https://huggingface.co/blog/gemma4), which
explicitly freezes both vision and audio towers during text-focused
fine-tuning.

Companion 26B A4B LoRA is published at
[`Truthseeker87/solarhive-26b-a4b-lora`](https://huggingface.co/Truthseeker87/solarhive-26b-a4b-lora).

The dataset uses the project archive for its 14 image-grounded Q&A
turns (7 Ann Arbor sky photos × 2 turns). Image-source planning pivoted
twice: the SWIM corpora (NUS) were rejected for CC BY-NC licensing, and
NREL SRRL was rejected because the legacy MIDC SkyCam image archive
ended May 2017 (modern ASI-16 only exposes derived measurements). The
shipped dataset uses the project archive only — fewer images, but every
label is human-confirmed and every paired Q&A traces back to the same
GHI / temperature-derating formula used elsewhere in the dataset.

The fine-tune notebook has been pre-aligned with the official Unsloth
Gemma 4 documentation
([train guide](https://unsloth.ai/docs/models/gemma-4/train),
[bug fixes & tips](https://unsloth.ai/docs/models/gemma-4/train#bug-fixes--tips)):
explicit loader arguments (`max_seq_length`, `dtype`,
`full_finetuning=False`), explicit `SFTConfig` arguments
(`weight_decay`, `lr_scheduler_type`, `max_grad_norm`), and
`chat_template="gemma-4"` per Tip #1 (the simpler template is
recommended for E2B/E4B; `gemma-4-thinking` is reserved for 26B/31B
reasoning-class variants). The change makes the embedded chat
template more robust across downstream Ollama / llama.cpp runtimes
that don't expose `enable_thinking=False` at the runtime layer.

---

## Citation

```bibtex
@misc{solarhive2026,
  title={SolarHive: AI-Powered Community Solar Energy Intelligence},
  author={Youshen Lim},
  year={2026},
  url={https://github.com/youshen-lim/the-gemma4-good-hackathon-solarhive},
  note={Gemma 4 Good Hackathon submission — Google DeepMind x Kaggle}
}
```

---

## Links

- **GitHub:** [youshen-lim/the-gemma4-good-hackathon-solarhive](https://github.com/youshen-lim/the-gemma4-good-hackathon-solarhive)
- **Kaggle:** [The Gemma 4 Good Hackathon](https://kaggle.com/competitions/gemma-4-good-hackathon)
- **Base Model:** [google/gemma-4-e4b-it](https://kaggle.com/models/google/gemma-4)
- **Unsloth Gemma 4 docs:** [unsloth.ai/docs/models/gemma-4](https://unsloth.ai/docs/models/gemma-4)

*Built with Gemma 4 in Ann Arbor, Michigan. May 2026.*

*Gemma is a trademark of Google LLC.*