---
license: apache-2.0
language:
- en
tags:
- agriculture
- farmer-advisory
- on-device
- edge-ai
- LiteRT-LM
- LiteRT
- text-generation
- qwen
- LoRA
- Indian-agriculture
base_model:
- unsloth/Qwen2.5-1.5B-Instruct
inference: false
library_name: litert-lm
pipeline_tag: text-generation
---

# Qwen-2.5-1.5B-KCC-LiteRT-LM

This is an on-device farmer advisory language model fine-tuned on cleaned Kisan Call Centre (KCC) question–answer pairs from Indian smallholder farmers, then converted and packaged for efficient offline inference using Google's LiteRT / LiteRT-LM stack.

It is intended for low-connectivity, edge scenarios — such as mobile advisory apps for Indian farmers.

## Model Details

- **Base model**: [unsloth/Qwen2.5-1.5B-Instruct](https://huggingface.co/unsloth/Qwen2.5-1.5B-Instruct)
- **Fine-tuning method**: LoRA (parameter-efficient) via Unsloth + TRL SFTTrainer
- **Dataset**: Cleaned ["Farmers Call Query Data"](https://www.kaggle.com/datasets/daskoushik/farmers-call-query-data-qa) by Das Koushik, based on data from [data.gov.in](https://www.data.gov.in/)
  → Only null/empty rows removed; no synthetic data, paraphrasing, or external augmentation
- **Training regime**: Very short, step-limited runs (max_steps=60, warmup_steps=5) on Colab Tesla T4 due to free-tier constraints  
  → Pilot for pipeline validation, **not** full convergence
- **Conversion**: PyTorch → LiteRT (.tflite) using [Google AI Edge Torch](https://github.com/google-ai-edge/ai-edge-torch) (v0.7.1)  
  → Static KV cache: 4096 tokens  
  → Result: ~1.6 GB .tflite artifact
- **Packaging**: .tflite → .litertlm using [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) (v0.8.1) tools
- **Quantization**: Quantized graph as produced by AI Edge Torch conversion
- **Context length**: 4096 tokens (fixed/static KV cache)
- **Intended use**: Offline, interactive agricultural advisory in low-resource settings
- **Out-of-scope**: General-purpose chat, high-precision agronomy, multi-turn memory beyond context limit, production-grade fluency

## Performance on Consumer Hardware

Tested on Mac mini (Apple M4, 16 GB unified memory) using LiteRT-LM with GPU backend:

- **Time-to-first-token (TTFT)**: < 1 second
- **End-to-end response time** (50–150 token advisory answers): ~2.5–4 seconds
- **Throughput**: Stable incremental decoding

Suitable for real-time, offline farmer-facing tools.

## Important Limitations & Known Behaviors

**This is an early engineering validation release — not a production model.**

Due to extremely short training (Colab constraints):

- Strong mirroring of original KCC terse, bullet-list style → outputs often lack natural conversational flow
- Occasional near-verbatim reuse of training phrases with limited adaptation to query variations
- Mild repetition / incomplete reasoning (undertraining artifact)

**LiteRT-LM specific observations** (compared to PyTorch inference):

- Noticeably reduced coherence
- Increased repetition, fragmentation, or looping in some generations
- Responses sometimes feel more generic / less tightly grounded

→ These are runtime-specific behaviors (not present in original PyTorch checkpoint).  
Root cause not yet isolated due to lack of controlled ablation; likely contributors include decoding configuration, stop-token alignment, fixed KV-cache constraints, and runtime-specific sampling behavior.

The project deliberately prioritizes **reproducible deployment path + failure mode transparency** over peak quality. Full multi-epoch training + runtime debugging expected to improve results significantly.

## Comparison with Prior Gemma-3n Effort

Compared to earlier Gemma-3n-E2B fine-tuning on the same task:

- **Qwen2.5-1.5B** wins on: conversion success, long-context stability, deployment reliability
- **Gemma-3n-E2B** wins on: more natural dialogue style, broader multilingual starting point
- Deciding factor: Gemma-3n could not be reliably converted to LiteRT-LM with public tooling → hard dead-end

See full comparison in the project documentation.

## Usage

This model is packaged in **.litertlm** format for use with **LiteRT-LM** runtime (preview stage as of Dec 2025).

Refer to:

- [LiteRT-LM GitHub](https://github.com/google-ai-edge/LiteRT-LM)
- Google AI Edge Gallery app (Android) for quick testing
- LiteRT documentation for integration into Android/iOS/macOS/Linux apps
- [UAI.LiteRTLM](https://github.com/Uralstech/UAI.LiteRTLM) — a Unity package wrapping LiteRT-LM inference, useful for building Android/Quest apps with this model

## Reproducibility

Full pipeline (data cleaning → LoRA fine-tuning → merge → LiteRT conversion → LiteRT-LM packaging) is documented with scripts and exact commands in the associated repository.

Conversion requires high-RAM CPU instance (~128 GB recommended). No GPUs needed for conversion/packaging.

See the project documentation for step-by-step instructions (AWS EC2 r6i instances used in original work):  
https://uralstech.github.io/Qwen-KCC-On-Device-Pipeline