---
license: apache-2.0
language:
  - en
  - ta
  - hi
pipeline_tag: text-generation
tags:
  - education
  - academic
  - tutor
  - concept-first
  - india
  - from-scratch
---

# AQ-Academic-AI — Academic Quotient v1 (Tutor)

**AQ (Academic Quotient) — India's Concept-First Academic AI. v1 now live.**
*Raising the Academic Quotient of every student.*

AQ is a **1.26B-parameter academic tutor built completely from scratch** by Zyora Labs — proprietary architecture, own training code (pure PyTorch), own tokenizer, own data pipeline, own tutor fine-tune. No fine-tune of any existing model.

This is the **tutor (instruct) model**: it answers student questions directly with explanations, numbered steps, and worked examples. The pretrained base model is available at [zyoralabs/AQ-academic-ai-base](https://huggingface.co/zyoralabs/AQ-academic-ai-base).

## Training

1. **Pretraining — 20B tokens**, knowledge-dense and concept-first: real textbooks, course notes, scientific papers, encyclopedic text, mathematical reasoning — in English, Tamil, and Hindi. Grown progressively 75M → 300M → 1.26B, finished with a quality anneal (LR → 0 on the highest-quality academic text).
2. **Tutor fine-tune — 400M tokens** of educator-style instruction data (explanations, step-by-step math, knowledge Q&A, Hindi instructions), loss masked to tutor responses.

## Prompt format

```
### Student:
{your question}

### Tutor:
```

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

# Pin a released revision so the loaded code + weights are immutable
# (trust_remote_code executes this repo's modeling files).
REV = "v2.0"

tok = AutoTokenizer.from_pretrained("zyoralabs/AQ-academic-ai", revision=REV)
model = AutoModelForCausalLM.from_pretrained(
    "zyoralabs/AQ-academic-ai", revision=REV, trust_remote_code=True)

prompt = "### Student:\nWhat is a stack in data structures?\n\n### Tutor:\n"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=200, do_sample=True,
                     temperature=0.7, top_k=40, repetition_penalty=1.3)
print(tok.decode(out[0][ids.shape[1]:]))
```

Tested environment: `transformers==4.53.0`, `torch==2.7.1`, `tokenizers==0.21`, dtype `bfloat16`, single GPU or CPU.

## Architecture (proprietary, from scratch)

| | |
|---|---|
| Parameters | 1.26B |
| Layers | 48 |
| Hidden size | 1536 |
| Attention heads | 24 (grouped-query, 8 KV heads) |
| Feed-forward | SwiGLU, 4096 |
| Positional encoding | Rotary (RoPE) |
| Normalization | RMSNorm |
| Context length | 2048 |
| Vocabulary | 32,000 (byte-level BPE, English + Tamil + Hindi) |

## Benchmarks (0-shot, lm-evaluation-harness)

| Benchmark | AQ v2 Tutor | Notes |
|---|---|---|
| SciQ | 70.0 | strong science knowledge for the size/data budget |
| PIQA | 61.9 | |
| ARC-easy | 45.2 | |
| Winogrande | 50.1 | |
| HellaSwag | 29.6 | |
| MMLU | 25.2 | at-chance, like all ~1B-class models |
| ARC-challenge | 20.8 | |

### Reproducing these numbers

Scores were produced with **lm-evaluation-harness 0.4.8** (`pip install lm-eval==0.4.8`),
0-shot, default task configs, on a single H100 (bf16):

```bash
lm_eval --model hf \
  --model_args pretrained=zyoralabs/AQ-academic-ai,revision=v2.0,trust_remote_code=True,dtype=bfloat16 \
  --tasks mmlu,arc_easy,arc_challenge,hellaswag,piqa,winogrande,sciq \
  --batch_size auto
```

Environment: `transformers==4.53.0`, `torch==2.7.1` (cu128), `datasets==3.2.0`, `accelerate`.
The machine-readable harness output is published in this repo at
[`eval/aq_v2_tutor_results.json`](./eval/aq_v2_tutor_results.json).
Reported metric is `acc` (see the artifact for `acc_norm` and per-subtask MMLU results).

For context: models of this size trained on **15×–150× more tokens** (e.g. 300B–3T) reach SciQ ~84–89. AQ reaches ~70 on just **20B tokens** — the concept-first, knowledge-dense corpus is the point.

**Transparency:** as a 1.26B model, AQ v1 has real limits — multi-step arithmetic word problems and MMLU-style abstract reasoning are weak (these unlock at larger scale, on our roadmap). In the AQ product, answers are additionally grounded with retrieval over real study material.

## Intended use

The tutor layer of the AQ educator stack. Best used with the Student/Tutor prompt format, sampling enabled, and (in production) retrieval grounding over curriculum material.

## Team

| Name | Role | Affiliation |
|---|---|---|
| **Vasanth** | Chief AI Researcher | Zyora Labs |
| **Adithi Sreedhar** | Jr AI Engineer | AI & DS, Arunachala College of Engineering for Women |

## About

Built in India by [Zyora Labs](https://zyora.in). AQ v1 is the first release of the Academic Quotient model family.