---
license: apache-2.0
language:
  - en
  - ta
  - hi
pipeline_tag: text-generation
tags:
  - education
  - academic
  - concept-first
  - india
  - from-scratch
---

# AQ-1B — Academic Quotient v1 (Base)

**AQ (Academic Quotient) — India's Concept-First Academic AI.**
*Raising the Academic Quotient of every student.*

This is the **pretrained base model**. For the tutor (instruct) model — the one that answers student questions — see [zyoralabs/AQ-academic-ai](https://huggingface.co/zyoralabs/AQ-academic-ai).

AQ-1B is a **1.26B-parameter foundation model built completely from scratch** by Zyora Labs — proprietary architecture, own training code (pure PyTorch), own tokenizer, own data pipeline. No fine-tune of any existing model.

It is trained **concept-first**: the model learns the *concepts* of mathematics, physics, chemistry, biology, engineering, history, geography, civics and economics — from foundations to advanced — rather than curriculum checklists.

## Highlights

- **From scratch, end to end** — architecture, tokenizer (32k byte-level BPE), training loop, and data pipeline all built in-house
- **20B tokens** of knowledge-dense pretraining: encyclopedic text, real textbooks and course notes, scientific papers, and mathematical reasoning corpora
- **Final quality anneal** — the last 1.5B tokens use only the highest-quality sources (textbooks, course material, scientific papers, encyclopedic facts) with learning rate annealed to zero
- **Tamil + Hindi inclusive** — trained with native Tamil and Hindi text alongside English
- **Progressive growth training** — grown and continually trained through 75M → 300M → 1.26B parameter stages, each stage inheriting the previous stage's knowledge

## Architecture (proprietary, from scratch)

| | |
|---|---|
| Parameters | 1.26B |
| Layers | 48 |
| Hidden size | 1536 |
| Attention heads | 24 (grouped-query, 8 KV heads) |
| Feed-forward | SwiGLU, 4096 |
| Positional encoding | Rotary (RoPE) |
| Normalization | RMSNorm |
| Context length | 2048 |
| Vocabulary | 32,000 (byte-level BPE, English + Tamil + Hindi) |
| Embeddings | Tied |

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("zyoralabs/AQ-academic-ai-base")
model = AutoModelForCausalLM.from_pretrained("zyoralabs/AQ-academic-ai-base", trust_remote_code=True)

ids = tok("Photosynthesis is the process", return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=60)
print(tok.decode(out[0]))
```

## Intended use

AQ-1B is a **base (pretrained) model** — the foundation of the AQ educator stack (instruct tuning, retrieval grounding, and the AQ Playground sit on top of it). As a raw base model it predicts text continuations; it is not yet instruction-tuned.

## Team

| Name | Role | Affiliation |
|---|---|---|
| **Vasanth** | Chief AI Researcher | Zyora Labs |
| **Adithi Sreedhar** | Jr AI Engineer | AI & DS, Arunachala College of Engineering for Women |

## About

Built in India by [Zyora Labs](https://zyora.in). AQ v1 is the first release of the Academic Quotient model family.