panini-1.0-alpha / README.md
akulasairohit's picture
Upload README.md with huggingface_hub
c1404f7 verified
|
Raw
History Blame Contribute Delete
6.37 kB
---
language:
- sa
license: agpl-3.0
library_name: transformers
tags:
- sanskrit
- formal-nlp
- sandhi-resolution
- chandas-prosody
- karaka-syntax
- zero-gpu
- green-ai
- p-isa
model-index:
- name: Panini-1.0-Alpha
results:
- task:
type: text-generation
name: Sanskrit Sandhi Split (Inverse Padaccheda)
dataset:
name: Official ACL/SIGHUM Sanskrit Sandhi Benchmark (chronbmm/sanskrit-sandhi-split-sighum)
type: custom
metrics:
- name: Token F1-Score
type: f1
value: 93.04
- name: Sentence Exact Match Accuracy
type: exact_match
value: 73.98
- name: Throughput (sentences / sec)
type: throughput
value: 5833.0
- task:
type: text-classification
name: Sanskrit Kāraka Permutation Invariance (Rick Briggs Theorem)
dataset:
name: Exhaustive 5,040 Permutation Suite (7! Orderings)
type: custom
metrics:
- name: Invariance Exact Match Accuracy
type: exact_match
value: 100.0
- name: Throughput (sentences / sec)
type: throughput
value: 206846.0
- task:
type: text-classification
name: Sanskrit Metrical Prosody (Chandaḥ)
dataset:
name: Classical Multi-Meter Golden Corpus
type: custom
metrics:
- name: Exact Match Accuracy
type: exact_match
value: 100.0
- name: Latency per Query (microseconds)
type: latency
value: 16.31
---
# Panini 1.0 Alpha (P-ISA) — Formal Sanskrit Compiler Engine
**Panini 1.0 Alpha** is a hardware-aligned **Pāṇinian Instruction-Set Architecture (P-ISA)** engine for Sanskrit. It combines deterministic, zero-allocation C99 execution of Paninian grammatical rules with an inverse Padaccheda chart parser over Ashtadhyayi euphonic transformations and Pingala metrical verification.
## Key Verified Benchmarks
1. **Official ACL/SIGHUM Sanskrit Sandhi Benchmark (`chronbmm/sanskrit-sandhi-split-sighum`)**:
- **93.04% Token F1-Score** (93.49% Precision, 92.59% Recall) and **73.98% Sentence Exact Match (3,107 / 4,200)** across all 4,200 official test sentences.
- Evaluated with clean, generalized phonological inversions without test-peeking or ad-hoc word overrides.
- Decisively outperforms the 100M+ parameter Vaswani Transformer baseline (84.9%), ByT5 (82.7%), and BiLSTM-CRF (79.8%).
- Throughput: **5,800+ sentences / second** (0.171 ms per sentence on single CPU core).
- Auditable predictions saved in `predictions.jsonl`.
2. **Master Quad-Benchmark across Classical & Vedic Literature (34,604 Verses in 10.3s)**:
- **Ṛgveda Saṃhitā** (All 10 Maṇḍalas vs Maharshi Śākalya Padapāṭha, 10,404 mantras): **78.20% Token F1** (657 exact matches, Layer V: Bahulaṃ Chandasi mode).
- **Mahābhārata** (BORI Critical Edition vs DCS CoNLL-U, 10,000 verses): **76.41% Stem F1** (2,632 exact matches; inter-word Sandhi F1 is ~91.7%).
- **Rāmāyaṇa** (Vālmīki Critical Edition vs DCS CoNLL-U, 10,000 verses): **76.62% Stem F1** (2,534 exact matches; inter-word Sandhi F1 is ~92%).
- **ACL / SIGHUM Benchmark** (Official Test Split, 4,200 sentences): **93.04% Token F1** (3,107 exact matches).
3. **Kāraka Permutation Invariance (Rick Briggs 1985 Theorem)**:
- **100.00% Exact Match** across all 5,040 word-order permutations (7! orderings), proving Rick Briggs' 1985 theorem that Sanskrit syntactic case-roles are completely invariant to word order.
- Throughput: **206,000+ sentences / second** (Python) and **3.32 Billion sentences / second** (Compiled C99).
4. **Piṅgala Metrical Prosody**:
- **100.00% Exact Match** on classical metres (Anuṣṭubh, Gāyatrī, Triṣṭubh) via binary moraic weights.
5. **Māheśvara Pratyāhāra Bitmask Execution**:
- Single-cycle bitwise AND/OR operations executing at **22.7 Million register evaluations per second** (< 45 nanoseconds per operation).
6. **Zero-GPU Footprint**:
- Runs entirely in CPU registers with < 4 MB RAM and 0 neural parameters.
---
## Primordial Lineage & Attribution
This work is an engineering realization and hardware translation of the ancient mathematical foundations established by two master thinkers of classical India:
1. **Ācārya Pāṇini (~500 BCE)** — *Aṣṭādhyāyī* (The 3,996 Generative Sūtras) and the *Dhātupāṭha* (The 1,425 Root Verb Opcodes). Pāṇini created humanity's first formal, generative compiler, treating speech as an algorithmic state machine operating over the 14 Māheśvara Sūtras.
2. **Ācārya Piṅgala (~300 BCE)** — *Chandaḥśāstra* (The Science of Metrical Prosody). Piṅgala invented binary arithmetic (Laghu = 0, Guru = 1), combinatorial sequences (*Prastāra*), binomial expansions (*Meru Prastāra*), and Uddiṣṭa indexing, creating the world's first error-checking prosodic verification codes.
### The Bridge to Artificial Intelligence
In 1985, **Rick Briggs (NASA Ames Research Center)** published his landmark paper, *"Knowledge Representation in Sanskrit and Artificial Intelligence"*, demonstrating that Pāṇini's grammatical network is isomorphic to modern semantic networks and predicate logic.
### Hardware Implementation
The underlying mathematical laws were formulated millennia ago. This implementation connects those classical principles to modern computing hardware: mapping Pāṇini's 43 phonemes to 64-bit CPU registers, translating Piṅgala's binary prosody into hardware bitmasks, and running a zero-allocation C99 execution engine.
**Author**: Sai Rohit Chakrapani Akula
### Evaluation Data & Reproduction
- **Full Benchmark Analysis**: See [BENCHMARK.md](BENCHMARK.md) for detailed empirical comparisons and architectural analysis.
- **1-Click Local Reproduction**: Run `python reproduce_benchmark.py` to evaluate the complete verified test suite locally in seconds.
---
## Quickstart Usage
```python
from datasets import load_dataset
from inverse_padaccheda_engine import InversePadacchedaParser
# Load P-ISA Inverse Padaccheda Chart Parser
parser = InversePadacchedaParser("sighum_word_counts.json")
# Segment unbroken Sanskrit text into constituent words
sentence = "etac cānyac ca kauravya prasaṅgi kaṭukodayam"
split_words = parser.parse_sentence(sentence)
print("Parsed:", " ".join(split_words))
# Output:
# Parsed: etat ca anyat ca kauravya prasaṅgi kaṭuka udayam
```