🏛️ FiscMind-Gemma4-26B-MoE (Frontier Romanian Fiscal & Accounting Assistant)

FiscMind-Gemma4-26B-MoE is an elite Romanian fiscal and accounting AI model adapted from Google's Gemma 4 (26B A4B MoE) architecture on high-performance NVIDIA A100-80GB SXM4 infrastructure via Modal.com.

It merges the parametric capacity of a 26-Billion parameter foundation model with the ultra-low latency and high token throughput of a 4-Billion active parameter sparse routing network, specialized for Romanian Statutory GAAP (OMFP 1802/2014), the Romanian Tax Code (Legea 227/2015), recent fiscal austerity packages (Legea 296/2023, OUG 115/2023), and mandatory digital compliance frameworks (RO e-Factura UBL 2.1, SAF-T D406).

---

⚡ Technical Highlights

Attribute	Specification
Base Model	google/gemma-4-26B-A4B-it (Google DeepMind)
Architecture	Sparse Mixture-of-Experts (gemma4_unified) with 128 experts, top-8 routing
Active Parameters	~4 Billion active parameters per token (25.82B total parameters)
Hardware Used	NVIDIA A100-SXM4-80GB (Modal.com Serverless Cluster)
Quantization at Training	4-bit NF4 with native bfloat16 compute (BitsAndBytesConfig)
LoRA Config	$r=16$, $\alpha=32$, targeting language model linear projections (q, k, v, o, gate, up, down)
Training Steps	100 steps (effective batch size 8) over 1,719 statutory curriculum items
Training Duration	979.9 seconds (16.33 min)
Final Loss	0.6011 (Mean token accuracy >99.9%)
Adapter Size	35.51 MB SafeTensors

---

📚 Domain Competencies & Curriculum

1. OMFP 1802/2014 RO GAAP Accounting:
   • Double-entry accounting monographies: purchases (371, 401, 4426), sales (4111, 707, 4427, 607), trade margins (378, 4428).
   • Fixed assets & leasing (213, 167, 666, 6811, 2813).
   • Foreign exchange variations (401, 5124, 7651, 6651).
   • Closing operations & dividends (121, 129, 1061, 463, 457).
2. Fiscal Code (Legea 227/2015 & Amending Legislation):
   • Corporate Income Tax (Impozit pe Profit 16%, protocol 2%, sponsorships credit).
   • New Minimum Turnover Tax (IMCA 1% for entities exceeding 50,000,000 EUR turnover).
   • Fiscal loss carryforward limit (70% cap over 5 consecutive fiscal years).
   • VAT reverse charge (Taxare inversă 4426 = 4427) & Intra-community acquisitions (AIC/LIC).
3. Payroll & Extrasalary Benefits (OUG 115/2023 & Legea 296/2023):
   • IT sector tax exemption cap (10,000 lei gross salary ceiling).
   • Meal vouchers mandatory CASS (10% health contribution retention).
   • 33% monthly tax-free extrasalary benefits threshold.
4. Digital Tax Compliance:
   • RO e-Factura (5 calendar days transmission deadline, 15% sanction for unregistered invoices).
   • RO e-Transport (UIT code validity of 5 calendar days, 500 kg / 10,000 lei thresholds).
   • SAF-T (D406) (General ledger taxonomy mapping, monthly/quarterly obligations).

---

🚀 Quickstart: Inference with Hugging Face & PEFT

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE_MODEL = "google/gemma-4-26B-A4B-it"
ADAPTER_REPO = "iulio/FiscMind-Gemma4-26B-MoE"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    device_map="auto",
    dtype=torch.bfloat16,
)

model = PeftModel.from_pretrained(base_model, ADAPTER_REPO)
model.eval()

sys_msg = (
    "Esti FiscMind AI, expert contabil CECCAR si consultant fiscal CCF conform "
    "legislatiei din Romania actualizate 2025/2026. Raspunde precis cu temei legal si formule contabile complete."
)
prompt = f"""<start_of_turn>user
{sys_msg}

Care este monografia contabila pentru inregistrarea dividendelor interimare trimestriale si ce impozit se retine?<end_of_turn>
<start_of_turn>model
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.2,
        do_sample=True,
    )

print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

---

🛡️ Safety, Guardrails & Human-in-the-Loop Standard

• Strict Non-Autonomous Final Approval: All accounting and fiscal recommendations require human chartered accountant (CECCAR/CCF) verification before statutory declaration submission or general ledger booking.
• Audit Trail: Every monetary suggestion maintains strict double-entry balance: $$\sum \text{Debit} \equiv \sum \text{Credit}$$
• Data Privacy: No client confidential data or real company tax numbers are committed to public models.

---

📜 Citation & Credits

Developed by Iulian Amaricai as part of the FiscMind AI Romanian Statutory Accounting Assistant initiative. Base model courtesy of Google DeepMind (Gemma 4 Family).