🚀 Nero-Qwen2.5-1.5B-Surgical

Overview

This model is a surgically optimized version of Qwen2.5-1.5B-Instruct. It was compressed using the Nero Hybrid Engine, an experimental framework designed to reduce VRAM footprint while preserving core logic and reasoning capabilities.

📊 Surgical Report

• Original Size: ~3.09 GB
• Compressed Size: ~1.74 GB
• VRAM Savings: 39.32%
• Method: Hybrid Low-Rank SVD Decomposition + INT8 Quantization.

🧠 The Nero Algorithm

Unlike traditional "blind" quantization, the Nero Engine applies a surgical approach:

1. SVD Filtering: It decomposes weights into smaller matrices to filter out redundant parameters (Noise) using an "Elbow Method" for optimal rank selection.
2. Dynamic Protection: Critical layers like self_attn and lm_head are kept at higher precision to ensure the model doesn't lose its "common sense."
3. Hybrid INT8: The remaining MLP weights are quantized to INT8, achieving massive storage gains.

🛠️ Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "neroai14/Nero-Qwen2.5-1.5B-Surgical"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    "yassin/Nero-Quantizer", 
    dtype="auto",  # <--- Make sure this says 'dtype', NOT 'torch_dtype'
    device_map="auto"
)