Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+granite-guardian-4.1-8b_IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+granite-guardian-4.1-8b_IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
+granite-guardian-4.1-8b_IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,148 @@
 ---
 license: apache-2.0
+language:
+    - en
+base_model:
+    - ibm-granite/granite-guardian-4.1-8b
+tags:
+    - text-generation
+    - safety
+    - content-moderation
+    - guardrails
+    - instruction-following
+    - reasoning
+    - efficient-model
 ---
+
+## Granite-Guardian-4.1-8B
+
+Granite-Guardian-4.1-8B is a safety-focused large language model developed for content moderation, policy evaluation, risk detection, and safe conversational workflows. This repository contains GGUF quantized variants of the model optimized for efficient local inference using llama.cpp.
+
+The quantized formats significantly reduce memory requirements while preserving strong classification and moderation performance, enabling practical deployment on consumer hardware and edge environments.
+
+---
+
+## Model Overview
+
+- **Model Name**: Granite-Guardian-4.1-8B
+- **Base Model**: ibm-granite/granite-guardian-4.1-8b
+- **Architecture**: Decoder-only Transformer
+- **Parameter Count**: 8 Billion
+- **Modalities**: Text
+- **Primary Languages**: English
+- **Developer**: IBM Granite
+- **License**: Apache 2.0
+
+---
+
+## Quantization Formats
+
+This repository provides various GGUF quantized versions of the `Granite-Guardian-4.1-8B` model, optimized for efficient local inference using `llama.cpp`. Below are the details of the available I-Matrix (IQ) formats.
+
+### IQ3_M
+
+- Size reduction of approx 76.68% (3.64 GB) compared to 16-bit (15.61 GB)
+- Aggressive 3-bit quantization optimized for maximum memory reduction
+- Suitable for low-memory systems and CPU-based inference
+- Maintains lightweight deployment capability for moderation pipelines
+- Output quality may degrade on nuanced reasoning or complex safety classification tasks
+
+### IQ4_NL
+
+- Size reduction of approx 70.92% (4.54 GB) compared to 16-bit (15.61 GB)
+- Advanced 4-bit non-linear quantization designed to better preserve output quality
+- More suitable for structured moderation workflows and detailed classification tasks
+- Typically provides stronger consistency compared to lower-bit formats
+- Slightly increased computational overhead during inference
+
+### IQ4_XS
+
+- Size reduction of approx 72.33% (4.32 GB) compared to 16-bit (15.61 GB)
+- Balanced 4-bit quantization focused on efficiency and stable inference performance
+- Good trade-off between model size, speed, and moderation quality
+- Suitable for general-purpose deployment across constrained hardware
+- Maintains reliable generation and classification behavior for most practical workloads
+
+---
+
+## Training Background (Original Model)
+
+Granite-Guardian-4.1-8B is trained with an emphasis on AI safety, risk evaluation, and policy-aware conversational analysis.
+
+### Pretraining
+
+- Large-scale language pretraining across diverse textual domains
+- Focus on contextual understanding and robust text representations
+- Optimized for downstream moderation and classification workflows
+
+### Alignment and Safety Tuning
+
+- Refined using safety-focused datasets and moderation objectives
+- Enhanced for harmful content detection and policy evaluation
+- Improved reliability for instruction compliance and risk-aware outputs
+
+---
+
+## Key Capabilities
+
+- **Content Moderation**
+  Detects unsafe, harmful, or policy-violating content across diverse inputs.
+
+- **Risk and Safety Evaluation**
+  Supports moderation pipelines and conversational safety workflows.
+
+- **Instruction Understanding**
+  Handles structured prompts and classification-oriented tasks effectively.
+
+- **Efficient Local Deployment**
+  Quantized variants enable practical offline inference on consumer hardware.
+
+- **Reliable Text Classification**
+  Suitable for filtering, moderation, and safety-oriented NLP applications.
+
+---
+
+## Usage Example
+
+### Using llama.cpp
+
+```
+./llama-cli \
+  -m SandlogicTechnologies/granite-guardian-4.1-8b_IQ4_NL.gguf \
+  -p "Explain the concept of knowledge distillation in detail"
+```
+
+---
+
+## Recommended Usecases
+
+- **AI Safety and Moderation Systems**
+  Build local moderation and filtering pipelines without cloud dependencies.
+
+- **Risk Classification Workflows**
+  Analyze prompts and outputs for harmful or unsafe content patterns.
+
+- **Enterprise Safety Layers**
+  Integrate guardrails into conversational AI systems and assistants.
+
+- **Research and Evaluation**
+  Study model alignment, moderation behavior, and safety-focused prompting strategies.
+
+---
+
+## Acknowledgments
+
+
+These quantized models are based on the original work by the **IBM Granite** development team.
+
+Special thanks to:
+
+- The [IBM Granite](https://huggingface.co/ibm-granite) team for developing and releasing the [Granite-Guardian-4.1-8B](https://huggingface.co/ibm-granite/granite-guardian-4.1-8b) model.
+
+- **Georgi Gerganov** and the `llama.cpp` open-source community for enabling efficient quantization and inference via the GGUF format.
+
+---
+
+## Contact
+
+For questions, feedback, or support, please reach out at [support@sandlogic.com](mailto:support@sandlogic.com) or visit https://www.sandlogic.com/

granite-guardian-4.1-8b_IQ3_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da4bf7d5eca8899dae49bbae727c7ef77a9acb5080237bd61643e37f92d37646
+size 3912561728

granite-guardian-4.1-8b_IQ4_NL.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa06ff4f3f46504c753e39792ab146bc556a0927c79bd3f006b16fb8a51b77ff
+size 4878480448

granite-guardian-4.1-8b_IQ4_XS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a3150672f8b6d333d6094ddacf296ece6ebdbb581c72b01d3efb67132a24c49
+size 4642878528