Text Generation
MLX
Safetensors
English
nemotron_h
nemotron
mamba
mamba2
mixture-of-experts
6bit
quantized
apple-silicon
conversational
reasoning
lm-studio
custom_code
6-bit
Instructions to use FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Faisal commited on
Upload explainability.md with huggingface_hub
Browse files- explainability.md +14 -0
explainability.md
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
| Field | Response |
|
| 2 |
+
| :---- | :---- |
|
| 3 |
+
| Intended Task/Domain: | Text generation, reasoning, and chat |
|
| 4 |
+
| Model Type: | Text-to-text Mamba2-Transformer Hybrid |
|
| 5 |
+
| Intended Users: | Generative AI creators working with conversational AI models and image content. |
|
| 6 |
+
| Output: | Text |
|
| 7 |
+
| Tools used to evaluate datasets to identify synthetic data and ensure data authenticity. | We used a Gemma-3 4B-based filtering model fine-tuned on [Nemotron Content Safety Dataset v2](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0) to ensure the quality of synthetic data. |
|
| 8 |
+
| Describe how the model works: | Generates text by predicting the next word or token based on the context provided in the input sequence using multiple self-attention layers. |
|
| 9 |
+
| Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of: | Age, Disability Status, Gender Identity, Nationality, Physical Appearance, Ethnicity, Socioeconomic Status, Sexual Orientation, Religion |
|
| 10 |
+
| Technical Limitations & Mitigation: | This model performs particularly well in instruction following regimes, as such may be strongly influenced by untrusted inputs and should be paired with appropriate guardrails and data filtering to better align use-case behaviors when exposed to such data. |
|
| 11 |
+
| Verified to have met prescribed NVIDIA quality standards: | Yes |
|
| 12 |
+
| Performance Metrics: | Accuracy, Throughput, and User-side throughput |
|
| 13 |
+
| Potential Known Risks: | The model was optimized explicitly for instruction following and as such may be influenced by untrusted inputs (prompt injection, indirect prompt injection, jailbreaking, web search, etc.) as a result of its instruction tuning that may degrade safety alignment and other training efforts. This model should be paired with additional guardrails and data filtering to limit exposure to instructions from malicious sources. Bypassing of safety alignment, system guardrails, and filters may allow harmful outcomes up to and including remote code execution in some agentic systems when effective security controls are not in place. The model was trained on data that contains toxic language and societal biases originally crawled from the internet. Therefore, the model may generate and amplify harmful, biased, or otherwise unsafe content reinforcing these biases and return toxic responses especially when prompted with toxic prompts. The model may also generate answers that may be inaccurate, omit key information, or include irrelevant or redundant text producing socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive. The model may exhibit self-anthropomorphism (e.g., displaying human-like characteristics in dialogue, such as expressing preferences and emotions). In integrated system contexts, the model could potentially be exploited to access or disclose information beyond the model’s intended permissions or scope of operation. |
|
| 14 |
+
| Licensing: | [NVIDIA Nemotron Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/) |
|