Text Generation
MLX
Safetensors
English
NemotronH_Nano_Omni_Reasoning_V3
nemotron
multimodal
mamba2
Mixture of Experts
quantized
turboquant
apple-silicon
mlx-lm
text-tower-only
conversational
custom_code
3-bit
Instructions to use majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit"
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit
Run Hermes
hermes
- OpenClaw new
How to use majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- MLX LM
How to use majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-TurboQuant-MLX-3bit", "messages": [ {"role": "user", "content": "Hello"} ] }'
| NVIDIA Open Model License Agreement | |
| This NVIDIA Open Model License Agreement (the “Agreement”) is a legal agreement between the Legal Entity You represent, or if no | |
| entity is identified, You and NVIDIA Corporation and its Affiliates (“NVIDIA”) and governs Your use of the Models that NVIDIA | |
| provides to You under this Agreement. NVIDIA and You are each a “party” and collectively the “parties.” | |
| NVIDIA models released under this Agreement are intended to be used permissively and enable the further development of AI | |
| technologies. Subject to the terms of this Agreement, NVIDIA confirms that: | |
| • | |
| Models are commercially useable. | |
| • | |
| You are free to create and distribute Derivative Models. | |
| • | |
| NVIDIA does not claim ownership to any outputs generated using the Models or Model Derivatives. | |
| By using, reproducing, modifying, distributing, performing or displaying any portion or element of the Model or Derivative Model, or | |
| otherwise accepting the terms of this Agreement, you agree to be bound by this Agreement. | |
| 1. | |
| Definitions. The following definitions apply to this Agreement: | |
| 1.1. | |
| “Derivative Model” means all (a) modifications to the Model, (b) works based on the Model, and (c) any other derivative | |
| works of the Model. An output is not a Derivative Model. | |
| 1.2. | |
| “Legal Entity” means the union of the acting entity and all other entities that control, are controlled by, or are under common | |
| control with that entity. For the purposes of this definition, “control” means (a) the power, direct or indirect, to cause the | |
| direction or management of such entity, whether by contract or otherwise, or (b) ownership of fifty percent (50%) or more | |
| of the outstanding shares, or (c) beneficial ownership of such entity. | |
| 1.3. | |
| “Model” means the machine learning model, software, checkpoints, learnt weights, algorithms, parameters, configuration | |
| files and documentation shared under this Agreement. | |
| 1.4. | |
| “You” or “Your” means an individual or Legal Entity exercising permissions granted by this Agreement. | |
| 2. | |
| Conditions for Use, License Grant, AI Ethics and IP Ownership. | |
| 2.1. | |
| Conditions for Use. The Model and any Derivative Model are subject to additional terms as described in Section 2 and | |
| Section 3 of this Agreement and govern Your use. If You institute copyright or patent litigation against any entity (including a crossclaim or counterclaim in a lawsuit) alleging that the Model or a Derivative Model constitutes direct or contributory copyright or | |
| patent infringement, then any licenses granted to You under this Agreement for that Model or Derivative Model will terminate as of | |
| the date such litigation is filed. NVIDIA may update this Agreement to comply with legal and regulatory requirements at any time | |
| and You agree to either comply with any updated license or cease Your copying, use, and distribution of the Model and any | |
| Derivative Model. | |
| 2.2. | |
| License Grant. The rights granted herein are explicitly conditioned on Your full compliance with the terms of this | |
| Agreement. Subject to the terms and conditions of this Agreement, NVIDIA hereby grants to You a perpetual, worldwide, nonexclusive, no-charge, royalty-free, revocable (as stated in Section 2.1) license to publicly perform, publicly display, reproduce, use, | |
| create derivative works of, make, have made, sell, offer for sale, distribute (through multiple tiers of distribution) and import the | |
| Model. | |
| 2.3. | |
| AI Ethics. NVIDIA is committed to safety, trust and transparency in AI development. NVIDIA encourages You to (a) ensure | |
| that the product or service You develop, use, offer as a service or distributes meets the legal and ethical requirements of the | |
| relevant industry or use case, (b) take reasonable measures to address unintended bias and to mitigate harm to others, including | |
| underrepresented or vulnerable groups, and (c) inform users of the nature and limitations of the product or service. NVIDIA | |
| expressly prohibits the use of its products or services for any purpose in violation of applicable law or regulation, including but not | |
| limited to (a) illegal surveillance, (b) illegal collection or processing of biometric information without the consent of the subject | |
| where required under applicable law, or (c) illegal harassment, abuse, threatening or bullying of individuals or groups of individuals | |
| or intentionally misleading or deceiving others. | |
| 2.4. | |
| NVIDIA owns the Model and any Model Derivatives created by NVIDIA. Subject to NVIDIA’s underlying ownership rights in | |
| the Model or its Model Derivatives, You are and will be the owner of Your Model Derivatives. NVIDIA claims no ownership rights in | |
| outputs. You are responsible for outputs and their subsequent uses. Except as expressly granted in this Agreement, (a) NVIDIA | |
| reserves all rights, interests and remedies in connection with the Model and (b) no other license or right is granted to you by | |
| implication, estoppel or otherwise. | |
| 3. | |
| Redistribution. You may reproduce and distribute copies of the Model or Derivative Models thereof in any medium, with or | |
| without modifications, provided that You meet the following conditions: | |
| 3.1. | |
| If you distribute the Model, You must give any other recipients of the Model a copy of this Agreement and include the | |
| following attribution notice within a “Notice” text file with such copies: “Licensed by NVIDIA Corporation under the NVIDIA Open | |
| Model License”; and | |
| 3.2. | |
| You may add Your own copyright statement to Your modifications and may provide additional or different license terms | |
| and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Models as a whole, provided | |
| Your use, reproduction, and distribution of the Model otherwise complies with the conditions stated in this Agreement. | |
| 4. | |
| Trademarks. This Agreement does not grant permission to use the trade names, trademarks, service marks, or product | |
| names of NVIDIA, except as required for reasonable and customary use in describing the origin of the Model and reproducing the | |
| content of the “Notice” text file. | |
| 5. | |
| Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, NVIDIA provides the Model on an “AS | |
| IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any | |
| warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are | |
| solely responsible for determining the appropriateness of using or redistributing the Model, Derivative Models and outputs and | |
| assume any risks associated with Your exercise of permissions under this Agreement. | |
| 6. | |
| Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or | |
| otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, will NVIDIA be | |
| liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a | |
| result of this Agreement or out of the use or inability to use the Model, Derivative Models or outputs (including but not limited to | |
| damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or | |
| losses), even if NVIDIA has been advised of the possibility of such damages. | |
| 7. | |
| Indemnity. You will indemnify and hold harmless NVIDIA from and against any claim by any third party arising out of or | |
| related to your use or distribution of the Model, Model Derivatives or outputs. | |
| 8. | |
| You. | |
| Feedback. NVIDIA appreciates your feedback, and You agree that NVIDIA may use it without restriction or compensation to | |
| 9. | |
| Governing Law. This Agreement will be governed in all respects by the laws of the United States and the laws of the State | |
| of Delaware, without regard to conflict of laws principles or the United Nations Convention on Contracts for the International Sale of | |
| Goods. The state and federal courts residing in Santa Clara County, California will have exclusive jurisdiction over any dispute or | |
| claim arising out of or related to this Agreement, and the parties irrevocably consent to personal jurisdiction and venue in those | |
| courts; except that, either party may apply for injunctive remedies or an equivalent type of urgent legal relief in any jurisdiction. | |
| 10. | |
| Trade and Compliance. You agree to comply with all applicable export, import, trade and economic sanctions laws and | |
| regulations, as amended, including without limitation U.S. Export Administration Regulations and Office of Foreign Assets Control | |
| regulations. These laws include restrictions on destinations, end-users and end-use. | |
| Version Release Date: June 14, 2024 | |