Image-Text-to-Text
MLX
Safetensors
English
cohere_compass
mlx-vlm
vision-language
multimodal
cohere
north
quantized
conversational
5-bit
Instructions to use mlx-community/North-Micro-Vision-Instruct-5bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/North-Micro-Vision-Instruct-5bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("mlx-community/North-Micro-Vision-Instruct-5bit") config = load_config("mlx-community/North-Micro-Vision-Instruct-5bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Restore 5-bit affine model card after CLI reconversion
Browse files
README.md
CHANGED
|
@@ -4,38 +4,72 @@ license: apache-2.0
|
|
| 4 |
pipeline_tag: image-text-to-text
|
| 5 |
language:
|
| 6 |
- en
|
| 7 |
-
- de
|
| 8 |
-
- fr
|
| 9 |
-
- es
|
| 10 |
-
- it
|
| 11 |
-
- pt
|
| 12 |
-
- hi
|
| 13 |
-
- ja
|
| 14 |
-
- ko
|
| 15 |
-
- zh
|
| 16 |
-
- ar
|
| 17 |
tags:
|
| 18 |
-
- vision
|
| 19 |
-
- multimodal
|
| 20 |
-
- conversational
|
| 21 |
-
- multilingual
|
| 22 |
-
- native-resolution
|
| 23 |
- mlx
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
base_model: CohereLabs/North-Micro-Vision-Instruct
|
|
|
|
| 25 |
---
|
| 26 |
|
| 27 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
-
|
| 34 |
|
| 35 |
-
|
| 36 |
-
pip install -U mlx-vlm
|
| 37 |
-
```
|
| 38 |
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
pipeline_tag: image-text-to-text
|
| 5 |
language:
|
| 6 |
- en
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
tags:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
- mlx
|
| 9 |
+
- mlx-vlm
|
| 10 |
+
- vision-language
|
| 11 |
+
- multimodal
|
| 12 |
+
- cohere
|
| 13 |
+
- north
|
| 14 |
+
- quantized
|
| 15 |
base_model: CohereLabs/North-Micro-Vision-Instruct
|
| 16 |
+
base_model_relation: quantized
|
| 17 |
---
|
| 18 |
|
| 19 |
+
# North Micro Vision Instruct — 5-bit affine (MLX)
|
| 20 |
+
|
| 21 |
+
This repository contains an Apple MLX conversion of
|
| 22 |
+
[CohereLabs/North-Micro-Vision-Instruct](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct). 5-bit affine MLX quantization with group size 64.
|
| 23 |
+
|
| 24 |
+
It belongs to the
|
| 25 |
+
[North Vision MLX collection](https://huggingface.co/collections/mlx-community/north-vision-6a7c9be6ccc1cd992a83aecb), which includes BF16, affine
|
| 26 |
+
4/5/6/8-bit, MXFP4, MXFP8, and NVFP4 variants.
|
| 27 |
+
|
| 28 |
+
## Conversion details
|
| 29 |
+
|
| 30 |
+
- Source: [CohereLabs/North-Micro-Vision-Instruct](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct)
|
| 31 |
+
- Format: MLX / MLX-VLM
|
| 32 |
+
- Quantization: bits: 5; group size: 64; mode: affine
|
| 33 |
+
- MLX-VLM source revision: main at 7ee8eba3
|
| 34 |
+
|
| 35 |
+
The repository was regenerated and uploaded directly with the MLX-VLM
|
| 36 |
+
conversion CLI:
|
| 37 |
+
|
| 38 |
+
~~~bash
|
| 39 |
+
python -m mlx_vlm convert \
|
| 40 |
+
--hf-path CohereLabs/North-Micro-Vision-Instruct \
|
| 41 |
+
--mlx-path North-Micro-Vision-Instruct-5bit \
|
| 42 |
+
--quantize --q-bits 5 --q-group-size 64 --q-mode affine \
|
| 43 |
+
--upload-repo mlx-community/North-Micro-Vision-Instruct-5bit
|
| 44 |
+
~~~
|
| 45 |
+
|
| 46 |
+
## Usage
|
| 47 |
+
|
| 48 |
+
Cohere Compass support is available on the current MLX-VLM main branch.
|
| 49 |
+
Install it directly from GitHub:
|
| 50 |
+
|
| 51 |
+
~~~bash
|
| 52 |
+
pip install -U "mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm.git"
|
| 53 |
+
~~~
|
| 54 |
+
|
| 55 |
+
Run vision-language inference:
|
| 56 |
|
| 57 |
+
~~~bash
|
| 58 |
+
mlx_vlm.generate \
|
| 59 |
+
--model mlx-community/North-Micro-Vision-Instruct-5bit \
|
| 60 |
+
--image /path/to/image.jpg \
|
| 61 |
+
--prompt "Describe this image." \
|
| 62 |
+
--max-tokens 512 \
|
| 63 |
+
--temperature 0.0
|
| 64 |
+
~~~
|
| 65 |
|
| 66 |
+
You can also pass an image URL to --image.
|
| 67 |
|
| 68 |
+
## Notes
|
|
|
|
|
|
|
| 69 |
|
| 70 |
+
- MLX is optimized for Apple silicon.
|
| 71 |
+
- This repository changes the storage precision/quantization, not the source
|
| 72 |
+
model architecture or intended behavior.
|
| 73 |
+
- Refer to the
|
| 74 |
+
[original model card](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct) for capabilities,
|
| 75 |
+
limitations, licensing context, and responsible-use guidance.
|