mlx-community

/

Nemotron-3-Ultra-550B-A55B

like 0

Follow

MLX Community 13.3k

try go into settings, which i guess its the button in the down-left

its beside word called "1 CONVERSATION", its on the

"settings button" "1 CONVERSATION" "other button"

on top of-it there is "DELETE ALL CHATS"

please go into it, and screenshot me it,

my whole current priority is to solve that problem

if you did, i may suggest to edit the temp into 1 or 0.7

just show me it

also did you try other models?

did they work?

Here are the screenshots of settings menu.

I just found out that the model gives proper output if pipeline parallelism is used. But gives gibberish output in tensor parallelism.

did you try other models on same setup?

try

mlx-community/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16-mlx-8Bit

its famous and alot of people have running it

if it didn't work it should be not the model's problem

if it did work, its the model of "mlx-community/Nemotron-3-Ultra-550B-A55B" 's problem

i just got into your settings just for searching about the settings called "top-p" and "top-k" and "temp"

and also the model i suggest it to you to try is just a suggestion, not a force, you can try any other model, i didn't force you into something you don't want, which trying a specific model

no i didn't

im sorry if looks like i did

in meanwhile just try any other model

don't let the time gets wasted!!

i hope Deeply from my heart it works the setup for running the models you want

please let me know if you have any issues,

As i said , currently my whole priority is to fix that problem, which also other users may gets affected from it,

i really doesn't want people lose interest in digital intelligence...

I tried with another model.
https://huggingface.co/mlx-community/Nemotron-3-Ultra-550B-A55B-4bit

Same issue. Works with pipeline parallel but not tensor parallel.
I think this is issue with EXO and not with the model.

https://huggingface.co/mlx-community/Nemotron-3-Ultra-550B-A55B-4bit

is the same model of

https://huggingface.co/mlx-community/Nemotron-3-Ultra-550B-A55B

beacuse

also
https://huggingface.co/mlx-community/Nemotron-3-Ultra-550B-A55B

is 4bit

all same...

use a different model, not just a different repositorys for the same model.....

I checked with https://huggingface.co/mlx-community/MiniMax-M2.7-8bit

It works fine in EXO with tensor parallel.

nice, now we know its from the model....

i recommend to you using this model

https://huggingface.co/spicyneuron/Nex-N2-Pro-MLX-5.3bit-vision
or
better fully
https://huggingface.co/usermma/Nex-N2-mini-mlx-fp16

this is good until someone or you, convert the model you want into mlx,

you have good unified memory,

which could so easily and much better than anyone else with just lower than half a hour,
the model would be converted!

this is full, good for your hardware,
this the currently the best for using by now... "unless you are ready for converting models into mlx in your own hardware"

https://huggingface.co/usermma/Nex-N2-mini-mlx-fp16

and also its great,

or try convert it yourself from
nex-agi/Nex-N2-mini

maybe you learn something new.... who knows?

Look... at Nex-N2-mini , it would be fast on your hardware, or even slower but better which is Nex-N2-Pro

can may i know what are you working on? i may help, giving some ideas... or even whole plans...

looks like you are working on something big that would make the world much better place...

if i had the same hardware as yours, i may try everything, not just stuck to one model which is "Nemotron" series

no quants: only full:

looks like there is a typo in GDPVal....