Does not work in LM Studio (NB - update CUDA, use NOXZURE1's answer)
Tried to use this as it was listed in the model search. However it is not supported yet.
The underlying problem is that LM Studio does not natively support custom ternary weights yet. This is for anyone else who tries on the current lm studio. I'm only running on this rather than getting into alternatives - I'm just playing around.
Your LM studio is out of date.
Your LM studio is out of date.
It's 4.25 and I got it to download all it's items it wanted to update. I double checked, restarted etc.
LM Studio uses llama.cpp as their model runner. AFAIK, they no longer use mainline, but maintain their own fork. Until the edits that prism made to llama.cpp land in LM Studios fork of llama.cpp, this model will not work in LMS.
It does work with LM Studios with custom runtime forks. like how i made one. Yall can use this one as i uploaded it. Please verify you got the req it needs. Like Cuda 13 or 14. Ima also soon make a release for the cpu version which is already in the repo. just not in releases.
https://github.com/NOXZURE/llama.cpp-PrismML-Fork-LmStudio
Hope this works for yall. Has worked for basically all the 180 people that already downloaded it. Atleast only got one issue till now. All love ❤️
make sure to provide feedback if it works or not
It does work with LM Studios with custom runtime forks. like how i made one. Yall can use this one as i uploaded it. Please verify you got the req it needs. Like Cuda 13 or 14. Ima also soon make a release for the cpu version which is already in the repo. just not in releases.
https://github.com/NOXZURE/llama.cpp-PrismML-Fork-LmStudio
Hope this works for yall. Has worked for basically all the 180 people that already downloaded it. Atleast only got one issue till now. All love ❤️make sure to provide feedback if it works or not
It worked like a bloody charm.
I followed the steps - CUDA itself needed updating too, something your release makes clear, so thank you for that too. Its now running on my old 3080ti/athlon 3900x at 47tk/s - quite, quite acceptable.
So thank you very much
No problem. The cuda that needs to be updated comes with drivers anyways so for the people reading this. update yalls drivers.
im guessing it wont work with rocm?
It does work with LM Studios with custom runtime forks. like how i made one. Yall can use this one as i uploaded it. Please verify you got the req it needs. Like Cuda 13 or 14. Ima also soon make a release for the cpu version which is already in the repo. just not in releases.
https://github.com/NOXZURE/llama.cpp-PrismML-Fork-LmStudio
Hope this works for yall. Has worked for basically all the 180 people that already downloaded it. Atleast only got one issue till now. All love ❤️make sure to provide feedback if it works or not
im guessing it wont work with rocm?
im guessing it wont work with rocm?
It does work with LM Studios with custom runtime forks. like how i made one. Yall can use this one as i uploaded it. Please verify you got the req it needs. Like Cuda 13 or 14. Ima also soon make a release for the cpu version which is already in the repo. just not in releases.
https://github.com/NOXZURE/llama.cpp-PrismML-Fork-LmStudio
Hope this works for yall. Has worked for basically all the 180 people that already downloaded it. Atleast only got one issue till now. All love ❤️make sure to provide feedback if it works or not
im guessing it wont work with rocm?
nope sadly not yet. I could try to make an experimental one with the rocm one on the release page here. but i can't test it myself as i don't got a Radeon GPU.
I'll try to do it but i can't promise things
You can try this one and if it works. make sure you tell me like in here or smth so i can release this one aswell. but idk if it does or doesn't as i don't got a RDNA GPU.
https://drive.google.com/file/d/1e2Y6-PtDMvhscl5OxVQe22rhRXWthThi/view?usp=sharing
For anyone new. Use this for Cuda / CPU. AMD GPUs aren't supported.
https://github.com/NOXZURE/llama.cpp-PrismML-Fork-LmStudio