LoLLMS Web UI which binding?
#1
by Silverspoon7 - opened
Sorry for asking, the q3_k isn‘t recognized (unknown safetensors ID 11) with pyllama
q4_0-Version: error loading model: missing tok_embeddings.weight
llama_init_from_file: failed to load model
ok think i made it with c_transformers
Yeah ctransformers is correct. I will make that clearer in the README
By the way, don‘t know if it cares, but I compared German usage in PyTorch model (online on huggingface, h2o chat), your 4 Bit and 3 Bit conversion. The 3 bit conversion is like testing a random 7B Model. Maybe, it does better in english. But in german , it’s like …damaged.