layerx-labs/AMALIA-9B-0626-SFT-MLX-4bit

AMALIA-9B-0626-SFT — MLX 4-bit

4-bit MLX quantization of amalia-llm/AMALIA-9B-0626-SFT (~5.1 GB), for Apple Silicon Macs via mlx-lm.

AMALIA is an open-source language model targeting European Portuguese, developed by a consortium of Portuguese universities and research centres and funded by the Government of Portugal. This is the SFT (supervised fine-tuned, instruction-following) variant. See the original model card and the technical report for full details.

Converted with mlx_lm.convert (4-bit, group size 64).

Usage

pip install mlx-lm
mlx_lm.generate --model layerx-labs/AMALIA-9B-0626-SFT-MLX-4bit --prompt "Explica o que é o Arquivo.pt."

# or chat
mlx_lm.chat --model layerx-labs/AMALIA-9B-0626-SFT-MLX-4bit

from mlx_lm import load, generate
model, tokenizer = load("layerx-labs/AMALIA-9B-0626-SFT-MLX-4bit")
prompt = tokenizer.apply_chat_template([{"role": "user", "content": "Explica o que é o Arquivo.pt."}], add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))

License

Apache 2.0, same as the original model. All credit for the model goes to the AMALIA project.