Text Generation
GGUF
Portuguese
llama-cpp
gemma3
aac
augmentative-alternative-communication
arasaac
board-generation
brazilian-portuguese
cpu
Instructions to use tardellirs/aac-board-generator-770m-ptbr-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use tardellirs/aac-board-generator-770m-ptbr-GGUF with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="tardellirs/aac-board-generator-770m-ptbr-GGUF", filename="aac-board-generator-770m-ptbr-Q4_K_M.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use tardellirs/aac-board-generator-770m-ptbr-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M
Use Docker
docker model run hf.co/tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use tardellirs/aac-board-generator-770m-ptbr-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tardellirs/aac-board-generator-770m-ptbr-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tardellirs/aac-board-generator-770m-ptbr-GGUF", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M
- Ollama
How to use tardellirs/aac-board-generator-770m-ptbr-GGUF with Ollama:
ollama run hf.co/tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M
- Unsloth Studio
How to use tardellirs/aac-board-generator-770m-ptbr-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for tardellirs/aac-board-generator-770m-ptbr-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for tardellirs/aac-board-generator-770m-ptbr-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for tardellirs/aac-board-generator-770m-ptbr-GGUF to start chatting
- Atomic Chat new
- Docker Model Runner
How to use tardellirs/aac-board-generator-770m-ptbr-GGUF with Docker Model Runner:
docker model run hf.co/tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M
- Lemonade
How to use tardellirs/aac-board-generator-770m-ptbr-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull tardellirs/aac-board-generator-770m-ptbr-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.aac-board-generator-770m-ptbr-GGUF-Q4_K_M
List all available models
lemonade list
Upload folder using huggingface_hub
Browse files
.gitattributes
CHANGED
|
@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
aac-board-generator-770m-ptbr-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
aac-board-generator-770m-ptbr-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
aac-board-generator-770m-ptbr-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
aac-board-generator-770m-ptbr-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
aac-board-generator-770m-ptbr-f16.gguf filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- pt
|
| 4 |
+
license: gemma
|
| 5 |
+
library_name: gguf
|
| 6 |
+
pipeline_tag: text-generation
|
| 7 |
+
base_model: tardellirs/aac-board-generator-770m-ptbr
|
| 8 |
+
tags:
|
| 9 |
+
- gguf
|
| 10 |
+
- llama-cpp
|
| 11 |
+
- text-generation
|
| 12 |
+
- gemma3
|
| 13 |
+
- aac
|
| 14 |
+
- augmentative-alternative-communication
|
| 15 |
+
- arasaac
|
| 16 |
+
- board-generation
|
| 17 |
+
- brazilian-portuguese
|
| 18 |
+
- cpu
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
# AAC Board Generator 770M (pt-BR) — GGUF
|
| 22 |
+
|
| 23 |
+

|
| 24 |
+
[](https://huggingface.co/tardellirs/aac-board-generator-770m-ptbr)
|
| 25 |
+
[](https://ai.google.dev/gemma/terms)
|
| 26 |
+
[](https://www.papuguinho.com)
|
| 27 |
+
|
| 28 |
+
`llama.cpp` **GGUF builds** of [**aac-board-generator-770m-ptbr**](https://huggingface.co/tardellirs/aac-board-generator-770m-ptbr)
|
| 29 |
+
— a 771.6M Brazilian-Portuguese **multi-function AAC** model that drafts **pictogram-board** word lists *and* follows
|
| 30 |
+
in-assistant board-editing instructions (add / remove / generate N), on **CPU**. See the
|
| 31 |
+
[**model card**](https://huggingface.co/tardellirs/aac-board-generator-770m-ptbr) for what it does, training, and
|
| 32 |
+
evaluation.
|
| 33 |
+
|
| 34 |
+
## Files
|
| 35 |
+
|
| 36 |
+
| File | Quant | Size | Notes |
|
| 37 |
+
|---|---|---|---|
|
| 38 |
+
| `aac-board-generator-770m-ptbr-Q4_K_M.gguf` | Q4_K_M | 561 MB | smallest; slight quality drop |
|
| 39 |
+
| `aac-board-generator-770m-ptbr-Q5_K_M.gguf` | Q5_K_M | 606 MB | good size/quality balance |
|
| 40 |
+
| `aac-board-generator-770m-ptbr-Q6_K.gguf` | Q6_K | 766 MB | near-lossless |
|
| 41 |
+
| `aac-board-generator-770m-ptbr-Q8_0.gguf` | Q8_0 | 824 MB | **recommended** — safe near-lossless |
|
| 42 |
+
| `aac-board-generator-770m-ptbr-f16.gguf` | f16 | 1.55 GB | full precision reference |
|
| 43 |
+
|
| 44 |
+
At **Q8_0** the model needs on the order of **~0.9–1.1 GB RAM** to serve (weights + a small context) and produces a
|
| 45 |
+
~12-item board in **about 4 s on 4 CPU threads**.
|
| 46 |
+
|
| 47 |
+
## Usage
|
| 48 |
+
|
| 49 |
+
```bash
|
| 50 |
+
llama-server -m aac-board-generator-770m-ptbr-Q8_0.gguf -t 4 -c 2048 --host 127.0.0.1 --port 8080
|
| 51 |
+
```
|
| 52 |
+
|
| 53 |
+
Gemma has no system role — put the instruction in the user turn and call `/completion`:
|
| 54 |
+
|
| 55 |
+
```python
|
| 56 |
+
import requests
|
| 57 |
+
INSTR = ("Você monta pranchas de CAA (pictogramas, pt-BR). Para o PEDIDO, liste ~12 itens concretos e relevantes, "
|
| 58 |
+
"um por linha, no formato palavra|tipo|sinônimos (tipo: v/s/a/e/l/p). Só a lista.")
|
| 59 |
+
pedido = "monta uma prancha de brincar no parquinho"
|
| 60 |
+
prompt = f"<start_of_turn>user\n{INSTR}\n\nPEDIDO: {pedido}<end_of_turn>\n<start_of_turn>model\n"
|
| 61 |
+
print(requests.post("http://127.0.0.1:8080/completion",
|
| 62 |
+
json={"prompt": prompt, "temperature": 0, "n_predict": 320}).json()["content"])
|
| 63 |
+
```
|
| 64 |
+
|
| 65 |
+
Generation is **greedy / deterministic** (`temperature 0`). Output is one item per line, `word|type|synonyms`.
|
| 66 |
+
|
| 67 |
+
## Notes on the build (why these GGUFs are faithful)
|
| 68 |
+
|
| 69 |
+
This model is a **vocabulary-trimmed** Gemma 3 (262k → 64k tokens). A reconstructed SentencePiece tokenizer is *not*
|
| 70 |
+
bit-exact for a trimmed vocabulary and would silently degrade greedy generation. These GGUFs instead use the
|
| 71 |
+
**BPE `tokenizer.json`** path, which tokenizes **identically** to the original — so the quantized model reproduces the
|
| 72 |
+
untrimmed parent's outputs. Verified: identical token counts and identical boards vs. the fp16 reference.
|
| 73 |
+
|
| 74 |
+
## License & attribution
|
| 75 |
+
|
| 76 |
+
Released under the **[Gemma license](https://ai.google.dev/gemma/terms)** (inherited from Gemma 3). Built for the
|
| 77 |
+
**ARASAAC** ecosystem (CC BY-NC-SA content). See the
|
| 78 |
+
[model card](https://huggingface.co/tardellirs/aac-board-generator-770m-ptbr) for full terms.
|
| 79 |
+
|
| 80 |
+
Developed for and used in **Papuguinho** — [www.papuguinho.com](https://www.papuguinho.com).
|
aac-board-generator-770m-ptbr-Q4_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:12af32ebb0193e434d6509dc530f7e066bad8d521fd53b9689108641101fc044
|
| 3 |
+
size 560686304
|
aac-board-generator-770m-ptbr-Q5_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3a5e2902588bbce59bfa1e64435b419c24ab384bde44da67b7044d42d742a562
|
| 3 |
+
size 605973728
|
aac-board-generator-770m-ptbr-Q6_K.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4db4f8f5c3786e110f5ec9d2fb121ff94e6b28c6ce32b057ec39a82798ede73a
|
| 3 |
+
size 766366688
|
aac-board-generator-770m-ptbr-Q8_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6a245b424d904a9a1e50e20aec3ec90e49a40f78f9898b0c55bc4ca251b28968
|
| 3 |
+
size 823934432
|
aac-board-generator-770m-ptbr-f16.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:79da518283053b1dc9b0a3589fd092b7ecba60c9a0c622c46963142973447838
|
| 3 |
+
size 1547209344
|