menezesbruno commited on
Commit
33b6d82
·
verified ·
1 Parent(s): ce3dfb2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +31 -1
README.md CHANGED
@@ -19,6 +19,20 @@ datasets:
19
 
20
  # Manacá-1B (base)
21
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  **[🇧🇷 Português](#português)** · **[🇬🇧 English](#english)**
23
 
24
  Manacá-1B é um modelo de linguagem decoder-only de ~1,72 bilhão de parâmetros,
@@ -85,6 +99,10 @@ warmup de 2.000), z-loss, bf16, *recompute* full, seed 1234. Treinado em 2 GPUs
85
  24 GB (sem NVLink) com paralelismo de dados (ZeRO-1). A corrida foi estável: 0
86
  iterações puladas e 0 NaN; loss de treino 11,41 → 2,48; loss de validação 2,07 nats.
87
 
 
 
 
 
88
  ### Avaliação
89
 
90
  Acurácia (%) em quatro benchmarks de português, mesmo harness. CALAME-PT por geração
@@ -104,6 +122,10 @@ de acaso em ARC-Challenge-PT (como todo modelo base nessa escala). Detalhes, tes
104
  pareados de McNemar e validação do harness no
105
  [repositório](https://github.com/Instituto-IA-LNCC/manaca-1b-base) e no [preprint](https://github.com/Instituto-IA-LNCC/manaca-1b-base/blob/main/paper/manaca_1b_base_arxiv.pdf).
106
 
 
 
 
 
107
  ### Uso
108
 
109
  ```python
@@ -186,6 +208,10 @@ warmup), z-loss, bf16, full recompute, seed 1234. Trained on 2 GPUs of 24 GB (no
186
  NVLink) with data parallelism (ZeRO-1). The run was stable: 0 skipped and 0 NaN
187
  steps; training loss 11.41 → 2.48; validation loss 2.07 nats.
188
 
 
 
 
 
189
  ### Evaluation
190
 
191
  Accuracy (%) on four Portuguese benchmarks, one harness. CALAME-PT by last-word
@@ -205,6 +231,10 @@ does every base model at this scale). Details, paired McNemar tests, and harness
205
  validation are in the
206
  [repository](https://github.com/Instituto-IA-LNCC/manaca-1b-base) and the [preprint](https://github.com/Instituto-IA-LNCC/manaca-1b-base/blob/main/paper/manaca_1b_base_arxiv.pdf).
207
 
 
 
 
 
208
  ### Usage
209
 
210
  ```python
@@ -244,7 +274,7 @@ on multiple choice). Evaluate before any sensitive use.
244
  Porto, Fabio Andr\'e Machado},
245
  year = {2026},
246
  note = {LNCC (AI Institute) $\times$ NII/LLM-jp},
247
- url = {https://github.com/brunoleomenezes/manaca-1b-base}
248
  }
249
  ```
250
 
 
19
 
20
  # Manacá-1B (base)
21
 
22
+ <p align="center">
23
+ <img src="manacaidentity.png" width="340" alt="Manacá — Tibouchina mutabilis: os três estágios florais como metáfora do treinamento do LLM / the three flowering colours as a metaphor for language-model maturation"/>
24
+ </p>
25
+
26
+ <p align="center"><em>LLM base aberto e reprodutível para o português do Brasil · Open, reproducible Brazilian-Portuguese base LLM</em></p>
27
+
28
+ <p align="center">
29
+ <a href="https://creativecommons.org/licenses/by/4.0/"><img src="https://img.shields.io/badge/License-CC%20BY%204.0-lightgrey.svg" alt="License: CC BY 4.0"></a>
30
+ <a href="https://github.com/Instituto-IA-LNCC/manaca-1b-base"><img src="https://img.shields.io/badge/Code-GitHub-181717.svg?logo=github" alt="Code"></a>
31
+ <img src="https://img.shields.io/badge/Language-PT--BR-009c3b.svg" alt="PT-BR">
32
+ <img src="https://img.shields.io/badge/Params-1.72B-purple.svg" alt="1.72B">
33
+ <img src="https://img.shields.io/badge/Base-pretrained-8A2BE2.svg" alt="base">
34
+ </p>
35
+
36
  **[🇧🇷 Português](#português)** · **[🇬🇧 English](#english)**
37
 
38
  Manacá-1B é um modelo de linguagem decoder-only de ~1,72 bilhão de parâmetros,
 
99
  24 GB (sem NVLink) com paralelismo de dados (ZeRO-1). A corrida foi estável: 0
100
  iterações puladas e 0 NaN; loss de treino 11,41 → 2,48; loss de validação 2,07 nats.
101
 
102
+ <p align="center">
103
+ <img src="training_dynamics_en.png" width="760" alt="Dinâmica de pré-treino do Manacá-1B: loss de treino e validação, norma do gradiente e learning rate ao longo de ~42B tokens"/>
104
+ </p>
105
+
106
  ### Avaliação
107
 
108
  Acurácia (%) em quatro benchmarks de português, mesmo harness. CALAME-PT por geração
 
122
  pareados de McNemar e validação do harness no
123
  [repositório](https://github.com/Instituto-IA-LNCC/manaca-1b-base) e no [preprint](https://github.com/Instituto-IA-LNCC/manaca-1b-base/blob/main/paper/manaca_1b_base_arxiv.pdf).
124
 
125
+ <p align="center">
126
+ <img src="benchmarks_paper_en.png" width="640" alt="Manacá-1B em quatro benchmarks de português, acurácia vs. parâmetros (escala log), com IC95%"/>
127
+ </p>
128
+
129
  ### Uso
130
 
131
  ```python
 
208
  NVLink) with data parallelism (ZeRO-1). The run was stable: 0 skipped and 0 NaN
209
  steps; training loss 11.41 → 2.48; validation loss 2.07 nats.
210
 
211
+ <p align="center">
212
+ <img src="training_dynamics_en.png" width="760" alt="Manacá-1B pretraining dynamics: training and validation loss, gradient norm, and learning rate over ~42B tokens"/>
213
+ </p>
214
+
215
  ### Evaluation
216
 
217
  Accuracy (%) on four Portuguese benchmarks, one harness. CALAME-PT by last-word
 
231
  validation are in the
232
  [repository](https://github.com/Instituto-IA-LNCC/manaca-1b-base) and the [preprint](https://github.com/Instituto-IA-LNCC/manaca-1b-base/blob/main/paper/manaca_1b_base_arxiv.pdf).
233
 
234
+ <p align="center">
235
+ <img src="benchmarks_paper_en.png" width="640" alt="Manacá-1B across four Portuguese benchmarks, accuracy vs. parameters (log scale), with 95% CIs"/>
236
+ </p>
237
+
238
  ### Usage
239
 
240
  ```python
 
274
  Porto, Fabio Andr\'e Machado},
275
  year = {2026},
276
  note = {LNCC (AI Institute) $\times$ NII/LLM-jp},
277
+ url = {https://github.com/Instituto-IA-LNCC/manaca-1b-base}
278
  }
279
  ```
280