Upload PIBot Joint BERT model package

README.md

@@ -1,154 +1,164 @@
 ---
 language: es
-license: mit
 tags:
-  - pytorch
-  - bert
-  - joint-bert
-  - intent-classification
-  - slot-filling
-  - named-entity-recognition
-  - multi-head-classification
-  - spanish
-  - macroeconomics
+- intent-classification
+- slot-filling
+- joint-bert
+- spanish
+- economics
+- chile
+- multi-head
+license: mit
+base_model: dccuchile/bert-base-spanish-wwm-cased
 pipeline_tag: token-classification
-library_name: transformers
 ---
 
 # PIBot Joint BERT
 
-Modelo **JointBERT multi-cabeza** para consultas macroeconómicas en español.  
-Predice simultáneamente **5 cabezas de intención** + **slot filling BIO** por token.
+Modelo **Joint BERT multi-head** para clasificación de intención y slot filling,
+especializado en consultas sobre indicadores macroeconómicos del Banco Central de Chile.
 
-## Configuración del checkpoint
+## Arquitectura
 
-| Parámetro | Valor |
+| Componente | Detalle |
 |---|---|
-| **Task** | `pibimacecv5` |
-| **Base model** | `dccuchile/bert-base-spanish-wwm-cased` (BETO) |
-| **Model type** | `bert` |
-| **Custom code** | Sí (`trust_remote_code=True`) |
+| Base | `dccuchile/bert-base-spanish-wwm-cased` |
+| Task | `pibimacecv3` |
+| Intent heads | 5 (`activity`, `calc_mode`, `investment`, `region`, `req_form`) |
+| Slot labels | 15 (BIO) |
+| Custom code | `modeling_jointbert.py`, `module.py` |
 
-## Cabezas de intención
+### Intent Heads
 
-| Cabeza | Archivo de labels |
-|---|---|
-| `calc_mode` | `labels/calc_mode_label.txt` |
-| `activity` | `labels/activity_label.txt` |
-| `region` | `labels/region_label.txt` |
-| `investment` | `labels/investment_label.txt` |
-| `req_form` | `labels/req_form_label.txt` |
+| Head | Clases | Valores |
+|---|---|---|
+| `activity` | 3 | `none`, `specific`, `general` |
+| `calc_mode` | 4 | `original`, `prev_period`, `yoy`, `contribution` |
+| `investment` | 3 | `none`, `specific`, `general` |
+| `region` | 3 | `none`, `specific`, `general` |
+| `req_form` | 3 | `latest`, `point`, `range` |
 
-## Slot filling
+### Slot Entities (BIO)
 
-- Etiquetado BIO por token (`labels/slot_label.txt`)
+Entidades extraídas: `activity`, `frequency`, `indicator`, `investment`, `period`, `region`, `seasonality`
 
-## Estructura del repositorio
+Esquema BIO completo: 15 etiquetas (`O`, `B-*`, `I-*`).
 
-```text
-├── config.json                 # Configuración del modelo
-├── model.safetensors           # Pesos del modelo
-├── training_args.bin           # Argumentos de entrenamiento
-├── tokenizer.json              # Tokenizer
-├── tokenizer_config.json
-├── vocab.txt
-├── special_tokens_map.json
-├── modeling_jointbert.py       # Código custom (AutoModel)
-├── module.py                   # Clasificadores por cabeza
-├── __init__.py
-├── labels/
-│   ├── calc_mode_label.txt
-│   ├── activity_label.txt
-│   ├── region_label.txt
-│   ├── investment_label.txt
-│   ├── req_form_label.txt
-│   └── slot_label.txt
-└── README.md
-```
+## Uso
+
+### Instalación
 
-## Uso rápido
+```bash
+pip install torch transformers
+```
 
-### Descargar y cargar el modelo
+### Carga del Modelo
 
 ```python
 import torch
-from pathlib import Path
-from transformers import AutoConfig, AutoTokenizer, AutoModel
-
-repo_id = "tu-usuario/pibot-jointbert"  # Cambiar por tu repo
+from transformers import AutoTokenizer, AutoConfig
 
-# 1. Cargar tokenizer y config
-tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
-config = AutoConfig.from_pretrained(repo_id, trust_remote_code=True)
+# Cargar tokenizer y config
+tokenizer = AutoTokenizer.from_pretrained("smenaaliaga/pibert", trust_remote_code=True)
+config = AutoConfig.from_pretrained("smenaaliaga/pibert", trust_remote_code=True)
 
-# 2. Cargar training args y labels
+# Cargar labels desde el repo
 from huggingface_hub import hf_hub_download
+import os
 
-args_path = hf_hub_download(repo_id, "training_args.bin")
-train_args = torch.load(args_path, weights_only=False)
+label_dir = os.path.dirname(hf_hub_download("smenaaliaga/pibert", "labels/slot_label.txt"))
 
-def read_labels(repo_id, filename):
-    path = hf_hub_download(repo_id, f"labels/{filename}")
-    return [line.strip() for line in open(path, encoding="utf-8") if line.strip()]
+# Leer intent y slot labels
+def read_labels(path):
+    with open(path) as f:
+        return [line.strip() for line in f if line.strip()]
 
-calc_mode_labels = read_labels(repo_id, "calc_mode_label.txt")
-activity_labels = read_labels(repo_id, "activity_label.txt")
-region_labels = read_labels(repo_id, "region_label.txt")
-investment_labels = read_labels(repo_id, "investment_label.txt")
-req_form_labels = read_labels(repo_id, "req_form_label.txt")
-slot_labels = read_labels(repo_id, "slot_label.txt")
+slot_labels = read_labels(os.path.join(label_dir, "slot_label.txt"))
 
-# 3. Instanciar modelo
-model = AutoModel.from_pretrained(
-    repo_id,
+# Preparar intent_label_lst para cada head
+intent_label_lst = []
+for head in ['activity', 'calc_mode', 'investment', 'region', 'req_form']:
+    intent_label_lst.append(read_labels(os.path.join(label_dir, f"{head}_label.txt")))
+
+# Cargar modelo con custom code
+from transformers import AutoModelForTokenClassification
+from modeling_jointbert import JointBERT  # auto-cargado con trust_remote_code
+
+model = JointBERT.from_pretrained(
+    "smenaaliaga/pibert",
     config=config,
-    trust_remote_code=True,
-    args=train_args,
-    calc_mode_label_lst=calc_mode_labels,
-    activity_label_lst=activity_labels,
-    region_label_lst=region_labels,
-    investment_label_lst=investment_labels,
-    req_form_label_lst=req_form_labels,
+    intent_label_lst=intent_label_lst,
     slot_label_lst=slot_labels,
+    trust_remote_code=True,
 )
 model.eval()
 ```
 
-### Inferencia
+### Predicción
 
 ```python
-text = "cual fue el ultimo imacec"
-inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=64)
+text = "cuál fue el imacec de agosto 2024"
+tokens = tokenizer(text, return_tensors="pt", padding=True, truncation=True)
 
 with torch.no_grad():
-    outputs = model(**inputs)
-
-# outputs contiene logits de cada cabeza de intención y slot filling
-calc_mode_logits = outputs[1]   # (batch, num_calc_mode_labels)
-activity_logits = outputs[2]    # (batch, num_activity_labels)
-region_logits = outputs[3]      # (batch, num_region_labels)
-investment_logits = outputs[4]  # (batch, num_investment_labels)
-req_form_logits = outputs[5]    # (batch, num_req_form_labels)
-slot_logits = outputs[6]        # (batch, seq_len, num_slot_labels)
+    outputs = model(**tokens)
+    # outputs contiene intent_logits (lista) y slot_logits
 ```
 
-### Uso en endpoint (snapshot local)
+## Estructura del Paquete
 
-```python
-from huggingface_hub import snapshot_download
+```
+model_package/
+├── config.json              # Configuración BERT + task
+├── model.safetensors        # Pesos del modelo
+├── tokenizer.json           # Tokenizer
+├── tokenizer_config.json
+├── special_tokens_map.json
+├── vocab.txt
+├── modeling_jointbert.py    # Arquitectura JointBERT (custom)
+├── module.py                # CRF y módulos auxiliares
+├── __init__.py
+├── README.md                # Este archivo
+└── labels/
+    ├── slot_label.txt
+    ├── activity_label.txt
+    ├── calc_mode_label.txt
+    ├── investment_label.txt
+    ├── region_label.txt
+    ├── req_form_label.txt
+```
+
+## Datos de Entrenamiento
+
+Entrenado con datos de consultas sobre indicadores macroeconómicos chilenos:
+- **IMACEC** (Indicador Mensual de Actividad Económica)
+- **PIB** (Producto Interno Bruto)
+- Sectores económicos, frecuencias, períodos, regiones
+
+## Limitaciones
 
-# Descargar checkpoint completo
-model_dir = snapshot_download("tu-usuario/pibot-jointbert")
+- Especializado en consultas macroeconómicas del Banco Central de Chile
+- Mejor rendimiento en consultas cortas (< 50 tokens)
+- Requiere `trust_remote_code=True` por la arquitectura custom
 
-# Cargar labels desde carpeta local
-from pathlib import Path
-labels_dir = Path(model_dir) / "labels"
-labels = {}
-for f in labels_dir.glob("*_label.txt"):
-    labels[f.stem] = [l.strip() for l in f.read_text(encoding="utf-8").splitlines() if l.strip()]
+## Cita
+
+```bibtex
+@misc{pibot-jointbert,
+  author = {Banco Central de Chile},
+  title = {PIBot Joint BERT - Multi-head Intent + Slot Filling},
+  year = {2025},
+  publisher = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/smenaaliaga/pibert}}
+}
 ```
 
-## Referencia
+## Referencias
+
+- [BERT for Joint Intent Classification and Slot Filling](https://arxiv.org/abs/1902.10909)
+- [JointBERT implementation](https://github.com/monologg/JointBERT)
+- [BETO: Spanish BERT](https://github.com/dccuchile/beto)
+
+## Licencia
 
-- Paper base: [BERT for Joint Intent Classification and Slot Filling](https://arxiv.org/abs/1810.04805v2)
-- Implementación original: [monologg/JointBERT](https://github.com/monologg/JointBERT)
+MIT License

config.json

@@ -5,7 +5,7 @@
   ],
   "attention_probs_dropout_prob": 0.1,
   "classifier_dropout": null,
-  "finetuning_task": "pibimacecv5",
+  "finetuning_task": "pibimacecv3",
   "gradient_checkpointing": false,
   "hidden_act": "gelu",
   "hidden_dropout_prob": 0.1,

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1d6084a789c8845d2d09b6ff593aff3639c66db677c9c7bd7399fca3eed7b902
-size 439525264
+oid sha256:f55af5f294478cb5085ae95a082a81bdaa637ce671d5cd466629b19db1e7f7fb
+size 439524020

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:91f6bf5e8606b9c4a901c47a0cde8195ea2546e401ff0cef645286e98ffab140
+oid sha256:68a843321a67970998f1578e010bdfed6eff0a16ca74cbfb6cbf8b6394f6e2dc
 size 2040