Instructions to use xaviviro/FLAMA-0.5-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use xaviviro/FLAMA-0.5-3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="xaviviro/FLAMA-0.5-3B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("xaviviro/FLAMA-0.5-3B")
model = AutoModelForCausalLM.from_pretrained("xaviviro/FLAMA-0.5-3B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use xaviviro/FLAMA-0.5-3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "xaviviro/FLAMA-0.5-3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xaviviro/FLAMA-0.5-3B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/xaviviro/FLAMA-0.5-3B

SGLang

How to use xaviviro/FLAMA-0.5-3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "xaviviro/FLAMA-0.5-3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xaviviro/FLAMA-0.5-3B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "xaviviro/FLAMA-0.5-3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xaviviro/FLAMA-0.5-3B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use xaviviro/FLAMA-0.5-3B with Docker Model Runner:
```
docker model run hf.co/xaviviro/FLAMA-0.5-3B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

FLAMA: Model 3B ChatML en Català i Castellà. Versió 0.5

👉🏻 Format GGUF i quantitzat

FLAMA és el primer model petit 3B bilingüe en català i castellà. És el resultat de finetunejar el model open_llama_3b_v2 amb les instruccions d'OpenAssistant v2 traduïdes automàticament al català i al castellà amb recursos de Helsinki-NLP i tractades en format ChatML.

Novetats de la versió 0.5

Català millorat
Afegit el Castellà

Prompt Template

FLAMA usa ChatML com a prompt template:

<|im_start|>user
Qui va ser Isaac Newton?<|im_end|>
<|im_start|>assistant\n

<|im_start|>user
Quien fué Isaac Newton?<|im_end|>
<|im_start|>assistant\n

Referències

@software{xaviviro2023flama,
  author = {xaviviro},
  title = {FLAMA: Model 3B ChatML en Català. Versió 0.5},
  month = January,
  year = 2024,
  url = {https://huggingface.co/xaviviro/FLAMA-0.5-3B}
}

@software{openlm2023openllama,
  author = {Geng, Xinyang and Liu, Hao},
  title = {OpenLLaMA: An Open Reproduction of LLaMA},
  month = May,
  year = 2023,
  url = {https://github.com/openlm-research/open_llama}
}

@software{together2023redpajama,
  author = {Together Computer},
  title = {RedPajama-Data: An Open Source Recipe to Reproduce LLaMA training dataset},
  month = April,
  year = 2023,
  url = {https://github.com/togethercomputer/RedPajama-Data}
}

@article{touvron2023llama,
  title={Llama: Open and efficient foundation language models},
  author={Touvron, Hugo and Lavril, Thibaut and Izacard, Gautier and Martinet, Xavier and Lachaux, Marie-Anne and Lacroix, Timoth{\'e}e and Rozi{\`e}re, Baptiste and Goyal, Naman and Hambro, Eric and Azhar, Faisal and others},
  journal={arXiv preprint arXiv:2302.13971},
  year={2023}
}

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	39.23
AI2 Reasoning Challenge (25-Shot)	37.97
HellaSwag (10-Shot)	67.65
MMLU (5-Shot)	25.73
TruthfulQA (0-shot)	41.11
Winogrande (5-shot)	62.12
GSM8k (5-shot)	0.83

Downloads last month: 9

Model tree for xaviviro/FLAMA-0.5-3B

Base model

openlm-research/open_llama_3b_v2

Finetuned

(15)

this model

Quantizations

3 models

Datasets used to train xaviviro/FLAMA-0.5-3B

Paper for xaviviro/FLAMA-0.5-3B

LLaMA: Open and Efficient Foundation Language Models

Paper • 2302.13971 • Published Feb 27, 2023 • 25