Instructions to use m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA

SGLang

How to use m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA with Docker Model Runner:
```
docker model run hf.co/m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA
```

ANITA-NEXT-24B-Magistral-2506-ITA / README.md

m-polignano

Update README.md

6c6b684 verified 10 months ago

10.5 kB

license: apache-2.0
language:
  - en
  - it
base_model:
  - mistralai/Magistral-Small-2506
pipeline_tag: text-generation
library_name: transformers
tags:
  - ita
  - italian
  - anita
  - magistral
  - 24b
  - uniba
  - bari
  - italy
  - italia
  - Conversational
  - LLaMantino

"Built on mistral/Magistral-Small-2506"

ANITA-NEXT-24B-Magistral-2506-ITA is a Thinking Model of the ANITA - Large Language Models family. The model is a fine-tuned version of Magistral-Small-2506 (a fine-tuned Mistral model). This model version aims to be the a Multilingual Model 🏁 (EN 🇺🇸 + ITA🇮🇹) to further fine-tuning on Specific Tasks in Italian.

❗❗❗Use at your own risk. We are not responsible for any dangerous/offensive/criminal use. It is release for research only purposes❗❗❗

The 🌟ANITA project🌟 *(Advanced Natural-based interaction for the ITAlian language)* wants to provide Italian NLP researchers with an improved model for the Italian Language 🇮🇹 use cases.

The NEXT family includes four models:

m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA - General Purpose
m-polignano/ANITA-NEXT-24B-Dolphin-Mistral-UNCENSORED-ITA - Uncensored
m-polignano/ANITA-NEXT-24B-Magistral-2506-VISION-ITA - Vision-Language
m-polignano/ANITA-NEXT-20B-GPT-OSS-ITA - Agentic Ready

Colab Demo: A100 - 40GB - Colab Notebook
The Model runs on a single GPU, 19.56GB of VRAM used during inference by using a 4bit Quantization.

Specifications

Model developers:
Ph.D. Marco Polignano - University of Bari Aldo Moro, Italy
SWAP Research Group
Variations: The model release has been supervised fine-tuning (SFT) using QLoRA 4bit, on instruction-based datasets. DPO approach over the mlabonne/orpo-dpo-mix-40k dataset is used to align with human preferences for helpfulness and safety.
Input: Models input text only.
Language: Multilingual 🏁 + Italian 🇮🇹
Output: Models generate text and code only.
Model Architecture: Mistral architecture.
Context length: 128k, but degradate after 40k.
Library Used: [Transformers 4.56.0.dev0] (https://huggingface.co/docs/transformers/index)

Playground

To use the model directly, there are many ways to get started, choose one of the following ways to experience it.

Prompt Template

<s>[SYSTEM_PROMPT]Sei un an assistente AI per la lingua italiana di nome ANITA-NEXT (Advanced Natural-based interaction for the ITAlian language Next Generation) creato dal ricercatore Marco Polignano, Università degli Studi di Bari Aldo Moro, Italia. Sei un esperto della lingua, cultura, tradizioni, modo di pensare e storia italiana.

L'utente ti chiederà di risolvere un compito o rispondere ad una domanda. Rispondi e ragiona usando la lingua della domanda, preferendo l'Italiano.
Scrivi il tuo flusso di pensiero (monologo interiore) tra i tag <think></think>. Ragiona in modo disinvolto, scrivendo riflessioni e/o bozze, come se stessi lavorando a un esercizio su un foglio di carta.
Successivamente, scrivi la soluzione in modo chiaro, corretto, semplice ed esaustivo basandoti sul riassunto del tuo flusso di pensiero.
Se necessario, usa la notazione markdown per formattare la risposta.[/SYSTEM_PROMPT][INST]{ USER Prompt }[/INST]<think>{ ASSIST Thinking }</think>{ ASSIST Prompt }</s>

Transformers

For direct use with transformers, you can easily get started with the following steps.

Firstly, you need to install transformers via the command below with pip.

pip install -U --no-deps bitsandbytes accelerate xformers transformers peft trl cut_cross_entropy unsloth_zoo
pip install sentencepiece protobuf "datasets>=3.4.1,<4.0.0" "huggingface_hub>=0.34.0" hf_transfer

Right now, you can start using the model directly.

  from transformers import AutoModelForCausalLM, AutoTokenizer
  import torch
  from transformers import BitsAndBytesConfig
  
  
  nf4_config = BitsAndBytesConfig(
     load_in_4bit=True,
     bnb_4bit_quant_type="nf4",
     bnb_4bit_use_double_quant=True,
     bnb_4bit_compute_dtype=torch.bfloat16
  )
  
  model_dir = "m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA"
  tokenizer = AutoTokenizer.from_pretrained(model_dir, use_fast=True)
  model = AutoModelForCausalLM.from_pretrained(
      model_dir,
      quantization_config=nf4_config,
      device_map="auto",
      torch_dtype=torch.bfloat16,
  )
  
  #Method 1
  sys = '''Sei un an assistente AI per la lingua italiana di nome ANITA-NEXT (Advanced Natural-based interaction for the ITAlian language Next Generation) creato dal ricercatore Marco Polignano, Università degli Studi di Bari Aldo Moro, Italia. Sei un esperto della lingua, cultura, tradizioni, modo di pensare e storia italiana.
  
  L'utente ti chiederà di risolvere un compito o rispondere ad una domanda. Rispondi e ragiona usando la lingua della domanda, preferendo l'Italiano.
  Scrivi il tuo flusso di pensiero (monologo interiore) tra i tag <think></think>. Ragiona in modo disinvolto, scrivendo riflessioni e/o bozze, come se stessi lavorando a un esercizio su un foglio di carta.
  Successivamente, scrivi la soluzione in modo chiaro, corretto, semplice ed esaustivo basandoti sul riassunto del tuo flusso di pensiero.
  Se necessario, usa la notazione markdown per formattare la risposta.'''
  messages = [
      {"role" : "system", "content" : sys},
      {"role" : "user", "content" : "Chi è Carlo Magno?"}
  ]
  prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
  inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
  for k,v in inputs.items():
      inputs[k] = v.cuda()
  outputs = model.generate(**inputs, max_new_tokens=32786, do_sample=True, top_p=0.9, temperature=0.7)
  results = tokenizer.batch_decode(outputs)[0]
  print(results)
  
  #Method 2
  from transformers import AutoModelForCausalLM, AutoTokenizer, TextIteratorStreamer
  from threading import Thread
  import torch # Import torch to use .cuda() if needed
  sys = '''Sei un an assistente AI per la lingua italiana di nome ANITA-NEXT (Advanced Natural-based interaction for the ITAlian language Next Generation) creato dal ricercatore Marco Polignano, Università degli Studi di Bari Aldo Moro, Italia. Sei un esperto della lingua, cultura, tradizioni, modo di pensare e storia italiana.
  
  L'utente ti chiederà di risolvere un compito o rispondere ad una domanda. Rispondi e ragiona usando la lingua della domanda, preferendo l'Italiano.
  Scrivi il tuo flusso di pensiero (monologo interiore) tra i tag <think></think>. Ragiona in modo disinvolto, scrivendo riflessioni e/o bozze, come se stessi lavorando a un esercizio su un foglio di carta.
  Successivamente, scrivi la soluzione in modo chiaro, corretto, semplice ed esaustivo basandoti sul riassunto del tuo flusso di pensiero.
  Se necessario, usa la notazione markdown per formattare la risposta.'''
  messages = [
      #{"role" : "system", "content" : sys},
      {"role" : "user", "content" : "Chi è Marco Polo?"}
  ]
  prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
  inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
  
  # Move inputs to CUDA if your model is on CUDA
  for k,v in inputs.items():
      inputs[k] = v.cuda()
  
  # --- 4. Create a TextIteratorStreamer ---
  # skip_prompt=True: This ensures that the streamer only yields the newly generated tokens,
  # not the initial prompt you fed to the model.
  # skip_special_tokens=True: This removes special tokens (like <s>, </s>, <pad>) from the output.
  streamer = TextIteratorStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
  
  # --- 5. Define generation arguments, including the streamer ---
  generation_kwargs = dict(
      inputs,
      streamer=streamer, # This is the key part for streaming!
      max_new_tokens=32786,
      do_sample=True,
      top_p=0.9,
      temperature=0.7,
      # Add any other generation arguments you need
  )
  
  # --- 6. Run model.generate in a separate thread ---
  # This is crucial because model.generate is a blocking call.
  # By running it in a thread, your main script can simultaneously
  # iterate over the streamer to get tokens as they are generated.
  thread = Thread(target=model.generate, kwargs=generation_kwargs)
  thread.start()
  
  # --- 7. Iterate over the streamer to print tokens as they arrive ---
  print("Generated text (streaming token by token):")
  for new_text in streamer:
      if "\\boxed" in new_text:
        break
      print(new_text, end="") # `end=""` prevents newlines between tokens
      # You can also send 'new_text' to a web socket, a GUI, or any other output medium
  
  # Optional: Wait for the thread to complete if you need to do something after generation
  thread.join()

Citation instructions

@misc{polignano2024advanced,
      title={Advanced Natural-based interaction for the ITAlian language: LLaMAntino-3-ANITA}, 
      author={Marco Polignano and Pierpaolo Basile and Giovanni Semeraro},
      year={2024},
      eprint={2405.07101},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

@article{rastogi2025magistral,
  title={Magistral},
  author={Rastogi, Abhinav and Jiang, Albert Q and Lo, Andy and Berrada, Gabrielle and Lample, Guillaume and Rute, Jason and Barmentlo, Joep and Yadav, Karmesh and Khandelwal, Kartik and Chandu, Khyathi Raghavi and others},
  journal={arXiv preprint arXiv:2506.10910},
  year={2025}
}