Instructions to use jon-fernandes/noteworthy with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jon-fernandes/noteworthy with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="jon-fernandes/noteworthy")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("jon-fernandes/noteworthy")
model = AutoModelForMultimodalLM.from_pretrained("jon-fernandes/noteworthy")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jon-fernandes/noteworthy with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jon-fernandes/noteworthy"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jon-fernandes/noteworthy",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/jon-fernandes/noteworthy

SGLang

How to use jon-fernandes/noteworthy with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jon-fernandes/noteworthy" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jon-fernandes/noteworthy",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jon-fernandes/noteworthy" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jon-fernandes/noteworthy",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use jon-fernandes/noteworthy with Docker Model Runner:
```
docker model run hf.co/jon-fernandes/noteworthy
```

minicpmv46_sheetmusic_full

This model is a fine-tuned version of openbmb/MiniCPM-V-4.6 on the sheetmusic_train dataset. It achieves the following results on the evaluation set:

Loss: 0.0018

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 1
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 16
total_train_batch_size: 64
total_eval_batch_size: 4
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 0.05
num_epochs: 4.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.4366	0.1664	100	0.4435
0.0412	0.3327	200	0.0465
0.0205	0.4991	300	0.0163
0.0080	0.6655	400	0.0084
0.0080	0.8319	500	0.0065
0.0072	0.9982	600	0.0072
0.0094	1.1630	700	0.0046
0.0038	1.3294	800	0.0048
0.0070	1.4958	900	0.0042
0.0013	1.6622	1000	0.0032
0.0020	1.8285	1100	0.0032
0.0011	1.9949	1200	0.0028
0.0030	2.1597	1300	0.0023
0.0018	2.3261	1400	0.0024
0.0007	2.4925	1500	0.0022
0.0048	2.6588	1600	0.0021
0.0017	2.8252	1700	0.0020
0.0019	2.9916	1800	0.0019
0.0006	3.1564	1900	0.0019
0.0004	3.3228	2000	0.0018
0.0007	3.4891	2100	0.0018
0.0004	3.6555	2200	0.0018
0.0009	3.8219	2300	0.0018
0.0005	3.9882	2400	0.0018
0.0001	4.0	2408	0.0018

Framework versions

Transformers 5.7.0
Pytorch 2.8.0+cu128
Datasets 4.0.0
Tokenizers 0.22.2

Downloads last month: 239

Safetensors

Model size

1B params

Tensor type

BF16

Model tree for jon-fernandes/noteworthy

Base model

openbmb/MiniCPM-V-4.6

Finetuned

(15)

this model

jon-fernandes
/

noteworthy

minicpmv46_sheetmusic_full

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Model tree for jon-fernandes/noteworthy

Spaces using jon-fernandes/noteworthy 4