Instructions to use nvidia/NVIDIA-Nemotron-Parse-v1.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nvidia/NVIDIA-Nemotron-Parse-v1.2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="nvidia/NVIDIA-Nemotron-Parse-v1.2", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("nvidia/NVIDIA-Nemotron-Parse-v1.2", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nvidia/NVIDIA-Nemotron-Parse-v1.2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nvidia/NVIDIA-Nemotron-Parse-v1.2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-Parse-v1.2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/nvidia/NVIDIA-Nemotron-Parse-v1.2

SGLang

How to use nvidia/NVIDIA-Nemotron-Parse-v1.2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nvidia/NVIDIA-Nemotron-Parse-v1.2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-Parse-v1.2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nvidia/NVIDIA-Nemotron-Parse-v1.2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-Parse-v1.2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use nvidia/NVIDIA-Nemotron-Parse-v1.2 with Docker Model Runner:
```
docker model run hf.co/nvidia/NVIDIA-Nemotron-Parse-v1.2
```

NVIDIA-Nemotron-Parse-v1.2 / golden_outputs.json

emelryan

Update with support for transformers 5.x compatibility (#4)

8b65d28 about 2 months ago

raw

history blame

2.08 kB

	{
	"metadata": {
	"transformers_version": "4.51.3",
	"torch_version": "2.11.0a0+a6c236b9fd.nv26.03.46836102",
	"device": "cuda:0",
	"dtype": "torch.bfloat16",
	"model_path": "/workspace"
	},
	"image_processing": {
	"shape": [
	1,
	3,
	2048,
	1664
	],
	"mean": 0.06537187099456787,
	"std": 0.2386379987001419,
	"first_20_values": [
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0,
	0.0
	]
	},
	"encoder_output": {
	"shape": [
	1,
	3329,
	1024
	],
	"mean": -0.0001354874111711979,
	"std": 0.9438313841819763,
	"token0_first16": [
	-0.1884765625,
	0.81640625,
	-0.37890625,
	-0.46875,
	0.5078125,
	-0.89453125,
	0.71484375,
	-0.11865234375,
	-0.484375,
	-0.169921875,
	0.173828125,
	0.6640625,
	1.5625,
	-0.283203125,
	-0.9921875,
	0.10009765625
	]
	},
	"forward_pass": {
	"logits_shape": [
	1,
	1,
	52352
	],
	"top_k_indices": [
	0,
	38394,
	159,
	1972,
	31501,
	5007,
	25230,
	49726,
	12301,
	40617
	],
	"top_k_values": [
	56.0,
	38.0,
	38.0,
	37.75,
	37.75,
	37.75,
	37.75,
	37.75,
	37.5,
	37.5
	]
	},
	"generation": {
	"max_new_tokens": 50,
	"token_ids": [
	2,
	0,
	50004,
	50008,
	50001,
	50010,
	50412,
	51799,
	82,
	2722,
	113,
	18121,
	579,
	115,
	113,
	19321,
	89,
	115,
	221,
	82,
	493,
	113,
	18121,
	579,
	115,
	50633,
	51850,
	52327,
	2
	],
	"decoded_text": "</s><s><predict_bbox><predict_classes><output_markdown><predict_no_text_in_pic><x_0.3916><y_0.5969>\\begin{tabular}{ccccc}\n\\end{tabular}<x_0.6074><y_0.6367><class_Table></s>"
	}
	}