[INFO|2025-02-12 17:41:57] configuration_utils.py:695 >> loading configuration file config.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/config.json

[INFO|2025-02-12 17:41:57] configuration_utils.py:762 >> Model config LlavaNextVideoConfig {
  "_name_or_path": "llava-hf/LLaVA-NeXT-Video-7B-hf",
  "architectures": [
    "LlavaNextVideoForConditionalGeneration"
  ],
  "ignore_index": -100,
  "image_grid_pinpoints": [
    [
      336,
      672
    ],
    [
      672,
      336
    ],
    [
      672,
      672
    ],
    [
      1008,
      336
    ],
    [
      336,
      1008
    ]
  ],
  "image_seq_length": 576,
  "image_token_index": 32001,
  "model_type": "llava_next_video",
  "projector_hidden_act": "gelu",
  "spatial_pool_mode": "average",
  "spatial_pool_out_channels": 1024,
  "spatial_pool_stride": 2,
  "text_config": {
    "_name_or_path": "lmsys/vicuna-7b-v1.5",
    "architectures": [
      "LlamaForCausalLM"
    ],
    "max_position_embeddings": 4096,
    "model_type": "llama",
    "pad_token_id": 0,
    "rms_norm_eps": 1e-05,
    "rope_scaling": {
      "factor": 2.5,
      "rope_type": "linear",
      "type": "linear"
    },
    "torch_dtype": "float16",
    "type": "linear",
    "vocab_size": 32064
  },
  "tie_word_embeddings": false,
  "torch_dtype": "bfloat16",
  "transformers_version": "4.47.1",
  "use_image_newline_parameter": true,
  "video_seq_length": 288,
  "video_token_index": 32000,
  "vision_config": {
    "hidden_size": 1024,
    "image_size": 336,
    "intermediate_size": 4096,
    "model_type": "clip_vision_model",
    "num_attention_heads": 16,
    "num_hidden_layers": 24,
    "patch_size": 14,
    "projection_dim": 768,
    "vocab_size": 32000
  },
  "vision_feature_layer": -2,
  "vision_feature_select_strategy": "default"
}


[INFO|2025-02-12 17:41:57] tokenization_utils_base.py:2030 >> loading file tokenizer.model from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/tokenizer.model

[INFO|2025-02-12 17:41:57] tokenization_utils_base.py:2030 >> loading file tokenizer.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/tokenizer.json

[INFO|2025-02-12 17:41:57] tokenization_utils_base.py:2030 >> loading file added_tokens.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/added_tokens.json

[INFO|2025-02-12 17:41:57] tokenization_utils_base.py:2030 >> loading file special_tokens_map.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/special_tokens_map.json

[INFO|2025-02-12 17:41:57] tokenization_utils_base.py:2030 >> loading file tokenizer_config.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/tokenizer_config.json

[INFO|2025-02-12 17:41:57] tokenization_utils_base.py:2030 >> loading file chat_template.jinja from cache at None

[INFO|2025-02-12 17:41:57] tokenization_utils_base.py:2300 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

[INFO|2025-02-12 17:41:59] processing_utils.py:720 >> loading configuration file processor_config.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/processor_config.json

[INFO|2025-02-12 17:42:00] image_processing_base.py:378 >> loading configuration file preprocessor_config.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/preprocessor_config.json

[INFO|2025-02-12 17:42:00] image_processing_base.py:432 >> Image processor LlavaNextVideoImageProcessor {
  "crop_size": {
    "height": 336,
    "width": 336
  },
  "do_center_crop": true,
  "do_convert_rgb": true,
  "do_normalize": true,
  "do_pad": true,
  "do_rescale": true,
  "do_resize": true,
  "image_grid_pinpoints": [
    [
      336,
      672
    ],
    [
      672,
      336
    ],
    [
      672,
      672
    ],
    [
      1008,
      336
    ],
    [
      336,
      1008
    ]
  ],
  "image_mean": [
    0.48145466,
    0.4578275,
    0.40821073
  ],
  "image_processor_type": "LlavaNextVideoImageProcessor",
  "image_std": [
    0.26862954,
    0.26130258,
    0.27577711
  ],
  "processor_class": "LlavaNextVideoProcessor",
  "resample": 3,
  "rescale_factor": 0.00392156862745098,
  "size": {
    "shortest_edge": 336
  }
}


[INFO|2025-02-12 17:42:00] image_processing_base.py:378 >> loading configuration file preprocessor_config.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/preprocessor_config.json

[INFO|2025-02-12 17:42:00] image_processing_base.py:432 >> Image processor LlavaNextImageProcessor {
  "crop_size": {
    "height": 336,
    "width": 336
  },
  "do_center_crop": true,
  "do_convert_rgb": true,
  "do_normalize": true,
  "do_pad": true,
  "do_rescale": true,
  "do_resize": true,
  "image_grid_pinpoints": [
    [
      336,
      672
    ],
    [
      672,
      336
    ],
    [
      672,
      672
    ],
    [
      1008,
      336
    ],
    [
      336,
      1008
    ]
  ],
  "image_mean": [
    0.48145466,
    0.4578275,
    0.40821073
  ],
  "image_processor_type": "LlavaNextImageProcessor",
  "image_std": [
    0.26862954,
    0.26130258,
    0.27577711
  ],
  "processor_class": "LlavaNextVideoProcessor",
  "resample": 3,
  "rescale_factor": 0.00392156862745098,
  "size": {
    "shortest_edge": 336
  }
}


[INFO|2025-02-12 17:42:00] tokenization_utils_base.py:2030 >> loading file tokenizer.model from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/tokenizer.model

[INFO|2025-02-12 17:42:00] tokenization_utils_base.py:2030 >> loading file tokenizer.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/tokenizer.json

[INFO|2025-02-12 17:42:00] tokenization_utils_base.py:2030 >> loading file added_tokens.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/added_tokens.json

[INFO|2025-02-12 17:42:00] tokenization_utils_base.py:2030 >> loading file special_tokens_map.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/special_tokens_map.json

[INFO|2025-02-12 17:42:00] tokenization_utils_base.py:2030 >> loading file tokenizer_config.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/tokenizer_config.json

[INFO|2025-02-12 17:42:00] tokenization_utils_base.py:2030 >> loading file chat_template.jinja from cache at None

[INFO|2025-02-12 17:42:01] tokenization_utils_base.py:2300 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

[INFO|2025-02-12 17:42:01] processing_utils.py:720 >> loading configuration file processor_config.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/processor_config.json

[INFO|2025-02-12 17:42:02] processing_utils.py:780 >> Processor LlavaNextVideoProcessor:
- video_processor: LlavaNextVideoImageProcessor {
  "crop_size": {
    "height": 336,
    "width": 336
  },
  "do_center_crop": true,
  "do_convert_rgb": true,
  "do_normalize": true,
  "do_pad": true,
  "do_rescale": true,
  "do_resize": true,
  "image_grid_pinpoints": [
    [
      336,
      672
    ],
    [
      672,
      336
    ],
    [
      672,
      672
    ],
    [
      1008,
      336
    ],
    [
      336,
      1008
    ]
  ],
  "image_mean": [
    0.48145466,
    0.4578275,
    0.40821073
  ],
  "image_processor_type": "LlavaNextVideoImageProcessor",
  "image_std": [
    0.26862954,
    0.26130258,
    0.27577711
  ],
  "processor_class": "LlavaNextVideoProcessor",
  "resample": 3,
  "rescale_factor": 0.00392156862745098,
  "size": {
    "shortest_edge": 336
  }
}

- image_processor: LlavaNextImageProcessor {
  "crop_size": {
    "height": 336,
    "width": 336
  },
  "do_center_crop": true,
  "do_convert_rgb": true,
  "do_normalize": true,
  "do_pad": true,
  "do_rescale": true,
  "do_resize": true,
  "image_grid_pinpoints": [
    [
      336,
      672
    ],
    [
      672,
      336
    ],
    [
      672,
      672
    ],
    [
      1008,
      336
    ],
    [
      336,
      1008
    ]
  ],
  "image_mean": [
    0.48145466,
    0.4578275,
    0.40821073
  ],
  "image_processor_type": "LlavaNextImageProcessor",
  "image_std": [
    0.26862954,
    0.26130258,
    0.27577711
  ],
  "processor_class": "LlavaNextVideoProcessor",
  "resample": 3,
  "rescale_factor": 0.00392156862745098,
  "size": {
    "shortest_edge": 336
  }
}

- tokenizer: LlamaTokenizerFast(name_or_path='llava-hf/LLaVA-NeXT-Video-7B-hf', vocab_size=32000, model_max_length=4096, is_fast=True, padding_side='left', truncation_side='right', special_tokens={'bos_token': '<s>', 'eos_token': '</s>', 'unk_token': '<unk>', 'pad_token': '<unk>', 'image_token': '<image>', 'video_token': '<video>'}, clean_up_tokenization_spaces=False, added_tokens_decoder={
	0: AddedToken("<unk>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	1: AddedToken("<s>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	2: AddedToken("</s>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	32000: AddedToken("<video>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
	32001: AddedToken("<image>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
}
)

{
  "image_token": "<image>",
  "num_additional_image_tokens": 1,
  "patch_size": 14,
  "processor_class": "LlavaNextVideoProcessor",
  "video_token": "<video>",
  "vision_feature_select_strategy": "default"
}


[INFO|2025-02-12 17:42:02] logging.py:157 >> Loading dataset qwen2_vl_dora.json...

[INFO|2025-02-12 17:42:14] configuration_utils.py:695 >> loading configuration file config.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/config.json

[INFO|2025-02-12 17:42:14] configuration_utils.py:762 >> Model config LlavaNextVideoConfig {
  "_name_or_path": "llava-hf/LLaVA-NeXT-Video-7B-hf",
  "architectures": [
    "LlavaNextVideoForConditionalGeneration"
  ],
  "ignore_index": -100,
  "image_grid_pinpoints": [
    [
      336,
      672
    ],
    [
      672,
      336
    ],
    [
      672,
      672
    ],
    [
      1008,
      336
    ],
    [
      336,
      1008
    ]
  ],
  "image_seq_length": 576,
  "image_token_index": 32001,
  "model_type": "llava_next_video",
  "projector_hidden_act": "gelu",
  "spatial_pool_mode": "average",
  "spatial_pool_out_channels": 1024,
  "spatial_pool_stride": 2,
  "text_config": {
    "_name_or_path": "lmsys/vicuna-7b-v1.5",
    "architectures": [
      "LlamaForCausalLM"
    ],
    "max_position_embeddings": 4096,
    "model_type": "llama",
    "pad_token_id": 0,
    "rms_norm_eps": 1e-05,
    "rope_scaling": {
      "factor": 2.5,
      "rope_type": "linear",
      "type": "linear"
    },
    "torch_dtype": "float16",
    "type": "linear",
    "vocab_size": 32064
  },
  "tie_word_embeddings": false,
  "torch_dtype": "bfloat16",
  "transformers_version": "4.47.1",
  "use_image_newline_parameter": true,
  "video_seq_length": 288,
  "video_token_index": 32000,
  "vision_config": {
    "hidden_size": 1024,
    "image_size": 336,
    "intermediate_size": 4096,
    "model_type": "clip_vision_model",
    "num_attention_heads": 16,
    "num_hidden_layers": 24,
    "patch_size": 14,
    "projection_dim": 768,
    "vocab_size": 32000
  },
  "vision_feature_layer": -2,
  "vision_feature_select_strategy": "default"
}


[INFO|2025-02-12 17:42:14] modeling_utils.py:3953 >> loading weights file model.safetensors from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/model.safetensors.index.json

[INFO|2025-02-12 17:42:14] modeling_utils.py:1641 >> Instantiating LlavaNextVideoForConditionalGeneration model under default dtype torch.float16.

[INFO|2025-02-12 17:42:14] configuration_utils.py:1140 >> Generate config GenerationConfig {}


[INFO|2025-02-12 17:42:14] modeling_utils.py:1641 >> Instantiating CLIPVisionModel model under default dtype torch.float16.

[INFO|2025-02-12 17:42:14] modeling_utils.py:1641 >> Instantiating LlamaForCausalLM model under default dtype torch.float16.

[INFO|2025-02-12 17:42:14] configuration_utils.py:1140 >> Generate config GenerationConfig {
  "bos_token_id": 1,
  "eos_token_id": 2,
  "pad_token_id": 0
}


[INFO|2025-02-12 17:42:20] modeling_utils.py:4849 >> All model checkpoint weights were used when initializing LlavaNextVideoForConditionalGeneration.


[INFO|2025-02-12 17:42:20] modeling_utils.py:4857 >> All the weights of LlavaNextVideoForConditionalGeneration were initialized from the model checkpoint at llava-hf/LLaVA-NeXT-Video-7B-hf.
If your task is similar to the task the model of the checkpoint was trained on, you can already use LlavaNextVideoForConditionalGeneration for predictions without further training.

[INFO|2025-02-12 17:42:20] configuration_utils.py:1095 >> loading configuration file generation_config.json from cache at /home/intern01/.cache/huggingface/hub/models--llava-hf--LLaVA-NeXT-Video-7B-hf/snapshots/df44d4db0beb3e5b8b85173e45cc47f0ef0f7754/generation_config.json

[INFO|2025-02-12 17:42:20] configuration_utils.py:1140 >> Generate config GenerationConfig {
  "bos_token_id": 1,
  "eos_token_id": 2,
  "pad_token_id": 0
}


[INFO|2025-02-12 17:42:20] logging.py:157 >> Gradient checkpointing enabled.

[INFO|2025-02-12 17:42:20] logging.py:157 >> Using torch SDPA for faster training and inference.

[INFO|2025-02-12 17:42:20] logging.py:157 >> Upcasting trainable params to float32.

[INFO|2025-02-12 17:42:20] logging.py:157 >> Fine-tuning method: LoRA

[INFO|2025-02-12 17:42:20] logging.py:157 >> Found linear modules: q_proj,k_proj,o_proj,gate_proj,v_proj,down_proj,up_proj

[INFO|2025-02-12 17:42:20] logging.py:157 >> Set vision model not trainable: ['vision_tower'].

[INFO|2025-02-12 17:42:21] logging.py:157 >> trainable params: 19,988,480 || all params: 7,083,419,648 || trainable%: 0.2822

[INFO|2025-02-12 17:42:21] trainer.py:734 >> Using auto half precision backend

[INFO|2025-02-12 17:42:21] trainer.py:2362 >> ***** Running training *****

[INFO|2025-02-12 17:42:21] trainer.py:2363 >>   Num examples = 452

[INFO|2025-02-12 17:42:21] trainer.py:2364 >>   Num Epochs = 5

[INFO|2025-02-12 17:42:21] trainer.py:2365 >>   Instantaneous batch size per device = 2

[INFO|2025-02-12 17:42:21] trainer.py:2368 >>   Total train batch size (w. parallel, distributed & accumulation) = 16

[INFO|2025-02-12 17:42:21] trainer.py:2369 >>   Gradient Accumulation steps = 8

[INFO|2025-02-12 17:42:21] trainer.py:2370 >>   Total optimization steps = 140

[INFO|2025-02-12 17:42:21] trainer.py:2371 >>   Number of trainable parameters = 19,988,480

[WARNING|2025-02-12 17:42:25] logging.py:328 >> `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.

[INFO|2025-02-12 17:47:02] logging.py:157 >> {'loss': 4.7686, 'learning_rate': 4.9899e-05, 'epoch': 0.18, 'throughput': 851.14}

[INFO|2025-02-12 17:51:43] logging.py:157 >> {'loss': 3.5672, 'learning_rate': 4.9492e-05, 'epoch': 0.35, 'throughput': 851.99}

[INFO|2025-02-12 17:56:23] logging.py:157 >> {'loss': 3.3447, 'learning_rate': 4.8776e-05, 'epoch': 0.53, 'throughput': 852.22}

[INFO|2025-02-12 18:01:04] logging.py:157 >> {'loss': 2.9905, 'learning_rate': 4.7762e-05, 'epoch': 0.71, 'throughput': 852.15}

[INFO|2025-02-12 18:05:46] logging.py:157 >> {'loss': 2.4303, 'learning_rate': 4.6461e-05, 'epoch': 0.88, 'throughput': 852.03}

[INFO|2025-02-12 18:09:44] logging.py:157 >> {'loss': 1.6583, 'learning_rate': 4.4891e-05, 'epoch': 1.04, 'throughput': 852.05}

[INFO|2025-02-12 18:09:44] trainer.py:4203 >> 
***** Running Evaluation *****

[INFO|2025-02-12 18:09:44] trainer.py:4205 >>   Num examples = 113

[INFO|2025-02-12 18:09:44] trainer.py:4208 >>   Batch size = 2

[INFO|2025-02-12 18:11:37] trainer.py:3887 >> Saving model checkpoint to saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-30

[INFO|2025-02-12 18:11:37] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-30/tokenizer_config.json

[INFO|2025-02-12 18:11:37] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-30/special_tokens_map.json

[INFO|2025-02-12 18:11:38] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-30/preprocessor_config.json

[INFO|2025-02-12 18:11:38] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-30/preprocessor_config.json

[INFO|2025-02-12 18:11:38] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-30/tokenizer_config.json

[INFO|2025-02-12 18:11:38] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-30/special_tokens_map.json

[INFO|2025-02-12 18:11:38] processing_utils.py:546 >> chat template saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-30/chat_template.json

[INFO|2025-02-12 18:11:38] processing_utils.py:552 >> processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-30/processor_config.json

[INFO|2025-02-12 18:16:18] logging.py:157 >> {'loss': 1.4066, 'learning_rate': 4.3070e-05, 'epoch': 1.21, 'throughput': 804.78}

[INFO|2025-02-12 18:20:59] logging.py:157 >> {'loss': 2.0231, 'learning_rate': 4.1448e-05, 'epoch': 1.39, 'throughput': 810.64}

[INFO|2025-02-12 18:25:39] logging.py:157 >> {'loss': 1.3326, 'learning_rate': 3.9237e-05, 'epoch': 1.57, 'throughput': 815.30}

[INFO|2025-02-12 18:30:19] logging.py:157 >> {'loss': 1.3365, 'learning_rate': 3.7338e-05, 'epoch': 1.74, 'throughput': 819.05}

[INFO|2025-02-12 18:34:59] logging.py:157 >> {'loss': 1.0757, 'learning_rate': 3.4826e-05, 'epoch': 1.92, 'throughput': 822.22}

[INFO|2025-02-12 18:38:57] logging.py:157 >> {'loss': 0.6975, 'learning_rate': 3.2190e-05, 'epoch': 2.07, 'throughput': 824.53}

[INFO|2025-02-12 18:38:57] trainer.py:4203 >> 
***** Running Evaluation *****

[INFO|2025-02-12 18:38:57] trainer.py:4205 >>   Num examples = 113

[INFO|2025-02-12 18:38:57] trainer.py:4208 >>   Batch size = 2

[INFO|2025-02-12 18:40:50] trainer.py:3887 >> Saving model checkpoint to saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-60

[INFO|2025-02-12 18:40:50] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-60/tokenizer_config.json

[INFO|2025-02-12 18:40:50] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-60/special_tokens_map.json

[INFO|2025-02-12 18:40:51] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-60/preprocessor_config.json

[INFO|2025-02-12 18:40:51] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-60/preprocessor_config.json

[INFO|2025-02-12 18:40:51] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-60/tokenizer_config.json

[INFO|2025-02-12 18:40:51] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-60/special_tokens_map.json

[INFO|2025-02-12 18:40:51] processing_utils.py:546 >> chat template saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-60/chat_template.json

[INFO|2025-02-12 18:40:51] processing_utils.py:552 >> processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-60/processor_config.json

[INFO|2025-02-12 18:45:31] logging.py:157 >> {'loss': 0.8230, 'learning_rate': 2.9464e-05, 'epoch': 2.25, 'throughput': 802.11}

[INFO|2025-02-12 18:50:11] logging.py:157 >> {'loss': 1.2483, 'learning_rate': 2.6682e-05, 'epoch': 2.42, 'throughput': 805.76}

[INFO|2025-02-12 18:54:51] logging.py:157 >> {'loss': 1.2269, 'learning_rate': 2.3878e-05, 'epoch': 2.60, 'throughput': 808.91}

[INFO|2025-02-12 18:59:31] logging.py:157 >> {'loss': 0.8036, 'learning_rate': 2.1089e-05, 'epoch': 2.78, 'throughput': 811.69}

[INFO|2025-02-12 19:04:11] logging.py:157 >> {'loss': 0.9940, 'learning_rate': 1.8349e-05, 'epoch': 2.96, 'throughput': 814.15}

[INFO|2025-02-12 19:08:09] logging.py:157 >> {'loss': 0.7200, 'learning_rate': 1.5693e-05, 'epoch': 3.11, 'throughput': 816.02}

[INFO|2025-02-12 19:08:09] trainer.py:4203 >> 
***** Running Evaluation *****

[INFO|2025-02-12 19:08:09] trainer.py:4205 >>   Num examples = 113

[INFO|2025-02-12 19:08:09] trainer.py:4208 >>   Batch size = 2

[INFO|2025-02-12 19:10:01] trainer.py:3887 >> Saving model checkpoint to saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-90

[INFO|2025-02-12 19:10:01] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-90/tokenizer_config.json

[INFO|2025-02-12 19:10:01] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-90/special_tokens_map.json

[INFO|2025-02-12 19:10:02] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-90/preprocessor_config.json

[INFO|2025-02-12 19:10:02] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-90/preprocessor_config.json

[INFO|2025-02-12 19:10:02] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-90/tokenizer_config.json

[INFO|2025-02-12 19:10:02] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-90/special_tokens_map.json

[INFO|2025-02-12 19:10:02] processing_utils.py:546 >> chat template saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-90/chat_template.json

[INFO|2025-02-12 19:10:02] processing_utils.py:552 >> processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-90/processor_config.json

[INFO|2025-02-12 19:14:42] logging.py:157 >> {'loss': 0.4563, 'learning_rate': 1.3153e-05, 'epoch': 3.28, 'throughput': 801.37}

[INFO|2025-02-12 19:19:21] logging.py:157 >> {'loss': 0.7728, 'learning_rate': 1.0763e-05, 'epoch': 3.46, 'throughput': 803.97}

[INFO|2025-02-12 19:24:01] logging.py:157 >> {'loss': 0.7990, 'learning_rate': 8.5515e-06, 'epoch': 3.64, 'throughput': 806.30}

[INFO|2025-02-12 19:28:41] logging.py:157 >> {'loss': 0.5341, 'learning_rate': 6.5470e-06, 'epoch': 3.81, 'throughput': 808.45}

[INFO|2025-02-12 19:33:21] logging.py:157 >> {'loss': 0.7853, 'learning_rate': 4.7746e-06, 'epoch': 3.99, 'throughput': 810.43}

[INFO|2025-02-12 19:37:19] logging.py:157 >> {'loss': 0.1667, 'learning_rate': 3.2565e-06, 'epoch': 4.14, 'throughput': 812.01}

[INFO|2025-02-12 19:37:19] trainer.py:4203 >> 
***** Running Evaluation *****

[INFO|2025-02-12 19:37:19] trainer.py:4205 >>   Num examples = 113

[INFO|2025-02-12 19:37:19] trainer.py:4208 >>   Batch size = 2

[INFO|2025-02-12 19:39:11] trainer.py:3887 >> Saving model checkpoint to saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-120

[INFO|2025-02-12 19:39:11] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-120/tokenizer_config.json

[INFO|2025-02-12 19:39:11] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-120/special_tokens_map.json

[INFO|2025-02-12 19:39:12] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-120/preprocessor_config.json

[INFO|2025-02-12 19:39:12] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-120/preprocessor_config.json

[INFO|2025-02-12 19:39:12] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-120/tokenizer_config.json

[INFO|2025-02-12 19:39:12] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-120/special_tokens_map.json

[INFO|2025-02-12 19:39:12] processing_utils.py:546 >> chat template saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-120/chat_template.json

[INFO|2025-02-12 19:39:12] processing_utils.py:552 >> processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-120/processor_config.json

[INFO|2025-02-12 19:43:51] logging.py:157 >> {'loss': 0.8620, 'learning_rate': 2.0118e-06, 'epoch': 4.32, 'throughput': 801.12}

[INFO|2025-02-12 19:48:31] logging.py:157 >> {'loss': 0.6083, 'learning_rate': 1.0562e-06, 'epoch': 4.50, 'throughput': 803.14}

[INFO|2025-02-12 19:53:10] logging.py:157 >> {'loss': 0.3353, 'learning_rate': 4.0176e-07, 'epoch': 4.67, 'throughput': 805.03}

[INFO|2025-02-12 19:57:50] logging.py:157 >> {'loss': 0.6499, 'learning_rate': 5.6628e-08, 'epoch': 4.85, 'throughput': 806.80}

[INFO|2025-02-12 19:57:50] trainer.py:3887 >> Saving model checkpoint to saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-140

[INFO|2025-02-12 19:57:50] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-140/tokenizer_config.json

[INFO|2025-02-12 19:57:50] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-140/special_tokens_map.json

[INFO|2025-02-12 19:57:51] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-140/preprocessor_config.json

[INFO|2025-02-12 19:57:51] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-140/preprocessor_config.json

[INFO|2025-02-12 19:57:51] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-140/tokenizer_config.json

[INFO|2025-02-12 19:57:51] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-140/special_tokens_map.json

[INFO|2025-02-12 19:57:51] processing_utils.py:546 >> chat template saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-140/chat_template.json

[INFO|2025-02-12 19:57:51] processing_utils.py:552 >> processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/checkpoint-140/processor_config.json

[INFO|2025-02-12 19:57:51] trainer.py:2636 >> 

Training completed. Do not forget to share your model on huggingface.co/models =)



[INFO|2025-02-12 19:57:51] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/preprocessor_config.json

[INFO|2025-02-12 19:57:51] image_processing_base.py:261 >> Image processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/preprocessor_config.json

[INFO|2025-02-12 19:57:51] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/tokenizer_config.json

[INFO|2025-02-12 19:57:51] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/special_tokens_map.json

[INFO|2025-02-12 19:57:51] processing_utils.py:546 >> chat template saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/chat_template.json

[INFO|2025-02-12 19:57:51] processing_utils.py:552 >> processor saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/processor_config.json

[INFO|2025-02-12 19:57:51] trainer.py:3887 >> Saving model checkpoint to saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B

[INFO|2025-02-12 19:57:51] tokenization_utils_base.py:2485 >> tokenizer config file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/tokenizer_config.json

[INFO|2025-02-12 19:57:51] tokenization_utils_base.py:2494 >> Special tokens file saved in saves/LLaVA-NeXT-Video-7B-Chat/lora/250212_Abroad_LLaVANeXTVideo_7B/special_tokens_map.json

[WARNING|2025-02-12 19:57:52] logging.py:162 >> No metric eval_accuracy to plot.

[INFO|2025-02-12 19:57:52] trainer.py:4203 >> 
***** Running Evaluation *****

[INFO|2025-02-12 19:57:52] trainer.py:4205 >>   Num examples = 113

[INFO|2025-02-12 19:57:52] trainer.py:4208 >>   Batch size = 2

[INFO|2025-02-12 19:59:44] modelcard.py:449 >> Dropping the following result as it does not have all the necessary fields:
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}