RuntimeError: The expanded size of the tensor (753) must match the existing size (752) at non-singleton dimension 0. Target sizes: [753, 1280]. Tensor sizes: [752, 1]
I encountered the above error when running your reference inference code, but with eager attention implementation (I can't get FA2 to run properly - same issue as listed here: https://huggingface.co/deepseek-ai/DeepSeek-OCR/discussions/7 )
Traceback:
File "deepseekocr/modeling_deepseekocr.py", line 506, in forward
inputs_embeds[idx].masked_scatter_(images_seq_mask[idx].unsqueeze(-1), images_in_this_batch)
System specs:
WSL (Ubuntu)
Python 3.12
Torch 2.6 + cu124
Transformers 4.57.1
Some help would be appreciated.
Hi. You are using the latest transformers (4.57.1), but Deepseek-OCR uses 4.46.3 . The latest version uses -1 to indicate an infinite kvcache space, instead of None type. So this slight type difference makes attention_mask is 1 shorter than inputs_embeds, i.e. 752 vs 753 in your log.
To solve it, you can downgrade your transformers version. Or checkout my PR/96.