Is MTP supported by this model?

#10

by Bronyaaa - opened Apr 9

Apr 9

I add --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}' (in VLLM), but error occurs:ValueError: could not determine the shape of object type 'torch.storage.UntypedStorage'.

Bronyaaa changed discussion title from Is MTP supported in this model? to Is MTP supported by this model? Apr 9

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment