mhenrichsen's picture
Use convert_tokens_to_ids for prompt (string tokenization shatters markers into subwords)
f54119b verified