How to use erbacher/VQ-PDETokenizer with Transformers:
# Load model directly from transformers import Tokenizer model = Tokenizer.from_pretrained("erbacher/VQ-PDETokenizer", dtype="auto")
No model card
# Load model directly from transformers import Tokenizer model = Tokenizer.from_pretrained("erbacher/VQ-PDETokenizer", dtype="auto")