How to change the batch size? or any tips to make the model faster?
#7
by sdvfsfwfgfwf - opened
How to change the batch size? or any tips to make the model faster?
If you are referring to accelerating the inference phase of the embedding model, you can refer to the inference acceleration solution provided by Hugging Face at https://github.com/huggingface/text-embeddings-inference