Error in running the model in CPU
#17
by Navanit-AI - opened
while running the same code below for bge-large-en-v1.55 and gte-large-en-v1.5. I didn't faced any error. But for this model I am facing the error
from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim
from tqdm.autonotebook import tqdm, trange
import torch
device = torch.device("cpu")
sentences = ['That is a happy person', 'That is a very happy person']
model = SentenceTransformer('dunzhang/stella_en_400M_v5',
trust_remote_code=True,
device ="cpu"
)
embeddings = model.encode(sentences)
print(cos_sim(embeddings[0], embeddings[1]))
The error is
"name": "ValueError",
"message": "Attention bias and Query/Key/Value should be on the same device
query.device: cpu
attn_bias : cuda:0
@infgrad Kindly help please.
Did you find a solution?
in config.json change both to false:
"unpad_inputs": true,
"use_memory_efficient_attention": true,
and then you also do not need xformers package.
It then runs on CPU