allenai

/

Molmo-7B-O-0924

like 164

Follow

Ai2 6.03k

I've been trying to run the example code from Molmo-7B-O-0924 model. However, the model is generating only unintelligible text rather than the indicated output "This image features an adorable black Labrador puppy, captured from a top-down perspective. The puppy is sitting on a wooden deck, which is composed ...".

Now, I'm running in a conda environment with python 3.11 and transformers==4.48.2. I have already tried other transformers version, but none of them worked... Below is the exact code snippet I'm running and the model's output.

from transformers import AutoModelForCausalLM, AutoProcessor, GenerationConfig
from PIL import Image
import requests
import torch 
# load the processor
gpu = 3
device = torch.device(f"cuda:{gpu}" if torch.cuda.is_available() else "cpu")
processor = AutoProcessor.from_pretrained(
    'allenai/Molmo-7B-O-0924',
    trust_remote_code=True,
    torch_dtype='auto',
    device=device
)


# load the model
model = AutoModelForCausalLM.from_pretrained(
    'allenai/Molmo-7B-O-0924',
    trust_remote_code=True,
    torch_dtype='auto',    
).to(device)


# process the image and text
inputs = processor.process(images=[Image.open(requests.get("https://picsum.photos/id/237/536/354", stream=True).raw)], text="Describe this image.")

# move inputs to the correct device and make a batch of size 1
inputs = {k: v.to(model.device).unsqueeze(0) for k, v in inputs.items()}

# generate output; maximum 200 new tokens; stop generation when <|endoftext|> is generated
output = model.generate_from_batch(inputs, GenerationConfig(max_new_tokens=200, stop_strings="<|endoftext|>"), tokenizer=processor.tokenizer)

# only get generated tokens; decode them to text
generated_tokens = output[0, inputs["input_ids"].size(1) :]
generated_text = processor.tokenizer.decode(generated_tokens, skip_special_tokens=True)

# print the generated text
print("Output:", generated_text)

# >>>  This image features an adorable black Labrador puppy, captured from a top-down
#      perspective. The puppy is sitting on a wooden deck, which is composed ...

Output: The rest of the a't, which it on the. The (A (i. The two. The time to the. The first. The image of the a, by a "s of the "A-2. The rest of the, The "The number "I,. It's, which is a "A to the the the "The two.

Any suggestions to fix it? :'(