Safetensors
molmo
custom_code
4-bit precision
auto-round
cicdatopea commited on
Commit
1ab8840
·
verified ·
1 Parent(s): 07ae75b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -12,12 +12,14 @@ This model is an int4 model with group_size 128 and symmetric quantization of [a
12
 
13
 
14
  ## How To Use
 
 
15
  ### INT4 Inference
16
  ```python
17
  from transformers import AutoModelForCausalLM, AutoProcessor, GenerationConfig
18
  from PIL import Image
19
  import requests
20
-
21
  quantized_model_path = "OPEA/Molmo-7B-D-0924-int4-sym-inc"
22
 
23
  # load the processor
 
12
 
13
 
14
  ## How To Use
15
+ transformers<=4.48.3, auto-round>0.5.1
16
+
17
  ### INT4 Inference
18
  ```python
19
  from transformers import AutoModelForCausalLM, AutoProcessor, GenerationConfig
20
  from PIL import Image
21
  import requests
22
+ from auto_round import AutoRound # must import for auto-round format if transformers <= 4.51.3
23
  quantized_model_path = "OPEA/Molmo-7B-D-0924-int4-sym-inc"
24
 
25
  # load the processor