BiliSakura commited on
Commit
22b9fbc
·
verified ·
1 Parent(s): 930dfab

Sync updated code and configs (no weight re-upload)

Browse files
README.md CHANGED
@@ -1,20 +1,3 @@
1
- ---
2
- license: mit
3
- language:
4
- - en
5
- tags:
6
- - remote-sensing
7
- - earth-observation
8
- - vision
9
- - feature-extraction
10
- - croma
11
- - sentinel-1
12
- - sentinel-2
13
- - multimodal
14
- library_name: transformers
15
- pipeline_tag: feature-extraction
16
- ---
17
-
18
  # CROMA Transformers Models
19
 
20
  Self-contained HuggingFace model checkpoints for [CROMA](https://arxiv.org/pdf/2311.00566.pdf). Each subfolder ships its own model, image processor, and custom pipeline code — no external `croma` package required.
@@ -30,6 +13,8 @@ CROMA expects dual-modality Sentinel-1 (2-channel SAR) and Sentinel-2 (12-channe
30
 
31
  ## Usage
32
 
 
 
33
  ```python
34
  from transformers import pipeline
35
 
@@ -43,8 +28,8 @@ pipe = pipeline(
43
 
44
  import numpy as np
45
 
46
- sar = np.random.randn(2, 120, 120).astype(np.float32)
47
- optical = np.random.randn(12, 120, 120).astype(np.float32)
48
 
49
  joint_gap = pipe(
50
  sar_images=sar,
@@ -62,6 +47,19 @@ sar_tokens = pipe(
62
  )
63
  ```
64
 
 
 
 
 
 
 
 
 
 
 
 
 
 
65
  Or load components directly:
66
 
67
  ```python
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # CROMA Transformers Models
2
 
3
  Self-contained HuggingFace model checkpoints for [CROMA](https://arxiv.org/pdf/2311.00566.pdf). Each subfolder ships its own model, image processor, and custom pipeline code — no external `croma` package required.
 
13
 
14
  ## Usage
15
 
16
+ Processors default to **`do_resize: false`**. CROMA expects SAR `(2, H, W)` and optical `(12, H, W)` at the same native spatial size (120×120 at pretraining; larger sizes work without forced resize).
17
+
18
  ```python
19
  from transformers import pipeline
20
 
 
28
 
29
  import numpy as np
30
 
31
+ sar = np.random.randn(2, 256, 256).astype(np.float32)
32
+ optical = np.random.randn(12, 256, 256).astype(np.float32)
33
 
34
  joint_gap = pipe(
35
  sar_images=sar,
 
47
  )
48
  ```
49
 
50
+ Opt in to 120×120 resize:
51
+
52
+ ```python
53
+ joint_gap = pipe(
54
+ sar_images=sar,
55
+ optical_images=optical,
56
+ use_8_bit=True,
57
+ pool=True,
58
+ return_tensors=True,
59
+ image_processor_kwargs={"do_resize": True},
60
+ )
61
+ ```
62
+
63
  Or load components directly:
64
 
65
  ```python
croma-base-patch8-120/image_processing_croma.py CHANGED
@@ -113,7 +113,7 @@ class CromaImageProcessor(BaseImageProcessor):
113
 
114
  def __init__(
115
  self,
116
- do_resize: bool = True,
117
  size: Optional[dict[str, int]] = None,
118
  resample: PILImageResampling = PILImageResampling.BILINEAR,
119
  do_rescale: bool = True,
 
113
 
114
  def __init__(
115
  self,
116
+ do_resize: bool = False,
117
  size: Optional[dict[str, int]] = None,
118
  resample: PILImageResampling = PILImageResampling.BILINEAR,
119
  do_rescale: bool = True,
croma-base-patch8-120/preprocessor_config.json CHANGED
@@ -4,7 +4,7 @@
4
  "height": 120,
5
  "width": 120
6
  },
7
- "do_resize": true,
8
  "do_rescale": true,
9
  "do_normalize": true,
10
  "use_8_bit": true,
@@ -13,4 +13,4 @@
13
  "auto_map": {
14
  "AutoImageProcessor": "image_processing_croma.CromaImageProcessor"
15
  }
16
- }
 
4
  "height": 120,
5
  "width": 120
6
  },
7
+ "do_resize": false,
8
  "do_rescale": true,
9
  "do_normalize": true,
10
  "use_8_bit": true,
 
13
  "auto_map": {
14
  "AutoImageProcessor": "image_processing_croma.CromaImageProcessor"
15
  }
16
+ }
croma-large-patch8-120/image_processing_croma.py CHANGED
@@ -113,7 +113,7 @@ class CromaImageProcessor(BaseImageProcessor):
113
 
114
  def __init__(
115
  self,
116
- do_resize: bool = True,
117
  size: Optional[dict[str, int]] = None,
118
  resample: PILImageResampling = PILImageResampling.BILINEAR,
119
  do_rescale: bool = True,
 
113
 
114
  def __init__(
115
  self,
116
+ do_resize: bool = False,
117
  size: Optional[dict[str, int]] = None,
118
  resample: PILImageResampling = PILImageResampling.BILINEAR,
119
  do_rescale: bool = True,
croma-large-patch8-120/preprocessor_config.json CHANGED
@@ -4,7 +4,7 @@
4
  "height": 120,
5
  "width": 120
6
  },
7
- "do_resize": true,
8
  "do_rescale": true,
9
  "do_normalize": true,
10
  "use_8_bit": true,
@@ -13,4 +13,4 @@
13
  "auto_map": {
14
  "AutoImageProcessor": "image_processing_croma.CromaImageProcessor"
15
  }
16
- }
 
4
  "height": 120,
5
  "width": 120
6
  },
7
+ "do_resize": false,
8
  "do_rescale": true,
9
  "do_normalize": true,
10
  "use_8_bit": true,
 
13
  "auto_map": {
14
  "AutoImageProcessor": "image_processing_croma.CromaImageProcessor"
15
  }
16
+ }