dronefreak commited on
Commit
bfb73a9
·
verified ·
1 Parent(s): b9aefb5

Upload 3 files

Browse files
Files changed (3) hide show
  1. README.md +20 -0
  2. country_breakdown.json +32 -0
  3. domain_metadata.json +0 -0
README.md CHANGED
@@ -109,6 +109,24 @@ Every model DetectionBench has trained and evaluated on Global Wheat Head Datase
109
  | wheat_head | 53.82 | 19.65 |
110
  ---
111
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
112
  ## Evaluation Visualizations
113
 
114
  This model was evaluated with [Supervision](https://github.com/roboflow/supervision)'s detection metrics, which report mAP/Precision/Recall directly but don't produce PR-curve, F1-curve, or confusion-matrix plot images the way Ultralytics' validator does. See the Performance table above for Precision/Recall/F1 and the per-class table above for the full per-class mAP breakdown.
@@ -178,6 +196,8 @@ detections = model.predict("image.jpg", threshold=0.25)
178
  checkpoint_best_total.pth
179
  metrics.csv
180
  config.json
 
 
181
  gwhd_rfdetr-nano_showcase.jpg
182
  README.md
183
  ```
 
109
  | wheat_head | 53.82 | 19.65 |
110
  ---
111
 
112
+ ## Per-Country Performance
113
+
114
+ Domain shift can matter more than the aggregate score above for field deployment, so this evaluates the same test split broken down by the contributing country/institution, using per-image domain metadata (`domain_metadata.json` in this repository) compiled by this project for this stratified evaluation -- not a file shipped with the original GWHD release. Each row below is computed by re-running this exact model's evaluation restricted to that country's images only -- the same mAP definition as the aggregate number above (Ultralytics' `model.val()` for YOLO, Supervision's `MeanAveragePrecision` for RF-DETR), just on a filtered subset, not a separate metric implementation. One test image with no resolvable country in the source metadata (a documented upstream duplicate-filename quirk) is excluded from every row below.
115
+
116
+ | Country | mAP@50 | mAP@50-95 | Test Images |
117
+ | --------------------------- | --------------- | ----------------- | ------------------ |
118
+ | Australia | 35.4 | 10.86 | 281 |
119
+ | China | 79.78 | 33.58 | 200 |
120
+ | Japan | 61.07 | 30.42 | 60 |
121
+ | Mexico | 54.51 | 19.27 | 205 |
122
+ | Sudan | 61.42 | 24.15 | 30 |
123
+ | US | 58.7 | 20.9 | 605 |
124
+
125
+
126
+ See `country_breakdown.json` (results) and `domain_metadata.json` (the country/growth-stage mapping used to compute them) in this repository for the raw data behind this table.
127
+
128
+ ---
129
+
130
  ## Evaluation Visualizations
131
 
132
  This model was evaluated with [Supervision](https://github.com/roboflow/supervision)'s detection metrics, which report mAP/Precision/Recall directly but don't produce PR-curve, F1-curve, or confusion-matrix plot images the way Ultralytics' validator does. See the Performance table above for Precision/Recall/F1 and the per-class table above for the full per-class mAP breakdown.
 
196
  checkpoint_best_total.pth
197
  metrics.csv
198
  config.json
199
+ country_breakdown.json
200
+ domain_metadata.json
201
  gwhd_rfdetr-nano_showcase.jpg
202
  README.md
203
  ```
country_breakdown.json ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "Australia": {
3
+ "mAP50": 0.3540254533290863,
4
+ "mAP50_95": 0.10858235508203506,
5
+ "num_images": 281
6
+ },
7
+ "China": {
8
+ "mAP50": 0.7978321313858032,
9
+ "mAP50_95": 0.3357720375061035,
10
+ "num_images": 200
11
+ },
12
+ "Japan": {
13
+ "mAP50": 0.6107019186019897,
14
+ "mAP50_95": 0.30420443415641785,
15
+ "num_images": 60
16
+ },
17
+ "Mexico": {
18
+ "mAP50": 0.5450758934020996,
19
+ "mAP50_95": 0.1927119940519333,
20
+ "num_images": 205
21
+ },
22
+ "Sudan": {
23
+ "mAP50": 0.6141822934150696,
24
+ "mAP50_95": 0.24153614044189453,
25
+ "num_images": 30
26
+ },
27
+ "US": {
28
+ "mAP50": 0.5870068073272705,
29
+ "mAP50_95": 0.2090480625629425,
30
+ "num_images": 605
31
+ }
32
+ }
domain_metadata.json ADDED
The diff for this file is too large to render. See raw diff