Publish TextSeg-refined v2 checkpoint and SFX 0.20 settings

Replaces the deployment weights with the epoch-7 best checkpoint from the 8xH100 Manga109 Segmentation v2.0.0 run. Updates validation metrics, provenance, and class-specific inference thresholds.

README.md

@@ -37,9 +37,9 @@ The model does **not** perform OCR or reading-order prediction.
 - `inference_config.json` — class mapping and recommended inference settings
 - `validation_metrics.json` — final held-out validation results
 
-The deployment weights contain only RF-DETR tensors. The auxiliary dense
-typography branch used during fine-tuning has been removed because it is not
-part of RF-DETR inference.
+The SafeTensors weights are the epoch-7 best checkpoint from the TextSeg-refined
+Manga109 Segmentation v2.0.0 training run. They contain only the standard
+RF-DETR Seg 2XL model state; no custom dense head is required.
 
 ## Usage
 
@@ -51,6 +51,7 @@ pip install "rfdetr==1.7.0" "safetensors>=0.5" huggingface_hub pillow
 from huggingface_hub import hf_hub_download
 from PIL import Image
 import importlib.util
+import numpy as np
 
 weights = hf_hub_download(
     repo_id="mayocream/koharu-layout-rfdetr-seg-2xl-1152",
@@ -68,11 +69,18 @@ model = loader.load_model(weights)
 image = Image.open("page.jpg").convert("RGB")
 detections = model.predict(
     image,
-    threshold=0.25,
+    threshold=0.20,
     shape=(1152, 1152),
     include_source_image=False,
 )
 
+class_thresholds = {0: 0.25, 1: 0.20, 2: 0.50, 3: 0.50}
+keep = np.asarray([
+    score >= class_thresholds[int(class_id)]
+    for class_id, score in zip(detections.class_id, detections.confidence)
+])
+detections = detections[keep]
+
 print(detections.xyxy)       # bounding boxes
 print(detections.mask)       # instance masks
 print(detections.class_id)   # 0=text, 1=COO, 2=bubble, 3=panel
@@ -83,43 +91,46 @@ CUDA is strongly recommended. The model was trained and evaluated at 1152 px.
 
 ## Recommended thresholds
 
-A single `0.25` threshold maximizes recall, but class-specific filtering is
-usually better:
+Run prediction at the lowest threshold below, then apply class-specific
+filtering:
 
 | Class | Suggested threshold |
 |---|---:|
-| Text | 0.25–0.30 |
-| COO | 0.40–0.50 |
+| Text | 0.25 |
+| COO / SFX | 0.20 |
 | Bubble | 0.50 |
 | Panel | 0.50 |
 
-The lower text threshold helps retain titles, credits, and back matter. Highly
-decorative pages may require tiling or a separate fallback detector.
+The lower SFX threshold is intentional. On a 291-page out-of-domain comparison,
+changing only SFX from 0.40 to 0.20 raised typography-mask recall against
+TextSeg from 72.98% to 79.51% and raw mask IoU from 58.37% to 61.75%. It is
+particularly helpful for large stylized effects. For applications that favor
+precision over pre-inpainting recall, raise the SFX threshold toward 0.40.
 
 ## Validation results
 
-The final checkpoint was evaluated on the held-out 1,070-page Manga109 test
-split at 1152 px.
+The final checkpoint was evaluated on the held-out 1,001-page TextSeg-refined
+Manga109 validation split at 1152 px.
 
 | Metric | Score |
 |---|---:|
-| Box mAP50–95 | 0.7929 |
-| Box mAP50 | 0.8941 |
-| Box mAP75 | 0.8373 |
-| Mask mAP50–95 | 0.5187 |
-| Mask mAP50 | 0.7206 |
-| Detection precision | 0.8061 |
-| Detection recall | 0.7452 |
-| Detection F1 | 0.7672 |
+| Box mAP50–95 | 0.7969 |
+| Box mAP50 | 0.8970 |
+| Box mAP75 | 0.8425 |
+| Mask mAP50–95 | 0.5713 |
+| Mask mAP50 | 0.8256 |
+| Detection precision | 0.8767 |
+| Detection recall | 0.8177 |
+| Detection F1 | 0.8392 |
 
 Per-class box AP:
 
 | Class | AP |
 |---|---:|
-| Text | 0.8762 |
-| COO | 0.4285 |
-| Bubble | 0.9094 |
-| Panel | 0.9575 |
+| Text | 0.8779 |
+| COO | 0.4431 |
+| Bubble | 0.9092 |
+| Panel | 0.9574 |
 
 ## Out-of-domain review
 
@@ -129,31 +140,32 @@ annotations, so the observations below are visual rather than accuracy claims.
 
 - Dialogue text, bubbles, and conventional panel layouts generalized well.
 - Marriage Toxin dialogue, bubbles, and panels were particularly consistent.
-- Large isolated COO was often detected, but smaller effects were fragmented or
-  confused with ordinary text.
+- SFX at threshold 0.20 recovered substantially more large stylized effects on
+  the Blue Archive pages, with a small precision cost.
 - Covers, logos, credits, and dense collage layouts remain difficult.
-- A highly decorative collage page produced 140 low-confidence predictions,
-  approaching the configured 160-candidate limit.
+- Very decorative marks can be accepted as SFX at the recommended low threshold.
+
+TextSeg was used as the comparison reference for this review, not human ground
+truth. After 4 px reference-scale dilation and hole filling, RF-DETR/TextSeg
+mask IoU increased from 79.18% at SFX 0.40 to 82.27% at SFX 0.20.
 
 ## Training
 
 - Architecture: RF-DETR Seg 2XL (`rfdetr==1.7.0`)
 - Resolution: 1152 × 1152
-- Main supervision: Manga109 Segmentation v1.1.0
+- Main supervision: Manga109 Segmentation v2.0.0, with PP-DocLayoutV3 boxes and
+  TextSeg-refined typography masks
 - Classes: text, onomatopoeia, bubble, panel
-- Final continuation stage: 6 epochs
+- Final continuation stage: 8 epochs; best checkpoint at epoch 7
 - Final-stage global batch: 32 (8 GPUs × 2 images × 2 accumulation steps)
 - Optimizer learning rate: `3e-5`; encoder learning rate: `1e-5`
 - EMA enabled; seed 42
-- Additional normal-text supervision was distilled from PP-DocLayoutV3
-
-The auxiliary typography head used for training improved mask recall but did
-not materially improve box AP. It is intentionally excluded from this
-deployment checkpoint.
+- Training hardware: 8 × NVIDIA H100 80 GB
+- No custom dense head or typography-distillation branch
 
 ## Limitations
 
-- COO is substantially weaker than text, bubble, and panel detection.
+- COO remains weaker than text, bubble, and panel detection.
 - Large display titles and credits may be missed or assigned low confidence.
 - Character nameplates and small icons can be confused with bubbles.
 - Dense collages can create many duplicate or low-confidence instances.
@@ -178,5 +190,5 @@ images or to third-party comic content.
 SHA-256:
 
 ```text
-0d730ea52064051f3a1669bd146a2625d511b78072f1c176e200f70cbdb9042c
+9bf6d2cbd7793c956d8c857bb1672a396eb7f100eb0682f86830d05e31168efb
 ```

inference_config.json

@@ -11,10 +11,11 @@
   },
   "recommended_thresholds": {
     "text": 0.25,
-    "onomatopoeia": 0.4,
+    "onomatopoeia": 0.2,
     "bubble": 0.5,
     "panel": 0.5
   },
   "checkpoint": "model.safetensors",
-  "checkpoint_sha256": "0d730ea52064051f3a1669bd146a2625d511b78072f1c176e200f70cbdb9042c"
+  "checkpoint_sha256": "9bf6d2cbd7793c956d8c857bb1672a396eb7f100eb0682f86830d05e31168efb",
+  "source_checkpoint_sha256": "b1f8c81e41e376342cb9e236be265505a96dc0c3774916df17cad4d720e3b67a"
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0d730ea52064051f3a1669bd146a2625d511b78072f1c176e200f70cbdb9042c
+oid sha256:9bf6d2cbd7793c956d8c857bb1672a396eb7f100eb0682f86830d05e31168efb
 size 161292684

validation_metrics.json

@@ -1,24 +1,25 @@
 {
-  "dataset": "Manga109 Segmentation v1.1.0 held-out test split",
-  "pages": 1070,
+  "dataset": "Manga109 Segmentation v2.0.0 TextSeg-refined validation split",
+  "pages": 1001,
   "resolution": 1152,
-  "checkpoint_epoch": 5,
+  "checkpoint_epoch": 7,
   "metrics": {
-    "box_map_50_95": 0.79288250207901,
-    "box_map_50": 0.8940883278846741,
-    "box_map_75": 0.8372876644134521,
-    "box_mar": 0.850690484046936,
-    "precision": 0.806089460849762,
-    "recall": 0.745160698890686,
-    "f1": 0.767235279083252,
-    "mask_map_50_95": 0.5187143087387085,
-    "mask_map_50": 0.7205575704574585,
+    "box_map_50_95": 0.7968884706497192,
+    "box_map_50": 0.8969718217849731,
+    "box_map_75": 0.8424826264381409,
+    "box_mar": 0.8537694215774536,
+    "precision": 0.8767244219779968,
+    "recall": 0.8176969885826111,
+    "f1": 0.8392113447189331,
+    "mask_map_50_95": 0.5712965726852417,
+    "mask_map_50": 0.8256368041038513,
     "box_ap_by_class": {
-      "text": 0.8761724829673767,
-      "onomatopoeia": 0.4284520149230957,
-      "bubble": 0.9094059467315674,
-      "panel": 0.9574995636940002
+      "text": 0.8779019117355347,
+      "onomatopoeia": 0.4430747330188751,
+      "bubble": 0.9092085957527161,
+      "panel": 0.9573686122894287
     }
-  }
+  },
+  "checkpoint_sha256": "b1f8c81e41e376342cb9e236be265505a96dc0c3774916df17cad4d720e3b67a",
+  "validation_mode": "single GPU after 8-GPU DDP training"
 }
-