kris-0918/lidar-food-measurable-yolo11n-seg

LiDAR Food Measurable YOLO11n-Seg

A one-class YOLO11n segmentation model fine-tuned on FoodSeg103 and selected COCO 2017 subsets. It produces an RGB food-foreground support mask for an iPhone RGB–LiDAR food-volume research pipeline.

中文简介：这是一个面向毕设研究的单类别食物前景分割模型，用于辅助 iPhone LiDAR 体积估算。它不是食物分类、质量/热量预测或医学诊断模型。

Model summary

• Base model: Ultralytics yolo11n-seg.pt
• Framework: Ultralytics 8.3.0
• Task: one-class segmentation (measurable_object)
• Input size used for training/evaluation: 640 × 640
• Parameters: 2,834,763
• Compute: 9.6 GFLOPs (Ultralytics fused-model summary)
• Formats: PyTorch (best.pt) and Core ML (coreml/best.mlpackage)
• Training hardware: Google Colab NVIDIA T4
• Training: batch 16, maximum 60 epochs, stopped after 42 epochs; trainer time 2.505 hours and full notebook job time about 3 h 12 min

Intended use

The intended use is academic research and non-commercial prototyping of foreground masks for local RGB–LiDAR food-volume estimation. The mask can restrict which RGB-D points are treated as candidate food points before geometric volume estimation.

This model is not:

• a food identity classifier;
• a mass, density, nutrition, or calorie estimator;
• a replacement for LiDAR geometry or ground-plane estimation;
• a medical device or a basis for clinical decisions;
• validated for safety-critical or production use.

Training data

No source images or annotations are redistributed in this repository.

The conversion pipeline merged all non-background FoodSeg103 semantic labels into one foreground class and converted selected COCO 2017 instance masks into the same class.

Split/source	Written image entries	Converted polygons	Notes
FoodSeg103 train	5,583	8,607	4,983 original entries plus 600 repeated focus entries
FoodSeg103 validation subset	800	1,230	Held-out subset used in the combined validation set
COCO 2017 train	1,190	4,046	990 food-positive images plus 200 container hard negatives
COCO 2017 validation	247	765	197 food-positive images plus 50 hard negatives

The combined training set contains 6,773 written entries, approximately 6,173 independent images plus 600 repeated focus entries. The validation set contains 1,047 images. Ultralytics reported 1,994 validation instances; conversion metadata recorded 1,995 polygons, indicating that one converted instance was filtered or otherwise not counted by the trainer.

Selected COCO positive categories were apple, banana, broccoli, cake, carrot, donut, hot dog, orange, pizza, and sandwich. Bottle, bowl, cup, and wine glass images were used as container-focused hard negatives.

Because FoodSeg103 semantic masks are unioned into one class, touching items can merge into a single region. The model must not be described as reliably recovering individual identities in dense piles.

Evaluation

Final revalidation of best.pt on the combined 1,047-image validation set (1,994 trainer-reported instances):

Output	Precision	Recall	mAP@0.50	mAP@0.50:0.95
Bounding boxes	0.830	0.633	0.755	0.641
Segmentation masks	0.852	0.622	0.747	0.598

These are validation-set detection/segmentation metrics, not a direct measurement of food volume, mass, or calorie accuracy. The reported T4 validation inference time (about 2.6 ms/image in the retained run) is not an iPhone latency benchmark.

The notebook's uploaded-photo qualitative test did not complete because that Colab runtime developed a NumPy binary-ABI incompatibility after training. Training, checkpoint validation, and Core ML export had already completed; this repository does not claim success for that qualitative photo test.

PyTorch usage

Install the recorded framework version in an isolated environment:

pip install ultralytics==8.3.0

from ultralytics import YOLO

model = YOLO("best.pt")
results = model.predict("food.jpg", imgsz=640, conf=0.25)

for result in results:
    if result.masks is not None:
        masks = result.masks.data
        print(masks.shape)

Inference users should tune confidence and NMS thresholds against their own capture geometry rather than treating 0.25 as a calibrated operating point.

Core ML deployment notes

The included Core ML package was exported without built-in NMS. Its retained raw-output contract is:

• detection tensor: [1, 37, 8400];
• prototype tensor: [1, 32, 160, 160];
• one object class plus four box channels and 32 mask coefficients.

The consuming iOS code must perform confidence filtering, NMS, prototype-mask reconstruction, crop/resize reversal, and alignment with the camera/depth coordinate system. This package is therefore not necessarily a drop-in replacement for a Core ML model that embeds NMS or returns already-rendered masks.

Limitations and failure modes

• Transparent, translucent, glossy, very dark, or strongly reflective foods may be unreliable in RGB and LiDAR sensing.
• Dense piles, touching pieces, severe occlusion, and unseen cuisines may be under-segmented or merged.
• Containers and food with similar appearance may still be confused despite container hard negatives.
• A 2D mask cannot recover hidden food volume or correct a bad depth map, camera calibration, ground plane, or coordinate transform.
• The model has not been validated against project-specific LiDAR volume ground truth.
• The selected/duplicated training entries mean that raw entry counts should not be presented as the number of independent images.

Repository contents

• best.pt — best PyTorch checkpoint.
• coreml/best.mlpackage/ — Core ML export with raw detection/prototype outputs.
• training/lidar_food_yolo_seg_colab7_11.ipynb — executed Colab notebook, including retained logs.
• training/food_measurable_object_yolo_seg_job.py — dataset conversion and training job.
• training/args.yaml, results.csv, run_manifest.json, experiment_summary.json — run configuration, curves, manifest, and summary.
• MODEL_CHECKSUMS.txt — SHA-256 checksums for release artifacts.
• THIRD_PARTY_NOTICES.md — dataset/framework provenance and license notes.

License and data provenance

This model repository is released under GNU AGPL-3.0. It was fine-tuned from an Ultralytics YOLO11 segmentation checkpoint using the Ultralytics training stack. See the Ultralytics licensing page and the included LICENSE and THIRD_PARTY_NOTICES.md.

FoodSeg103 is referenced through its Hugging Face dataset card, which identifies Apache-2.0 for that dataset repository. COCO-derived selection used COCO 2017 annotations and images; individual COCO image licenses may vary. See the COCO website and COCO API repository.

This repository does not contain the original FoodSeg103/COCO images, their full annotations, or personal test photos. The repository license does not override third-party rights, dataset terms, image licenses, or trademark rights. This notice is a provenance record, not legal advice and not a warranty of non-infringement.

Citation

If this model supports academic work, cite this model repository and the upstream datasets/framework used in the experiment. A formal project paper citation can be added after the FYP thesis is published.