Instructions to use mcity-data-engine/fisheye8k_jozhang97_deta-swin-large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mcity-data-engine/fisheye8k_jozhang97_deta-swin-large with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("object-detection", model="mcity-data-engine/fisheye8k_jozhang97_deta-swin-large")# Load model directly from transformers import AutoModelForObjectDetection model = AutoModelForObjectDetection.from_pretrained("mcity-data-engine/fisheye8k_jozhang97_deta-swin-large", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Improve model card: Add object-detection pipeline tag, license, links, and usage example
Browse filesThis PR significantly improves the model card for `fisheye8k_jozhang97_deta-swin-large` by:
* Adding the `pipeline_tag: object-detection` to the metadata, ensuring better discoverability (e.g., via https://huggingface.co/models?pipeline_tag=object-detection).
* Adding the `license: mit` to the metadata, as explicitly stated in the paper and GitHub repository.
* Adding direct links to the associated research paper, the project homepage, and the GitHub repository.
* Providing a clear, runnable Python code snippet for inference using the `transformers` library, replacing the generic "More information needed" sections.
* Enriching the model description, intended uses, and training/evaluation data sections with context from the research paper and project details.
* Removing redundant auto-generated comments and technical file information.
* Adding an Acknowledgements and Citation section for proper attribution.
|
@@ -1,35 +1,83 @@
|
|
| 1 |
---
|
| 2 |
-
library_name: transformers
|
| 3 |
base_model: jozhang97/deta-swin-large
|
| 4 |
-
tags:
|
| 5 |
-
- generated_from_trainer
|
| 6 |
datasets:
|
| 7 |
- Voxel51/fisheye8k
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
model-index:
|
| 9 |
- name: fisheye8k_jozhang97_deta-swin-large
|
| 10 |
results: []
|
| 11 |
---
|
| 12 |
|
| 13 |
-
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
| 14 |
-
should probably proofread and complete it, then remove this comment. -->
|
| 15 |
-
|
| 16 |
# fisheye8k_jozhang97_deta-swin-large
|
| 17 |
|
| 18 |
-
This model is a fine-tuned version of [jozhang97/deta-swin-large](https://huggingface.co/jozhang97/deta-swin-large) on the
|
|
|
|
| 19 |
It achieves the following results on the evaluation set:
|
| 20 |
- Loss: 17.9701
|
| 21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
## Model description
|
| 23 |
|
| 24 |
-
|
|
|
|
|
|
|
| 25 |
|
| 26 |
## Intended uses & limitations
|
| 27 |
|
| 28 |
-
|
|
|
|
|
|
|
| 29 |
|
| 30 |
## Training and evaluation data
|
| 31 |
|
| 32 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
|
| 34 |
## Training procedure
|
| 35 |
|
|
@@ -65,4 +113,18 @@ The following hyperparameters were used during training:
|
|
| 65 |
- Datasets 3.2.0
|
| 66 |
- Tokenizers 0.21.0
|
| 67 |
|
| 68 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
base_model: jozhang97/deta-swin-large
|
|
|
|
|
|
|
| 3 |
datasets:
|
| 4 |
- Voxel51/fisheye8k
|
| 5 |
+
library_name: transformers
|
| 6 |
+
tags:
|
| 7 |
+
- generated_from_trainer
|
| 8 |
+
pipeline_tag: object-detection
|
| 9 |
+
license: mit
|
| 10 |
model-index:
|
| 11 |
- name: fisheye8k_jozhang97_deta-swin-large
|
| 12 |
results: []
|
| 13 |
---
|
| 14 |
|
|
|
|
|
|
|
|
|
|
| 15 |
# fisheye8k_jozhang97_deta-swin-large
|
| 16 |
|
| 17 |
+
This model is a fine-tuned version of [jozhang97/deta-swin-large](https://huggingface.co/jozhang97/deta-swin-large) on the [Fisheye8K dataset](https://huggingface.co/datasets/Voxel51/fisheye8k). It was developed as part of the **Mcity Data Engine** project, an open-source system designed for iterative model improvement through open-vocabulary data selection.
|
| 18 |
+
|
| 19 |
It achieves the following results on the evaluation set:
|
| 20 |
- Loss: 17.9701
|
| 21 |
|
| 22 |
+
**Paper**: [Mcity Data Engine: Iterative Model Improvement Through Open-Vocabulary Data Selection](https://huggingface.co/papers/2504.21614)
|
| 23 |
+
**Project Page**: [Mcity Data Engine Docs](https://mcity.github.io/mcity_data_engine/)
|
| 24 |
+
**Code**: [GitHub Repository](https://github.com/mcity/mcity_data_engine)
|
| 25 |
+
|
| 26 |
## Model description
|
| 27 |
|
| 28 |
+
This model is a key component of the **Mcity Data Engine**, a comprehensive, open-source system for the complete data-based development cycle of machine learning models. It specifically targets challenges in Intelligent Transportation Systems (ITS), where the goal is to detect rare and novel classes in vast amounts of unlabeled data, such as those generated by vehicle fleets and roadside perception systems.
|
| 29 |
+
|
| 30 |
+
This `fisheye8k_jozhang97_deta-swin-large` model is an object detection model fine-tuned using the Mcity Data Engine's methodologies. It focuses on identifying specific object categories relevant to ITS, trained on data from fisheye cameras. The engine facilitates iterative model improvements by intelligently selecting and labeling data, especially for long-tail classes.
|
| 31 |
|
| 32 |
## Intended uses & limitations
|
| 33 |
|
| 34 |
+
**Intended Uses**: This model is primarily intended for object detection tasks within Intelligent Transportation Systems (ITS). It is designed to identify objects such as `Bus`, `Bike`, `Car`, `Pedestrian`, and `Truck` in visual data, particularly from fisheye camera perspectives, as part of the iterative data selection and model training processes facilitated by the Mcity Data Engine. It serves as a practical demonstration and artifact of the engine's capabilities.
|
| 35 |
+
|
| 36 |
+
**Limitations**: As a model fine-tuned on a specific dataset (Fisheye8K), its performance may vary when applied to datasets with significantly different characteristics, environmental conditions, or object distributions. Its optimal utility is achieved when integrated within the broader Mcity Data Engine framework for continuous improvement and adaptation to novel classes.
|
| 37 |
|
| 38 |
## Training and evaluation data
|
| 39 |
|
| 40 |
+
This model was fine-tuned on the [Voxel51/fisheye8k](https://huggingface.co/datasets/Voxel51/fisheye8k) dataset. This dataset is crucial for the model's application in Intelligent Transportation Systems, providing data from fisheye cameras. The training process leverages the open-vocabulary data selection capabilities of the Mcity Data Engine to identify and incorporate relevant samples, including rare and long-tail classes. The model detects the following classes: `Bus`, `Bike`, `Car`, `Pedestrian`, `Truck`.
|
| 41 |
+
|
| 42 |
+
## Sample Usage
|
| 43 |
+
|
| 44 |
+
You can use this model directly with the Hugging Face `transformers` library for object detection:
|
| 45 |
+
|
| 46 |
+
```python
|
| 47 |
+
import torch
|
| 48 |
+
from transformers import AutoImageProcessor, AutoModelForObjectDetection
|
| 49 |
+
from PIL import Image
|
| 50 |
+
import requests
|
| 51 |
+
|
| 52 |
+
# Load an example image (replace with your fisheye image if available)
|
| 53 |
+
# This example uses a standard COCO image for demonstration purposes.
|
| 54 |
+
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
|
| 55 |
+
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")
|
| 56 |
+
|
| 57 |
+
# Load image processor and model from the Hugging Face Hub
|
| 58 |
+
model_name = "jozhang97/fisheye8k_jozhang97_deta-swin-large"
|
| 59 |
+
image_processor = AutoImageProcessor.from_pretrained(model_name)
|
| 60 |
+
model = AutoModelForObjectDetection.from_pretrained(model_name)
|
| 61 |
+
|
| 62 |
+
# Process image and get predictions
|
| 63 |
+
inputs = image_processor(images=image, return_tensors="pt")
|
| 64 |
+
|
| 65 |
+
with torch.no_grad():
|
| 66 |
+
outputs = model(**inputs)
|
| 67 |
+
|
| 68 |
+
# Post-process outputs to get bounding boxes, labels, and scores
|
| 69 |
+
target_sizes = torch.tensor([image.size[::-1]]) # (height, width) for post-processing
|
| 70 |
+
results = image_processor.post_process_object_detection(outputs, target_sizes=target_sizes, threshold=0.9)[0]
|
| 71 |
+
|
| 72 |
+
# Print detected objects
|
| 73 |
+
print("Detected objects:")
|
| 74 |
+
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
|
| 75 |
+
box = [round(i, 2) for i in box.tolist()]
|
| 76 |
+
print(
|
| 77 |
+
f" Detected {model.config.id2label[label.item()]} "
|
| 78 |
+
f"with confidence {round(score.item(), 3)} at location {box}"
|
| 79 |
+
)
|
| 80 |
+
```
|
| 81 |
|
| 82 |
## Training procedure
|
| 83 |
|
|
|
|
| 113 |
- Datasets 3.2.0
|
| 114 |
- Tokenizers 0.21.0
|
| 115 |
|
| 116 |
+
## Acknowledgements
|
| 117 |
+
Mcity would like to thank Amazon Web Services (AWS) for their pivotal role in providing the cloud infrastructure on which the Data Engine depends. We couldn’t have done it without their tremendous support!
|
| 118 |
+
|
| 119 |
+
## Citation
|
| 120 |
+
|
| 121 |
+
If you use the Mcity Data Engine in your research, feel free to cite the project:
|
| 122 |
+
|
| 123 |
+
```bibtex
|
| 124 |
+
@article{bogdoll2025mcitydataengine,
|
| 125 |
+
title={Mcity Data Engine: Iterative Model Improvement Through Open-Vocabulary Data Selection},
|
| 126 |
+
author={Bogdoll, Daniel and Anata, Rajanikant Patnaik and Stevens, Gregory},
|
| 127 |
+
journal={arXiv preprint arXiv:2504.21614},
|
| 128 |
+
year={2025}
|
| 129 |
+
}
|
| 130 |
+
```
|