Takayanagi-sensei/Text-to-Vehicle-Retrieval-Model

Usage restriction: this model was trained in part on CityFlow-NL (NVIDIA AI City Challenge). Per the dataset's license, models developed using CityFlow-NL — including these weights — may be used for non-commercial, academic purposes only. See LICENSE.md in this repo for the exact clause. This is not a general-purpose open license; commercial use is not permitted.

Text-to-Vehicle Retrieval Model

Joint visual/text embedding model for retrieving vehicle tracks from CCTV footage using natural-language queries (e.g. "a black SUV turning left").

Architecture

• Visual encoder: ResNet-50 (via timm), fine-tuned with triplet + ID loss on VeRi-776 for vehicle re-identification, then projected into a 256-dim shared embedding space.
• Text encoder: OpenCLIP ViT-B-32 text tower, projected into the same 256-dim space.
• Training objective: contrastive (InfoNCE) loss on (vehicle crop, NL description) pairs.

Training data

• VeRi-776 (public) — Re-ID pretraining for the visual encoder.
• CityFlow-NL (AI City Challenge Track 2, private/access-restricted) — contrastive training of the joint embedding space.

This repository contains only the trained model weights. No CityFlow-NL data (video, images, or annotations) is included or redistributed here — CityFlow-NL requires its own separate, approved access request from the AI City Challenge organizers if you want to retrain or evaluate on it yourself: https://www.aicitychallenge.org/

Usage

from huggingface_hub import hf_hub_download
import torch

ckpt_path = hf_hub_download(repo_id="Takayanagi-sensei/Text-to-Vehicle-Retrieval-Model",
                             filename="joint_model_epoch20.pt")
checkpoint = torch.load(ckpt_path, map_location="cpu")
# checkpoint["visual_encoder"], checkpoint["text_encoder_proj"] -- see the demo notebook
# in the linked GitHub repo for the full model class definitions and inference pipeline.

License

This model is released for non-commercial, academic use only, consistent with the license terms of CityFlow-NL, one of its training datasets. The relevant clause from NVIDIA's CityFlow-NL / AI City Challenge dataset license:

Upon the conclusion of a challenge or event ... you may use the DATASET, including models developed using the DATASET, for non-commercial, academic purposes only.

Full license terms: see LICENSE.md in this repo, or the official AI City Challenge dataset license page. If you need to use this model (or a similarly-trained model) commercially, you would need to retrain on data that doesn't carry this restriction — e.g. VeRi-776 alone, without CityFlow-NL in the training pipeline — and confirm VeRi-776's own license terms separately, as they are not addressed here.

Limitations

• Retrieval quality reflects CityFlow-NL's camera domain (fixed traffic intersections); performance on unrelated camera angles/lighting is untested.
• During the contrastive stage, the visual encoder's ResNet-50 backbone was fine-tuned further (unfrozen), while the CLIP text backbone was kept frozen and only its projection head was trained.