---
base_model: unsloth/Qwen2.5-VL-3B-Instruct-bnb-4bit
library_name: peft
pipeline_tag: image-text-to-text
license: other
license_name: qwen-research
license_link: https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE
datasets:
- abejarano/container-defects-qwen
language:
- en
tags:
- base_model:adapter:unsloth/Qwen2.5-VL-3B-Instruct-bnb-4bit
- lora
- sft
- transformers
- trl
- unsloth
- vision
- container-inspection
---

# Qwen2.5-VL-3B Container Defect Detector (LoRA)

LoRA adapter for `Qwen2.5-VL-3B-Instruct`, fine-tuned to detect and classify visible
defects on shipping containers (dents, rust, scratches, holes, frame damage) directly
from a photo. Built for **ContainerHub**.

## Model Details

### Model Description

Given a photo of a shipping container, the model returns a comma-separated list of
the visible defect labels (e.g. `Dent, Rusty, Scratch`). It is a LoRA adapter — load
it on top of the base vision-language model, not standalone.

- **Developed by:** abejarano (ContainerHub)
- **Model type:** Vision-language model, LoRA adapter (PEFT)
- **Language(s):** English
- **License:** [qwen-research](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE) (inherited from the base model)
- **Finetuned from model:** [unsloth/Qwen2.5-VL-3B-Instruct-bnb-4bit](https://huggingface.co/unsloth/Qwen2.5-VL-3B-Instruct-bnb-4bit) (base: `Qwen/Qwen2.5-VL-3B-Instruct`)

### Model Sources

- **Training data:** [abejarano/container-defects-qwen](https://huggingface.co/datasets/abejarano/container-defects-qwen)

## Uses

### Direct Use

Multi-label defect classification of shipping container photos, prompted as:
`"What defect on the shipping container?"` → free-text answer listing defect labels.

### Out-of-Scope Use

Not intended for defect severity grading, safety certification, or any decision where
a missed or false defect classification has safety/financial consequences without
human review. Not validated on container types, lighting conditions, or camera angles
outside the training distribution.

## Bias, Risks, and Limitations

- Trained on 1,016 images total, with **no held-out validation split** — reported loss
  reflects training-set fit, not generalization to unseen containers.
- Early experiments (3 epochs, LoRA r=16) showed the model could hallucinate
  near-miss labels (e.g. "Dentation" instead of "Dent") even under greedy decoding.
  This version (6 epochs, LoRA r=32) fixes that on the training examples checked, but
  a proper eval split is still recommended before production use.
- Label vocabulary is fixed to the 5 classes present in the source dataset (Dent,
  Rusty, Scratch, Hole, Deframe); it will not recognize defect types outside this set.

### Recommendations

Evaluate on a held-out set of container images before relying on this model for
automated decisions. Keep a human in the loop for edge cases.

## How to Get Started with the Model

```python
from unsloth import FastVisionModel
from PIL import Image

model, tokenizer = FastVisionModel.from_pretrained(
    "abejarano/qwen25vl-container-defects-lora",
    load_in_4bit=True,
)
FastVisionModel.for_inference(model)

image = Image.open("container.jpg").convert("RGB")
messages = [
    {"role": "user", "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "What defect on the shipping container?"},
    ]}
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to("cuda")

output = model.generate(**inputs, max_new_tokens=150, do_sample=False)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

## Training Details

### Training Data

[abejarano/container-defects-qwen](https://huggingface.co/datasets/abejarano/container-defects-qwen) —
1,016 container images converted to Qwen conversational format, derived from
[howell0123/shipping_container](https://huggingface.co/datasets/howell0123/shipping_container).
Labels: `Dent`, `Rusty`, `Scratch`, `Hole`, `Deframe` (multi-label, comma-separated).

### Training Procedure

Fine-tuned with [Unsloth](https://github.com/unslothai/unsloth) + LoRA on a single
RTX 4090 (RunPod, `unsloth/unsloth:latest` template).

#### Preprocessing

Images decoded to PIL from the dataset's embedded bytes at load time; conversation
format `{"role": ..., "content": [{"type": "image"/"text", ...}]}` fed directly to
`UnslothVisionDataCollator`.

#### Training Hyperparameters

- **LoRA rank (r):** 32
- **LoRA alpha:** 32
- **Target modules:** all-linear (vision + language + attention + MLP layers)
- **Trainable params:** 82,169,856 (2.14% of 3.8B)
- **Epochs:** 6
- **Batch size:** 2 per device, gradient accumulation 4 (effective batch size 8)
- **Learning rate:** 2e-4, cosine schedule, 3% warmup
- **Optimizer:** adamw_8bit
- **Training regime:** bf16 mixed precision

#### Speeds, Sizes, Times

- **Train runtime:** 1942.7s (~32 min)
- **Train samples/sec:** 3.14
- **Final train loss:** 0.128 (per-step loss ~0.0002–0.001 by the last epoch)
- **Adapter size:** ~329MB (`adapter_model.safetensors`)

## Evaluation

Informal validation only: predictions (greedy decoding) were spot-checked against
ground-truth labels on training examples. No dedicated held-out evaluation split
exists yet — see Limitations above.

## Environmental Impact

- **Hardware Type:** NVIDIA RTX 4090
- **Hours used:** ~0.85h GPU time across training runs
- **Cloud Provider:** RunPod
- **Compute Region:** RunPod on-demand pod (region not pinned)

## Technical Specifications

### Model Architecture and Objective

Qwen2.5-VL-3B-Instruct (vision-language transformer) with LoRA adapters, trained via
supervised fine-tuning (SFT) framed as multi-label defect classification through
free-text generation.

### Compute Infrastructure

- **Hardware:** 1x RTX 4090 24GB, RunPod pod
- **Software:** unsloth 2026.5.9, transformers 4.57.6, trl (`SFTTrainer`), peft 0.18.1, torch 2.10.0+cu128

### Framework versions

- PEFT 0.18.1