OpenSparX-7b-cabin-sft-v2-gptq-int8

简介 | Overview

这是一个基于 Qwen2.5-VL-7B-Instruct 的多模态座舱助手模型，采用 GPTQ INT8 量化，适合更关注精度保留的高算力域控或独立 GPU 推理场景。

This is a multimodal in-cabin assistant model based on Qwen2.5-VL-7B-Instruct and quantized with GPTQ INT8. It is better suited for higher-performance deployments where accuracy retention is more important.

适用场景 | Use Cases

• 座舱情绪识别与安抚建议。

• 驾驶分心行为识别与安全提醒。

• 儿童安全带佩戴检测与提醒。

• 输出自然语言回复和结构化控制标签，便于接入车机控制链路。

• Emotion recognition and comfort suggestions in the cabin.

• Driver distraction detection and safety reminders.

• Child seat-belt wearing detection and reminders.

• Natural-language responses plus structured action tags for downstream vehicle control integration.

推荐输入格式 | Recommended Input Format

推荐使用 Qwen2.5-VL 的多模态消息格式，图像和文本通过 content 数组一起传入。

Use the native Qwen2.5-VL multimodal message format, where image and text are passed together in the content array.

{
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "image"},
        {"type": "text", "text": "请根据图像描述驾驶员状态并给出功能建议。"}
      ]
    }
  ]
}

Python 推理示例 | Python Inference Example

该模型是视觉语言模型，推理时应使用 AutoProcessor 和 Qwen2_5_VLForConditionalGeneration。

This model is a vision-language model, so inference should use AutoProcessor and Qwen2_5_VLForConditionalGeneration.

from PIL import Image
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration

repo_id = "qualcomm-ai-hub-community/OpenSparX-7b-cabin-sft-v2-gptq-int8"
processor = AutoProcessor.from_pretrained(repo_id, trust_remote_code=True)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    repo_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

image = Image.open("/path/to/your/image.jpg").convert("RGB")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": "Describe the driver state and suggest actions."},
        ],
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt")
inputs = inputs.to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=128)
generated_ids = [
    output_ids[len(input_ids):]
    for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
]
print(processor.batch_decode(generated_ids, skip_special_tokens=False)[0])

文件说明 | File Overview

• model-00001-of-00003.safetensors、model-00002-of-00003.safetensors、model-00003-of-00003.safetensors、model.safetensors.index.json：模型权重与索引。

• config.json：模型结构配置，基座为 Qwen2.5-VL-7B-Instruct。

• quantize_config.json：GPTQ INT8 量化配置。

• preprocessor_config.json：视觉预处理配置。

• chat_template.json：对话模板。

• model-00001-of-00003.safetensors, model-00002-of-00003.safetensors, model-00003-of-00003.safetensors, and model.safetensors.index.json: model shards and index.

• config.json: model architecture config based on Qwen2.5-VL-7B-Instruct.

• quantize_config.json: GPTQ INT8 quantization settings.

• preprocessor_config.json: visual preprocessing config.

• chat_template.json: chat template used for prompt construction.

资源需求 | Requirements

建议使用支持 GPTQ 的推理环境，显存建议 16 GB 以上，实际需求受输入分辨率和生成长度影响。

Use a GPTQ-capable inference stack. A GPU with at least 16 GB VRAM is recommended, although actual usage depends on image resolution and generation length.

注意事项 | Notes

模型仅用于研究与演示。若要部署到真实驾驶环境，请补充安全评估、异常回退和权限控制。

This model is intended for research and demo use only. For real driving deployment, add safety validation, fallback handling, and permission control.