Sky-Kim

/

fastvlm-0.5b-unity

like 3

Hi shuowu123, I tested it on iOS. It builds, but unfortunately, it hits a memory limit error by exceeding 3GB.
Loading.PreloadManager: EXC_RESOURCE (RESOURCE_TYPE_MEMORY: high watermark memory limit exceeded) (limit=3072 MB)
Extra optimization is needed to fix the memory issues on this platform.

Hi,I have encountered the same problem as you.I am also considering how to optimize it

Hi, FastVLM doesn't seem feasible on iOS at the moment.
I do have SmolVLM2 working on the iOS CPU backend, though.
(it does run on GPUCompute backend as well if quantized to fp16 .sentis)
Would that be a helpful alternative?

Thank you for your suggestion! I have replaced the models with those from SmolVLM2，But meet error like :Generation error: Cannot set input tensor 0 as shapes are not compatible, expected (d0, d1, 3, 512, 512) received (1, 3, 256, 256)
Assertion failure. Value was False

Hi, just changing the model file won't work because of the differences in network layers and tensor shapes.
So, I am making another demo now which runs on the iPhone using both GPUCompute and CPU.
Stay tuned!

Please check this repo:
https://huggingface.co/Sky-Kim/smolvlm2-256m-unity

Hi, I would like to know the differences between the decoderer_model_merged bnb4, fp16, and other models in FastVLM-0.5B-ONNX/onnx? They look relatively small, can I use them?

That means quantized models. I would recommend using decoder_model_merged.onnx, which is for fp32, and you can quantize the model in the Sentis Package.
https://docs.unity3d.com/Packages/com.unity.ai.inference@2.5/manual/quantize-a-model.html

Thank you, I tried smolVLM and it can indeed run on the iOS platform. However, the support for Chinese is not very good. It is estimated that the model training is only for English.

Yes, it is not multilingual model, so it only support English.