Inquiry about TPU support on Pixel 10 (Tensor G5) for Gemma 4 E2B
Hi LiteRT Community team,
First of all, thank you so much for your continuous work and for maintaining this repository!
I recently noticed that the "gemma-4-E2B-it_Google_Tensor_G5.litertlm" file has been uploaded here.
Since the filename specifically mentions "Tensor G5," I was wondering: does this mean it is currently possible to utilize the TPU via LiteRT-LM on the Pixel 10 series?
I have been trying to run Gemma 4 E2B on a Pixel 10, but I was quite surprised to find that the GPU and CPU performance is lower than I initially expected.
I am highly anticipating that leveraging the TPU will significantly improve the inference speed.
If TPU execution is indeed supported for this model on the Pixel 10, could you please share some guidance, examples, or documentation on how to properly set it up and use it?
Thank you in advance for your time and help!
It's great to hear your interests in running Gemma on Pixel 10 TPU
For a general instruction of how to set up LiteRT-LM, see it here: https://developers.google.com/edge/litert/next/litert_lm_npu#tensor
Yes Tensor_G5 is pixel 10. You can download the model and run it on your device. Download the file, put it in a folder, I named mine models, located at /storage/emulated/0/Models, then in Google edge gallery at the top left select the icon, then select models, scroll down to the bottom, select import, then select only the npu as the accelerator and upload the model. Then select ai chat, you should see Gemma4 E2b G5 as an option, select it and try it out.
Thanks for the quick reply.
I'll give it a try!
Hi,
I am testing the Google AI Edge Gallery app on a Pixel 10 Pro via Remote Device Streaming.
App crash on Pixel 10 Pro (Tensor G5) with Gemma-4-E2B - Missing NPU Dispatch Library
I followed the instructions to import the Gemma-4-E2B model tailored for the Google Tensor G5 (gemma-4-E2B-it_Google_Tensor_G5.litertlm) and attempted to run it using the NPU as the accelerator.
However, the app crashes immediately when starting the AI Chat.
Symptom & Logcat Analysis
While the model seems to import and load successfully, the crash occurs when initializing the NPU delegate.
The logcat indicates that the dispatch library for the Tensor G5 NPU cannot be found:
06-15 06:26:47.859 23088 23105 I native : I0000 00:00:1781530007.859187 23105 model_resources_litert_lm.cc:68] litert model size: 1899251616
06-15 06:26:47.863 23088 23105 I tflite : Initialized TensorFlow Lite runtime.
06-15 06:26:47.865 23088 23105 E litert : [litert_dispatch.cc:112] No dispatch library found in /data/app/~~AySovX2HxNrJ5U6SGpouTA==/com.google.ai.edge.
gallery-GZDDTSZ18cF9xUczq21X4A==/lib/arm64
06-15 06:26:47.867 23088 23105 E litert : [dispatch_delegate.cc:115] Failed to initialize Dispatch API: ERROR:
[third_party/odml/litert/litert/runtime/dispatch/dispatch_delegate.cc:176]
06-15 06:26:47.867 23088 23105 E litert : [dispatch_delegate.cc:130] Failed to create a dispatch delegate kernel: No usable Dispatch runtime found
06-15 06:26:47.867 23088 23105 F libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 23105 (DefaultDispatch), pid 23088 (ai.edge.gallery)
Feedback & Questions
I reviewed the documentation regarding LiteRT-LM NPU compilation for Tensor https://developers.google.com/edge/litert/next/litert_lm_npu#tensor, which mentions building the Dispatch API.
However, manually building the shared library ( libLiteRtDispatch_GoogleTensor.so ) and injecting it into the gallery app is extremely difficult for general testing.
I actually attempted this build process earlier but could not successfully complete it.
- Is the Tensor G5 dispatch library supposed to be pre-packaged in the Google AI Edge Gallery app?
- If not, could you provide an updated APK/build of the AI Edge Gallery that already contains the Tensor G5 dispatch libraries?
- Or, is there a simpler way to verify Gemma-4-E2B NPU execution on the Pixel 10 Pro without having to manually build and bundle the dispatch library?
Thank you for your help.
Best regards,
This guy made a fork of edge gallery that should run your g5 models, Google Tensor runtime is packaged in his app - https://github.com/jegly/Box
Thank you!
I've been keeping an eye on that repository.
I'll give it a try.