i tried on a macmini m4 32G , but i meet something wrong ,could someone help ?
#4
by mktime - opened
I build finished , logs like this below:
~/sources/llama.cpp | pr24975 cmake --build build -j --target llama-mtmd-cli llama-server ok | base py | 18:54:32
[ 0%] Built target llama-common-base
[ 0%] Built target cpp-httplib
[ 3%] Built target ggml-base
[ 3%] Built target ggml-blas
[ 7%] Built target ggml-metal
[ 13%] Built target ggml-cpu
[ 15%] Built target ggml
[ 74%] Built target llama
[ 86%] Built target llama-common
[100%] Built target mtmd
[100%] Built target llama-mtmd-cli
[ 0%] Built target llama-common-base
[ 0%] Built target cpp-httplib
[ 1%] Built target llama-ui-embed
[ 5%] Built target ggml-base
[ 5%] Provisioning UI assets
-- UI: downloading from b9771: https://huggingface.co/buckets/llama-ui/resolve/b9771/dist.tar.gz
[ 5%] Built target ggml-blas
[ 9%] Built target ggml-metal
[ 14%] Built target ggml-cpu
[ 16%] Built target ggml
[ 70%] Built target llama
[ 81%] Built target llama-common
[ 94%] Built target mtmd
[ 98%] Built target server-context
-- UI: download dist.tar.gz from b9771 failed: "HTTP response code said error"
-- UI: downloading from latest: https://huggingface.co/buckets/llama-ui/resolve/latest/dist.tar.gz
-- UI: download dist.tar.gz from latest failed: "HTTP response code said error"
-- UI: HF download failed
CMake Warning at /Users/apple/sources/llama.cpp/scripts/ui-assets.cmake:353 (message):
UI: no assets available - building without an embedded UI. In a
disconnected environment, download the pre-built UI from a llama.cpp
release at https://github.com/ggml-org/llama.cpp/releases and extract to
tools/ui/dist.
embed: write output file /Users/apple/sources/llama.cpp/build/tools/ui/ui.h
embed: write output file /Users/apple/sources/llama.cpp/build/tools/ui/ui.cpp
[ 98%] Built target llama-ui-assets
[ 98%] Building CXX object tools/ui/CMakeFiles/llama-ui.dir/ui.cpp.o
[ 98%] Linking CXX static library libllama-ui.a
[ 98%] Built target llama-ui
[ 98%] Building CXX object tools/server/CMakeFiles/llama-server-impl.dir/server-models.cpp.o
[ 98%] Building CXX object tools/server/CMakeFiles/llama-server-impl.dir/server.cpp.o
[ 98%] Building CXX object tools/server/CMakeFiles/llama-server-impl.dir/server-http.cpp.o
[ 98%] Linking CXX shared library ../../bin/libllama-server-impl.dylib
[ 98%] Built target llama-server-impl
[ 98%] Building CXX object tools/server/CMakeFiles/llama-server.dir/main.cpp.o
[100%] Linking CXX executable ../../bin/llama-server
[100%] Built target llama-server
then i run command:
~/sources/llama.cpp | pr24975 ./build/bin/llama-mtmd-cli \ ok | 8s | base py | 18:54:42
-m /Users/apple/Documents/models/Unlimited-OCR-GGUF/Unlimited-OCR-Q4_K_M.gguf \
--mmproj /Users/apple/Documents/models/Unlimited-OCR-GGUF/mmproj-Unlimited-OCR-F16.gguf \
--image /Users/apple/Downloads/奥赛数学题.png \
-p "<|grounding|>Convert the document to markdown." \
--temp 0
it ouptuts:
~/sources/llama.cpp | pr24975 ./build/bin/llama-mtmd-cli \ ok | 8s | base py | 18:54:42
-m /Users/apple/Documents/models/Unlimited-OCR-GGUF/Unlimited-OCR-Q4_K_M.gguf \
--mmproj /Users/apple/Documents/models/Unlimited-OCR-GGUF/mmproj-Unlimited-OCR-F16.gguf \
--image /Users/apple/Downloads/奥赛数学题.png \
-p "<|grounding|>Convert the document to markdown." \
--temp 0
0.06.506.568 I common_init_result: fitting params to device memory ...
0.06.506.572 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.07.793.341 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
WARNING: Using native backtrace. Set GGML_BACKTRACE_LLDB for more info.
WARNING: GGML_BACKTRACE_LLDB may cause native MacOS Terminal.app to crash.
See: https://github.com/ggml-org/llama.cpp/pull/17869
0 libggml-base.0.15.2.dylib 0x0000000100826e74 ggml_print_backtrace + 276
1 libggml-base.0.15.2.dylib 0x000000010083cdc4 _ZL23ggml_uncaught_exceptionv + 12
2 libc++abi.dylib 0x0000000186310698 _ZSt11__terminatePFvvE + 16
3 libc++abi.dylib 0x0000000186313c30 __cxa_get_exception_ptr + 0
4 libc++abi.dylib 0x0000000186313bd8 _ZN10__cxxabiv1L12failed_throwEPNS_15__cxa_exceptionE + 0
5 libllama-common.0.0.9771.dylib 0x0000000100dd8aec _Z27common_chat_templates_applyPK21common_chat_templatesRK28common_chat_templates_inputs + 2172
6 libllama-common.0.0.9771.dylib 0x0000000100dddc84 _Z26common_chat_format_examplePK21common_chat_templatesbRKNSt3__13mapINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEES9_NS2_4lessIS9_EENS7_INS2_4pairIKS9_S9_EEEEEE + 648
7 llama-mtmd-cli 0x000000010041c63c _ZN16mtmd_cli_contextC2ER13common_params + 516
8 llama-mtmd-cli 0x0000000100419024 main + 208
9 dyld 0x0000000185fbab98 start + 6076
libc++abi: terminating due to uncaught exception of type std::runtime_error: this custom template is not supported, try using --jinja
[1] 9415 abort ./build/bin/llama-mtmd-cli -m --mmproj --image -p --temp 0
the example 奥赛数学题.png is below:
when i tried --jinja flag, it outputs repeat contents:
~/sources/llama.cpp | pr24975 ./build/bin/llama-mtmd-cli \ ABRT | base py | 19:00:11
-m /Users/apple/Documents/models/Unlimited-OCR-GGUF/Unlimited-OCR-Q4_K_M.gguf \
--mmproj /Users/apple/Documents/models/Unlimited-OCR-GGUF/mmproj-Unlimited-OCR-F16.gguf \
--image /Users/apple/Downloads/奥赛数学题.png \
--jinja \
-p "<|grounding|>Convert the document to markdown." \
--temp 0
0.00.052.143 I common_init_result: fitting params to device memory ...
0.00.052.146 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.399.695 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.00.436.680 I mtmd_cli_context: chat template example:
You are a helpful assistantHelloHi thereHow are you?
0.00.944.054 I main: loading model: /Users/apple/Documents/models/Unlimited-OCR-GGUF/Unlimited-OCR-Q4_K_M.gguf
0.00.944.330 W WARN: This is an experimental CLI for testing multimodal capability.
0.00.944.332 W For normal use cases, please use the standard llama-cli
0.00.952.838 I encoding mtmd batch, n_chunks = 1 (done = 1, total = 3)
0.02.950.100 I mtmd batch encoding done in 1997 ms
The following table provides the original text: "1. The English text in the source image is a single, continuous string. It is not a standard character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual representation of any specific character. It is not a visual
I tried it too. unfortunately, the quantization versions aren't cutting it! The Q8_0 failed to OCR a 9-pages pdf properly. I had to give up and download the safetensors and do it from the source directly