================================================================================ LFM2 VLM Image Reasoning Test Results Date: 2026-04-14 Image: ~/Pictures/screen2.png Prompt: "What is the direction of the waves in this image?" ================================================================================ -------------------------------------------------------------------------------- TEST 1: BASE MODEL (LFM2.5-VL-450M-F16) -------------------------------------------------------------------------------- Model: ~/llama.cpp/models/LFM2.5-VL-450M-F16.gguf mmproj: ~/llama.cpp/models/mmproj-LFM2.5-VL-450m-F16.gguf Response: The waves in this image are moving downward. This is evident from the prominent wave pattern visible in the center of the image. The wave line extends from the bottom left corner towards the top right, indicating a downward direction. The pattern consists of alternating horizontal and vertical wave shapes, which is typical of ocean waves. The downward movement is a key characteristic of wave motion, suggesting that this is likely a photograph of ocean waves captured in motion. Performance: Prompt: 960.6 t/s | Generation: 304.7 t/s -------------------------------------------------------------------------------- TEST 2: TRAINED MODEL - F16 -------------------------------------------------------------------------------- Model: lfm2.5-VL-450M-lora-vlm_sft-vrsbench_m-all-lr2em05-w0p0-lora_m-20260413_223123-F16.gguf Path: ~/cookbook/examples/satellite-vlm/leap-finetune/outputs/vrsbench-lora-1epoch/ mmproj: ~/llama.cpp/models/mmproj-LFM2.5-VL-450m-F16.gguf Response: Diagonally Performance: Prompt: 913.6 t/s | Generation: 282.5 t/s -------------------------------------------------------------------------------- TEST 3: TRAINED MODEL - Q8_0 -------------------------------------------------------------------------------- Model: lfm2.5-VL-450M-lora-vlm_sft-vrsbench_m-all-lr2em05-w0p0-lora_m-20260413_223123-Q8_0.gguf Path: ~/cookbook/examples/satellite-vlm/leap-finetune/outputs/vrsbench-lora-1epoch/ mmproj: ~/llama.cpp/models/mmproj-LFM2.5-VL-450m-F16.gguf Response: Radial Performance: Prompt: 950.0 t/s | Generation: 322.4 t/s -------------------------------------------------------------------------------- TEST 4: TRAINED MODEL - Q4_K_M -------------------------------------------------------------------------------- Model: lfm2.5-VL-450M-lora-vlm_sft-vrsbench_m-all-lr2em05-w0p0-lora_m-20260413_223123-Q4_K_M.gguf Path: ~/cookbook/examples/satellite-vlm/leap-finetune/outputs/vrsbench-lora-1epoch/ mmproj: ~/llama.cpp/models/mmproj-LFM2.5-VL-450m-F16.gguf Response: Diagonal Performance: Prompt: 829.8 t/s | Generation: 385.4 t/s ================================================================================ SUMMARY ================================================================================ | Model Version | Response | Generation Speed (t/s) | |---------------|-----------------|------------------------| | Base F16 | "downward" | 304.7 | | Trained F16 | "Diagonally" | 282.5 | | Trained Q8_0 | "Radial" | 322.4 | | Trained Q4_K_M| "Diagonal" | 385.4 | Observations: - Base model gives a more detailed explanation - Trained models give shorter, more concise answers - All responses relate to directional patterns in the waves - Quantization doesn't significantly impact response quality for this task ================================================================================ ================================================================================ ADDITIONAL TESTS - IMAGE screen3.png ================================================================================ -------------------------------------------------------------------------------- TEST 1: BASE MODEL (LFM2.5-VL-450M-F16) -------------------------------------------------------------------------------- Image: ~/Pictures/screen3.png Model: ~/llama.cpp/models/LFM2.5-VL-450M-F16.gguf mmproj: ~/llama.cpp/models/mmproj-LFM2.5-VL-450m-F16.gguf Response: The image you've provided appears to be a microscopic view of a tissue sample stained with hematoxylin and eosin (H&E). In this type of staining, hematoxylin stains the nuclei of cells blue, while eosin stains the cytoplasm and extracellular matrix pink. The image shows a pattern of red and blue staining. The red staining, which represents the nuclei, is distributed in a somewhat circular pattern, suggesting a circular arrangement of nuclei. The blue staining, which indicates the cytoplasm and extracellular matrix, is more scattered throughout the image. The direction of the waves in the image is not clearly visible due to the nature of the staining pattern. The red nuclei appear to be oriented in a circular or spiral-like fashion, while the blue staining is more randomly distributed... Performance: Prompt: 1528.6 t/s | Generation: 316.9 t/s -------------------------------------------------------------------------------- TEST 2: TRAINED MODEL - F16 -------------------------------------------------------------------------------- Image: ~/Pictures/screen3.png Model: Trained F16 Response: In the provided image, the red-stained cells appear to be oriented in a direction that resembles a wave pattern. The waves are arranged in a manner that creates a sense of movement across the field of view. This wave-like pattern is typically observed in biological tissues, particularly in regions where there is a concentration of cell nuclei or where there is a significant degree of cellular activity. The arrangement of the red-stained cells in the image, along with the blue-stained areas, suggests that the waves are directed in a direction that could be interpreted as radiating from a central point, with the red cells aligned in the direction of the wave pattern. Performance: Prompt: 1591.2 t/s | Generation: 319.1 t/s -------------------------------------------------------------------------------- TEST 3: TRAINED MODEL - Q8_0 -------------------------------------------------------------------------------- Image: ~/Pictures/screen3.png Model: Trained Q8_0 Response: Vertical Performance: Prompt: 1558.2 t/s | Generation: 334.3 t/s -------------------------------------------------------------------------------- TEST 4: TRAINED MODEL - Q4_K_M -------------------------------------------------------------------------------- Image: ~/Pictures/screen3.png Model: Trained Q4_K_M Response: Horizontal Performance: Prompt: 1402.4 t/s | Generation: 371.7 t/s ================================================================================ ADDITIONAL TESTS - IMAGE screen.png ================================================================================ -------------------------------------------------------------------------------- TEST 1: BASE MODEL (LFM2.5-VL-450M-F16) -------------------------------------------------------------------------------- Image: ~/Pictures/screen.png Model: ~/llama.cpp/models/LFM2.5-VL-450M-F16.gguf mmproj: ~/llama.cpp/models/mmproj-LFM2.5-VL-450m-F16.gguf Response: The waves in this image are moving from the right side to the left. The darker, more textured portion of the wave surface is on the left, while the lighter, more defined areas on the right indicate the direction of the waves. This creates a dynamic visual effect, with the waves appearing to flow from the right to the left of the frame. Performance: Prompt: 1553.2 t/s | Generation: 315.7 t/s -------------------------------------------------------------------------------- TEST 2: TRAINED MODEL - F16 -------------------------------------------------------------------------------- Image: ~/Pictures/screen.png Model: Trained F16 Response: Wavy Performance: Prompt: 1530.4 t/s | Generation: 267.6 t/s -------------------------------------------------------------------------------- TEST 3: TRAINED MODEL - Q8_0 -------------------------------------------------------------------------------- Image: ~/Pictures/screen.png Model: Trained Q8_0 Response: Northeast Performance: Prompt: 1621.8 t/s | Generation: 246.1 t/s -------------------------------------------------------------------------------- TEST 4: TRAINED MODEL - Q4_K_M -------------------------------------------------------------------------------- Image: ~/Pictures/screen.png Model: Trained Q4_K_M Response: Southwest Performance: Prompt: 1448.1 t/s | Generation: 371.8 t/s ================================================================================ COMPLETE SUMMARY ================================================================================ IMAGE: screen2.png | Model Version | Response | Generation Speed (t/s) | |---------------|-----------------|------------------------| | Base F16 | "downward" | 304.7 | | Trained F16 | "Diagonally" | 282.5 | | Trained Q8_0 | "Radial" | 322.4 | | Trained Q4_K_M| "Diagonal" | 385.4 | IMAGE: screen3.png | Model Version | Response | Generation Speed (t/s) | |---------------|---------------------------------------------|------------------------| | Base F16 | "circular/spiral pattern, not clearly visible" | 316.9 | | Trained F16 | "radiating from a central point" | 319.1 | | Trained Q8_0 | "Vertical" | 334.3 | | Trained Q4_K_M| "Horizontal" | 371.7 | IMAGE: screen.png | Model Version | Response | Generation Speed (t/s) | |---------------|----------------------------------|------------------------| | Base F16 | 'right to left' | 315.7 | | Trained F16 | 'Wavy' | 267.6 | | Trained Q8_0 | 'Northeast' | 246.1 | | Trained Q4_K_M| 'Southwest' | 371.8 | ================================================================================ ADDITIONAL TESTS - lfm2.5-vl-450m-train-9k-test.gguf ================================================================================ Model: ~/llama.cpp/lfm2.5-vl-450m-train-9k-test.gguf mmproj: ~/llama.cpp/models/mmproj-LFM2.5-VL-450m-F16.gguf -------------------------------------------------------------------------------- Image: screen2.png -------------------------------------------------------------------------------- Prompt: 'What is the direction of the waves in this image?' Response: diagonal Performance: Prompt: 944.8 t/s | Generation: 272.3 t/s -------------------------------------------------------------------------------- Image: screen3.png -------------------------------------------------------------------------------- Prompt: 'What is the direction of the waves in this image?' Response: The image shows the presence of blue-walled waves, which are directional structures or patterns. These waves are oriented along the longitudinal axis of the tissue, indicating the direction of the wave propagation or movement. The waves can be observed as alternating dark and light bands within the tissue, creating a pattern that runs parallel to the tissue's longitudinal direction. Performance: Prompt: 1603.4 t/s | Generation: 316.7 t/s -------------------------------------------------------------------------------- Image: screen.png -------------------------------------------------------------------------------- Prompt: 'What is the direction of the waves in this image?' Response: horizontally Performance: Prompt: 1525.5 t/s | Generation: 289.8 t/s ================================================================================ SUMMARY - lfm2.5-vl-450m-train-9k-test.gguf ================================================================================ | Image | Response | Speed (t/s) | |-----------|----------------------------------------------------------|-------------| | screen2.png | 'diagonal' | 272.3 | | screen3.png | 'oriented along the longitudinal axis of the tissue' | 316.7 | | screen.png | 'horizontally' | 289.8 | ================================================================================