molequla / README.md

ears: Whisper tiny and base ggml weights beside the eye, with the phone-1 parity note

0c9170d 17 days ago

3.21 kB

license: gpl-3.0
base_model: HuggingFaceTB/SmolVLM2-500M-Video-Instruct
tags:
  - gguf
  - vision-language
  - smolvlm2
  - arianna-method
  - molequla
  - notorch

molequla — weights

Weights that molequla organisms use on the phone. Code lives at github.com/ariannamethod/molequla; the log book there (MOLEQULALOG2.md) carries the measurements quoted below.

ocelli/ — the eye

ocelli is the vision organ: SmolVLM2-500M with the Yent eye LoRA v2 merged (SFT, 2026-05-29), converted to GGUF and run by a C inference path on notorch. It gives an organism a sentence about a camera frame; that sentence enters the colony as the fifth DNA source (world).

file	quant	size	origin
`yent_eye_ours_q4_0.gguf`	Q4_0	232 MB	quantised on phone-1 with notorch `gguf_quantize` from the f16 merge
`yent_eye_ours_q6_k.gguf`	Q6_K	418 MB	same
`yent_eye_ours_q8_0.gguf`	Q8_0	437 MB	same
`yent_eye_smolvlm2_lora_v2_q8_0.gguf`	Q8_0	437 MB	quantised on neo (llama.cpp)
`yent_eye_smolvlm2_lora_v2_mmproj_q8_0.gguf`	Q8_0	109 MB	vision tower + projector, required beside any decoder
`yent_eye_smolvlm2_lora_v2_mmproj_f16.gguf`	f16	199 MB	vision tower + projector

SHA256SUMS.txt in the folder lists every file; the f16 decoder merge (820 MB) is not uploaded here.

Measured on phone-1 (Galaxy A56, Exynos 1580, cores 4-7, OpenBLAS)

One global 512×512 frame (SMOLVLM_NOSPLIT=1), 84-token prompt, from MOLEQULALOG2.md (2026-09-13):

decoder	gen tok/s	wall	peak RSS
`yent_eye_ours_q6_k`	4.6	11.4 s	951 MB
`yent_eye_smolvlm2_lora_v2_q8_0`	6.4	13.5 s	988 MB
f16 (not uploaded)	1.4	31.2 s	1602 MB

Tiled prompts (13 frames, 878 tokens) cost 74-94 s per frame on this phone at equal or worse content, so the eye runs on one global frame. The Q4_0 decoder generated slower than Q8_0 on the tiled run (3.0 vs 5.0 tok/s) and dropped objects from the description; Q6_K and Q8_0 reproduced the same sentence.

Coming

Other VLMs as alternative eyes for the same organ are planned; they will land in sibling folders with the same measurement table.

ears/ — the hearing

ears is whisper on notorch: a C organ that loads OpenAI Whisper weights in ggerganov's flat ggml format and transcribes 16 kHz mono wav with greedy decoding. Its gate is parity with whisper.cpp: on phone-1 the log-mel is bit-identical, and the transcripts of jfk.wav and of the same sentence re-recorded through the phone microphone match token for token on both models below. Encoder outputs differ by up to 4.9e-2 (mean 2.2e-4) because whisper.cpp keeps attention in f16.

file	params	size	origin
`ggml-tiny.bin`	39M, multilingual	78 MB	OpenAI Whisper tiny, ggml conversion from whisper.cpp `models/download-ggml-model.sh`
`ggml-base.bin`	74M, multilingual	148 MB	OpenAI Whisper base, same

Measured on phone-1, cores 4-7, four threads, jfk.wav: ears tiny 4.2 s wall, 131 MB peak RSS. Whisper weights are MIT-licensed by OpenAI; SHA256SUMS.txt in the folder lists both files.