Sync model repo (text/metadata)

.python-version

@@ -0,0 +1 @@
+3.13

README.md

@@ -0,0 +1,193 @@
+---
+license: mit
+pipeline_tag: text-generation
+library_name: llama.cpp
+tags:
+  - arm
+  - arm-optimized
+  - premium-smartphone
+  - llamacpp
+  - gguf
+base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
+base_model_relation: quantized
+---
+
+# DeepSeek-R1-Distill-Qwen-1.5B optimized for Arm-based Premium Smartphone
+
+DeepSeek-R1-Distill-Qwen-1.5B text generation, quantized to a Q4_K_M GGUF for the llama.cpp runtime and optimized for Arm-based Premium Smartphone systems.
+
+## Summary
+
+This repository contains an Arm-optimized version of deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B for text generation. The model is provided as a GGUF `.gguf` file for the llama.cpp runtime, targeting Premium Smartphone systems.
+
+The recipe is a mixed llama.cpp k-quant, not a uniform 4-bit one. Weights are 4-bit (Q4_K) for the default tensor type and the token embeddings and 6-bit (Q6_K) for the output weight. Activations are not stored in the file: the llama.cpp CPU kernels dynamically quantize them to 8-bit per block at runtime. Weights are quantized with an importance matrix (imatrix-weighted k-quant), per-group and asymmetric.
+
+This version is intended to demonstrate efficient inference on Arm-based platforms while preserving the original model's intended behavior. Arm has evaluated this model on MATH-500 and measured performance with llama-bench synthetic tokens on a Vivo X300.
+
+The model is reasoning-distilled: the embedded chat template opens the assistant turn with a think tag, the model emits a chain of thought, and the final answer follows the closing think tag. It needs a large generation budget to be useful.
+
+**Key results**
+
+| Area | Result |
+|---|---|
+| Model format | GGUF `.gguf` (llama.cpp) |
+| Target device class | Premium Smartphone |
+| Reference device | Vivo X300 (C1-Ultra, C1-Premium, C1-Pro; android Android 16 / OriginOS 6) |
+| Primary performance result | 41.21 tokens/sec decode, 2655.50 ms prefill |
+| Accuracy result | MATH-500 80.0% |
+| Size / memory result | 1065.56 MB model size, 3.19x smaller than the f16 baseline |
+| Precision | Mixed: 4-bit weights (Q4_K default tensor type and token embeddings), 6-bit output weight (Q6_K); activations dynamically quantized to 8-bit at runtime |
+
+## Original model
+
+| Field | Value |
+|---|---|
+| Original model | deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B |
+| Original source | Hugging Face |
+| Original developer | DeepSeek AI |
+| Original model card | [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) |
+| Original license | [MIT](https://opensource.org/licenses/MIT) |
+
+## Model files
+
+| File | Description |
+|---|---|
+| `deepseek-ai__DeepSeek-R1-Distill-Qwen-1.5B_llamacpp_Q4_K_M.gguf` | Arm-optimized model for deployment |
+| `example.py` | Minimal inference example |
+| `pyproject.toml` | Pinned runtime dependencies for `example.py`, resolved with uv |
+| `uv.lock` | Locked dependency resolution for `pyproject.toml` |
+| `config.yaml` | Model I/O contract used by the example |
+| `benchmarks/` | F16 baseline and Arm-optimized benchmark records |
+
+## Performance
+
+Performance was measured on the reference configuration below. Results are intended to make the optimization reproducible but do not guarantee identical performance on every Arm-based system.
+
+**Reference configuration**
+
+| Field | Value |
+|---|---|
+| Device / platform | Vivo X300 |
+| CPU / accelerator | C1-Ultra, C1-Premium, C1-Pro (8 cores), cpu |
+| OS | android — Android 16 / OriginOS 6 |
+| Runtime | llama.cpp |
+| Backend / delegate | cpu, ggml-cpu-repack |
+| Batch size | 1 |
+| Precision | Mixed: 4-bit weights (Q4_K default tensor type and token embeddings), 6-bit output weight (Q6_K); activations dynamically quantized to 8-bit at runtime; imatrix-weighted k-quant, per-group, asymmetric |
+| Runs | 5 measured |
+
+**Performance results**
+
+| Metric | Original / baseline | Arm-optimized | Improvement |
+|---|---:|---:|---:|
+| Decode throughput | 10.01 tokens/sec | 41.21 tokens/sec | 4.12x |
+| Prefill time | 19284.20 ms | 2655.50 ms | 7.26x faster |
+| p50 latency | 32073.58 ms | 5761.18 ms | 5.57x faster |
+| Model size | 3395.48 MB | 1065.56 MB | 3.19x smaller |
+| Peak memory | 3529.36 MB | 2012.06 MB | 1.75x less |
+
+Latency here is the arithmetic consequence of the two measured llama-bench throughputs at the benched shape (512-token prompt, 128 generated tokens), not a directly timed wall-clock figure. Peak memory is Android USS (private clean plus private dirty), not peak RSS.
+
+## Accuracy
+
+Accuracy was evaluated using the same preprocessing, input resolution, and evaluation protocol described below. Where possible, the optimized model is compared against the original model under the same evaluation conditions.
+
+**Evaluation setup**
+
+| Field | Value |
+|---|---|
+| Dataset | MATH-500 |
+| Split | 0-shot |
+| Number of samples | 250 |
+| Metric(s) | MATH-500 accuracy (pass@1) |
+| Evaluation runtime | llama.cpp |
+
+The accuracy run was executed on AWS Graviton4, not on the Vivo X300 used for the performance figures above; quality was never scored on device. Quantization is deterministic, so the accuracy figure carries across hosts, but the two blocks describe the same artifact measured on two different machines.
+
+**Accuracy results**
+
+| Metric | Original / baseline | Arm-optimized | Change |
+|---|---:|---:|---:|
+| MATH-500 accuracy | 82.8% | 80.0% | -2.80 pp |
+
+Accuracy was measured using the evaluation setup described above. Users should re-evaluate the model on their own data before production use.
+
+## Arm optimization approach
+
+Arm optimized this model for efficient inference on Arm-based platforms using a hardware-aware conversion and validation flow.
+
+For this release, Arm used:
+
+| Optimization area | Applied? | Notes |
+|---|---|---|
+| Model conversion | Yes | Converted to GGUF `.gguf` for llama.cpp |
+| Quantization | Yes | Imatrix-weighted llama.cpp k-quant, per-group and asymmetric. Mixed weight precision: 4-bit (Q4_K) default tensor type and token embeddings, 6-bit (Q6_K) output weight; activations dynamically quantized to 8-bit at runtime by the CPU kernels. Importance matrix calibrated on 32 randomly selected WikiText-2 samples |
+| Runtime/backend selection | Yes | llama.cpp CPU backend with ggml-cpu-repack, built with -march=armv8.6-a+dotprod+i8mm |
+| Graph/runtime compatibility updates | Yes | Performed as part of the llama.cpp GGUF conversion pipeline |
+| Accuracy validation | Yes | Compared against the original model or published baseline |
+| Performance validation | Yes | Measured on the reference Arm platform |
+
+The goal of this process is to improve deployment characteristics such as latency, memory use, model size, and runtime compatibility while preserving the model's intended behavior. Detailed conversion scripts, calibration configuration, or backend-specific implementation details may be provided separately where appropriate.
+
+## Using this model
+
+### Install dependencies
+
+Dependencies are declared in `pyproject.toml`, which ships with this repository. Resolve and install them into a local virtual environment with uv:
+
+```bash
+uv python install
+uv sync --frozen
+```
+
+### Run the example
+
+```bash
+uv run example.py
+```
+
+### Expected input
+
+| Property | Value |
+|---|---|
+| Input shape | [1, T], where T is the runtime sequence length |
+| Input type | int32 token ids |
+| Input range | vocabulary ids, 0 to 151935 |
+| Preprocessing | Apply the deepseek_r1 chat template, then tokenize with the BPE tokenizer embedded in the `.gguf` (add_bos, bos id 151646) |
+
+### Expected output
+
+| Property | Value |
+|---|---|
+| Output shape | Token ids generated one at a time |
+| Output type | int32 token ids, decoded to text by llama.cpp |
+| Postprocessing | Decode with the embedded tokenizer; the final answer is the span after the closing think tag (id 151649); stop on the EOS token (id 151643) or max_tokens |
+
+## Intended use
+
+This model is intended for developers evaluating text-generation workloads on Arm-based platforms. It is suitable as a reference implementation for benchmarking, prototyping, and integration exploration.
+
+## Limitations
+
+- Performance depends on the target device, runtime version, backend/delegate support, memory configuration, and system load.
+- Accuracy was evaluated on MATH-500 (250-problem stratified subset) and may not generalize to all domains.
+- This release preserves the original model's intended task and behavior, but users should validate it for their own application, data, and deployment environment.
+- This repository is not a replacement for the original model documentation.
+
+## Additional notes
+
+- Peak memory is Android USS read from /proc/<pid>/smaps_rollup, not the peak RSS that server-class evaluators report. The two are not interchangeable across targets.
+- Reasoning budget matters. Traces are long and the reported score was obtained with max_tokens 12288 at n_ctx 14336, while the shipped serving window is n_ctx 4096, sized for the phone's KV-cache budget. Smaller budgets truncate traces, and a truncated trace yields no answer at all. DeepSeek advises against a system prompt and against greedy decoding for the R1-distill family.
+
+## About this version
+
+This repository contains a converted version of the deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B model, originally developed by DeepSeek AI.
+Arm has converted the model to enable efficient execution on Arm-based platforms. No changes have been made to the model's intended behavior.
+
+## Original model and documentation
+
+For full details of the original model, please refer to the original [model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B).
+
+## Purpose of this release
+
+This version is provided by Arm as a reference implementation to demonstrate performance on Arm-based systems. It is not a production-ready or supported solution. Users should evaluate the model independently for their use-case. Arm provides no warranties or ongoing support for this version.

benchmarks/deepseek-r1-distill-qwen-1-5b-llamacpp-vivo-x300-fp32.yaml

@@ -0,0 +1,32 @@
+version: 1.2.0
+report_type: llm-generative
+profile: Baseline
+created_at: '2026-08-11T03:16:44Z'
+context:
+  target_ref: vivo-x300
+  runtime:
+    name: llama.cpp
+    execution_backend: cpu
+    config:
+      n_threads: 4
+      optimisations:
+      - ggml-cpu-repack
+      compile_flags:
+      - -march=armv8.6-a+dotprod+i8mm
+      - GGML_CPU_KLEIDIAI=ON
+  dataset:
+    name: llama-bench synthetic tokens
+    sample_count: 5
+    slice: -p 512 -n 128 -r 5 -t 4
+  benchmark:
+    batch_size: 1
+    prompt_length_tokens: 512
+    generation_length_tokens: 128
+    deployment_mode: chat
+    num_runs: 5
+performance:
+  end_to_end_latency_ms:
+    p50: 32073.58
+  prefill_encode_time_ms: 19284.2
+  tokens_per_second: 10.008307
+  peak_memory_mb: 3529.35546875

benchmarks/deepseek-r1-distill-qwen-1-5b-llamacpp-vivo-x300-int4.yaml

@@ -0,0 +1,32 @@
+version: 1.2.0
+report_type: llm-generative
+profile: Arm-Optimized
+created_at: '2026-08-11T03:16:44Z'
+context:
+  target_ref: vivo-x300
+  runtime:
+    name: llama.cpp
+    execution_backend: cpu
+    config:
+      n_threads: 4
+      optimisations:
+      - ggml-cpu-repack
+      compile_flags:
+      - -march=armv8.6-a+dotprod+i8mm
+      - GGML_CPU_KLEIDIAI=ON
+  dataset:
+    name: llama-bench synthetic tokens
+    sample_count: 5
+    slice: -p 512 -n 128 -r 5 -t 4
+  benchmark:
+    batch_size: 1
+    prompt_length_tokens: 512
+    generation_length_tokens: 128
+    deployment_mode: chat
+    num_runs: 5
+performance:
+  end_to_end_latency_ms:
+    p50: 5761.18
+  prefill_encode_time_ms: 2655.5
+  tokens_per_second: 41.214719
+  peak_memory_mb: 2012.0625

config.yaml

@@ -0,0 +1,70 @@
+input:
+  shape: [1, "T"]
+  dtype: int32
+  name: tokens
+  preprocessing:
+    - apply_chat_template: deepseek_r1   # <|begin_of_sentence|>{system}<|User|>{prompt}<|Assistant|><think>\n
+    - tokenize:
+        tokenizer: embedded in the .gguf (tokenizer.ggml.*, BPE / gpt2 model, qwen2 pre-tokenizer)
+        add_bos: true                    # bos id 151646
+  notes: |
+    `T` is the runtime sequence length. The GGUF carries its own vocab, merges
+    and chat template, so no external tokenizer files are needed — llama.cpp
+    tokenizes the prompt string internally. `[1, 4096]` in the run report is the
+    pipeline's nominal export shape, not a fixed graph input: the trained
+    context is 131072 tokens and the served window is whatever `n_ctx` the
+    runtime is started with (4096 in the shipped recipe).
+
+output:
+  format: "Token ids generated one at a time; decoded to text by llama.cpp"
+  postprocessing:
+    decode:
+      tokenizer: embedded in the .gguf
+    reasoning_trace:
+      opening_tag: "<think>"             # id 151648, emitted by the generation prompt
+      closing_tag: "</think>"            # id 151649
+      note: "final answer is the span after </think>"
+    stop_on:
+      - eos_token                        # id 151643 (<|end_of_sentence|>)
+      - max_tokens
+
+generation:
+  # Sampling settings the reported MATH-500 pass@1 was measured under.
+  temperature: 0.6
+  top_p: 0.95
+  max_tokens: 12288
+  # n_ctx below is the shipped serving window, sized for the phone's KV-cache
+  # budget. The MATH-500 evaluation ran at n_ctx 14336 instead — a 12288-token
+  # generation budget does not fit in 4096.
+  n_ctx: 4096
+  n_threads: 4
+
+model:
+  format: gguf
+  architecture: qwen2
+  num_layers: 28
+  hidden_size: 1536
+  feed_forward_length: 8960
+  num_attention_heads: 12
+  num_kv_heads: 2
+  rope_freq_base: 10000.0
+  vocab_size: 151936
+  trained_context_length: 131072
+
+quantization:
+  variant: Q4_K_M                        # general.file_type = 15 (MOSTLY_Q4_K_M)
+  method: llama.cpp k-quant, imatrix-weighted
+  default_tensor_type: Q4_K
+  output_weight_type: Q6_K               # llama-quantize's own Q4_K_M default
+  token_embd_type: Q4_K                  # ditto
+  fp32_tensors: norms and attention biases
+  imatrix_entries: 196
+  imatrix_chunks: 256
+  source_precision: f16
+
+runtime:
+  name: llama.cpp
+  execution_backend: cpu
+  target_hardware: vivo_x300
+  build_flags: -march=armv8.6-a+dotprod+i8mm, GGML_CPU_KLEIDIAI=ON
+  kleidiai_engaged: false                # no Q4_K tensor is eligible; ggml's own repack path runs instead

example.py

@@ -0,0 +1,131 @@
+"""Run one chat turn against the Q4_K_M GGUF of DeepSeek-R1-Distill-Qwen-1.5B with llama.cpp."""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+from llama_cpp import Llama
+
+
+HERE = Path(__file__).resolve().parent
+MODEL_FILENAME = "deepseek-ai__DeepSeek-R1-Distill-Qwen-1.5B_llamacpp_Q4_K_M.gguf"
+
+DEFAULT_PROMPT = (
+    "What is the smallest positive integer that is divisible by every integer "
+    "from 1 to 10? Put your final answer in \\boxed{}."
+)
+
+# The sampling settings the published MATH-500 pass@1 was measured under.
+TEMPERATURE = 0.6
+TOP_P = 0.95
+MAX_TOKENS = 12288
+# N_CTX is the shipped serving window, not the evaluation one: MATH-500 ran at
+# 14336 so the whole 12288-token budget fits. Raise this to reproduce that
+# contract — at 4096 a long trace hits the context wall before max_tokens.
+N_CTX = 4096
+N_THREADS = 4
+
+THINK_CLOSE = "</think>"
+
+
+def resolve_model_path(explicit: str | None) -> Path:
+    """Find the GGUF next to this script, or one directory up."""
+    if explicit:
+        path = Path(explicit).expanduser().resolve()
+        if not path.is_file():
+            raise FileNotFoundError(f"No GGUF at {path}")
+        return path
+
+    for candidate in (HERE / MODEL_FILENAME, HERE.parent / MODEL_FILENAME):
+        if candidate.is_file():
+            return candidate
+
+    raise FileNotFoundError(
+        f"{MODEL_FILENAME} not found in {HERE} or {HERE.parent}. "
+        "Pass --model with an explicit path."
+    )
+
+
+def split_reasoning(text: str) -> tuple[str, str]:
+    """Separate the chain of thought from the final answer.
+
+    Returns ``(reasoning, answer)``. When the trace was cut off before
+    ``</think>`` the whole output is reasoning and the answer is empty — that is
+    a truncation, not a refusal, and it means the token budget was too small.
+    """
+    if THINK_CLOSE in text:
+        reasoning, answer = text.split(THINK_CLOSE, 1)
+        return reasoning.strip(), answer.strip()
+    return text.strip(), ""
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--model", default=None, help="Path to the .gguf file")
+    parser.add_argument("--prompt", default=DEFAULT_PROMPT)
+    parser.add_argument("--max-tokens", type=int, default=MAX_TOKENS)
+    parser.add_argument("--temperature", type=float, default=TEMPERATURE)
+    parser.add_argument("--top-p", type=float, default=TOP_P)
+    parser.add_argument("--n-ctx", type=int, default=N_CTX)
+    parser.add_argument("--threads", type=int, default=N_THREADS)
+    args = parser.parse_args()
+
+    model_path = resolve_model_path(args.model)
+    print(f"Loading {model_path.name} ({model_path.stat().st_size / 1024**2:.2f} MB)")
+
+    llm = Llama(
+        model_path=str(model_path),
+        n_ctx=args.n_ctx,
+        n_threads=args.threads,
+        verbose=False,
+    )
+
+    # create_chat_completion applies the chat template stored in the GGUF, which
+    # is what appends the opening <think> tag that puts the model in reasoning
+    # mode. Hand-building the prompt string without it changes the behaviour.
+    completion = llm.create_chat_completion(
+        messages=[{"role": "user", "content": args.prompt}],
+        temperature=args.temperature,
+        top_p=args.top_p,
+        max_tokens=args.max_tokens,
+    )
+
+    text = completion["choices"][0]["message"]["content"]
+    finish_reason = completion["choices"][0]["finish_reason"]
+    reasoning, answer = split_reasoning(text)
+
+    print(f"\nPrompt: {args.prompt}")
+    print(f"\nReasoning ({len(reasoning)} chars, truncated below):\n{reasoning[:800]}")
+    print(f"\nFinal answer:\n{answer or '(none — the trace was truncated)'}")
+    print(f"\nfinish_reason: {finish_reason}")
+    print(f"tokens: {completion['usage']}")
+
+    output_path = HERE / "generation.json"
+    output_path.write_text(
+        json.dumps(
+            {
+                "model": model_path.name,
+                "prompt": args.prompt,
+                "reasoning": reasoning,
+                "answer": answer,
+                "finish_reason": finish_reason,
+                "usage": completion["usage"],
+                "sampling": {
+                    "temperature": args.temperature,
+                    "top_p": args.top_p,
+                    "max_tokens": args.max_tokens,
+                    "n_ctx": args.n_ctx,
+                    "n_threads": args.threads,
+                },
+            },
+            indent=2,
+        ),
+        encoding="utf-8",
+    )
+    print(f"\nWrote {output_path}")
+
+
+if __name__ == "__main__":
+    main()

metadata.yaml

@@ -0,0 +1,32 @@
+schema_version: 1.0.0
+report_type: llm-generative
+task_type: text-generation
+title: DeepSeek-R1-Distill-Qwen-1.5B optimized for Arm-based Premium Smartphone
+description: DeepSeek-R1-Distill-Qwen-1.5B text generation optimized as a Q4_K_M GGUF model for the llama.cpp runtime on Arm-based
+  Premium Smartphone systems.
+id: Arm/deepseek-r1-distill-qwen-1-5b-q4-k-m-llamacpp-vivo-x300
+filename: deepseek-ai__DeepSeek-R1-Distill-Qwen-1.5B_llamacpp_Q4_K_M.gguf
+base_model_id: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
+vendor: DeepSeek
+base_model_url: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
+profile: Arm-Optimized
+weight_dtype: q4_k_m
+quantization:
+  method: K-quant
+  symmetric: false
+  weight_granularity: per-group
+  variant: Q4_K_M
+  fp32_layers:
+  - attn_norm.weight
+  - ffn_norm.weight
+  - output_norm.weight
+  - attn_q.bias
+  - attn_k.bias
+  - attn_v.bias
+calibration:
+  dataset_name: WikiText-2
+  sample_count: 32
+  selection: random
+model_size_mb: 1065.56
+parameter_count: 1777088000
+format: gguf

pyproject.toml

@@ -0,0 +1,41 @@
+[project]
+name = "deepseek-r1-distill-qwen-1-5b-q4-k-m-llamacpp-vivo-x300-runtime"
+version = "0.1.0"
+requires-python = ">=3.13,<3.14"
+# example.py imports llama_cpp and nothing else outside the standard library.
+# llama-cpp-python publishes no wheels on PyPI, so this builds from the sdist,
+# which vendors llama.cpp and compiles it with scikit-build-core.
+dependencies = [
+    "llama-cpp-python==0.3.35",
+]
+
+[tool.uv]
+package = false
+
+[tool.ai-portal.deployment]
+schema-version = "1"
+runtime = "llama.cpp"
+# Tokens reported by llama_cpp.llama_print_system_info(). REPACK is the
+# ggml-cpu-repack path this deliverable actually runs on; KleidiAI is
+# deliberately absent because no Q4_K tensor is eligible for it
+# (config.yaml: kleidiai_engaged: false). llama_supports_gpu_offload() is False.
+required-capabilities = ["CPU", "REPACK"]
+
+[tool.ai-portal.deployment.ubuntu]
+# Stock Ubuntu 24.04 (noble) arm64 versions. ninja is resolved as a PyPI wheel
+# by scikit-build-core, so it is not an apt requirement; a C++ compiler is not
+# available from PyPI and always is.
+packages = [
+    "cmake=3.28.3-1build7",
+    "g++=4:13.2.0-7ubuntu1",
+]
+
+[tool.ai-portal.deployment.raspbian]
+# Raspbian bookworm armhf versions, confirmed against the archive package index.
+packages = [
+    "cmake=3.25.1-1",
+    "g++=4:12.2.0-3+rpi1",
+]
+
+[tool.ai-portal.deployment.files]
+expected = []

uv.lock

@@ -0,0 +1,105 @@
+version = 1
+revision = 3
+requires-python = "==3.13.*"
+
+[[package]]
+name = "deepseek-r1-distill-qwen-1-5b-q4-k-m-llamacpp-vivo-x300-runtime"
+version = "0.1.0"
+source = { virtual = "." }
+dependencies = [
+    { name = "llama-cpp-python" },
+]
+
+[package.metadata]
+requires-dist = [{ name = "llama-cpp-python", specifier = "==0.3.35" }]
+
+[[package]]
+name = "diskcache"
+version = "5.6.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/3f/21/1c1ffc1a039ddcc459db43cc108658f32c57d271d7289a2794e401d0fdb6/diskcache-5.6.3.tar.gz", hash = "sha256:2c3a3fa2743d8535d832ec61c2054a1641f41775aa7c556758a109941e33e4fc", size = 67916, upload-time = "2023-08-31T06:12:00.316Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/3f/27/4570e78fc0bf5ea0ca45eb1de3818a23787af9b390c0b0a0033a1b8236f9/diskcache-5.6.3-py3-none-any.whl", hash = "sha256:5e31b2d5fbad117cc363ebaf6b689474db18a1f6438bc82358b024abd4c2ca19", size = 45550, upload-time = "2023-08-31T06:11:58.822Z" },
+]
+
+[[package]]
+name = "jinja2"
+version = "3.1.6"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "markupsafe" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" },
+]
+
+[[package]]
+name = "llama-cpp-python"
+version = "0.3.35"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "diskcache" },
+    { name = "jinja2" },
+    { name = "numpy" },
+    { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/1a/71/5ecdad726b87527b361d9d95313c6591f27bb3fad79c9ab524ad4250b4c9/llama_cpp_python-0.3.35.tar.gz", hash = "sha256:1139dbb54509074b70893fab8554e3b079aa9f4d312058ce4018ef0019e3de12", size = 74868054, upload-time = "2026-08-17T09:41:34.939Z" }
+
+[[package]]
+name = "markupsafe"
+version = "3.0.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622, upload-time = "2025-09-27T18:36:41.777Z" },
+    { url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029, upload-time = "2025-09-27T18:36:43.257Z" },
+    { url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374, upload-time = "2025-09-27T18:36:44.508Z" },
+    { url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980, upload-time = "2025-09-27T18:36:45.385Z" },
+    { url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990, upload-time = "2025-09-27T18:36:46.916Z" },
+    { url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784, upload-time = "2025-09-27T18:36:47.884Z" },
+    { url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588, upload-time = "2025-09-27T18:36:48.82Z" },
+    { url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041, upload-time = "2025-09-27T18:36:49.797Z" },
+    { url = "https://files.pythonhosted.org/packages/19/bc/e7140ed90c5d61d77cea142eed9f9c303f4c4806f60a1044c13e3f1471d0/markupsafe-3.0.3-cp313-cp313-win32.whl", hash = "sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed", size = 14543, upload-time = "2025-09-27T18:36:51.584Z" },
+    { url = "https://files.pythonhosted.org/packages/05/73/c4abe620b841b6b791f2edc248f556900667a5a1cf023a6646967ae98335/markupsafe-3.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5", size = 15113, upload-time = "2025-09-27T18:36:52.537Z" },
+    { url = "https://files.pythonhosted.org/packages/f0/3a/fa34a0f7cfef23cf9500d68cb7c32dd64ffd58a12b09225fb03dd37d5b80/markupsafe-3.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485", size = 13911, upload-time = "2025-09-27T18:36:53.513Z" },
+    { url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658, upload-time = "2025-09-27T18:36:54.819Z" },
+    { url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066, upload-time = "2025-09-27T18:36:55.714Z" },
+    { url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639, upload-time = "2025-09-27T18:36:56.908Z" },
+    { url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569, upload-time = "2025-09-27T18:36:57.913Z" },
+    { url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284, upload-time = "2025-09-27T18:36:58.833Z" },
+    { url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801, upload-time = "2025-09-27T18:36:59.739Z" },
+    { url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769, upload-time = "2025-09-27T18:37:00.719Z" },
+    { url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642, upload-time = "2025-09-27T18:37:01.673Z" },
+    { url = "https://files.pythonhosted.org/packages/80/d6/2d1b89f6ca4bff1036499b1e29a1d02d282259f3681540e16563f27ebc23/markupsafe-3.0.3-cp313-cp313t-win32.whl", hash = "sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354", size = 14612, upload-time = "2025-09-27T18:37:02.639Z" },
+    { url = "https://files.pythonhosted.org/packages/2b/98/e48a4bfba0a0ffcf9925fe2d69240bfaa19c6f7507b8cd09c70684a53c1e/markupsafe-3.0.3-cp313-cp313t-win_amd64.whl", hash = "sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218", size = 15200, upload-time = "2025-09-27T18:37:03.582Z" },
+    { url = "https://files.pythonhosted.org/packages/0e/72/e3cc540f351f316e9ed0f092757459afbc595824ca724cbc5a5d4263713f/markupsafe-3.0.3-cp313-cp313t-win_arm64.whl", hash = "sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287", size = 13973, upload-time = "2025-09-27T18:37:04.929Z" },
+]
+
+[[package]]
+name = "numpy"
+version = "2.5.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/9a/80/db0b4559e57ec36362bedbb05530a87fafbcb6067708c946967a41d449e7/numpy-2.5.2.tar.gz", hash = "sha256:d482d171c406ae88c5b19cad3b6a1c4c5209f886ab74bc44c2c865c23f52d860", size = 20773161, upload-time = "2026-08-09T13:48:27.962Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/f5/d2/6b24738a0ef4557d189b150046cd07823c50e4273e8aebd651222e24306f/numpy-2.5.2-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8e4cb9a754c8a0c62eaa88273a5fba3391f4a610d1dee893c0755da31c083f15", size = 16886595, upload-time = "2026-08-09T13:45:27.323Z" },
+    { url = "https://files.pythonhosted.org/packages/65/60/f2d208d366f263f39c6e69ed309290717aab41078b6d04c9be2a84fa2a07/numpy-2.5.2-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:52c808f96484f5571a5cc863775ce50247c17dfb3b0361f8ed6b4b0456f80080", size = 11896845, upload-time = "2026-08-09T13:45:31.638Z" },
+    { url = "https://files.pythonhosted.org/packages/3c/79/81e0bf24f4d020a2b1d5cd297a9f60c3f24eeb116f9bba5870443f7b6a4a/numpy-2.5.2-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:29d81e97f668489cba8ebfd796b9bdd453525d35dd9e162e2daec94bf3fc7740", size = 5343880, upload-time = "2026-08-09T13:45:34.373Z" },
+    { url = "https://files.pythonhosted.org/packages/ba/cc/e3141cf06d1a8a2c7e107543fe1269c1d1af760d4d683c0794a4ee1127c2/numpy-2.5.2-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:afb3f0632d6b2e3ba04dbce8d1e48d321b369138b73830b5ca371a0e8d479d56", size = 6682264, upload-time = "2026-08-09T13:45:36.7Z" },
+    { url = "https://files.pythonhosted.org/packages/29/f1/2a64a307d92c5d98f5255a4014eb43bb6103ee477087b61ecae44a3aa9b9/numpy-2.5.2-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0aadf13b60048d501e05fa699efaf7734e2494f3498a4c2a5521d822640324f3", size = 15609566, upload-time = "2026-08-09T13:45:39.518Z" },
+    { url = "https://files.pythonhosted.org/packages/7b/44/59a1eb68e773c4098d107ef34a0dbdeca501d72ffcfbff9a7707343921ce/numpy-2.5.2-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:29b86ff8a6cc556b47ec6b64b194815cc80e6bf5eedcc6cddfd65318cb0b4eee", size = 16709995, upload-time = "2026-08-09T13:45:43.661Z" },
+    { url = "https://files.pythonhosted.org/packages/8a/4c/3e54d4ddbc359a1295f8b633e8106bcd4d7d4a206e82df051bdfb3058755/numpy-2.5.2-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:6950c4b7dd562453090548ba7f5da7e59f57f85663f15d5dcc60e249192f7e59", size = 16972511, upload-time = "2026-08-09T13:45:47.094Z" },
+    { url = "https://files.pythonhosted.org/packages/f2/9f/02e371638ebf19b66d46231e4be52999e87f32d1961b113bc45656608b22/numpy-2.5.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b9727f472d2f3888053b8a75ab0cb94745a9de224bb5846dbadc0092101bc71d", size = 18465609, upload-time = "2026-08-09T13:45:50.808Z" },
+    { url = "https://files.pythonhosted.org/packages/eb/ae/ad6645abc7a3510fe48e8ea1ab4598166f500057ef4ebf38bfad4f1577de/numpy-2.5.2-cp313-cp313-win32.whl", hash = "sha256:4f9744f9fbdcea0bc552e8f19e1f141f811a3f9bc2be2cc6e86d982cab23e3f4", size = 6070204, upload-time = "2026-08-09T13:45:54.111Z" },
+    { url = "https://files.pythonhosted.org/packages/15/20/f3489f86d81ea460b2bcdceaed094142ca6579f6be0ec527b781d39afe68/numpy-2.5.2-cp313-cp313-win_amd64.whl", hash = "sha256:85aaccb24182c25df891ad0ec333585967e115269d5f1b17f2c9ae005bc96657", size = 12460532, upload-time = "2026-08-09T13:45:57.167Z" },
+    { url = "https://files.pythonhosted.org/packages/d5/21/35b31dde1b283b79de828b80f876afd8c94e28fe1e9c375f89e261cc4c0d/numpy-2.5.2-cp313-cp313-win_arm64.whl", hash = "sha256:bd68ece1553d2023c09a4226d9e41c586ad2d20594d1a456186c33513d2cb3f2", size = 10396725, upload-time = "2026-08-09T13:46:00.478Z" },
+]
+
+[[package]]
+name = "typing-extensions"
+version = "4.16.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/f6/cc/6253133b5bb138fc3306cebfbda2c520f545d36b5be2c7255cc528bb45d6/typing_extensions-4.16.0.tar.gz", hash = "sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5", size = 113555, upload-time = "2026-07-02T08:40:05.92Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl", hash = "sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8", size = 45571, upload-time = "2026-07-02T08:40:04.659Z" },
+]