Add files using upload-large-folder tool

README.md

@@ -0,0 +1,109 @@
+---
+license: cc-by-nc-4.0
+tags:
+  - larql
+  - vindex
+  - mechanistic-interpretability
+  - feature-extraction
+model_name: MedGemma 1.5-4B-IT
+base_model: google/medgemma-1.5-4b-it
+---
+
+# MedGemma 1.5-4B-IT — LarQL Vindex
+
+**Source model**: [google/medgemma-1.5-4b-it](https://huggingface.co/google/medgemma-1.5-4b-it)
+**Vindex short ID**: `75eeb232`
+**Layers**: 34  **Hidden size**: 2560  **Features per layer**: 128
+
+## What This Is
+
+A **LarQL vindex** (vector index) — a compact binary representation of the feature geometry of `google/medgemma-1.5-4b-it`. It contains the top-128 SVD directions of every MLP gate_proj and down_proj matrix in the network, plus token embeddings, layer norms, and vocabulary projection metadata.
+
+## What This Is NOT
+
+This is **not** a model you can run for inference. It has no weights sufficient to generate text. It is a mechanistic interpretability artifact: a feature database for probing, editing, and comparing what `google/medgemma-1.5-4b-it` has learned.
+
+## Universal Constants (Phase 2 Measurements)
+
+Measured via forward-pass hooks on a 256-token factual probe text.
+
+| Constant | Symbol | Value | Interpretation |
+|----------|--------|-------|----------------|
+| FFN Sparsity | C1 | 0.181 | Fraction of near-zero SwiGLU activations |
+| Top-8 Prob Mass | C2 | 0.795 | Probability mass on top-8 output tokens |
+| Gate Coherence | C3 | 0.800 | Mean cosine sim of adjacent gate_proj directions |
+| Layer Temperature | C4 | 1.898 | Mean per-neuron SwiGLU activation variance |
+| Circuit Stages | C5 | 2 | CKA transition count + 1 |
+
+**Notes**: Multimodal PaliGemma 1.5 architecture — text-only probing may explain C4=1.898 anomaly (50× above Gemma4-E2B). C3=0.800 matches Llama-3.1-8B gate coherence pattern, not Qwen3 collapse. Phase 2 C4 should be treated as provisional until vision+text probing is implemented.
+
+## Gate 3 Status (DELETE Patch Test)
+
+PENDING (~7 days).
+
+Gate 3 tests whether a rank-1 ΔW patch to `gate_proj.weight` at the top Paris→capital feature layer suppresses P(Paris) by ≥70% with ≤30% Berlin collateral damage.
+
+## Files
+
+| File | Description |
+|------|-------------|
+| `gate_vectors.bin` | Top-128 SVD directions of gate_proj per layer \[L×F×H, f16\] |
+| `down_features.bin` | Top-128 SVD directions of down_proj per layer \[L×F×H, f16\] |
+| `embeddings.bin` | Token embedding matrix \[V×H, f16\] |
+| `norms.bin` | Layer norm weight vectors |
+| `down_meta.bin` | Per-feature top-k vocabulary projections |
+| `index.json` | Vindex metadata (layers, hidden_size, num_feats, etc.) |
+| `manifest.json` | Build provenance (source SHA, extraction timestamp) |
+| `SHA256SUMS` | File integrity checksums |
+
+## How to Use
+
+```python
+import numpy as np, json
+
+vindex_dir = "path/to/downloaded/vindex"
+
+with open(f"{vindex_dir}/index.json") as f:
+    idx = json.load(f)
+
+L, F, H = idx["num_layers"], idx["num_feats"], idx["hidden_size"]
+V = idx["vocab_size"]
+
+# Load gate feature directions [L, F, H]
+gate = np.frombuffer(
+    open(f"{vindex_dir}/gate_vectors.bin", "rb").read(),
+    dtype=np.float16
+).reshape(L, F, H).astype(np.float32)
+
+# Load embeddings [V, H]
+emb = np.frombuffer(
+    open(f"{vindex_dir}/embeddings.bin", "rb").read(),
+    dtype=np.float16
+).reshape(V, H).astype(np.float32)
+
+# Score a token against all features (cosine similarity)
+emb_n = emb / (np.linalg.norm(emb, axis=1, keepdims=True) + 1e-8)
+gate_n = gate / (np.linalg.norm(gate, axis=2, keepdims=True) + 1e-8)
+
+token_id = 12379  # e.g., " Paris"
+scores = gate_n @ emb_n[token_id]  # [L, F]
+l_max, f_max = np.unravel_index(scores.argmax(), scores.shape)
+print(f"Top feature: layer={l_max}, feature={f_max}, score={scores[l_max, f_max]:.4f}")
+```
+
+## License
+
+CC-BY-NC 4.0 — same terms as the source model. Research use only.
+
+## Citation
+
+If you use this vindex in published work, please cite:
+
+```
+@misc{divinci2026larql,
+  title  = {LarQL Vindex: MedGemma 1.5-4B-IT},
+  author = {Divinci AI},
+  year   = {2026},
+  url    = {https://huggingface.co/Divinci-AI/medgemma-1.5-4b-vindex}
+}
+```

SHA256SUMS

@@ -0,0 +1,9 @@
+24c6e21919c8e5f94125d18fbda6eb3f187cf77040fe5773f6013d03225be05b  README.md
+6f16a515fa02c81a5dad2210bdd6d9ccf84840b6d7c7fd84a7d6ea3bed001143  down_features.bin
+fbbbf54f41beb5f0450b99a8df5516a56a3d5f94be10625b70ba56a5f491d5ac  down_meta.bin
+278ab70f2bbd880bd6300d73bb2f8c161dd531fc6df3b3490348c72dacf991de  embeddings.bin
+6e4c8dc7eb32239504ecdc90455e979e193655d2aaef298fe39362da20e7b449  gate_vectors.bin
+9d0d16b4ac917a7a7d54fa4406bed3fe689dcc059278e351006c3ac22db868ed  index.json
+5450d33c20f16a92c13353e8a841c5a5268412b72411fb99f36714c5053bed8a  manifest.json
+17e5161c2a98abe0a66748aa9d256972fec6f049cf8b9db1c3962fedcd942835  norms.bin
+4f669d46559ca1da48b4f2414927d85ad19af0054791a506d84a90271614db87  router_weights.bin

down_features.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f16a515fa02c81a5dad2210bdd6d9ccf84840b6d7c7fd84a7d6ea3bed001143
+size 22282240

down_meta.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbbbf54f41beb5f0450b99a8df5516a56a3d5f94be10625b70ba56a5f491d5ac
+size 383128

embeddings.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:278ab70f2bbd880bd6300d73bb2f8c161dd531fc6df3b3490348c72dacf991de
+size 1342504960

gate_vectors.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e4c8dc7eb32239504ecdc90455e979e193655d2aaef298fe39362da20e7b449
+size 22282240

index.json

@@ -0,0 +1,327 @@
+{
+  "version": 1,
+  "model": "google/medgemma-1.5-4b-it",
+  "family": "gemma3",
+  "source": {
+    "huggingface_repo": "/home/ubuntu/hf/hub/models--google--medgemma-1.5-4b-it/snapshots/91850547d9f0b2fdd21aa7c5f4f3d1a8a52c243b",
+    "huggingface_revision": null,
+    "safetensors_sha256": null,
+    "extracted_at": "2026-04-21T09:08:17.402243Z",
+    "larql_version": "0.2.0-python"
+  },
+  "num_layers": 34,
+  "hidden_size": 2560,
+  "intermediate_size": 2560,
+  "vocab_size": 256000,
+  "embed_scale": 1.0,
+  "extract_level": "features",
+  "dtype": "f16",
+  "has_model_weights": false,
+  "down_top_k": 10,
+  "layer_bands": {
+    "syntax": [
+      0,
+      8
+    ],
+    "knowledge": [
+      8,
+      25
+    ],
+    "output": [
+      25,
+      33
+    ]
+  },
+  "layers": [
+    {
+      "layer": 0,
+      "num_features": 128,
+      "offset": 0,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 1,
+      "num_features": 128,
+      "offset": 655360,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 2,
+      "num_features": 128,
+      "offset": 1310720,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 3,
+      "num_features": 128,
+      "offset": 1966080,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 4,
+      "num_features": 128,
+      "offset": 2621440,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 5,
+      "num_features": 128,
+      "offset": 3276800,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 6,
+      "num_features": 128,
+      "offset": 3932160,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 7,
+      "num_features": 128,
+      "offset": 4587520,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 8,
+      "num_features": 128,
+      "offset": 5242880,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 9,
+      "num_features": 128,
+      "offset": 5898240,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 10,
+      "num_features": 128,
+      "offset": 6553600,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 11,
+      "num_features": 128,
+      "offset": 7208960,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 12,
+      "num_features": 128,
+      "offset": 7864320,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 13,
+      "num_features": 128,
+      "offset": 8519680,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 14,
+      "num_features": 128,
+      "offset": 9175040,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 15,
+      "num_features": 128,
+      "offset": 9830400,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 16,
+      "num_features": 128,
+      "offset": 10485760,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 17,
+      "num_features": 128,
+      "offset": 11141120,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 18,
+      "num_features": 128,
+      "offset": 11796480,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 19,
+      "num_features": 128,
+      "offset": 12451840,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 20,
+      "num_features": 128,
+      "offset": 13107200,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 21,
+      "num_features": 128,
+      "offset": 13762560,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 22,
+      "num_features": 128,
+      "offset": 14417920,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 23,
+      "num_features": 128,
+      "offset": 15073280,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 24,
+      "num_features": 128,
+      "offset": 15728640,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 25,
+      "num_features": 128,
+      "offset": 16384000,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 26,
+      "num_features": 128,
+      "offset": 17039360,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 27,
+      "num_features": 128,
+      "offset": 17694720,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 28,
+      "num_features": 128,
+      "offset": 18350080,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 29,
+      "num_features": 128,
+      "offset": 19005440,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 30,
+      "num_features": 128,
+      "offset": 19660800,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 31,
+      "num_features": 128,
+      "offset": 20316160,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 32,
+      "num_features": 128,
+      "offset": 20971520,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 33,
+      "num_features": 128,
+      "offset": 21626880,
+      "length": 655360,
+      "num_experts": null,
+      "num_features_per_expert": null
+    }
+  ],
+  "model_config": {
+    "model_type": "gemma3",
+    "hidden_size": 2560,
+    "moe": {
+      "num_experts": 1,
+      "top_k": 1,
+      "moe_intermediate_size": 2560,
+      "aggregated_features": 128,
+      "aggregation": "router_weighted_svd"
+    }
+  },
+  "checksums": {
+    "gate_vectors.bin": "6e4c8dc7eb32239504ecdc90455e979e193655d2aaef298fe39362da20e7b449",
+    "embeddings.bin": "278ab70f2bbd880bd6300d73bb2f8c161dd531fc6df3b3490348c72dacf991de",
+    "norms.bin": "17e5161c2a98abe0a66748aa9d256972fec6f049cf8b9db1c3962fedcd942835",
+    "down_features.bin": "6f16a515fa02c81a5dad2210bdd6d9ccf84840b6d7c7fd84a7d6ea3bed001143",
+    "down_meta.bin": "fbbbf54f41beb5f0450b99a8df5516a56a3d5f94be10625b70ba56a5f491d5ac"
+  }
+}

manifest.json

@@ -0,0 +1,16 @@
+{
+  "vindexSha256": "75eeb232eacc960fd77a2889ae9534511ad37858955bc40d656b1a69790cf9eb",
+  "shortId": "75eeb232",
+  "baseModel": "google/medgemma-1.5-4b-it",
+  "extractLevel": "features",
+  "f16": true,
+  "totalBytes": 1388328156,
+  "files": {
+    "gate_vectors.bin": "6e4c8dc7eb32239504ecdc90455e979e193655d2aaef298fe39362da20e7b449",
+    "embeddings.bin": "278ab70f2bbd880bd6300d73bb2f8c161dd531fc6df3b3490348c72dacf991de",
+    "norms.bin": "17e5161c2a98abe0a66748aa9d256972fec6f049cf8b9db1c3962fedcd942835",
+    "down_features.bin": "6f16a515fa02c81a5dad2210bdd6d9ccf84840b6d7c7fd84a7d6ea3bed001143",
+    "down_meta.bin": "fbbbf54f41beb5f0450b99a8df5516a56a3d5f94be10625b70ba56a5f491d5ac",
+    "router_weights.bin": "4f669d46559ca1da48b4f2414927d85ad19af0054791a506d84a90271614db87"
+  }
+}

norms.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17e5161c2a98abe0a66748aa9d256972fec6f049cf8b9db1c3962fedcd942835
+size 701440

router_weights.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f669d46559ca1da48b4f2414927d85ad19af0054791a506d84a90271614db87
+size 174148