ToricBLM: Thought Alcoves, Tropical Attention, and Small Models With Serious Geometry

---

---

TLDR: ToricBLM presents a fundamental evolution of the vanilla transformer, with nothing more than a clever new tokenization method that tokenizes edges in graphs along with the vertices, and subsequent orthonormal positional encodings that encode adjacency and other graph structure and properties, systematically extending it into a standardized, universal graph-to-graph function approximator capable of natively reasoning over arbitrary data structures, multimodal inputs, boolean circuits, and complex graph grammars. The framework critically addresses the limitations of current scalar alignment methodologies by rigorously generalizing Anthropic’s one-dimensional "assistant axis" into a multidimensional, geometrically bounded latent space; this structural shift allows for the precise disentanglement of competing scientific constraints within highly optimized, isolated "Thought Alcoves."

By embedding deep algebraic and topological priors into a remarkably compact 100M-parameter biomolecular model—specifically utilizing tropical attention for sparse, auditable inference, toric sidecars for algebraic certificates, and persistent homology to capture invariant structural memory—ToricBLM actively mitigates the representation collapse inherent in standard softmax attention and the brittleness of linear single-path chain-of-thought. Instead, it employs a robust, GFlowNet-guided "Forest-of-Thought" to navigate vast solution spaces, yielding profound, mathematically provable increases in both model expressiveness and out-of-distribution (OOD) generalization capacity. For researchers skeptical of the diminishing returns of brute-force parameter scaling and seeking a rigorous paradigm for machine intelligence, this blog post and its accompanying preprint provide a highly compelling, meticulously engineered blueprint for achieving true universal reasoning in targeted scientific discovery.

---

What would it take for a small biomolecular model to reason like a careful protein engineer, a medicinal chemist, a structural biologist, and a mathematically trained search agent at the same time?

That is the motivating question behind ToricBLM, a proposed 100M-parameter biomolecular language model for de novo design, scientific reasoning, dynamics-aware generation, and compact edge deployment. The short version is simple:

• ToricBLM is a small transformer.
• Its inputs are not only strings, but graphs.
• Its reasoning is not only a chain, but a forest.
• Its steering is not only a vector, but a geometry.
• Its memory is not only retrieval, but structured analogy.

The longer version is the fun part.

ToricBLM takes inspiration from UniGenX-style multimodal scientific generation, TokenGT-style graph tokenization, GraphCG-style disentangled latent bases, GFlowNets, flow matching, tropical geometry, toric geometry, persistent homology, sheaves, and algebraic/combinatorial tools that are usually not invited into machine learning architecture meetings nearly often enough. It is designed to be deceptively small: a model that can run in medical or research settings with modest hardware, while hiding a substantial amount of geometry and training discipline under the hood.

The goal is not to build a giant model that memorizes all of biology. The goal is to build a compact biomolecular reasoning agent that knows how to propose, critique, refine, validate, and remember.

This post explains the main ideas behind ToricBLM in a didactic way. It is meant for ML researchers, Hugging Face builders, computational biologists, medicinal chemists, protein designers, mathematicians, and anyone curious about what happens when we stop treating geometry as decorative and start treating it as an operating system for reasoning.

1. What Is ToricBLM?

ToricBLM is a proposed biomolecular language model with the following core ingredients:

• a decoder-only transformer backbone near 100M parameters,
• TokenGT-style vertex and edge tokens for scientific graphs,
• orthonormal endpoint encodings for graph incidence,
• tropical attention heads for hard support decisions,
• toric sidecars for finite algebraic/geometric certificates,
• GraphCG-style disentangled basis training,
• embedding-space Forest-of-Thought reasoning with GFlowNets,
• analogical memory retrieval over vectors, graphs, topology, and certificates,
• flow-matching heads for continuous structures and dynamics,
• optional UMA/PITA-inspired oracle calibration for force and energy feedback,
• a validation ladder using modern structure, affinity, dynamics, and atomistic models.

The model is called ToricBLM because it is a biomolecular language model whose internal reasoning geometry is organized using tropical and toric ideas. Its intended modalities include:

• proteins,
• antibodies,
• minibinders,
• enzymes,
• RNA,
• DNA,
• small molecules,
• materials,
• protein-ligand complexes,
• functional annotations,
• property annotations,
• assay facts,
• reasoning trajectories,
• structural and dynamical states.

The simplest input record looks like this:

$$r=(\mu ,S,X,A,G,M,T,Y,\mathrm{\Pi })\mathrm{.}r=(\backslash mu,S,X,A,G,M,T,Y,\backslash Pi).$$

Here $\mu \backslash mu$ is the modality, $SS$ is a symbolic stream, $XX$ is optional continuous data such as coordinates or forces, $AA$ is an annotation dictionary, $GG$ is a typed graph, $MM$ is memory, $TT$ is an optional trajectory, $YY$ is the training target, and $\mathrm{\Pi }\backslash Pi$ records provenance, licensing, leakage policy, and safety constraints.

The important point is that ToricBLM does not pretend biomolecules are just text.

A protein is a sequence, yes. But it is also a contact graph, a motif graph, a domain graph, an evolutionary family, an assay object, a structure, a set of dynamical states, and a collection of scientific claims with evidence. A molecule is a SMILES string, yes. But it is also an atom-bond graph, a conformer ensemble, a scaffold, a pharmacophore, a synthesis object, and a property vector. A material is a formula, but also a lattice, a periodic graph, a relaxation problem, and a phase-stability claim.

So ToricBLM uses a unified stream, but augments it with graph tokens. A vertex token might represent a residue, atom, nucleotide, material site, claim, memory item, or reasoning state. An edge token might represent a bond, contact, base pair, coevolution link, retrieval link, edit step, evidence relation, periodic neighbor, or contradiction.

The TokenGT-style representation is:

$$z_{v_i}=e_{\tau (i)}+\varphi (x_i)+2p_i+u_i,z\_\{v\_i\}=e\_\{\backslash tau(i)\}+\backslash phi(x\_i)+2p\_i+u\_i,$$

for a vertex, and

$$z_{e_{ij}^{(r)}}=e_{\tau (r)}+\psi (x_i,x_j,r)+p_i+p_j+{\omega }_r(p_j-p_i)+u_{ij}^{(r)}z\_\{e\_\{ij\}^\{(r)\}\}=e\_\{\backslash tau(r)\}+\backslash psi(x\_i,x\_j,r)+p\_i+p\_j+\backslash omega\_r(p\_j-p\_i)+u\_\{ij\}^\{(r)\}$$

for an edge. The vectors $p_{i}p\_i$ are endpoint identifiers. If they are orthonormal, then incidence is visible by inner product:

$$⟨2p_k,p_i+p_j⟩=2({\mathbf{1}}_{k=i}+{\mathbf{1}}_{k=j})\mathrm{.}\backslash langle\; 2p\_k,p\_i+p\_j\backslash rangle=2(\backslash mathbf\{1\}\_\{k=i\}+\backslash mathbf\{1\}\_\{k=j\}).$$

That little equation is a quiet but important design choice. It means the transformer does not need to infer from scratch which vertices are attached to which edges. The geometry tells it.

Inspirations

ToricBLM borrows from several families of models and methods:

• UniGenX: unified scientific generation over proteins, molecules, materials, structure, and function.
• NatureLM / scientific foundation models: cross-domain scientific tokenization and instruction mixtures.
• TokenGT: pure transformer graph learning by tokenizing both vertices and edges.
• GraphCG: disentangling steerable graph-generation factors.
• GFlowNets: sampling diverse high-reward objects through trajectory-balance training.
• Flow matching: fast continuous generation for coordinates, conformers, dynamics, and structures.
• UMA: a universal model for atoms, useful as a force/energy oracle in late-stage calibration.
• PITA: temperature-annealed force/energy-only sampling ideas that suggest how to use expensive atomistic oracles judiciously.
• Tropical and toric geometry: hard support, polyhedral cells, fans, divisors, cones, and algebraic certificates.
• Persistent homology, sheaves, and combinatorial commutative algebra: compact signatures of shape, overlap consistency, and constraint structure.

The result is not just “a protein model with RAG.” It is a small model trained to move through structured scientific design spaces.

2. Tropical Geometry in One Friendly Page

Tropical geometry begins with a strange but useful arithmetic. In the max-plus tropical semiring, addition is replaced by maximum and multiplication is replaced by ordinary addition:

$$a\oplus b=\max (a,b),a\odot b=a+b\mathrm{.}a\backslash oplus\; b=\backslash max(a,b),\backslash qquad\; a\backslash odot\; b=a+b.$$

At first this looks like a mathematical parlor trick. Then one notices that many neural operations already behave this way at important moments.

When a model chooses the strongest retrieved document, the most likely contact edge, the best reasoning branch, the lowest edit cost, the nearest analog, or the dominant attention support, it is doing something tropical in spirit. It is making a hard support decision.

Softmax attention is diffuse:

$$\mathrm{softmax}(z{)}_i=\frac{\exp (z_i)}{\sum _j\exp (z_j)}\mathrm{.}\backslash operatorname\{softmax\}(z)\_i=\backslash frac\{\backslash exp(z\_i)\}\{\backslash sum\_j\backslash exp(z\_j)\}.$$

Tropical attention is decisive:

$$y_i=\underset{j}{\max }\{s_{ij}+v_j\}\mathrm{.}y\_i=\backslash max\_j\backslash \{s\_\{ij\}+v\_j\backslash \}.$$

Softmax says, “blend many pieces of evidence.” Tropical attention says, “tell me which piece of evidence is active.”

Both are useful. ToricBLM uses both.

The tropical view becomes especially valuable when the model must explain why it believes something. If an enzyme design is scored highly because of a catalytic triad, a substrate pocket, a transition-state analog, and a retrieved homolog, we want to know which supports are active. If a minibinder looks good because of a particular interface patch, we want that patch to be auditable. If a molecule is optimized for affinity but quietly violates ADMET constraints, we want the hard contradiction to be visible.

Tropical heads provide margins. If the active candidate is $j^{\star }j^\backslash star$, define:

$$\gamma =z_{j^{\star }}-\underset{j\mathrm{\ne }{j}^{\star }}{\max }{z}_j\mathrm{.}\backslash gamma=z\_\{j^\backslash star\}-\backslash max\_\{j\backslash neq\; j^\backslash star\}z\_j.$$

If quantization, retrieval approximation, or numerical noise perturbs all scores by at most $ϵ\backslash epsilon$, then the active support stays fixed whenever:

$$\gamma >2ϵ\mathrm{.}\backslash gamma>2\backslash epsilon.$$

This gives a practical engineering metric. A tropical margin is not merely pretty geometry. It tells us when a support decision is stable under compression, pruning, low precision, or approximate retrieval.

3. Why Embed Tropical Geometry Into Toric Geometry?

Tropical geometry gives piecewise-linear structure: maxima, cones, fans, cells, hard supports, shortest paths, and polyhedral decompositions.

Toric geometry gives algebraic and combinatorial tools for working with that structure: toric varieties, divisors, Cox coordinates, one-dimensional cones, monomial maps, fans, initial ideals, semigroups, and compactifications.

The practical reason to embed tropical structure into toric structure is that tropical geometry tells us which support is active, while toric geometry gives us a richer language for organizing, comparing, transporting, and auditing those supports.

For a finite max-linear probe, suppose we have candidates:

$${\mathrm{\ell }}_r(u)=⟨a_r,u⟩+b_r\mathrm{.}\backslash ell\_r(u)=\backslash langle\; a\_r,u\backslash rangle+b\_r.$$

We can form a Laurent polynomial over a non-Archimedean field:

$$f=\sum _{r=1}^R{\tau }^{-b_r}{\chi }^{a_r}\mathrm{.}f=\backslash sum\_\{r=1\}^R\; \backslash tau^\{-b\_r\}\backslash chi^\{a\_r\}.$$

With the right sign convention, the active tropical candidate corresponds to an initial term of $ff$. That gives a bridge:

• neural max support,
• tropical initial form,
• toric/algebraic certificate.

ToricBLM uses finite toric sidecars on small audited windows:

• mutation semigroups,
• reaction-route semigroups,
• residue-contact supports,
• molecular scaffold edits,
• base-pair motif complexes,
• lattice-site moves,
• reasoning-action traces.

The student model receives compact labels:

• cone identifiers,
• Cox-degree summaries,
• active support ids,
• forbidden nonfaces,
• Hilbert-basis move ids,
• Betti vectors,
• rank-defect warnings,
• local cohomology warnings.

This is the central compromise: exact geometry in the teacher stack, compressed geometry in the student.

4. Anthropic’s Assistant Axis: Useful, But Not Enough

Anthropic’s “Assistant Axis” work studies a direction in activation space corresponding to whether a model is operating in its default helpful assistant persona. Their paper, The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models, argues that models have a leading persona direction that can help monitor and stabilize default assistant behavior.

This is important. It says there is structure in activation space. It also says model behavior can drift, and that white-box directions can help detect or constrain that drift.

But one axis is a very small instrument.

A single assistant direction might help answer questions like:

• Is the model acting like its default assistant persona?
• Is it drifting toward unusual roleplay?
• Is it becoming less helpful, less harmless, or more theatrical?

Those are useful questions. They are not enough for scientific design.

ToricBLM needs to ask much harder questions:

• Is this enzyme design preserving its catalytic geometry?
• Is the apparent activity improvement actually coming from a stability artifact?
• Did the binder improve affinity while losing specificity?
• Is this antibody moving into an aggregation-prone alcove?
• Is the model hallucinating an EC annotation because retrieval support is stale?
• Did a low-precision cache flip the active evidence edge?
• Is a reasoning branch improving because of a real analogy or because of superficial token overlap?
• Which local mechanism changed when the design jumped from one solution family to another?

These are not one-axis questions. They are questions about cells, cones, supports, graphs, analogies, topological signatures, and trajectories.

That is why ToricBLM uses a richer geometry. The Assistant Axis is a useful coordinate. It is not a coordinate system.

5. Thought and Reasoning Alcoves

This is the heart of ToricBLM.

A thought alcove is a stable region in the model’s embedding space where the same scientific and reasoning supports remain active.

Informally, a thought alcove is a place where the model is “thinking in the same mode.” Inside one alcove, the same catalytic residue, the same retrieved assay, the same binder hotspot, the same base-pair register, the same material coordination shell, or the same reasoning analogy remains active. Crossing an alcove wall changes the mechanism.

This is different from saying the model has “a direction” for helpfulness or truthfulness. Directions are useful, but they are too poor for the combinatorial richness of scientific reasoning. A model designing an enzyme is not merely moving toward “helpful.” It is moving through a structured design landscape with discrete supports and continuous refinements.

Definition: Thought Alcove

Let $z\in {\mathbb{R}}^{d}z\backslash in\backslash mathbb\{R\}^d$ be a pooled embedding state after GraphCG projection. Let $\mathcal{H}\backslash mathcal\{H\}$ be a hyperplane arrangement induced by:

• active tropical comparisons,
• oracle thresholds,
• GraphCG basis signs,
• toric cone labels,
• validation inequalities,
• safety and provenance constraints.

Write:

$$\mathcal{H}=\{⟨{\alpha }_{\mathrm{\ell }},z⟩+{\beta }_{\mathrm{\ell }}=0:\mathrm{\ell }=1,\dots ,L\}\mathrm{.}\backslash mathcal\{H\}=\backslash \{\backslash langle\; \backslash alpha\_\backslash ell,z\backslash rangle+\backslash beta\_\backslash ell=0:\backslash ell=1,\backslash dots,L\backslash \}.$$

The thought alcoves are connected components of:

$${\mathbb{R}}^d\setminus \bigcup _{\mathrm{\ell }=1}^L{H}_{\mathrm{\ell }}\mathrm{.}\backslash mathbb\{R\}^d\backslash setminus\; \backslash bigcup\_\{\backslash ell=1\}^L\; H\_\backslash ell.$$

In ToricBLM, these cells are refined by a toric fan $\mathrm{\Sigma }\backslash Sigma$, GraphCG basis chambers, and validator states.

A design thought alcove is a tuple:

$$A=(\mathrm{cell},{\sigma }_A,{\rho }_A,{\eta }_A,{\chi }_A)\mathrm{.}A=(\backslash operatorname\{cell\},\backslash sigma\_A,\backslash rho\_A,\backslash eta\_A,\backslash chi\_A).$$

Here:

• ${\sigma }_A\in \mathrm{\Sigma }\backslash sigma\_A\backslash in\backslash Sigma$ is a toric cone,
• ${\rho }_A\backslash rho\_A$ is the vector of active tropical supports,
• ${\eta }_A\backslash eta\_A$ is the GraphCG basis sign pattern,
• ${\chi }_A\backslash chi\_A$ is the validator state.

The alcove is admissible only if chemistry, structure, safety, and provenance constraints are satisfied.

Why Alcoves Matter

Suppose ToricBLM is designing an enzyme. It has a candidate sequence, a predicted fold, a candidate active site, a substrate, a retrieved family, and a catalytic reward. The model may improve the design by changing residues around the pocket. Some mutations preserve the catalytic story. Others break it and create a different mechanism.

Inside one thought alcove, the model can do local optimization:

• improve stability,
• reduce aggregation,
• tune pocket volume,
• improve expression,
• reduce off-target motifs,
• preserve catalytic residues.

Crossing an alcove wall is different. It might switch:

• which residue acts as the acid/base,
• which loop controls specificity,
• which homolog is the main analogy,
• which ligand pose is active,
• which interface patch anchors a minibinder,
• which base-pair register defines an RNA motif.

This gives the model a language for a distinction expert scientists already make intuitively:

Are we improving the same idea, or did we change the idea?

Thought alcoves make that distinction trainable.

Theorem: Alcove-Safe Steering

Let $AA$ be a thought alcove defined by signed inequalities:

$$s_{\mathrm{\ell }}(⟨{\alpha }_{\mathrm{\ell }},z⟩+{\beta }_{\mathrm{\ell }})>0.s\_\backslash ell(\backslash langle\backslash alpha\_\backslash ell,z\backslash rangle+\backslash beta\_\backslash ell)>0.$$

Assume every inequality has margin at least $\gamma >0\backslash gamma>0$ at $zz$. If an update $\mathrm{\Delta }z\backslash Delta\; z$ satisfies:

$$\underset{\mathrm{\ell }}{\max }\mathrm{\mid }⟨{\alpha }_{\mathrm{\ell }},\mathrm{\Delta }z⟩\mathrm{\mid }\le \gamma \mathrm{/}2,\backslash max\_\backslash ell\; |\backslash langle\backslash alpha\_\backslash ell,\backslash Delta\; z\backslash rangle|\backslash leq\; \backslash gamma/2,$$

then $z+\mathrm{\Delta }zz+\backslash Delta\; z$ remains in the same alcove.

Proof. Each signed inequality starts with value at least $\gamma \backslash gamma$. The update changes each signed value by at most $\gamma \mathrm{/}2\backslash gamma/2$. Therefore every signed value remains positive. Since tropical support changes occur only on equality walls, the active supports remain unchanged. $\mathrm{\square }\backslash square$

This theorem is small but powerful. It lets us steer the model while preserving mechanism.

If a latent edit stays inside the alcove, we can say:

• the same evidence edge stayed active,
• the same catalytic support stayed active,
• the same analogy stayed active,
• the same validator state stayed active.

If it crosses a wall, we know something discrete changed. That wall crossing can be allowed, but it should be logged, scored, and justified.

Forest-of-Thought Through Alcoves

ToricBLM does not use only one chain of reasoning. It uses a forest:

$$\mathfrak{F}=({\mathcal{T}}_1,\dots ,{\mathcal{T}}_B)\mathrm{.}\backslash mathfrak\{F\}=(\backslash mathcal\{T\}\_1,\backslash ldots,\backslash mathcal\{T\}\_B).$$

Each tree can specialize:

• catalysis,
• affinity,
• specificity,
• foldability,
• dynamics,
• developability,
• safety,
• evidence.

A node in the forest is:

$$q_{b,t}=(h_{b,t},G_{b,t},M_{b,t},C_{b,t},a_{b,t},{\xi }_{b,t})\mathrm{.}q\_\{b,t\}=(h\_\{b,t\},G\_\{b,t\},M\_\{b,t\},C\_\{b,t\},a\_\{b,t\},\backslash xi\_\{b,t\}).$$

That means the reasoning state includes an embedding, a graph, memory, certificate state, the latest action, and a role.

The GFlowNet reward for a forest trajectory $\mathrm{T}\backslash Tau$ can be written:

The model is not merely rewarded for sounding plausible. It is rewarded for producing designs that survive structure, dynamics, affinity, specificity, novelty, and safety checks.

Theorem: Forest Trajectory Balance

Assume every activated tree and merge edge belongs to an acyclic augmented state graph. If forward and backward policies satisfy trajectory balance:

$$\log Z_{\theta }+\sum _{e\in \mathrm{T}}\log P_F(e)-\sum _{e\in \mathrm{T}}\log P_B(e)-\log R_F(\mathrm{T})=0,\backslash log\; Z\_\backslash theta+\backslash sum\_\{e\backslash in\backslash Tau\}\backslash log\; P\_F(e)-\backslash sum\_\{e\backslash in\backslash Tau\}\backslash log\; P\_B(e)-\backslash log\; R\_F(\backslash Tau)=0,$$

then the terminal probability of a design $yy$ is proportional to the total reward of all forest trajectories ending at $yy$:

$$P(y)=\frac{\sum _{\mathrm{T}:\mathrm{term}(\mathrm{T})=y}{R}_F(\mathrm{T})}{\sum _{y^{\mathrm{\prime }}}\sum _{\mathrm{T}:\mathrm{term}(\mathrm{T})=y^{\mathrm{\prime }}}{R}_F(\mathrm{T})}\mathrm{.}P(y)=\; \backslash frac\{\backslash sum\_\{\backslash Tau:\backslash operatorname\{term\}(\backslash Tau)=y\}R\_F(\backslash Tau)\}\; \{\backslash sum\_\{y\text{'}\}\backslash sum\_\{\backslash Tau:\backslash operatorname\{term\}(\backslash Tau)=y\text{'}\}R\_F(\backslash Tau)\}.$$

Proof. Trajectory balance gives:

$$Z_{\theta }{P}_F(\mathrm{T})=R_F(\mathrm{T})P_B(\mathrm{T}\mid y)\mathrm{.}Z\_\backslash theta\; P\_F(\backslash Tau)=R\_F(\backslash Tau)P\_B(\backslash Tau\backslash mid\; y).$$

Sum over all complete forest trajectories ending at $yy$. The backward probabilities over histories ending at $yy$ sum to one. Therefore:

$$Z_{\theta }P(y)=\sum _{\mathrm{T}:\mathrm{term}(\mathrm{T})=y}{R}_F(\mathrm{T})\mathrm{.}Z\_\backslash theta\; P(y)=\backslash sum\_\{\backslash Tau:\backslash operatorname\{term\}(\backslash Tau)=y\}R\_F(\backslash Tau).$$

Normalizing over terminal designs gives the result. $\mathrm{\square }\backslash square$

GraphCG and Steering

GraphCG-style training gives ToricBLM a disentangled basis:

$$D=\left[\begin{array}{c}{d}_1\\ \mathrm{⋮}\\ d_k\end{array}\right]\mathrm{.}D=\backslash begin\{bmatrix\}\; d\_1\backslash \backslash \; \backslash vdots\backslash \backslash \; d\_k\; \backslash end\{bmatrix\}.$$

Each direction can correspond to a factor:

• stability,
• solubility,
• catalytic geometry,
• pocket volume,
• affinity,
• specificity,
• expression,
• aggregation,
• immunogenicity,
• RNA secondary structure,
• molecular polarity,
• material stability.

The basic losses are:

$${\mathcal{L}}_{\text{ortho}}=\mathrm{\parallel }D{D}^{\mathrm{\top }}-I{\mathrm{\parallel }}_F^2,\backslash mathcal\{L\}\_\{\backslash text\{ortho\}\}=\backslash |DD^\backslash top-I\backslash |\_F^2,$$

$${\mathcal{L}}_{\text{sparse}}=\mathrm{\parallel }D{\mathrm{\parallel }}_1,\backslash mathcal\{L\}\_\{\backslash text\{sparse\}\}=\backslash |D\backslash |\_1,$$

$${\mathcal{L}}_{\text{rank}}=\frac{1}{r}\sum _{i=1}^r[\eta -{\sigma }_i(D){]}_{+}^2\mathrm{.}\backslash mathcal\{L\}\_\{\backslash text\{rank\}\}=\backslash frac\{1\}\{r\}\backslash sum\_\{i=1\}^r[\backslash eta-\backslash sigma\_i(D)]\_+^2.$$

Inside an alcove, steering can be:

$$z^{\mathrm{\prime }}={\mathrm{\Pi }}_A^{\psi }(z+\eta D^{\mathrm{\top }}w)\mathrm{.}z\text{'}=\backslash Pi\_A^\backslash psi(z+\backslash eta\; D^\backslash top\; w).$$

Here $ww$ encodes the desired property shift and ${\mathrm{\Pi }}_A^{\psi }\backslash Pi\_A^\backslash psi$ projects the update back into the alcove.

This is where ToricBLM differs sharply from simple activation steering. We are not saying “push the model in the assistant direction.” We are saying:

• choose a cell,
• know its active supports,
• choose disentangled directions,
• steer within margins,
• log wall crossings,
• use validators,
• distill successful trajectories.

That is a much more serious control theory for neural behavior.

A Friendly but Firm Complaint

The current interpretability discourse is still too fond of one-dimensional stories.

One axis for refusal. One vector for truthfulness. One direction for helpfulness. One persona coordinate. These are not useless. They are useful first instruments. But if we are going to train models that reason about medicine, enzymes, molecular interactions, dynamics, assays, and design under uncertainty, one-dimensional steering is not enough.

The field needs more mathematicians in the room.

Not just people borrowing the word “manifold” as a vibe. Actual tropical geometers. Actual toric geometers. People who know what a fan is, what a divisor is, what a Cox ring is, what a Schubert problem feels like, why Young tableaux are combinatorial machines rather than antique notation, and how intersection-theoretic thinking can become a training signal. People who can look at a model’s branching reasoning and ask: what are the cells, what are the walls, what are the multiplicities, what are the degenerations, what are the invariants?

This is not academic ornamentation. It is engineering.

If a model crosses from one reasoning alcove into another, that is a discrete event. If two design paths are equivalent under a semigroup relation, that should be a training constraint. If a family of supports degenerates, that should be measured. If a retrieval graph has persistent holes, that is not just a retrieval bug; it is a topological signal. If a set of constraints has syzygies, those relations should not be thrown away because they look too algebraic for a leaderboard.

Modern ML does not need less abstraction. It needs better abstraction, disciplined enough to compile into losses, features, metrics, audits, and cheap student-side surrogates.

ToricBLM is one attempt at that.

6. Analogical Memory Beyond RAG

The second major idea in ToricBLM is analogical memory.

RAG was a necessary stage in model development. It taught us that models should not rely only on weights. They should retrieve. They should cite. They should ground. But ordinary RAG is a blunt instrument:

• retrieve chunks,
• stuff tokens into context,
• hope the model reads them correctly.

That works for many tasks. It is not enough for deep scientific reasoning.

Scientific analogy is not just “find a nearby paragraph.” It is more like:

• find a protein family with similar active-site geometry,
• find a previous mutation path with the same failure mode,
• find a molecule scaffold whose ADMET tradeoff resembles this one,
• find an antibody CDR loop with analogous interface topology,
• find a material coordination pattern with a matching local lattice environment,
• find a reasoning trajectory whose graph has the same shape.

So ToricBLM memory records are structured packets:

$$M=(e,\mathrm{\Gamma },\beta ,\rho ,\mathrm{\partial },\mu ,\pi )\mathrm{.}M=(e,\backslash Gamma,\backslash beta,\backslash rho,\backslash partial,\backslash mu,\backslash pi).$$

Here:

• $ee$ is an embedding,
• $\mathrm{\Gamma }\backslash Gamma$ is a graph signature,
• $\beta \backslash beta$ is topological data, such as persistent homology,
• $\rho \backslash rho$ is provenance,
• $\mathrm{\partial }\backslash partial$ is a chain or differential summary,
• $\mu \backslash mu$ is multiplicative or cochain data,
• $\pi \backslash pi$ is policy and licensing metadata.

Retrieval becomes:

$$S(q,M)=\alpha \cos (e_q,e_M)+\beta \cos ({\mathrm{\Gamma }}_q,{\mathrm{\Gamma }}_M)+\gamma \cos ({\beta }_q,{\beta }_M)+\delta S_{\text{prov}}-\lambda S_{\text{stale}}\mathrm{.}S(q,M)=\; \backslash alpha\backslash cos(e\_q,e\_M)\; +\backslash beta\backslash cos(\backslash Gamma\_q,\backslash Gamma\_M)\; +\backslash gamma\backslash cos(\backslash beta\_q,\backslash beta\_M)\; +\backslash delta\; S\_\{\backslash text\{prov\}\}\; -\backslash lambda\; S\_\{\backslash text\{stale\}\}.$$

This is a different philosophy from dumping markdown files into a vector database. The model stores high-value reasoning trajectories, graph signatures, topological summaries, and provenance-linked analogies.

Successful Trajectories as Optimization Templates

Analogical memory becomes especially powerful during iterative de novo design.

Suppose ToricBLM is trying to improve a new candidate $x_{\text{new}}x\_\{\backslash text\{new\}\}$ for a collection of properties:

• higher catalytic efficiency,
• better binding affinity,
• improved specificity,
• lower aggregation,
• better solubility,
• lower strain energy,
• improved thermal stability,
• safer off-target profile.

The model may not need to solve the optimization from scratch. It can retrieve a previous Forest-of-Thought trajectory that successfully optimized a different candidate $x_{\text{old}}x\_\{\backslash text\{old\}\}$ for a similar property bundle.

That retrieved trajectory is not copied blindly. It is used as a template, initial bias, or search prior.

Let a successful stored trajectory be:

$${\tau }^{\star }=(q_0,a_0,q_1,a_1,\dots ,q_T),\backslash tau^\backslash star=(q\_0,a\_0,q\_1,a\_1,\backslash ldots,q\_T),$$

where each $q_{t}q\_t$ is a reasoning/design state and each $a_{t}a\_t$ is an action such as mutate, retrieve, fold, dock, relax, verify, reject, or stop. The trajectory also stores:

• which thought alcoves were visited,
• which walls were crossed,
• which GraphCG directions were useful,
• which validators improved,
• which branches failed,
• which evidence sources mattered,
• which final design properties improved.

For a new candidate, ToricBLM retrieves ${\tau }^{\star }\backslash tau^\backslash star$ when a similarity score is high:

$$\mathrm{sim}(x_{\text{new}},{\tau }^{\star })=\alpha S_{\text{embed}}+\beta S_{\text{graph}}+\gamma S_{\text{alcove}}+\delta S_{\text{property}}+\eta S_{\text{validator}}\mathrm{.}\backslash operatorname\{sim\}(x\_\{\backslash text\{new\}\},\backslash tau^\backslash star)\; =\; \backslash alpha\; S\_\{\backslash text\{embed\}\}\; +\backslash beta\; S\_\{\backslash text\{graph\}\}\; +\backslash gamma\; S\_\{\backslash text\{alcove\}\}\; +\backslash delta\; S\_\{\backslash text\{property\}\}\; +\backslash eta\; S\_\{\backslash text\{validator\}\}.$$

The retrieved trajectory then biases the new GFlowNet policy:

$$\log P_F(a\mid q,x_{\text{new}})=\log P_F^0(a\mid q,x_{\text{new}})+{\lambda }_{\tau }{B}_{{\tau }^{\star }}(a,q)\mathrm{.}\backslash log\; P\_F(a\backslash mid\; q,x\_\{\backslash text\{new\}\})\; =\; \backslash log\; P\_F^\{0\}(a\backslash mid\; q,x\_\{\backslash text\{new\}\})\; +\backslash lambda\_\{\backslash tau\}\backslash ,B\_\{\backslash tau^\backslash star\}(a,q).$$

Here $P_F^{0}P\_F^\{0\}$ is the model’s base forward policy, while $B_{{\tau }^{\star }}B\_\{\backslash tau^\backslash star\}$ is a trajectory-template bias. It can encourage actions that were useful in the old optimization, discourage actions that failed, and suggest an order of operations.

For example:

• if an old enzyme optimization succeeded by first repairing a pocket wall and only then tuning active-site electrostatics, the new enzyme search can try that order earlier;
• if a minibinder optimization succeeded by rejecting high-affinity but low-specificity branches, the new binder search can inherit that skepticism;
• if an antibody optimization improved developability by staying inside one CDR-shape alcove while steering solubility, the new antibody search can reuse that pattern;
• if a molecule optimization learned that increasing affinity and lowering logP conflicted in one scaffold family, the new molecule search can test whether the same conflict appears in the new scaffold.

This is a large improvement over ordinary RAG. The model is not retrieving an answer. It is retrieving an optimization strategy.

Non-Orthogonal Property Bundles

The desired property list may or may not be orthogonal in GraphCG space.

If $DD$ is the GraphCG basis and $ww$ is the desired property vector, a steering update might look like:

$$\mathrm{\Delta }z=D^{\mathrm{\top }}w\mathrm{.}\backslash Delta\; z=D^\backslash top\; w.$$

If the relevant directions are orthogonal, life is pleasant:

$$D{D}^{\mathrm{\top }}\approx I\mathrm{.}DD^\backslash top\backslash approx\; I.$$

But biology is rarely that courteous. Catalytic activity may trade off with stability. Affinity may trade off with specificity. Solubility may trade off with membrane permeability. Lower energy may trade off with novelty. A desired property bundle can contain entangled directions:

$$G_D=D{D}^{\mathrm{\top }}\mathrm{\ne }I\mathrm{.}G\_D=DD^\backslash top\backslash neq\; I.$$

ToricBLM should therefore retrieve not only trajectories that optimized one property, but trajectories that navigated similar property conflicts.

The transferable object is not merely a direction $d_{k}d\_k$. It is a local policy:

$${\pi }_{{\tau }^{\star }}:(\text{alcove},\text{property conflict},\text{validator state})⟶\text{next useful action}\mathrm{.}\backslash pi\_\{\backslash tau^\backslash star\}:\; (\backslash text\{alcove\},\backslash text\{property\; conflict\},\backslash text\{validator\; state\})\; \backslash longrightarrow\; \backslash text\{next\; useful\; action\}.$$

This matters. If the old trajectory discovered that improving $p_{1}p\_1$ damages $p_{2}p\_2$ unless the model first crosses a particular alcove wall, that is precious information. The new candidate may live in a different sequence, scaffold, fold, or modality, but the shape of the optimization problem can still transfer.

A Template Is Not a Command

The retrieved FoT collection should bias, not dominate.

ToricBLM uses the memory trajectory as a prior, then lets validators and GFlowNet rewards decide whether it actually transfers. The policy can be written:

$$P_F^{\text{adapted}}(a\mid q)\propto P_F^0(a\mid q)\exp ({\lambda }_{\tau }{B}_{{\tau }^{\star }}(a,q))\exp ({\lambda }_R{R}_{\text{local}}(a,q))\mathrm{.}P\_F^\{\backslash text\{adapted\}\}(a\backslash mid\; q)\; \backslash propto\; P\_F^\{0\}(a\backslash mid\; q)\; \backslash exp(\backslash lambda\_\{\backslash tau\}B\_\{\backslash tau^\backslash star\}(a,q))\; \backslash exp(\backslash lambda\_R\; R\_\{\backslash text\{local\}\}(a,q)).$$

If the template helps, it will generate better branches and receive higher reward. If it misleads, oracle scores, support flips, failed validators, and low local reward will suppress it.

That is the right epistemic attitude for scientific memory:

• analogies are useful,
• analogies are dangerous,
• good models should exploit them,
• good models should also distrust them.

This gives ToricBLM a natural optimization loop:

1. propose a de novo candidate,
2. retrieve successful FoT trajectories with similar property geometry,
3. adapt their action patterns as search priors,
4. steer inside thought alcoves when mechanisms should be preserved,
5. cross walls deliberately when a mechanism change is justified,
6. score with structure, dynamics, energy, evidence, and safety validators,
7. store the new successful or failed trajectory back into memory.

Over time, the memory becomes less like a document store and more like a library of design moves.

From Knowledge Graphs to Graphs of Vectors

Knowledge graphs are useful, but many are still trapped in token-space or symbolic-space thinking. They encode entities and relations, but they often do not encode the geometry of reasoning.

ToricBLM suggests a different object: a graph-of-vectors database.

In a graph-of-vectors database:

• nodes have embeddings,
• edges have embeddings,
• subgraphs have embeddings,
• trajectories have embeddings,
• persistence summaries are attached,
• toric cone codes are attached,
• provenance and license metadata are first-class,
• retrieval can happen at the level of objects, paths, motifs, or reasoning episodes.

A knowledge graph says:

Enzyme A catalyzes reaction B.

A graph-of-vectors memory can say:

This enzyme-reaction-support trajectory has a catalytic geometry analogous to that earlier family, but its pocket-volume alcove differs, and its DMS evidence suggests the stabilizing mutation should come from a different branch.

That is a much richer retrieval object.

Category Theory View

Analogical reasoning can be described categorically without becoming mystical.

Let $\mathcal{C}\backslash mathcal\{C\}$ be a category of scientific objects: proteins, molecules, RNA motifs, materials, assays, or reasoning states. A morphism $f:X\to Yf:X\backslash to\; Y$ is a structure-preserving transformation: mutation, alignment, scaffold edit, fold mapping, reaction mapping, analogy, or evidence transfer.

An analogy is not just a nearest neighbor. It is a partially structure-preserving map:

$$F:{\mathcal{C}}_{\text{source}}\to {\mathcal{C}}_{\text{target}}\mathrm{.}F:\backslash mathcal\{C\}\_\{\backslash text\{source\}\}\backslash to\backslash mathcal\{C\}\_\{\backslash text\{target\}\}.$$

Good analogies preserve what matters:

• contact topology,
• catalytic roles,
• binding geometry,
• property tradeoffs,
• evidence structure,
• uncertainty,
• failure modes.

Bad analogies preserve only superficial text.

Persistent homology adds shape summaries. Sheaves add local-to-global consistency. Toric sidecars add combinatorial certificates. Semigroup relations add equivalence between edit paths. Together, these structures help the model distinguish deep analogy from mere resemblance.

Persistent Homology in Memory

Persistent homology summarizes how connected components, loops, and higher-dimensional holes appear and disappear across a filtration.

For a protein contact map, a filtration might be built from distance thresholds. For a molecule, it might use bond distance, conformer energy, or pharmacophore proximity. For a reasoning graph, it might use confidence, relevance, or time.

The barcode is not the full object. It is a compressed shape signature.

That compression is the point.

ToricBLM does not store every detail of every previous trajectory. It stores the parts likely to matter later:

• embeddings,
• graph signatures,
• persistence vectors,
• cone codes,
• support margins,
• validator states,
• provenance.

This improves efficiency and compression. It also gives the model mathematical priors: shape, relation, support, and trajectory matter.

Why This Beats Ordinary RAG

Ordinary RAG is good when the answer is in a document. ToricBLM-style analogical memory is better when the answer requires remembering a pattern.

Compare:

1. Token RAG

   • efficient for factual lookup,
   • weak at structural analogy,
   • often bloats context,
   • depends heavily on chunk quality.

2. Knowledge graphs

   • good for explicit relations,
   • brittle when relations are incomplete,
   • often symbolic but not geometric,
   • weak at continuous similarity and trajectory memory.

3. Long-context markdown piles

   • easy to build,
   • expensive to read,
   • hard to audit,
   • poor compression of reasoning structure.

4. ToricBLM analogical memory

   • stores high-value trajectories,
   • retrieves by vector, graph, topology, and provenance,
   • uses mathematical priors,
   • supports branch scoring and GFlowNet rewards,
   • compresses evidence into structured memory packets.

The point is not that RAG is obsolete everywhere. The point is that scientific design needs more than retrieval. It needs structured remembrance.

7. Flow Matching, UMA, and PITA-Inspired Calibration

ToricBLM uses flow matching for structures and dynamics. A conditional path can be:

$$x_t=(1-t)x_0+t{x}_1\mathrm{.}x\_t=(1-t)x\_0+tx\_1.$$

The target velocity is:

$$v^{\star }(x_t,t,c)=x_1-x_0\mathrm{.}v^\backslash star(x\_t,t,c)=x\_1-x\_0.$$

The model trains:

$${\mathcal{L}}_{\text{FM}}={\mathbb{E}}_{t,x_0,x_1}\mathrm{\parallel }{v}_{\theta }(x_t,t,c)-v^{\star }(x_t,t,c){\mathrm{\parallel }}_2^2\mathrm{.}\backslash mathcal\{L\}\_\{\backslash text\{FM\}\}\; =\; \backslash mathbb\{E\}\_\{t,x\_0,x\_1\}\; \backslash |v\_\backslash theta(x\_t,t,c)-v^\backslash star(x\_t,t,c)\backslash |\_2^2.$$

This is useful for:

• protein conformers,
• antibody loops,
• minibinder interfaces,
• molecule conformers,
• RNA folds,
• crystal structures,
• local pocket dynamics.

UMA enters later. It is too expensive to call inside every training step, but very useful as an oracle for forces and energies. A candidate state $xx$ under context $cc$ can be scored by:

$$E_U(x,c),F_U(x,c)=-{\mathrm{\nabla }}_x{E}_U(x,c)\mathrm{.}E\_U(x,c),\backslash qquad\; F\_U(x,c)=-\backslash nabla\_xE\_U(x,c).$$

Inspired by PITA, we can use a temperature ladder:

Define an annealed target:

$${\pi }_{\beta }(x\mid c)\propto \exp \{-\beta E_U(x,c)+\beta \lambda U_{\theta }(x,c)-\beta C(x,c)\}\mathrm{.}\backslash pi\_\backslash beta(x\backslash mid\; c)\; \backslash propto\; \backslash exp\backslash \{-\backslash beta\; E\_U(x,c)+\backslash beta\backslash lambda\; U\_\backslash theta(x,c)-\backslash beta\; C(x,c)\backslash \}.$$

At high temperature, the model explores. At lower temperature, it concentrates on low-energy, high-utility candidates. SMC-style reweighting and cached UMA calls let us spend expensive oracle compute only where it matters.

This gives a late-stage training phase:

• sample high-value Forest-of-Thought candidates,
• score or relax them with UMA,
• reweight with a PITA-style annealing schedule,
• distill energy and force heads,
• distill flow endpoints,
• update GFlowNet rewards.

The model does not need UMA at inference. It learns from UMA, then carries a compressed approximation.

8. Datasets and Training

ToricBLM’s training mixture is intentionally curated rather than maximal.

Important sources include:

• AmelieSchreiber/toricgt-curated-splits for graph-reasoning mechanics,
• reviewed UniProt and Swiss-Prot records with rich annotations,
• PDB experimental structures,
• AFDB and selected predicted structures,
• ReactZyme for enzyme-reaction prediction,
• ProteinGym, MaveDB, and BindingGYM for deep mutational scans,
• PDBbind and MISATO-style complex data,
• RNAcentral and Rfam for RNA,
• PubChem, ChEMBL, and GEOM for molecules,
• Materials Project, NOMAD, and OQMD for materials.

The toricgt-curated-splits dataset is used carefully. It is valuable because it stores reasoning as graph JSON with provenance, hashes, split assignments, and quality flags. But it is not treated as a biomedical truth source. Reasoning traces are verifier-gated. Unverified traces can teach graph syntax, but not scientific reward.

The reference run is small:

• 100M dense parameters,
• below 25B training tokens,
• preferred first run around 12-15B packed tokens,
• RTX 4090-class feasibility target,
• larger Soft-MoE and UMA/PITA phases only after the dense model proves itself.

9. Why Small Matters

Small models are not just cheaper. In medicine and biology, small models are often more usable.

A useful biomedical model should be able to run:

• near private patient or lab data,
• in constrained medical settings,
• on modest hardware,
• with auditable outputs,
• with explicit validation ladders,
• without requiring a datacenter for every idea.

ToricBLM aims for a compact model that can still reason over:

• sequence,
• structure,
• dynamics,
• evidence,
• analogy,
• topology,
• geometry,
• force and energy feedback,
• wet-lab validation.

The guiding belief is that quality does not come only from parameter count. It also comes from choosing the right structure.

10. Closing

ToricBLM is a proposal for a different kind of scientific model: small, graph-native, geometry-aware, memory-rich, and trained to reason through design spaces rather than merely complete strings.

The most important idea is the thought alcove.

Thought alcoves give us a language for stable reasoning modes. They let a model improve an idea without silently changing the idea. They let us distinguish local optimization from mechanism switching. They connect activation geometry to tropical supports, toric fans, GraphCG bases, GFlowNet trajectories, and validator states.

That is a lot of machinery. But biology is a lot of machinery too.

If we want models that design enzymes, antibodies, minibinders, molecules, RNA, DNA, and materials with real scientific judgment, we should stop pretending that one axis, one retrieval chunk, or one giant context window is enough.

The future of small scientific models may look less like a bigger autocomplete engine and more like a compact geometric laboratory:

• part transformer,
• part graph learner,
• part dynamical generator,
• part analogical memory,
• part tropical fan,
• part toric audit system,
• part experimental design assistant.

That is the wager of ToricBLM.

And if the model is only 100M parameters, all the better. The elegance is in making the small thing know where it is.

Further Reading

• The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models
• Persona Vectors: Monitoring and Controlling Character Traits in Language Models
• TokenGT: Pure Transformers are Powerful Graph Learners
• Trajectory Balance for GFlowNets
• Flow Matching for Generative Modeling
• FlowDock
• UMA: A Family of Universal Models for Atoms
• PITA: Progressive Inference-Time Annealing of Diffusion Models for Sampling from Boltzmann Densities
• Introduction to Tropical Geometry
• Toric Varieties