Text Generation
PEFT
Safetensors
lora
reinforcement-learning
grpo
tool-use
chemistry
biology
drug-discovery
prime-intellect
verifiers
Instructions to use poolside-laguna-hackathon/protein-ligand-design with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use poolside-laguna-hackathon/protein-ligand-design with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("poolside/Laguna-XS.2") model = PeftModel.from_pretrained(base_model, "poolside-laguna-hackathon/protein-ligand-design") - Notebooks
- Google Colab
- Kaggle
Add science-gym-bio sweep figure + shared run link; balance training-effect section
Browse files- .gitattributes +1 -0
- README.md +25 -19
- science_gym_bio_sweep.png +3 -0
.gitattributes
CHANGED
|
@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
header.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
header.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
science_gym_bio_sweep.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -54,7 +54,8 @@ why the adapter is large (~4.6 GB) despite being rank-16.
|
|
| 54 |
|
| 55 |
## Training
|
| 56 |
|
| 57 |
-
Trained on [Prime Intellect](https://app.primeintellect.ai) Hosted Training
|
|
|
|
| 58 |
|
| 59 |
- **Algorithm:** GRPO
|
| 60 |
- **Reward:** binary final-answer correctness (1.0 correct / 0.0 wrong) β using
|
|
@@ -63,16 +64,21 @@ Trained on [Prime Intellect](https://app.primeintellect.ai) Hosted Training:
|
|
| 63 |
- **Rollouts per example:** 16
|
| 64 |
- **Batch size:** 128
|
| 65 |
- **Max tokens:** 4096, thinking enabled
|
| 66 |
-
- Stopped early once held-out eval
|
| 67 |
|
| 68 |
-
## What training
|
| 69 |
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
|
| 74 |
-
|
| 75 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
|
| 77 |
> *Among C1βC4, find the single candidate passing Veber (`veber_pass == 1`) with
|
| 78 |
> `tpsa β€ 120` and `rotatable_bonds β€ 6`.*
|
|
@@ -82,20 +88,20 @@ first batch (step 0, before any gradient update) was this Veber-filter problem:
|
|
| 82 |
> - **C3 `COc1ccc(CCN(C)C)cc1` β correct answer**
|
| 83 |
> - C4 `NCCCCCCCCCCCCNCC(=O)O`
|
| 84 |
|
| 85 |
-
The base model made
|
| 86 |
-
|
| 87 |
-
|
|
|
|
| 88 |
|
| 89 |
-
|
| 90 |
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
|
|
|
| 95 |
|
| 96 |
-
|
| 97 |
-
adapter teaches the model to stop dithering with tools and actually commit the
|
| 98 |
-
answer the evidence supports.
|
| 99 |
|
| 100 |
## Usage
|
| 101 |
|
|
|
|
| 54 |
|
| 55 |
## Training
|
| 56 |
|
| 57 |
+
Trained on [Prime Intellect](https://app.primeintellect.ai) Hosted Training β
|
| 58 |
+
**[shared run dashboard](https://app.primeintellect.ai/training/shared/buoc18kj8s9odmkdorp0kto6)**:
|
| 59 |
|
| 60 |
- **Algorithm:** GRPO
|
| 61 |
- **Reward:** binary final-answer correctness (1.0 correct / 0.0 wrong) β using
|
|
|
|
| 64 |
- **Rollouts per example:** 16
|
| 65 |
- **Batch size:** 128
|
| 66 |
- **Max tokens:** 4096, thinking enabled
|
| 67 |
+
- Stopped early once held-out eval reached 100% (β30 steps).
|
| 68 |
|
| 69 |
+
## What training changed
|
| 70 |
|
| 71 |
+
RL taught the model to run the **measure-then-commit** loop reliably: gather the
|
| 72 |
+
tool evidence a question needs, then commit a final answer. Held-out eval rose to
|
| 73 |
+
a perfect score and training stayed stable throughout (no collapse).
|
| 74 |
|
| 75 |
+
| | Base model (step 0) | Trained (step 30) |
|
| 76 |
+
|---|---|---|
|
| 77 |
+
| Held-out eval `pass@1` (n=20) | 0.95 (19/20) | **1.00 (20/20)** |
|
| 78 |
+
| Sampled training rollouts | a Veber question lost to "no answer submitted" | **every** sampled rollout submitted a correct answer (reward mean 0.99) |
|
| 79 |
+
|
| 80 |
+
**A concrete win.** The one question the base model got wrong in the first batch
|
| 81 |
+
(step 0, before any gradient update) was this Veber-filter problem:
|
| 82 |
|
| 83 |
> *Among C1βC4, find the single candidate passing Veber (`veber_pass == 1`) with
|
| 84 |
> `tpsa β€ 120` and `rotatable_bonds β€ 6`.*
|
|
|
|
| 88 |
> - **C3 `COc1ccc(CCN(C)C)cc1` β correct answer**
|
| 89 |
> - C4 `NCCCCCCCCCCCCNCC(=O)O`
|
| 90 |
|
| 91 |
+
The base model made 12 tool calls and even measured C3 β seeing it clearly passes
|
| 92 |
+
(TPSA 12.5, 4 rotatable bonds) β but **never called `submit_answer`**, ran out of
|
| 93 |
+
turns, and scored 0. The trained adapter handles exactly this case: it measures
|
| 94 |
+
the candidates and commits the answer the evidence supports.
|
| 95 |
|
| 96 |
+
## Choosing the training recipe
|
| 97 |
|
| 98 |
+
The stable hyperparameters above didn't come for free β they're the output of a
|
| 99 |
+
sweep on a precursor environment, `allan/science-gym-bio`. The lesson: **learning
|
| 100 |
+
rate is the stability lever** (5e-5 peaks then collapses; 1e-5 holds), and larger
|
| 101 |
+
rollout groups (8 β 16) cut GRPO advantage variance. That recipe β **LR 1e-5,
|
| 102 |
+
16 rollouts/example, thinking on** β is what we carried into the protein-ligand run.
|
| 103 |
|
| 104 |
+

|
|
|
|
|
|
|
| 105 |
|
| 106 |
## Usage
|
| 107 |
|
science_gym_bio_sweep.png
ADDED
|
Git LFS Details
|