--- base_model: meta-llama/Llama-3.3-70B-Instruct library_name: peft license: apache-2.0 license_name: llama3.3 license_link: https://www.llama.com/llama3_3/license/ tags: - lora - peft - science - clinical-trials - biostatistics - statistical-power - reasoning - autoscientist-challenge --- # Model Card: Clinical Trial Power Analysis — Llama 3.3 70B LoRA ## Model Details ### Model Description This is a LoRA adapter fine-tuned on top of Llama-3.3-70B-Instruct to perform statistical power classification for clinical and research trial designs. Given a trial's design — test type, sample size, expected effect size, and significance level — the model computes the actual statistical power achieved, classifies the trial as adequately powered or underpowered, and states the minimum sample size that would be needed to reach the conventional 80% power threshold. This targets a well-documented, real problem in scientific credibility: a substantial share of irreproducible findings across psychology, medicine, and other fields trace back to studies that were underpowered from the start — enrolled too few participants to reliably detect the effect they set out to test. Correctly classifying power requires genuine computation (the noncentral-distribution relationship between sample size, effect size, and detectable power), not verbal pattern-matching, which is a documented weakness in how base LLMs reason about statistical adequacy. Training labels were generated by construction from the standard, textbook two-sample power approximation formula (`n_per_group = 2 × ((z_α/2 + z_β) / d)²`, the widely-taught closed-form approximation for two-sided tests at 80% power), so the classification, required sample size, and enrollment gap are all guaranteed internally consistent and independently re-verifiable from the reported numbers — not separately annotated. This formula was additionally sanity-checked against `statsmodels.stats.power` (the standard, peer-vetted Python implementation used in real biostatistics work) before dataset construction. Submitted to the **AutoScientist Challenge** (Science track) by Adaption Labs. - **Developed by:** Grold Otieno Mboya - **Shared by:** Grold Otieno Mboya - **Model type:** LoRA adapter for causal language modeling (statistical power / trial design reasoning) - **Language(s):** English - **License:** Llama 3.3 Community License Agreement (inherited from the base model — see https://www.llama.com/llama3_3/license/) - **Finetuned from model:** `meta-llama/Llama-3.3-70B-Instruct` ### Model Sources | Platform | Dataset | Model | |----------|---------|-------| | Hugging Face | [adaption-clinical-trial-power-analysis](https://huggingface.co/datasets/Gro97/adaption-clinical-trial-power-analysis) | [clinical-trial-power-analysis-llama3-3-70b-lora](https://huggingface.co/Gro97/clinical-trial-power-analysis-llama3-3-70b-lora) | | Kaggle | [adaption-clinical-trial-power-analysis](https://www.kaggle.com/datasets/groldotieno/adaption-clinical-trial-power-analysis) | [clinical-trial-power-analysis-llama3-3-70b-lora](https://www.kaggle.com/models/groldotieno/clinical-trial-power-analysis-llama3-3-70b-lora) | **Live Demo:** [trialpower.adaptionlabs.app](https://trialpower.adaptionlabs.app/) ## Uses ### Direct Use Checking whether a planned or completed trial's sample size is actually sufficient to detect its claimed effect size: useful for grant reviewers and funding committees screening proposals, IRB/ethics committees assessing trial design, journal reviewers evaluating submitted studies, and early-career researchers or PhD candidates building intuition around sample-size decisions before designing a study. ### Downstream Use Could be extended to additional test types beyond the current coverage (e.g. chi-square tests, survival/Cox regression, cluster-randomized or non-inferiority designs), or integrated into grant-submission or IRB-review tooling as an automated pre-screening step. ### Out-of-Scope Use Not a substitute for a full consultation with a professional biostatistician, particularly for complex designs (multi-arm, adaptive, cluster-randomized, or non-inferiority trials) that the closed-form approximation used here does not model. Not evaluated for general capability, factuality, or safety beyond the Science task category described here. The underlying formula is a standard *approximation*, not the exact noncentral-t solution — adequate for the two-sample/paired/one-sample designs it targets, but not a replacement for exact power software on borderline or unusual cases. ## Bias, Risks, and Limitations The training data combines 15 real, cited rows sourced from actual ClinicalTrials.gov filings (spot-checked for authenticity — e.g. NCT06983743, a real Phase 1 trial of ERAS-0015 in advanced solid tumors, confirmed against independent sources) with 3,100 synthetic rows using invented trial/company names but realistic parameter ranges. All labels — required sample size, enrollment gap, and power classification — were independently re-derived from the raw numbers using the stated formula and confirmed to match exactly across all 3,115 rows, with zero inconsistencies found. **Known class imbalance:** the dataset skews toward adequately-powered examples (2,134 adequately powered vs. 981 underpowered, roughly a 2:1 split) rather than an even balance — worth accounting for when interpreting edge-case performance. **Known approximation limitation:** the closed-form formula used for ground truth is the standard normal-approximation, not the exact noncentral-t calculation; for small sample sizes or extreme effect sizes, professional power-analysis software may compute a slightly different value than this model reports. **Known validation gap:** authenticity of the 15 real anchor rows was spot-checked for trial identity and design details, but exact enrollment figures were not independently re-confirmed against the live registry for every row. Performance has only been measured against Adaption Labs' own evaluation sets (see Results) — not yet tested against a larger, independent sample of real trial registry data. ### Recommendations Treat outputs as a rapid pre-screening signal, not a final determination — borderline classifications, complex trial designs, or high-stakes funding/publication decisions should be confirmed with exact power-analysis software or a professional biostatistician. ## How to Get Started with the Model ```python from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel base_model_id = "meta-llama/Llama-3.3-70B-Instruct" adapter_id = "Gro97/clinical-trial-power-analysis-llama3-3-70b-lora" tokenizer = AutoTokenizer.from_pretrained(base_model_id) base_model = AutoModelForCausalLM.from_pretrained(base_model_id, device_map="auto") model = PeftModel.from_pretrained(base_model, adapter_id) # NOTE: reconstructed to match the verified dataset schema and formula -- # check against the actual "system" field in your training file before # relying on this verbatim, since it was not authored in this session. system_prompt = ( "You are an expert biostatistician evaluating clinical trial designs. " "Given a trial's test type, sample size, expected effect size, and alpha " "level, compute the statistical power achieved, classify the trial as " "adequately_powered or underpowered (80% power is the standard threshold), " "and state the minimum sample size required for 80% power. Respond only " "with strict JSON with exactly these keys: trial_id, condition, phase, " "actual_enrollment, assumed_effect_size_label, assumed_cohens_d, " "required_n_for_80_percent_power, power_classification, enrollment_gap, " "interpretation." ) user_prompt = ( "Trial NCT-EXAMPLE-001, Phase 2, studying a novel treatment for a " "chronic condition. Actual enrollment: 120 participants. Assumed " "effect size: medium (Cohen's d = 0.5)." ) messages = [ {"role": "system", "content": system_prompt}, {"role": "user", "content": user_prompt}, ] inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device) output = model.generate(inputs, max_new_tokens=400) print(tokenizer.decode(output[0], skip_special_tokens=True)) ``` ## Training Details ### Training Data Custom-built dataset **`clinical_trial_power_analysis`**: 3,115 rows. | Component | Rows | Description | |---|---|---| | Real (cited) | 15 | Sourced from actual ClinicalTrials.gov filings, spot-checked for authenticity | | Synthetic (realistic) | 3,100 | Invented trial identifiers, realistic parameter ranges | Phase coverage spans Phase 1, Phase 1/Phase 2, Phase 2, Phase 3, Phase 4, and a small number of observational designs. Effect sizes are roughly evenly split across small/medium/large (Cohen's d conventions). Power classification skews toward adequately-powered (2,134) over underpowered (981) — see Limitations above. Every row's `required_n_for_80_percent_power`, `enrollment_gap`, and `power_classification` were independently recomputed from `assumed_cohens_d` and `actual_enrollment` using the stated formula and confirmed to match exactly, with zero inconsistencies across all 3,115 rows. - Hugging Face: https://huggingface.co/datasets/Gro97/adaption-clinical-trial-power-analysis - Kaggle: https://www.kaggle.com/datasets/groldotieno/adaption-clinical-trial-power-analysis ### Training Procedure Fine-tuned via Adaption Labs' AutoScientist platform using LoRA. #### Training Hyperparameters - **Training regime:** see `adapter_config.json` in this repository for the exact LoRA rank/alpha/target-module configuration used for this run. #### Speeds, Sizes, Times Not independently benchmarked by the author; training was run on Adaption Labs' hosted infrastructure (specific hardware not disclosed to the end user for this run). ## Evaluation ### Testing Data, Factors & Metrics #### Testing Data Two evaluation sets, as reported by Adaption Labs' AutoScientist platform: (1) a held-out split of this project's own dataset, (2) a broader set of unseen tasks from Adaption's internal Science category benchmark. #### Metrics Pairwise preference win rate against the un-adapted base model, as reported by Adaption Labs' AutoScientist platform (judge methodology not disclosed to the end user). ### Results | Evaluation set | Base model win rate | Adapted model win rate | |---|---|---| | This dataset's held-out samples | 23% | **77%** | | Broader Science category | 24% | **76%** | #### Summary The adapter shows a large improvement over the base model both on its own training distribution (77% win rate) and on the broader Science category benchmark (76% win rate) — the two numbers being close suggests the gains generalize well beyond the specific training distribution, consistent with the model having learned the underlying power-computation skill rather than memorizing dataset-specific patterns. This is the strongest result across this author's AutoScientist submissions, consistent with the hypothesis that tasks requiring genuine numeric computation (rather than pattern recognition of well-known concepts) produce the largest, most defensible base-to-adapted gaps. ## Environmental Impact - **Hardware Type:** Not disclosed by the training platform for this run. - **Hours used:** Not disclosed. - **Cloud Provider:** Adaption Labs' hosted infrastructure. - **Compute Region:** Not disclosed. - **Carbon Emitted:** Not calculated. ## Technical Specifications ### Model Architecture and Objective LoRA adapter applied to `meta-llama/Llama-3.3-70B-Instruct`, trained via supervised fine-tuning to compute and classify statistical power for clinical/research trial designs. ### Compute Infrastructure #### Software - PEFT 0.15.1 - Trained via Adaption Labs' AutoScientist platform ## Citation **BibTeX:** ```bibtex @misc{mboya2026trialpower, author = {Mboya, Grold Otieno}, title = {Clinical Trial Power Analysis: A LoRA Adapter for Statistical Power Classification of Trial Designs}, year = {2026}, howpublished = {AutoScientist Challenge submission, Adaption Labs}, url = {https://huggingface.co/Gro97/clinical-trial-power-analysis-llama3-3-70b-lora} } ``` ## Model Card Contact See author's Hugging Face profile: https://huggingface.co/Gro97 ### Framework versions - PEFT 0.15.1