README.md

---
license: gemma
language:
- ko
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- awaxis
- think
- gemma
- gemma-4
- gemma4
- reasoning
- distillation
- darwin-derived
- vidraft
- darwin-crossbreed
- ko
- en
base_model:
- TeichAI/gemma-4-31B-it-Claude-Opus-Distill-v2
- google/gemma-4-31B-it
model-index:
- name: AWAXIS-Think-31B
  results:
  - task:
      type: text-generation
      name: GPQA Diamond (20Q greedy, max_new_tokens=4096)
    dataset:
      name: GPQA Diamond (subset n=20, seed=42)
      type: Idavidrein/gpqa
      config: gpqa_diamond
    metrics:
    - type: accuracy
      value: 60.0
      name: accuracy
  - task:
      type: text-generation
      name: CLIcK (Korean cultural-linguistic, n=200, alpha grid best)
    dataset:
      name: CLIcK
      type: EunsuKim/CLIcK
    metrics:
    - type: accuracy
      value: 86.0
      name: accuracy
---

# AWAXIS-Think-31B

## Overview

**AWAXIS-Think-31B** is a 31B-parameter Korean/English reasoning model created through the **[VIDRAFT](https://huggingface.co/VIDraft) Darwin AI Model Breeding/Evolution Platform**. This model was produced using Darwin's proprietary **FFN-crossbreed merge engine (V8)**, which emulates biological crossbreeding between AI models to create offspring with combined strengths of both parents.

> AWAXIS-Think-31B은 **VIDRAFT Darwin AI 모델 교배/진화 플랫폼**을 통해 생성된 31B 파라미터 한국어/영어 추론 모델입니다.

---

## VIDRAFT Darwin Platform

**[VIDRAFT Darwin](https://huggingface.co/VIDraft)**은 AI 모델의 **교배(Crossbreeding)와 진화(Evolution)**를 통해 새로운 고성능 모델을 자동 생성하는 플랫폼입니다. 생물학적 유전 원리에서 영감을 받아, 두 개 이상의 부모 모델에서 각각의 장점을 선택적으로 결합하여 자식 모델을 탄생시킵니다.

### Darwin 교배/진화 핵심 기술

| 기술 | 설명 |
|------|------|
| **FFN Crossbreed Engine (V8)** | 부모 모델의 Feed-Forward Network(FFN) 레이어를 선택적으로 교차 결합하는 핵심 엔진. 어텐션·임베딩은 어머니(Mother)에서, FFN 시그널은 아버지(Father)에서 추출하여 블렌딩 |
| **Smart MRI (Model Resonance Imaging)** | 두 모델 간 레이어별 유사도·호환성을 분석하여 최적 교배 비율(alpha)을 자동 탐색하는 기술 |
| **Alpha Grid Search** | 교배 비율 alpha를 체계적으로 탐색(0.1~0.4)하여 벤치마크 성능이 최대화되는 최적점을 발견 |
| **Multi-Generation Breeding** | 1세대 교배 결과물을 다시 부모로 삼아 2세대, 3세대 교배를 수행하는 다세대 진화 |

### Darwin 교배 프로세스 (이 모델의 생성 과정)

```
[Step 1] 부모 선정 (Parent Selection)
   - Mother: TeichAI/gemma-4-31B-it-Claude-Opus-Distill-v2 (추론 능력 기반)
   - Father: google/gemma-4-31B-it (Gemma-4 원본 FFN 기여)

[Step 2] Smart MRI 호환성 분석
   - 두 모델의 60개 레이어별 FFN 텐서 유사도 스캔
   - 아키텍처 호환성 확인 (동일 Gemma-4 family = 100% 호환)

[Step 3] FFN Crossbreed (교배 실행)
   - 어머니의 어텐션, 임베딩, 라우팅 = 100% 보존
   - 아버지의 FFN (gate_proj, up_proj, down_proj) = alpha 비율로 블렌딩
   - 수식: w_child = w_mother * (1 - alpha) + w_father * alpha

[Step 4] Alpha Grid Search (최적 교배 비율 탐색)
   - alpha = {0.1, 0.2, 0.3, 0.4} 4종 생성
   - CLIcK-50 벤치마크로 각 alpha 평가
   - 최적: alpha = 0.1 (CLIcK-200 = 86.0%)

[Step 5] 검증 및 출시
   - GPQA Diamond, CLIcK 등 벤치마크 검증
   - HuggingFace 모델 허브 공개
```

### 왜 Darwin 교배인가?

기존 모델 합성 방식(단순 가중치 평균, SLERP, TIES 등)과 달리, Darwin 교배는:

1. **생물학적 유전 모방**: 어머니/아버지 역할을 명확히 분리하여 각 부모의 핵심 능력만 선택적으로 상속
2. **FFN 선택적 주입**: 어텐션(문맥 이해)은 어머니에서 100% 보존하고, FFN(지식·추론 패턴)만 아버지에서 교차 → 능력 충돌 최소화
3. **벤치마크 기반 자연선택**: alpha grid search로 여러 자식 후보를 생성한 뒤, 실측 벤치마크로 최적 개체를 선택 (= 자연선택 시뮬레이션)
4. **다세대 진화 가능**: 이 모델(AWAXIS-Think-31B)이 다시 AWAXIS-KR-31B의 아버지가 되어 2세대 교배 수행 → 능력 누적 진화

---

## Build Recipe (Honest Disclosure)

- **Mother (kept full)**: [TeichAI/gemma-4-31B-it-Claude-Opus-Distill-v2](https://huggingface.co/TeichAI/gemma-4-31B-it-Claude-Opus-Distill-v2) — reasoning-distill base, retained 100% (incl. `<think>` chain-of-thought style)
- **Father (FFN donor)**: [google/gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it) — base Gemma-4 FFN tensors blended at **alpha = 0.1**
- **Method**: Darwin V8 FFN-crossbreed — per-layer FFN blend `w = w_mother*(1-alpha) + w_father*alpha` on `mlp.{gate,up,down}_proj` + `pre/post_feedforward_layernorm` for all 60 language-model layers; grid search alpha in {0.1, 0.2, 0.3, 0.4} on CLIcK-50 — best alpha=0.1 (CLIcK-200 = 86.0%)
- **Platform**: **VIDRAFT Darwin AI Model Breeding/Evolution Platform** ([VIDraft on HuggingFace](https://huggingface.co/VIDraft))
- **Architecture**: `Gemma4ForConditionalGeneration` (multimodal wrapper; text generation primary)
- **Tokenizer**: Gemma-4 (vocab 262,144)

---

## Model Lineage (Genealogy)

```
AWAXIS-Think-31B  (this model -- Darwin V8 FFN-crossbreed)
|
+-- Mother (kept full, 100%)
|   TeichAI/gemma-4-31B-it-Claude-Opus-Distill-v2
|   -- Claude Opus reasoning distill base
|
+-- Father (FFN donor, alpha=0.1)
    google/gemma-4-31B-it
    -- Gemma-4 base FFN tensors
```

**Common ancestor**: Google **Gemma-4** architecture.

---

## Measured Benchmarks

| Benchmark | Setting | Result |
|-----------|---------|--------|
| GPQA Diamond 20Q (seed 42) | greedy, max_new_tokens=**4096**, 2-way DP | **12/20 = 60.0%** (16/20 still hit token cap, 0 null) |
| GPQA Diamond 20Q (seed 42) | greedy, max_new_tokens=**2048** | 9/20 = 45.0% (16/20 truncated, 2 null) — *truncation artifact, included for transparency* |
| CLIcK (Korean) 200Q | greedy alpha-grid winner | 86.0% |

### Honest Caveats
- GPQA 60% is from **n=20** (small sample). 16/20 still hit the 4096-token cap — real ceiling may be higher with longer generation budget.
- Comparison to random baseline: GPQA random 25% — +35pp clear learning signal.
- The full GPQA Diamond (198Q) and other broad suites have not yet been measured for this exact merged artifact.
- The model retains the **Mother's `<think>...</think>` reasoning template** — strip via post-processing if undesired.

---

## Intended Use

- Korean/English step-by-step reasoning, instruction following, knowledge QA
- The `Think` suffix reflects the inherited Opus-distilled chain-of-thought behavior
- **2nd-generation breeding parent**: This model served as the Father for [AWAXIS-KR-31B](https://huggingface.co/Anserwise/AWAXIS-KR-31B), demonstrating Darwin's multi-generation evolution capability

## Out-of-Scope / Limitations

- Not a final clinical/legal advisor; outputs may be confidently wrong on hard graduate-level questions
- Inherits Gemma-4 base limitations (multimodal wrapper retained; image inputs not the primary use-case here)
- Subject to Gemma Terms of Use; see parent model cards for derivative-use clauses

---

## Inference

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tok = AutoTokenizer.from_pretrained("Anserwise/AWAXIS-Think-31B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "Anserwise/AWAXIS-Think-31B",
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
    attn_implementation="eager",
)

msgs = [{"role": "user", "content": "양자역학의 불확정성 원리를 자세히 설명해 주세요."}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inp = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inp, max_new_tokens=2048, do_sample=False)
print(tok.decode(out[0][inp["input_ids"].shape[-1]:], skip_special_tokens=True))
```

---

## License

Gemma Terms of Use (inherited from base). Use of this model is bound by [Google Gemma Terms](https://ai.google.dev/gemma/terms).

## Acknowledgements

- **[VIDRAFT](https://huggingface.co/VIDraft)** — Darwin AI Model Breeding/Evolution Platform
- TeichAI for the Opus-Distill base
- Google DeepMind for Gemma-4

---

*Built with the **VIDRAFT Darwin AI Model Breeding/Evolution Platform** — FFN-crossbreed V8 engine. This model was generated through Darwin's automated crossbreeding process, which selectively combines the strengths of parent models using biologically-inspired genetic algorithms. Measured numbers above are exact; nothing inflated.*