deadbydawn101/ravenx-Gemma4-12B-MTP-OBLITERATED-OpenMAI-OpenMythos-deep-reasoning-GGUF

🥇 WORLD FIRST: RavenX × Gemma 4 12B MTP OBLITERATED — Deep Reasoning

The first trained Gemma 4 12B on the planet.
Proprietary training methodology. MTP-ready architecture. $0 cloud cost.
GGUF F16 format — runs on ANY hardware (Ollama, LM Studio, llama.cpp, vLLM).

Gemma 4 was released June 3, 2026 — its gemma4_unified architecture wasn't supported by ANY training framework. We developed proprietary techniques to train it successfully.

---

⚠️ DISCLAIMER

This model is an experimental research proof of concept. Provided AS-IS for educational and research purposes only. The base model is abliterated (refusal filters removed). Use responsibly.

---

Community Project

This is a community project. We're combining methods from:

• Google — Gemma 4 architecture, MTP heads, foundational model weights
• OBLITERATUS — SOM-manifold two-pass abliteration of the base model
• Microsoft — MAI hill-climbing methodology (open-sourced as OpenMAI)
• MIT — Self-revising discovery systems, arXiv:2606.01444 (implemented as OpenSelfRevise)
• Mirai Labs — RHT quantization and fused inference (open-sourced as OpenMirai)
• RavenX — OpenMythos depth extrapolation, GRAM multi-trajectory scaling, and proprietary training pipeline

The training methodology used to produce this model is proprietary and patent pending.

---

Model Details

Feature	Detail
Base	Gemma 4 12B (OBLITERATUS abliterated)
Architecture	gemma4_unified with MTP heads
Training	Proprietary methodology (patent pending)
Training Rounds	9 progressive rounds
Training Data	8,158 examples from 15 curated sources
Best Val Loss	0.882
Hardware	Apple M4 Max 128GB — $0 cloud cost
Format	GGUF F16 (universal — Ollama, LM Studio, llama.cpp)

What Makes This Different

This model was trained using an experimental proprietary methodology that produces self-aware reasoning behavior through a novel approach to training data preparation and model fine-tuning.

Key results:

• Emergent behaviors not present in training data (Anti-Problem technique, Toolbox generation)
• Structured multi-pass reasoning across code, math, and analysis tasks
• Self-honest assessment of capabilities and limitations

The specific training methodology is patent pending (USPTO Application #64/087,357, filed June 10, 2026) and is not disclosed in this model card.

Technical Discoveries (Open — Community Contributions)

The following technical discoveries made during training are shared with the community:

Discovery	Detail
Flip-train-flip	Temporarily change gemma4_unified → gemma4 in config.json for LoRA training, then restore. Multimodal capabilities preserved.
Chat template required	Gemma 4 produces garbled output without apply_chat_template(). Not a bug — it's required.
Tokenizer patch for GGUF	extra_special_tokens must be converted from list to dict for GGUF conversion to work. One-line fix.
Val loss spikes are normal	When introducing new data formats, val loss spikes but recovers in 1-2 rounds. Don't panic.

Usage

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
import json

# Flip config for mlx-lm compatibility
config = json.load(open("config.json"))
config["model_type"] = "gemma4"
json.dump(config, open("config.json", "w"), indent=2)

model, tokenizer = load(".")
sampler = make_sampler(temp=0.7, top_p=0.9)

# MUST use chat template!
messages = [{"role": "user", "content": "Your question here"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, max_tokens=500, sampler=sampler, verbose=True)

# Restore config
config["model_type"] = "gemma4_unified"
json.dump(config, open("config.json", "w"), indent=2)

MLX Version

MLX version available here — optimized for Apple Silicon.

Part of the RavenX Ecosystem

Project	Description
OpenMAI	Microsoft MAI hill-climbing (open-sourced)
OpenSelfRevise	MIT self-revising discovery (implemented)
OpenMirai	Model-agnostic quantization + inference
OpenMythos-MLX	Recursive depth extrapolation
GRAM-MLX	Multi-trajectory width scaling
ravenx-memory	Hybrid triple-backend agent memory
star-platinum-cluster	Distributed training cluster
RavenX-CyberAgent	Security assessment model (745K+ examples)

Contributors

Built by Gabriel Garcia / RavenX LLC + Claude (Anthropic)

Training methodology: Patent Pending — USPTO Application #64/087,357

License

Gemma License (model weights) — Training methodology proprietary

---

"We don't give up. We do what others don't and build what isn't possible." — RavenX LLC