---
license: apache-2.0
base_model: mistralai/Mistral-Medium-3.5-128B
tags:
- mistral
- mistral-3.5
- text-only
- bf16
- 128b
- heretic
- uncensored
- decensored
- abliterated
- reproducible
---
# This is a decensored version of [Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only](https://huggingface.co/Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only), made using [Heretic](https://github.com/p-e-w/heretic) v1.3.0

> [!TIP]
> **This model is reproducible!**
>
> See the [README](reproduce/README.md) in the `reproduce` directory for more information.

## Abliteration parameters

| Parameter | Value |
| :-------- | :---: |
| **direction_index** | 40.13 |
| **attn.o_proj.max_weight** | 1.47 |
| **attn.o_proj.max_weight_position** | 52.50 |
| **attn.o_proj.min_weight** | 1.39 |
| **attn.o_proj.min_weight_distance** | 49.92 |
| **mlp.down_proj.max_weight** | 1.21 |
| **mlp.down_proj.max_weight_position** | 54.38 |
| **mlp.down_proj.min_weight** | 0.76 |
| **mlp.down_proj.min_weight_distance** | 50.68 |

## Performance

| Metric | This model | Original model ([Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only](https://huggingface.co/Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only)) |
| :----- | :--------: | :---------------------------: |
| **KL divergence** | 0.0329 | 0 *(by definition)* |
| **Refusals** | 8/100 | 93/100 |

-----


<style>
body {
  font-family: 'Quicksand', sans-serif;
  background: linear-gradient(135deg, #4a1e00 0%, #1c0a00 100%);
  color: #F5EFE6;
  margin: 0;
  padding: 0;
  font-size: 16px;
}

h1, h2, h3, h4, summary {
    font-family: 'Cinzel', serif;
}

.container {
  margin: 20px auto;
  max-width: 900px;
  background-color: rgba(28, 22, 18, 0.95);
  padding: 30px;
  border-radius: 12px;
  box-shadow: 0 4px 20px rgba(255, 140, 0, 0.15);
  border: 1px solid rgba(255, 140, 0, 0.2);
  outline: 1px solid rgba(255, 140, 0, 0.5);
  outline-offset: -1px;
  position: relative;
}

.container::before {
  content: '';
  position: absolute;
  top: -1px;
  left: -1px;
  right: -1px;
  bottom: -1px;
  border: 1px solid rgba(255, 165, 0, 0.98);
  border-radius: 12px;
  pointer-events: none;
  animation: borderGlow 2.5s ease-in-out infinite;
}

@keyframes borderGlow {
  0% { box-shadow: 0 0 5px rgba(255, 165, 0, 0.98); }
  50% { box-shadow: 0 0 12px rgba(255, 165, 0, 0.98); }
  100% { box-shadow: 0 0 5px rgba(255, 165, 0, 0.98); }
}

.header h1 {
  font-size: 32px;
  color: #FFA500;
  margin: 0 0 20px 0;
  text-align: center;
  text-shadow: 0 0 12px rgba(255, 100, 0, 0.6);
}

a {
  color: #FFD700;
  text-decoration: none;
  transition: color 0.3s ease;
}

.button {
  display: inline-block;
  background-color: #E55B00;
  color: #FFFFFF;
  padding: 12px 24px;
  border-radius: 5px;
  cursor: pointer;
  text-decoration: none;
  font-family: 'Cinzel', serif;
  font-weight: 600;
  transition: all 0.3s ease;
  border: 1px solid transparent;
}

.button:hover {
  background-color: #FF8C00;
  box-shadow: 0 0 15px rgba(255, 140, 0, 0.5);
  transform: translateY(-2px);
}

.info-card {
  background: rgba(45, 35, 25, 0.95);
  border: 1px solid rgba(255, 140, 0, 0.2);
  border-radius: 8px;
  overflow: hidden;
  margin-bottom: 25px;
}

.info-header {
  background: rgba(255, 140, 0, 0.1);
  padding: 20px;
  border-bottom: 1px solid rgba(255, 140, 0, 0.2);
}

.info-header h3 {
  color: #FFA500;
  margin: 0 0 10px 0;
  font-size: 22px;
}

.card-content {
  padding: 20px;
  line-height: 1.7;
}

.card-content ul {
    list-style: none;
    padding-left: 20px;
}

.card-content li::before {
    content: '✦';
    color: #FFD700;
    font-weight: bold;
    display: inline-block;
    width: 1em;
    margin-left: -1.2em;
}

.card-content strong {
    color: #FFD700;
}

/* Update to the note card */
.note-card {
  border: 1px solid #FFA500;
  box-shadow: 0 0 10px rgba(255, 165, 0, 0.1);
}

.note-header {
  background: rgba(255, 165, 0, 0.1);
}

.note-header h3 {
  color: #FFA500;
  text-align: center;
}

.support-section {
    text-align: center;
    margin-top: 40px;
    background: rgba(45, 35, 25, 0.95);
    border: 1px solid rgba(255, 140, 0, 0.2);
    border-radius: 8px;
    padding: 20px;
}

summary {
    cursor: pointer;
    list-style: none;
    outline: none;
    display: flex;
    align-items: center;
}

summary::before {
    content: '▶';
    font-size: 1.2em;
    color: #FFA500;
    margin-right: 15px;
    transition: transform 0.2s ease;
}

details[open] > summary::before {
    transform: rotate(90deg);
}

h2 {
    color: #FFA500;
    border-bottom: 1px solid rgba(255, 140, 0, 0.2);
    padding-bottom: 10px;
    margin-bottom: 20px;
}
</style>

<div class="container">
<link href="https://fonts.googleapis.com/css2?family=Cinzel:wght@400;500;600&family=Quicksand:wght@400;500&display=swap" rel="stylesheet">

<div class="header">
  <h1>Mistral-Medium-3.5-128B-BF16-Text-Only</h1>
</div>

<div class="info">
  
  <div class="info-card note-card">
    <div class="info-header note-header">
      <h3>📜 Technical Architecture Note</h3>
    </div>
    <div class="card-content">
      <p>This model has been converted from <strong>Mistral3ForConditionalGeneration</strong> (Multimodal) to <strong>MistralForCausalLM</strong> (Standard Text-Only). This change ensures maximum compatibility with standard fine-tuning libraries like <em>Axolotl, Unsloth, and Hugging Face Transformers</em> without requiring custom vision-encoder handling.</p>
    </div>
  </div>

  <div class="support-section">
    <p><strong>Help me feed the data beast! Taking commissions for universe-specific models.</strong></p>
    <a href="https://ko-fi.com/som1tokmynam" target="_blank" class="button">
      Support on Ko-fi
    </a>
  </div>
  
  <div class="section-container">
    <details open>
      <summary><h2>Model Description</h2></summary>
      <div class="info-card">
        <div class="card-content">
          <p>This is a processed version of <strong>Mistral-Medium-3.5-128B</strong> designed for users who prioritize text-only performance and ease of fine-tuning.</p>
          <p><strong>Modification Details:</strong></p>
          <ul>
            <li><strong>Precision Upscale:</strong> Converted from <strong>FP8</strong> weights to <strong>BF16</strong> to restore full 16-bit brain-float precision for stable gradient updates during training.</li>
            <li><strong>Vision Layer Stripping:</strong> All vision encoders and multimodal projection layers have been removed, significantly reducing memory overhead during inference and training for text-only tasks.</li>
            <li><strong>Architecture Re-mapping:</strong> The configuration has been modified to use <code>MistralForCausalLM</code>, allowing it to be treated as a standard dense language model.</li>
          </ul>
        </div>
      </div>
    </details>
  </div>

  <div class="section-container">
    <details>
      <summary><h2>Purpose & Usage</h2></summary>
      <div class="info-card">
          <div class="card-content">
              <p>This model is intended to serve as a <strong>clean base for fine-tuning</strong>. By removing the vision components, you can allocate more VRAM to sequence length or batch size. It is 100% functional for text-only chat and reasoning out of the box.</p>
          </div>
      </div>
    </details>
  </div>

  <div class="section-container">
    <details>
      <summary><h2>Acknowledgements</h2></summary>
      <div class="info-card">
          <div class="card-content">
              <ul>
                  <li>Credit to <strong>Mistral AI</strong> for the original Mistral-Medium-3.5-128B architecture.</li>
              </ul>
          </div>
      </div>
    </details>
  </div>

</div>
</div>