---
license: agpl-3.0
base_model: Qwen/Qwen3-4B-Base
pipeline_tag: text-generation
language:
- en
tags:
- prose
- rewriting
- style-transfer
- creative-writing
- deslop
- qwen3
library_name: transformers
---

# prose-rewriter-4b-v1.3

A paragraph-level **prose rewriter**: it takes prose written by a large model and
re-renders it to be more human, preserving the semantics it was given.

`Qwen/Qwen3-4B-Base` with a rank-32 LoRA merged in at strength 1.2.

Successor to [prose-rewriter-4b-v1.2](https://huggingface.co/chartreuse-verte/prose-rewriter-4b-v1.2),
trained on a larger pool with two new data gates. It **invents less and copies
less** than v1.2; see [Evaluation](#evaluation). This is why it allows for a merge at higher strength.

## Variants

| Path | Format | Use with |
|---|---|---|
| `/` | safetensors bf16, `qwen3` arch | transformers |
| `GGUF/prose-rewriter-4b-v1.3-Q8_0.gguf` | GGUF Q8_0, 4.69 GB | llama.cpp / llama-cpp-python |
| `GGUF/prose-rewriter-4b-v1.3-Q4_K_M.gguf` | GGUF Q4_K_M, 2.72 GB | llama.cpp / llama-cpp-python |

The quants carry the chat template and stop on `<|im_end|>`, and the adapted
output head is kept separate from the token embeddings in both — Q8_0 stores it
at Q8_0, Q4_K_M at Q6_K.

## Prompt format

```
<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>edit
match<|im_end|>
<|im_start|>rewrite
```

The chat template in this repo builds exactly that string, byte for byte, from
two roles:

```python
messages = [
    {"role": "source", "content": paragraph},
    {"role": "edit",   "content": "match"},
]
tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
```

It is not a chat model. The template rejects an `edit` value outside the three
modes rather than quietly building a prompt the weights have never seen. Any
other role is treated as the source paragraph, so a runtime that probes the
template with a `user` message still gets a valid prompt.

## The `edit` block is mandatory

`edit` names which of three length transforms is being asked for. **The values
are the corruptor's, so they read backwards.** They name what was done to build
the *input*, not what the model should do to it:

| `edit` | what it says about the input | what the model does |
|---|---|---|
| `match` | the source is the human's length | rewrite in place |
| `inflate` | the source was padded relative to the human original | cut |
| `compress` | the source was flattened and shortened | open it back out |

`match` is the setting for "rewrite it, do not trim it". It's strongly recommended
you use this mode.

Sending **no** block is the worst thing you can do to this checkpoint. It was
trained with the block, so omitting it collapses the model onto its
deletion-heaviest mode.

## Serving recipe

Sampled at `temperature=0.9, top_p=0.9`.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "chartreuse-verte/prose-rewriter-4b-v1.3"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="cuda").eval()

def rewrite(paragraph, mode="match"):
    text = tok.apply_chat_template(
        [{"role": "source", "content": paragraph},
         {"role": "edit",   "content": mode}],
        tokenize=False, add_generation_prompt=True,
    )
    ids = tok(text, return_tensors="pt", add_special_tokens=False).input_ids.to(model.device)
    out = model.generate(ids, max_new_tokens=512, do_sample=True, temperature=0.9, top_p=0.9)
    return tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True).strip()
```

Same thing under llama.cpp. The roles are `source` and `edit`, which no chat API
models, so build the string yourself; `<|im_end|>` stops it:

```bash
llama-cli -m GGUF/prose-rewriter-4b-v1.3-Q8_0.gguf -no-cnv -n 512 --temp 0.9 --top-p 0.9 \
  -p '<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>edit
match<|im_end|>
<|im_start|>rewrite
'
```

## Input length

The training pool's median input is 50 words and most of it is under 80, so serve
it on anything from a full sentence up.

The practical floor is about **15 words**. Below it the failure mode is padding
and fabrication rather than gibberish: the model stretches the line toward its
learned length and adds material the input never supported. Below 80 bytes, pass
the text through unchanged.

## Evaluation

Final training probe, v1.2 against v1.3:

| | v1.2 | v1.3 |
|---|---|---|
| invention rate | 0.146 | **0.104** |
| copy rate | 0.062 | **0.042** |
| distinct-4 | 0.982 | **0.990** |
| copy excess vs target | −0.009 | +0.049 |
| entropy | 0.506 | 0.505 |
| length ratio | 1.004 | 1.016 |
| train loss | 0.9104 | 0.974 |
| val loss | 0.7539 | 0.8418 |

**Invention drops by 29% and copy rate by 32%, at slightly better diversity.**

A known quirk is that human-written texts use less rare vocabulary than LLM texts.
This is a trade-off that must be accepted.

## Training

**Corrupt forward, train backward.** The human paragraph is the target; an
on-policy LLM manufactures the input by slop-ifying it.

The target side is human prose: roughly 57/42 r/WritingPrompts
([`Mollymo/Human-to-AI-writing`](https://huggingface.co/datasets/Mollymo/Human-to-AI-writing))
and AO3
([`midwestern-simulation-active/ao3_random_subset`](https://huggingface.co/datasets/midwestern-simulation-active/ao3_random_subset)),
with a sliver of fanfiction.net
([`atom-in-the-universe/fanfics-10k-10k`](https://huggingface.co/datasets/atom-in-the-universe/fanfics-10k-10k)).

The input side was generated by eleven corruptor endpoints — a few of which are
two routes onto one set of weights — weighted and share-capped so no single
model's tics dominate:

| pool axis | composition |
|---|---|
| rows | 67,696 over 54,209 distinct targets |
| corruptor | artemis-local 15%, ds-flash-nano 15%, ds-flash 13%, gemma-nano 12%, gemma-local 12%, qwen-flash 9%, ox-alpha 8% + 7%, then artemis, muse, ds-pro |
| corruption band | heavy 39%, medium 38%, light 20% |
| `len_mode` | match 64%, inflate 22%, compress 8%, unmarked 5% |
| kind | prose 96%, dialogue 4%, structural no-ops 1% |

Pairs pass invariant gates before they reach the GPU: POV, tense, who is in the
scene, grammatical correctness on the target side, content recall stratified by
target length, and NLI entailment both ways.

**Loss on the target paragraph only.** Everything before `rewrite` is masked.

| | |
|---|---|
| LoRA | r=32, alpha=32, dropout 0.05 |
| target modules | q, k, v, o, gate, up, down, **and `lm_head`** |
| trainable | 71,004,160 params (1.73%) |
| schedule | 1 epoch, lr 1e-4 cosine, batch 12 × accum 3, seq 2048 |
| steps | 1,864 on one RTX 3090, 2h27 |
| loss | train 0.974, val 0.8418 (600 val rows, document-disjoint) |

## The merge

Merged at strength 1.2. Rank 32 with alpha 32 is a LoRA scaling of 1.0, so the
effective scaling is **1.2**: `W + (B @ A) * 1.2`. Merged in float32, stored
bfloat16.

`lm_head` is adapted, and `Qwen3-4B-Base` ties `lm_head.weight` to
`embed_tokens.weight`. **This checkpoint is untied**: the merged output head is
stored separately and the input embeddings are bit-identical to the base
model's, which is what training assumed. `config.json` says
`tie_word_embeddings: false` and it means it. Do not re-tie it, and if you
convert to another format, check that the head survived.

## Limitations

- **Not an instruct model.** It has one job and one prompt. There is nothing to
  ask it.
- **Works on fictional prose only.** May not work on technical documentation.
- **One paragraph per call.** Longer input degrades; split it.
- **Will not pass AI detectors.** Pangram and such will still know because this model
  preserves word choices and certain sentence structures.
- **English only**, narrative register (third and first person fiction, dialogue
  with quoted speech).
- **Short input pads and invents.** The floor is about 15 words, and below it
  the failure is fabrication rather than gibberish. See
  [Input length](#input-length).

## License

The weights in this repository are released under the **GNU Affero General
Public License, version 3**. The full text is in `LICENSE`.

This is a derivative of [`Qwen/Qwen3-4B-Base`](https://huggingface.co/Qwen/Qwen3-4B-Base),
which is licensed under **Apache License 2.0**. That license is preserved and
its terms continue to apply to the base weights this model was built from; the
AGPL covers the combined work as distributed here. Apache-2.0 is one-way
compatible with AGPLv3, which is what makes this combination possible.

If you run a modified version of this model as a network service, AGPL section
13 requires you to offer the corresponding source of your modifications to its
users.