---
license: agpl-3.0
base_model: Qwen/Qwen3-1.7B-Base
pipeline_tag: text-generation
language:
- en
tags:
- prose
- rewriting
- style-transfer
- creative-writing
- deslop
- qwen3
library_name: transformers
---

# prose-rewriter-1.7b-v2.1

A paragraph-level **prose rewriter**: it takes prose written by a large model and
re-renders it to be more human, preserving the semantics it was given.

`Qwen/Qwen3-1.7B-Base` with a rank-32 LoRA merged in at strength 1.10.

**Invented people, mostly gone.** v2 would would invent a person where there was none
-- `She offered a trembling smile` -> `She gave him a trembling smile`.
Training pairs that teach this are now rejected: the input names one gender and
the target brings in the other. On 400 single-gender roleplay paragraphs the
rate falls from 1.3% to 0.4%.

**Fixed dialogues randomly dropping double-quotes.**

**More published fiction.** Professionally edited novels are now 40% of the
target side, up from 33%.

## Variants

| Path | Format | Use with |
|---|---|---|
| `/` | safetensors bf16, `qwen3` arch | transformers |
| `GGUF/prose-rewriter-1.7b-v2.1-Q8_0.gguf` | GGUF Q8_0, 2.17 GB | llama.cpp / llama-cpp-python |

The quant carries the chat template and stops on `<|im_end|>`, and the adapted
output head is kept separate from the token embeddings, stored at Q8_0.

## Prompt format

```
<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>rewrite
```

The chat template in this repo builds exactly that string, byte for byte, from
one message:

```python
messages = [{"role": "source", "content": paragraph}]
tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
```

It is not a chat model. Every message is rendered as the source paragraph, so a
runtime that probes the template with a `user` message still gets a valid
prompt. Send one message per call.

## Serving recipe

Sampled at `temperature=0.9, top_p=0.9`. Temperature 0.9 has been tested
internally to be the most optimal value. It's recommended you use this.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "chartreuse-verte/prose-rewriter-1.7b-v2.1"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="cuda").eval()

def rewrite(paragraph):
    text = tok.apply_chat_template(
        [{"role": "source", "content": paragraph}],
        tokenize=False, add_generation_prompt=True,
    )
    ids = tok(text, return_tensors="pt", add_special_tokens=False).input_ids.to(model.device)
    out = model.generate(ids, max_new_tokens=512, do_sample=True, temperature=0.9, top_p=0.9)
    return tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True).strip()
```

Same thing under llama.cpp. The role is `source`, which no chat API models, so
build the string yourself; `<|im_end|>` stops it:

```bash
llama-cli -m GGUF/prose-rewriter-1.7b-v2.1-Q8_0.gguf -no-cnv -n 512 --temp 0.9 --top-p 0.9 \
  -p '<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>rewrite
'
```

## Input length

The training pool's median input is 37 words and 80% of it is under 70, so serve
it on anything from a full sentence up.

The practical floor is about **15 words**. Below it the failure mode is padding,
cutting and fabrication rather than gibberish. Below 80 bytes, pass the text
through unchanged.

## Evaluation

Both releases measured **as they ship**, baked at strength 1.10, on 365 held-out
paragraphs of LLM-written prose that neither model saw in training, at
`temperature=0.9, top_p=0.95`, three swipes each. 1,095 generations an arm.

Pronoun invention numbers for this release:

| | v2 | v2.1 |
|---|---|---|
| a gendered pronoun the input never licensed | 1.3% | **0.4%** |
| ... belonging to the other gender | 1.3% | **0.4%** |
| a proper noun the input does not have | 0.2% | 0.7% |

Paired over the 400 inputs, the phantom pronoun falls (t = -2.7) and invented
proper nouns rise (t = +2.3).

Structural, on the 365 paragraphs:

| | v2 | v2.1 |
|---|---|---|
| words changed | 44.3% | 43.7% |
| passed through unchanged | 3.1% | 2.9% |
| near-verbatim outputs | 2.1% | 2.0% |
| below the training edit floor | 36.1% | 36.5% |
| sentence count moved | 77.5% | 77.1% |
| sentence-length variety vs input | +0.130 | +0.133 |
| words kept from the input | 0.624 | 0.627 |
| length preserved | 0.880 | 0.879 |
| truncated below 0.75x | 18.7% | 19.2% |
| repeated 3-grams | 0.009 | 0.008 |

Paired over the 303 inputs of 51-90 words, no column separates the two.

Unsupported content -- a sentence in the rewrite the input does not entail,
scored by NLI with the input alone as the premise:

| | v2 | v2.1 |
|---|---|---|
| unsupported sentences, mean per rewrite | 13.6% | 13.3% |
| unsupported sentences, all rewrites | 13.0% | 12.4% |
| coverage of the input (reverse entailment) | 0.494 | 0.508 |

The register numbers on the same paragraphs, against the input:

| | input | v2 | v2.1 | human corpus |
|---|---|---|---|---|
| banned constructions /1k | 7.52 | 3.18 | 2.90 | 0.00 |
| slop lexicon density | 0.081 | 0.048 | 0.050 | 0.019 |
| purple score | 0.498 | 0.308 | 0.311 | -- |

Paired over all 365 inputs, none of the three separate.

Dialogue: the number of spoken lines changes on 2.5% of rewrites against v2's
3.2%, and of the rewrites whose input carried quoted speech, 2.3% lost the
quotes entirely -- the same for both.

Markup round-trip on 365 tagged paragraphs -- every marker handed back in order,
with the same counts: 70.1% for v2, 71.2% here.

### A note on strength

Every number above is measured at the strength the release is baked at, which is
not the same as measuring the adapter. LoRA strength is spent at merge time
(`W + (B @ A) * 2.20` here), and the metrics move with it: read at the adapter's
natural 1.0 this same checkpoint is a materially different artifact.

1.10 is the lowest strength at which the pass-through and near-verbatim rates
bottomed out on the strength sweep.

## Training

**Corrupt forward, train backward.** The human paragraph is the target; an
on-policy LLM manufactures the input by slop-ifying it.

The target side is human prose: published contemporary literary fiction,
r/WritingPrompts
([`Mollymo/Human-to-AI-writing`](https://huggingface.co/datasets/Mollymo/Human-to-AI-writing)),
AO3
([`midwestern-simulation-active/ao3_random_subset`](https://huggingface.co/datasets/midwestern-simulation-active/ao3_random_subset)),
a scrape of `bluemoonroleplaying.com` -- the only human writing in the pool
already in the deployment's own register -- and a sliver of fanfiction.net
([`atom-in-the-universe/fanfics-10k-10k`](https://huggingface.co/datasets/atom-in-the-universe/fanfics-10k-10k)).

The input side is manufactured from those targets, weighted and share-capped so
that no single generator's tics dominate:

| pool axis | composition |
|---|---|
| rows | 15,794 over 11,476 distinct target paragraphs |
| corruption band | medium 35%, heavy 31%, light 30%, identity/no-op 3%, curated real slop 1% |
| `len_mode` | match only |
| markup | 15% of rows, identical on both sides |
| kind | prose 91%, dialogue 8%, structural no-ops 1% |
| target source | published fiction 40%, r/WritingPrompts 29%, AO3 21%, roleplay forum 8%, other 2% |

Pairs pass invariant gates before they reach the GPU: POV, tense, who is in the
scene, the gender of who is in the scene, the number of spoken lines,
grammatical correctness on the target side, content recall stratified by target
length, and NLI entailment both ways. Two further screens shape what reaches
training -- a floor on how much a pair actually changes, and a floor on the
sentence-length variety of the target, applied at a higher threshold for long
paragraphs than short ones.

**Loss on the target paragraph only.** Everything before `rewrite` is masked.

| | |
|---|---|
| LoRA | r=32, alpha=64, dropout 0.05 |
| target modules | q, k, v, o, gate, up, down, **and `lm_head`** |
| trainable | 39,792,640 params (2.26%) |
| schedule | 2 epochs, lr 1e-4 cosine, batch 4 × accum 8, seq 2048 |
| steps | 950 on one RTX 3090 |

## The merge

Merged at strength 1.10. Rank 32 with alpha 64 is a LoRA scaling of 2.0, so the
effective scaling is **2.20**: `W + (B @ A) * 2.20`. Merged in float32, stored
bfloat16.

`lm_head` is adapted, and `Qwen3-1.7B-Base` ties `lm_head.weight` to
`embed_tokens.weight`. **This checkpoint is untied**: the merged output head is
stored separately and the input embeddings are bit-identical to the base
model's, which is what training assumed. `config.json` says
`tie_word_embeddings: false` and it means it. Do not re-tie it, and if you
convert to another format, check that the head survived.

## Limitations

- **Not an instruct model.** It has one job and one prompt. There is nothing to
  ask it.
- **Works on fictional prose only.** May not work on technical documentation.
- **One paragraph per call.** Longer input degrades; split it.
- **Still invents people occasionally.** The phantom pronoun is down, not gone,
  and invented proper nouns went up. Check the output when the cast matters.
- **Truncates.** It drops below 0.75x the input's length on 19.2% of paragraphs.
  Check the output when length matters.
- **Will not pass AI detectors.** Pangram and such will still know because this
  model preserves word choices and certain sentence structures.
- **English only**, narrative register (third and first person fiction, dialogue
  with quoted speech).
- **Short input pads, cuts and invents.** The floor is about 15 words. See
  [Input length](#input-length).
- **Repeats a noun sooner than an LLM would.** Human prose reuses a plain noun
  where generated prose uses a synonym, and this model has learned that habit.

## License

The weights in this repository are released under the **GNU Affero General
Public License, version 3**. The full text is in `LICENSE`.

This is a derivative of [`Qwen/Qwen3-1.7B-Base`](https://huggingface.co/Qwen/Qwen3-1.7B-Base),
which is licensed under **Apache License 2.0**. That license is preserved and
its terms continue to apply to the base weights this model was built from; the
AGPL covers the combined work as distributed here. Apache-2.0 is one-way
compatible with AGPLv3, which is what makes this combination possible.

If you run a modified version of this model as a network service, AGPL section
13 requires you to offer the corresponding source of your modifications to its
users.