---
tags:
- 128k-context
- 4b
- agentic-coding
- android
- apple-silicon
- attested
- bash
- c
- chain-of-custody
- chinese
- code
- code-completion
- code-generation
- code-infill
- coder
- coding
- consumer-gpu
- cpp
- cryptographically-verified
- css
- edge-inference
- embedded
- english
- extended-context
- forge-alloy
- function-calling
- go
- html
- iphone
- java
- javascript
- kotlin
- llama-cpp
- lm-studio
- local-inference
- long-context
- macbook
- mlx
- mobile
- multilingual
- ollama
- on-device
- php
- programming
- python
- qwen
- qwen3
- qwen3.5
- raspberry-pi
- reproducible
- ruby
- rust
- software-engineering
- sql
- swift
- text-generation
- typescript
- yarn
base_model: Qwen/Qwen3.5-4B
pipeline_tag: text-generation
license: apache-2.0
---

# 32K → 128K: Context Window Extended 4x

We took **Qwen3.5-4B** (32K context) and extended it to **128K context** via YaRN RoPE scaling — then trained it on code for 3 cycles to recover and improve quality.

**Paste your entire codebase, not just one file.**


<p align="center">
<a href="https://cambriantech.github.io/forge-alloy/verify/#hf.co/continuum-ai/qwen3.5-4b-code-128k-forged/resolve/main/qwen3.5-4b-code-128k-forged.alloy.json@a4da7dea5bb8d3d9">
<img src="alloy-qr.png" alt="Verify Chain of Custody" width="160"/>
</a>
</p>

<p align="center">
<a href="https://cambriantech.github.io/forge-alloy/verify/#hf.co/continuum-ai/qwen3.5-4b-code-128k-forged/resolve/main/qwen3.5-4b-code-128k-forged.alloy.json@a4da7dea5bb8d3d9"><b>Every claim on this card is verified</b></a><br>
<b>Trust: self-attested</b> · 3 benchmarks · 1 device tested<br>
<a href="https://github.com/CambrianTech/forge-alloy">ForgeAlloy</a> chain of custody · <a href="qwen3.5-4b-code-128k-forged.alloy.json">Download alloy</a> · Merkle-chained
</p>

---

**Qwen3.5-4B** with cryptographic provenance via the [ForgeAlloy](https://github.com/CambrianTech/forge-alloy) chain of custody.


## Benchmarks

| Benchmark | Result | Verified |
|---|---|---|
| **perplexity** | **18.7** | Self-reported |
| **humaneval** | **32.3** | Self-reported |
| **humaneval+** | **28.0** | Self-reported |


## What Changed (Base → Forged)

| | Base | Forged | Delta |
|---|---|---|---|
| **Perplexity** (code) | 3.04 | 2.47 | -18.7% ✅ |
| **Context Window** | 32,768 | **131,072** | **4x** via yarn ✅ |
| **Training** | General | code, 2000 steps | LR 2e-4, 3 cycles |
| **Pipeline** | | context-extend → train | 3 cycles |

## Runs On

| Device | Format | Size | Speed |
|--------|--------|------|-------|
| **RTX 5090** | fp16 | 8.4GB | **~23 tok/s** (verified) |
| MacBook Pro 32GB | fp16 | 8.4GB | Expected |
| MacBook Air 16GB | Q8_0 | ~4.2GB | Expected |
| MacBook Air 8GB | Q4_K_M | ~2.6GB | Expected |
| iPhone / Android | Q4_K_M | ~2.6GB | Expected |

## Quick Start

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("continuum-ai/qwen3.5-4b-code-128k-forged",
    torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("continuum-ai/qwen3.5-4b-code-128k-forged")

inputs = tokenizer("def merge_sort(arr):", return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```


## Methodology

Produced via YaRN context extension. Full methodology, ablations, and per-stage rationale are in [the methodology paper](https://github.com/CambrianTech/continuum/blob/main/docs/papers/PLASTICITY-COMPACTION.md) and the companion [`MODEL_METHODOLOGY.md`](MODEL_METHODOLOGY.md) in this repository. The pipeline ran as `context-extend → train` over 3 cycles on RTX 5090.


## Chain of Custody

Scan the QR or [verify online](https://cambriantech.github.io/forge-alloy/verify/#hf.co/continuum-ai/qwen3.5-4b-code-128k-forged/resolve/main/qwen3.5-4b-code-128k-forged.alloy.json@a4da7dea5bb8d3d9). Download the [alloy file](qwen3.5-4b-code-128k-forged.alloy.json) to verify independently.

| What | Proof |
|------|-------|
| Model weights | `sha256:fef00899cd644c6118c9fc63d1f1e6b63...` |
| Code that ran | [`sha256:8a421f10994ecb1b5...`](https://github.com/CambrianTech/sentinel-ai.git/tree/a6b200cb465e2f5b87fcd86d9ee92d15890ee37f) |
| Git commit | [`a6b200cb465e`](https://github.com/CambrianTech/sentinel-ai.git/commit/a6b200cb465e2f5b87fcd86d9ee92d15890ee37f) |
| Forged on | RTX 5090, 2026-04-02T02:38:44-0500 |
| Trust level | [`self-attested`](https://github.com/CambrianTech/forge-alloy/blob/main/docs/ATTESTATION.md) |
| Spec | [ForgeAlloy](https://github.com/CambrianTech/forge-alloy) — Rust/Python/TypeScript |

## Make Your Own

Forged with [Continuum](https://github.com/CambrianTech/continuum) — a distributed AI world that runs on your hardware.

<p align="center">
<a href="https://github.com/CambrianTech/continuum"><img src="https://raw.githubusercontent.com/CambrianTech/continuum/main/docs/images/factory.png" alt="Continuum Model Factory" width="400"/></a>
</p>

The Factory configurator lets you design and forge custom models visually — context extension, pruning, LoRA, quantization, vision/audio modalities. Pick your target devices, the system figures out what fits.

[GitHub](https://github.com/CambrianTech/continuum) · [All Models](https://huggingface.co/continuum-ai) · [Forge-Alloy](https://github.com/CambrianTech/forge-alloy)

## License

apache-2.0