---
license: cc-by-nc-4.0
language:
- en
tags:
- lora
- aya
- tiny-aya
- multilingual
- code
- legesher
- tiny-aya-expedition
- language-decoded
- unsloth
- phase-2
- the-stack-v1
- deprecated
library_name: transformers
base_model:
- CohereLabs/tiny-aya-base
pipeline_tag: text-generation
---

# Language Decoded LoRA — Condition 1: English Code (Phase 2 · The Stack v1)

## ⚠️ Deprecated — preliminary (Phase 2 · The Stack v1)

This adapter is the original **March-2026 hackathon (Phase 2)** model, trained on [`bigcode/the-stack`](https://huggingface.co/datasets/bigcode/the-stack) (v1, non-dedup). It is **superseded by the paper's Phase 3 adapter**, which was re-trained from scratch on the cleaner [`bigcode/the-stack-v2-dedup`](https://huggingface.co/datasets/bigcode/the-stack-v2-dedup) corpus. For paper-grade use, load the Phase 3 adapter from the umbrella repo:

```python
PeftModel.from_pretrained(base_model, "legesher/language-decoded-lora", subfolder="tiny-aya-base/condition-1-en-5k-seed42")
```

This repo is **kept for reproducibility of the preliminary results only — do not cite it for the paper.** It was renamed from `legesher/language-decoded-lora-condition-1-en-5k`; the old URL continues to resolve via a Hugging Face redirect.

Raw English Python from [`bigcode/the-stack`](https://huggingface.co/datasets/bigcode/the-stack) (v1, non-dedup), 5k subset. Tests whether code fine-tuning improves multilingual reasoning (replicates Aryabumi et al., 2024).

Part of the **Language Decoded** project ([Cohere's Tiny Aya Expedition](https://aya.for.ai)).

> For the full adapter inventory across both phases, see the [Language Decoded LoRA hub](https://huggingface.co/legesher/language-decoded-lora) and its [`MANIFEST.md`](https://huggingface.co/legesher/language-decoded-lora/blob/main/MANIFEST.md).

## Training Data

[`legesher/language-decoded-data` / `phase-2-the-stack-v1-condition-1-en-5k`](https://huggingface.co/datasets/legesher/language-decoded-data?config=phase-2-the-stack-v1-condition-1-en-5k) — the Phase 2 / The Stack v1 config.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-base")
tokenizer = AutoTokenizer.from_pretrained("CohereLabs/tiny-aya-base")
# Preliminary Phase 2 adapter (kept for reproducibility):
model = PeftModel.from_pretrained(base_model, "legesher/language-decoded-lora-phase-2-the-stack-v1-condition-1-en-5k")
```

## Citation

```bibtex
@misc{language-decoded-2026,
  title={Language, Decoded: Exploring the Impact of Fine-Tuning a Multilingual Model on Native-Language Code},
  author={Madison Edgar and Saad Ahmed Bazaz and Tom Sherborne and Rashik Shahjahan and Khojasteh Mirza and Sarah Jawaid and Rafay Mustafa and Sohaib Ahmed Bazaz},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/legesher/language-decoded-lora}
}
```

## License

CC-BY-NC-4.0. The adapters inherit the license of the base model,
[CohereLabs/tiny-aya-base](https://huggingface.co/CohereLabs/tiny-aya-base)
(CC-BY-NC-4.0). The training datasets
([legesher/language-decoded-data](https://huggingface.co/datasets/legesher/language-decoded-data))
are separately licensed under Apache-2.0.