---
license: mit
language:
  - en
base_model:
  - mistralai/Mistral-7B-v0.1
pipeline_tag: text-generation
tags:
  - academic-writing
  - mistral
  - qlora
  - fine-tuning
  - arxiv
  - llm
  - uc-davis
---

# Mistral-7B Fine-Tuned for Academic Style (QLoRA)

This is a parameter-efficient fine-tuning of `mistralai/Mistral-7B-v0.1` using QLoRA on 500K academic abstracts. It was built for the ECS 271 final project at UC Davis.

## Intended Use

The model is designed to generate formal academic paragraphs given a paper title, useful for research drafts, educational AI tools, and academic-style assistants.

## Training Details

- Base model: `mistralai/Mistral-7B-v0.1`
- Method: QLoRA (low-rank adapter)
- Prompt format: "Write an academic paragraph given the title: ..."
- Dataset: 500K arXiv abstracts
- Epochs: 1
- GPU: RTX 5070 Ti (~60 hours)

## Limitations

- Generic or templated outputs
- No citation support
- Frequent hallucinations

## Example

**Prompt:**  
"Write an academic paragraph given the title: LoRA for In-Context Learning"

**Output:**  
"We present LoRA (Loosely Regularized Adapters), a novel approach to fine-tune large language models for in-context learning tasks. Unlike traditional methods, LoRA updates only a small number of trainable parameters, achieving comparable performance while reducing training costs."

## More Info
Main Project: [Github](https://github.com/Joshua-Sun-CompSci/academic-style-llms)