---
license: apache-2.0
datasets:
- mayiwen/PaperAudit_Dataset
language:
- en
base_model:
- meta-llama/Llama-3.2-3B-Instruct
---
# PaperAudit Llama3.2 3B (SFT + RL)

## Model Overview

PaperAudit_Llama3.2_3B_sft_rl is a lightweight model specifically trained for academic paper error detection and automated review tasks. This model is based on Llama 3.2 3B Instruct and has been optimized through Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF).

## Model Information

- **Base Model**: Llama 3.2 3B Instruct
- **Model Parameters**: ~3 billion parameters
- **Training Method**: Supervised Fine-Tuning (SFT) + Reinforcement Learning (RLHF)
- **Model Architecture**: LlamaForCausalLM
- **Context Length**: 131,072 tokens
- **Data Type**: bfloat16

## Model Features

- **Lightweight and Efficient**: 3B parameter scale, suitable for resource-constrained environments
- **Specialized Optimization**: Specifically optimized for academic paper error detection and review tasks
- **Reinforcement Learning**: Aligned with human preferences through RLHF to improve review quality and error detection accuracy
- **Long Context**: Supports ultra-long context (131K tokens), suitable for processing complete academic papers

## Training Data

This model is trained on [PaperAudit_Dataset](https://huggingface.co/datasets/mayiwen/PaperAudit_Dataset). The dataset includes:

- Academic papers downloaded from OpenReview
- Structured paper content (processed via LlamaParse and LLM)
- Synthetic error data for training error detection models
- Human review feedback data

For more details about the dataset, please visit: https://huggingface.co/datasets/mayiwen/PaperAudit_Dataset

## Usage

### Install Dependencies

```bash
pip install transformers torch accelerate
```

### Load Model

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_path = "./llama3.2_3b_sft_rl"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
```

### Inference Example

```python
# Prepare input (paper error detection task)
prompt = """Please detect errors in the following academic paper paragraph:

[Paper content...]

Please identify errors and provide correction suggestions."""

# Encode input
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate response
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        do_sample=True,
        pad_token_id=tokenizer.pad_token_id
    )

# Decode output
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```

## Application Scenarios

- Academic paper error detection
- Automated paper review
- Academic writing quality assessment
- Paper content analysis and feedback generation

## Model Architecture Details

- **Hidden Size**: 3072
- **Intermediate Size**: 8192
- **Number of Attention Heads**: 24
- **Number of Key-Value Heads**: 8 (Grouped Query Attention)
- **Number of Hidden Layers**: 28
- **Vocabulary Size**: 128,256

## Notes

- This model is specifically optimized for academic paper review tasks and may require further fine-tuning for other domains
- It is recommended to use bfloat16 precision to save memory and improve inference speed
- For long document processing, appropriate context window management strategies are recommended

## Related Resources

- **Training Dataset**: [PaperAudit_Dataset](https://huggingface.co/datasets/mayiwen/PaperAudit_Dataset)
- **PaperAudit Project**: For more details, please refer to the PaperAudit project documentation

## License

Please refer to the license terms of the base model Llama 3.2.