---

library_name: transformers
base_model: facebook/bart-large
pipeline_tag: summarization
tags:
- bart
- title-generation
- scientific-text
- summarization
- scihigh-2026
- fire-2026


---

# SciHigh 2026 Task 2 — BART-large Title Generation Model

This repository contains a fine-tuned **BART-large** model for **Task 2: Title Generation from Abstracts** of the SciHigh 2026 shared task at FIRE 2026.

The model takes a scientific paper abstract as input and generates a concise research-paper title.

## Base Model

* Model: `facebook/bart-large`
* Architecture: BART encoder-decoder Transformer
* Parameters: approximately 406 million
* Framework: Hugging Face Transformers

## Task

**Input:** Scientific paper abstract
**Output:** Generated scientific paper title

The model was fine-tuned for highly compressed abstractive generation, where the goal is to capture the central topic and contribution of an abstract in title form.

## Dataset

The model was fine-tuned on the **SpringerSSAT** dataset released for SciHigh 2026 Task 2.

Dataset split used:

* Training: 2,778 abstract-title pairs
* Validation: 347 abstract-title pairs
* Test: 348 abstracts with masked reference titles

Only the training split was used for gradient updates. The validation set was used for evaluation and best-checkpoint selection.

## Preprocessing

* Maximum abstract length: 512 tokens
* Maximum target-title length: 64 tokens
* Input truncation: enabled
* Target truncation: enabled
* Dynamic batch padding using `DataCollatorForSeq2Seq`
* No task-specific instruction prefix was added

## Fine-Tuning Configuration

* Epochs: 3
* Learning rate: `3e-5`
* Weight decay: `0.01`
* Per-device training batch size: 1
* Per-device evaluation batch size: 1
* Gradient accumulation steps: 8
* Effective training batch size: 8
* Mixed-precision training: FP16 AMP
* Gradient checkpointing: enabled
* Evaluation strategy: once per epoch
* Checkpoint saving strategy: once per epoch
* Generation beam size during validation: 4
* Maximum generation length: 64
* Best-model selection metric: ROUGE-L
* Best model automatically restored after training

## Validation Results

| Epoch | Validation Loss |     ROUGE-1 |     ROUGE-2 |     ROUGE-L |  ROUGE-Lsum |
| ----- | --------------: | ----------: | ----------: | ----------: | ----------: |
| 1     |        2.282688 |     44.9926 |     21.3081 |     36.6377 |     36.6539 |
| 2     |    **2.217401** | **45.7446** | **22.2135** | **37.6433** | **37.6641** |
| 3     |        2.242662 |     45.2856 |     21.8391 |     37.2244 |     37.2773 |

### Best Checkpoint

The best model was obtained at **Epoch 2**.

* ROUGE-1: **45.7446**
* ROUGE-2: **22.2135**
* ROUGE-L: **37.6433**
* ROUGE-Lsum: **37.6641**
* Validation loss: **2.217401**

A separate evaluation over all 347 validation examples reproduced these results.

These are validation-set results and are not official hidden-test scores.

## Example Usage

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "kckrish21/SciHigh2026-Task2-BART-large"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

abstract = """
Insert a scientific paper abstract here.
"""

inputs = tokenizer(
    abstract,
    return_tensors="pt",
    max_length=512,
    truncation=True
)

generated_ids = model.generate(
    **inputs,
    max_length=64,
    num_beams=4,
    early_stopping=True
)

title = tokenizer.decode(
    generated_ids[0],
    skip_special_tokens=True
)

print(title)
```

## Test-Set Generation

For SciHigh 2026 test inference:

* Test examples: 348
* Generation batch size: 4
* Beam size: 4
* Maximum generation length: 64
* Early stopping: enabled
* Missing predictions: 0
* Empty predictions: 0

The reference test titles are masked by the task organizers, so no test-set metric is reported here.

## Intended Use

This model is intended for:

* scientific paper title generation from abstracts
* research on scientific text generation
* participation and reproducibility for SciHigh 2026 Task 2

Generated titles should be treated as model suggestions rather than authoritative paper titles.

## Limitations

The model was fine-tuned on a relatively small dataset of social-science research abstracts. Performance may differ for scientific domains or writing styles that are substantially different from the SpringerSSAT training distribution.

Like other neural text-generation systems, the model may generate wording that is incomplete, overly generic, or not fully supported by the abstract.

## Shared Task

**SciHigh 2026 — Research Highlight Generation from Scientific Papers**
**FIRE 2026**
Subtask 2: **Title Generation from Abstracts**

Ranking for the shared task is primarily based on ROUGE-L F1.