---
license: apache-2.0
tags:
- time-series
- forecasting
- seasonal
- monthly
- patchtst
- time-series-forecasting
- foundation models
datasets:
- autogluon/chronos_datasets
metrics:
- mase
- smape
pipeline_tag: time-series-forecasting
---

# PatchTST Monthly Seasonal Forecast Model

**State-of-the-art monthly seasonal time series forecasting model**, fine-tuned on the M4 Monthly competition dataset (48,000 time series).

## Architecture

Based on [PatchTST](https://arxiv.org/abs/2211.14730) (ICLR 2023) — a patch-based Transformer for time series that:
- Tokenizes time series into **6-month patches** to capture semi-annual seasonal patterns
- Uses **sinusoidal positional encoding** for temporal awareness
- Applies **standard normalization** (RevIN) for scale-invariant forecasting
- Achieves strong performance with only **603,538 parameters**

## Training Details

| Parameter | Value |
|-----------|-------|
| Dataset | M4 Monthly (48,000 series, 6 categories) |
| Context Length | 48 months (4 years) |
| Prediction Length | 18 months |
| Patch Length | 6 months |
| d_model | 128 |
| Attention Heads | 8 |
| Transformer Layers | 3 |
| Epochs | 2000 steps |
| Batch Size | 128 |
| Learning Rate | 0.001 (cosine schedule) |
| Optimizer | AdamW (weight_decay=0.01) |

## Evaluation Results (M4 Monthly Test Set, 18-month horizon)

| Model | MASE (mean) | sMAPE (mean) |
|-------|-------------|--------------|
| **PatchTST (ours)** | **1.0243** | **14.04** |
| Chronos-Bolt (zero-shot) | 0.9202 | 13.72 |
| Seasonal Naive | 1.2453 | 16.36 |

### Per-Category Results
| Category | MASE | sMAPE | # Series |
|----------|------|-------|----------|
| Macro | 1.0693 | 14.79 | 407 |
| Micro | 0.9918 | 17.13 | 472 |
| Finance | 1.0292 | 14.86 | 485 |
| Industry | 1.0650 | 13.75 | 410 |
| Demographic | 0.9344 | 4.33 | 214 |
| Other | 0.7960 | 17.58 | 12 |

## Usage

```python
from transformers import PatchTSTForPrediction
import torch
import numpy as np

model = PatchTSTForPrediction.from_pretrained("stevevaius/patchtst-monthly-seasonal")
model.eval()

# Input: monthly time series with at least 48 months of history
# Shape: [batch_size, context_length, 1] (univariate)
monthly_values = np.array([...])  # your monthly data
context = torch.tensor(monthly_values[-48:], dtype=torch.float32).unsqueeze(0).unsqueeze(-1)

with torch.no_grad():
    output = model(past_values=context)
    forecast = output.prediction_outputs.squeeze().numpy()

# forecast shape: [18] — next 18 months
print("Forecast:", forecast)
```

## References

- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://arxiv.org/abs/2211.14730) (ICLR 2023)
- [Chronos: Learning the Language of Time Series](https://arxiv.org/abs/2403.07815) (TMLR 2024)
- [M4 Competition](https://www.sciencedirect.com/science/article/pii/S0169207019301128)