# Data Attribution (CC BY-SA 4.0 & Compatible Licenses)

This project (`CreateLLM_JP`) utilizes the following datasets which are licensed under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) or compatible licenses (such as CC-BY, Public Domain, etc.).

## 1. Wikipedia (Japanese)
- **Dataset Source**: [Hugging Face Datasets: wikipedia](https://huggingface.co/datasets/wikipedia)
- **Original Source**: [Wikipedia, the free encyclopedia](https://ja.wikipedia.org/)
- **License**: [Creative Commons Attribution-ShareAlike 3.0 Unported License (CC BY-SA 3.0)](https://creativecommons.org/licenses/by-sa/3.0/) / [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)
- **Description**: SFT instructions for factual expansion were generated using the text extracts from Japanese Wikipedia articles. Under the ShareAlike provision, generated models utilizing significant portions of this text should carry compatible attribution.

## 2. Aozora Bunko Clean (Globis University)
- **Dataset Source**: [Hugging Face Datasets: globis-university/aozorabunko-clean](https://huggingface.co/datasets/globis-university/aozorabunko-clean)
- **Original Source**: [Aozora Bunko (青空文庫)](https://www.aozora.gr.jp/)
- **License**: Public Domain / Copyright Expired (Text).
- **Description**: Text chunks were formatted into instruction/output pairs for creative writing tuning.

## 3. xwinograd-ja (polm-stability)
- **Dataset Source**: [Hugging Face Datasets: polm-stability/xwinograd-ja](https://huggingface.co/datasets/polm-stability/xwinograd-ja)
- **License**: CC-BY 4.0
- **Description**: Used for coreference resolution and reasoning instructions.

## 4. Nemotron-Personas-Japan (Nvidia)
- **Dataset Source**: [Hugging Face Datasets: nvidia/Nemotron-Personas-Japan](https://huggingface.co/datasets/nvidia/Nemotron-Personas-Japan)
- **License**: CC-BY 4.0
- **Description**: Synthesized persona-driven instructions to generate stylistic system prompts.

## 5. Law QA (Digital Agency)
- **Dataset Source**: [github.com/digital-go-jp/lawqa_jp](https://github.com/digital-go-jp/lawqa_jp)
- **License**: Public Data License (Compatible with CC BY 4.0)
- **Description**: Used for legal reasoning and document-based question answering.

## 6. Tanaka Corpus (ja-en)
- **Dataset Source**: [Hugging Face Datasets: hpprc/tanaka-corpus](https://huggingface.co/datasets/hpprc/tanaka-corpus)
- **License**: Public Domain
- **Description**: Used for English-Japanese translation instructions.

## 7. Weather Forecast Japan (jniimi)
- **Dataset Source**: [Hugging Face Datasets: jniimi/weather_forecast_japan](https://huggingface.co/datasets/jniimi/weather_forecast_japan)
- **License**: CC-BY 4.0
- **Description**: Weather forecast data summarized into descriptive outputs.

## 8. Open-Platypus-Japanese (weblab-GENIAC)
- **Dataset Source**: [Hugging Face Datasets: weblab-GENIAC/Open-Platypus-Japanese-masked](https://huggingface.co/datasets/weblab-GENIAC/Open-Platypus-Japanese-masked)
- **License**: MIT / CC-BY-4.0
- **Description**: High-quality reasoning and math datasets.

## 9. Japanese Honorifics & Trending Words (ronantakizawa)
- **Source 1**: [ronantakizawa/japanese-honorifics](https://huggingface.co/datasets/ronantakizawa/japanese-honorifics)
- **Source 2**: [ronantakizawa/japanese-trending-words](https://huggingface.co/datasets/ronantakizawa/japanese-trending-words)
- **License**: CC-BY 4.0
- **Description**: Linguistic and cultural instructions focused on Japanese nuances.

---
*Note: This document ensures that all upstream creators are credited as per their license requirements. If the resulting model is distributed, compatibility with the ShareAlike (BY-SA) provision of the Wikipedia data must be maintained.*