| --- |
| license: llama3.1 |
| datasets: |
| - newmindai/EuroHPC-Legal |
| language: |
| - tr |
| - en |
| base_model: |
| - meta-llama/Llama-3.1-8B-Instruct |
| tags: |
| - EuroHPC |
| - Karolina |
| - Axolotl |
| - Unsloth |
| --- |
| |
| <div style="display: flex; justify-content: center; flex-wrap: wrap; gap: 15px; align-items: flex-start;"> |
|
|
| <div style="padding: 10px; border-radius: 8px; background-color: #000; |
| color-scheme: light dark; margin-top: 40px;"> |
| <img |
| src="https://cdn-uploads.huggingface.co/production/uploads/683d4880e639f8d647355997/iA0QsWk-k-fQ7CCt65znl.png" |
| style="width: 100%; max-width: 250px; height: 40px; |
| filter: invert(0) brightness(1) contrast(1);" /> |
| </div> |
| |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/683d4880e639f8d647355997/mqbOdFfrC7KjDZbQlLFFj.png" |
| style="width: 100%; max-width: 300px; height: auto;" /> |
| |
| <img |
| src="https://cdn-uploads.huggingface.co/production/uploads/683d4880e639f8d647355997/VGnh14pYg-SXSaEt640qz.png" |
| style="width: 100%; max-width: 150px; height: 100px; margin-top: 35px;" /> |
| |
| </div> |
| |
| ## Model Card |
|
|
| This document describes a parameter-efficient fine-tuning setup using LoRA on the EuroHPC Karolina system. Axolotl provides flexible orchestration and Unsloth supplies optimized kernels for high-throughput training on the newmindai/EuroHPC-Legal dataset. This model is fine-tuned using Parameter-Efficient Fine-Tuning (PEFT) with LoRA (Low-Rank Adaptation) on the EuroHPC dataset, specifically the kvkk subset. The fine-tuning leverages the Axolotl framework for orchestration and Unsloth library for optimized training kernels. |
|
|
| ### Hyperparameters |
|
|
| * **LoRA Rank**: 16 |
| * **LoRA Alpha**: 32 |
| * **LoRA Dropout**: 0.05 |
| * **Learning Rate**: 3×10⁻⁵ with cosine scheduling |
| * **Training Epochs**: 3 per domain |
| * **Batch Size**: Optimized for A100 memory capacity |
|
|
| ### Architecture |
|
|
| * **Base Model**: Llama-3.1-8B-Instruct (Meta) |
| * **Fine-tuning Method**: LoRA (Low-Rank Adaptation) |
| * **Parameter Efficiency**: Only trainable LoRA parameters, frozen base model |
| * **Model Size**: 8B parameters (base) + LoRA adapters |
|
|
| ## Hardware and Software |
|
|
| * **Orchestration**: [Axolotl framework](https://axolotl.ai/) |
| * **Acceleration**: [Unsloth library](https://unsloth.ai/) |
| * **Backend**: PyTorch with CUDA support |
| * **System**: EuroHPC Karolina supercomputer |
| * **GPUs**: NVIDIA A100 (8 × 40 GB per node, 320 GB HBM2 total) |
| * **Utilization**: 85–90% GPU and memory efficiency |
| * **Total Compute**: ~600 GPU hours |
|
|
| ## Data |
|
|
| ### Input Format |
|
|
| The dataset follows the **Alpaca format** with three key fields: |
|
|
| ```json |
| { |
| "instruction": "Task description or question", |
| "input": "Additional context or input data", |
| "output": "Expected response or answer" |
| } |
| ``` |
|
|
| **Dataset**: `newmindai/EuroHPC-Legal` (kvkk subset) |
|
|
| ## How to Use |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| from peft import PeftModel |
| |
| # Load base model and tokenizer |
| base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct") |
| tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct") |
| |
| # Load LoRA adapter |
| model = PeftModel.from_pretrained(base_model, "newmindai/Llama-3.1-8B-Instruct-kvkk-alpaca") |
| |
| # Format input according to Alpaca format |
| def format_prompt(instruction, input_text=""): |
| if input_text: |
| return f"### Instruction:\n{instruction}\n\n### Input:\n{input_text}\n\n### Response:\n" |
| else: |
| return f"### Instruction:\n{instruction}\n\n### Response:\n" |
| |
| # Example usage |
| prompt = format_prompt("Explain the benefits of regular exercise") |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=256) |
| response = tokenizer.decode(outputs[0], skip_special_tokens=True) |
| ``` |
|
|
| ## Acknowledgments |
|
|
| This research was supported by the EuroHPC Joint Undertaking (EuroHPC JU) under the Benchmark Access |
| grant agreement No EHPC-BEN-2024B11-003. The authors gratefully acknowledge the computational resources |
| provided by the IT4Innovations National Supercomputing Center (Czech Republic) on the Karolina supercomputer, |
| made available through the EuroHPC JU. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{newmind2025, |
| title={Tailoring AI for Turkish Law: Domain-Specific Fine-Tuning of Small Language Models for Legal Expertise}, |
| author={New Mind AI Team}, |
| journal={Procedia Computer Science}, |
| year={2025}, |
| volume={239}, |
| doi={10.1016/j.procs.2025.08.239}, |
| note={Available online 23 September 2025, Version of Record 23 September 2025} |
| } |
| ```--- |
| license: llama3.1 |
| datasets: |
| - newmindai/Euro_HPC |
| language: |
| - tr |
| - en |
| base_model: |
| - meta-llama/Llama-3.1-8B-Instruct |
| tags: |
| - EuroHPC |
| - Karolina |
| - Axolotl |
| - Unsloth |
| --- |
|
|
| <div style="display: flex; justify-content: center; flex-wrap: wrap; gap: 15px; align-items: flex-start;"> |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/683d4880e639f8d647355997/mqbOdFfrC7KjDZbQlLFFj.png" |
| style="width: 100%; max-width: 300px; height: auto;" /> |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/683d4880e639f8d647355997/VGnh14pYg-SXSaEt640qz.png" |
| style="width: 100%; max-width: 150px; height: auto;" /> |
| </div> |
|
|
| ## Model Card |
|
|
| This document describes a parameter-efficient fine-tuning setup using LoRA on the EuroHPC Karolina system. Axolotl provides flexible orchestration and Unsloth supplies optimized kernels for high-throughput training on the Euro_HPC dataset. This model is fine-tuned using Parameter-Efficient Fine-Tuning (PEFT) with LoRA (Low-Rank Adaptation) on the EuroHPC dataset, specifically the kvkk subset. The fine-tuning leverages the Axolotl framework for orchestration and Unsloth library for optimized training kernels. |
| |
| ### Hyperparameters |
| |
| * **LoRA Rank**: 16 |
| * **LoRA Alpha**: 32 |
| * **LoRA Dropout**: 0.05 |
| * **Learning Rate**: 3×10⁻⁵ with cosine scheduling |
| * **Training Epochs**: 3 per domain |
| * **Batch Size**: Optimized for A100 memory capacity |
| |
| ### Architecture |
| |
| * **Base Model**: Llama-3.1-8B-Instruct (Meta) |
| * **Fine-tuning Method**: LoRA (Low-Rank Adaptation) |
| * **Parameter Efficiency**: Only trainable LoRA parameters, frozen base model |
| * **Model Size**: 8B parameters (base) + LoRA adapters |
| |
| ## Hardware and Software |
| |
| * **Orchestration**: [Axolotl framework](https://axolotl.ai/) |
| * **Acceleration**: [Unsloth library](https://unsloth.ai/) |
| * **Backend**: PyTorch with CUDA support |
| * **System**: EuroHPC Karolina supercomputer |
| * **GPUs**: NVIDIA A100 (8 × 40 GB per node, 320 GB HBM2 total) |
| * **Utilization**: 85–90% GPU and memory efficiency |
| * **Total Compute**: ~600 GPU hours |
| |
| ## Data |
| |
| ### Input Format |
| |
| The dataset follows the **Alpaca format** with three key fields: |
| |
| ```json |
| { |
| "instruction": "Task description or question", |
| "input": "Additional context or input data", |
| "output": "Expected response or answer" |
| } |
| ``` |
| |
| **Dataset**: `newmindai/Euro_HPC` (kvkk subset) |
|
|
| ## How to Use |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| from peft import PeftModel |
| |
| # Load base model and tokenizer |
| base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct") |
| tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct") |
| |
| # Load LoRA adapter |
| model = PeftModel.from_pretrained(base_model, "newmindai/Llama-3.1-8B-Instruct-kvkk-alpaca") |
| |
| # Format input according to Alpaca format |
| def format_prompt(instruction, input_text=""): |
| if input_text: |
| return f"### Instruction:\n{instruction}\n\n### Input:\n{input_text}\n\n### Response:\n" |
| else: |
| return f"### Instruction:\n{instruction}\n\n### Response:\n" |
| |
| # Example usage |
| prompt = format_prompt("Explain the benefits of regular exercise") |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=256) |
| response = tokenizer.decode(outputs[0], skip_special_tokens=True) |
| ``` |
|
|
| ## Acknowledgments |
|
|
| This research was supported by the EuroHPC Joint Undertaking (EuroHPC JU) under the Benchmark Access |
| grant agreement No EHPC-BEN-2024B11-003. The authors gratefully acknowledge the computational resources |
| provided by the IT4Innovations National Supercomputing Center (Czech Republic) on the Karolina supercomputer, |
| made available through the EuroHPC JU. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{newmind2025, |
| title={Tailoring AI for Turkish Law: Domain-Specific Fine-Tuning of Small Language Models for Legal Expertise}, |
| author={New Mind AI Team}, |
| journal={Procedia Computer Science}, |
| year={2025}, |
| volume={239}, |
| doi={10.1016/j.procs.2025.08.239}, |
| note={Available online 23 September 2025, Version of Record 23 September 2025} |
| } |
| ``` |
