---
library_name: transformers
tags:
- financial
- swift
- mt564
- gemma
- lora
- corporate-actions
- causal-lm
- transformers
- huggingface
- fine-tuned
- finance
demo: https://huggingface.co/pareshmishra/mt564-gemma-lora-chat # Ensure this is your actual Space URL
license: mit
# Removed 'metrics: accuracy' unless you have specific accuracy scores.
base_model: google/gemma-2b-it
pipeline_tag: text-generation # Changed from text2text-generation
---


# Model Card: MT564-Gemma-LoRA

This model is a fine-tuned version of `google/gemma-3-1b-it` designed to **analyze SWIFT MT564 corporate action messages** and flag potential structural or compliance-related anomalies. It supports extracting sequences, identifying missing fields, and detecting risky patterns such as incorrect codes, unusual currencies, or sanctioned countries.

---

## Model Details

### Model Description

- **Developer:** Paresh Mishra
- **Model Type:** Causal Language Model (Instruction-tuned)
- **Language(s):** English, Financial NLP
- **Base Model:** `google/gemma-2b-it`
- **Fine-tuning:** PEFT / LoRA (r=16, alpha=32, dropout=0.05)
- **Framework:** Hugging Face Transformers

### Sources

- **Model Repo:** https://huggingface.co/pareshmishra/mt564-gemma-lora
- **Dataset:** Custom-crafted `.jsonl` from MT564 structure, PDF guidelines, and synthetic variations

---

## Uses

### Direct Use

- Identify anomalies in SWIFT MT564 messages
- Understand sequences (GENL, CAOPTN, etc.)
- Verify country/currency codes for compliance
- Detect missing mandatory fields or wrong order

### Downstream Use

- Can be integrated into:
  - Compliance tools
  - Audit automation platforms
  - Financial reporting systems

### Out-of-Scope Use

- General-purpose chat
- Legal or regulatory interpretation without human oversight

---

## Bias, Risks, and Limitations

This model:
- May not generalize beyond SWIFT MT564 unless retrained.
- May hallucinate anomalies when fields are non-standard but valid.
- Should **not be used in production without human validation**.

### Recommendations

- Always cross-validate flagged anomalies with domain experts.
- Extend dataset with more ISO20022-compliant and real-world examples.

---

## How to Use

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "pareshmishra/mt564-gemma-lora"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = """### Instruction:
Analyze this MT564 message for anomalies

### Input:
{1:F01TESTBANKXXXX0000000000}{2:I564CLIENTBANKXXXXN}{4:
:16R:GENL
:20C::CORP//CA20250501
:23G:NEWM
:22F::CAEV//DVCA
:16S:GENL
:16R:CAOPTN
:13A::CAON//001
:36B::ENTL//UNIT/5000000
:19A::SETT//ZAR/5000000
:95Q::RCPT//KP
:16S:CAOPTN
}

### Response:"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details
Training Data
80+ high-quality JSONL records crafted from:

ISO20022 documentation

swift_ISO20022.pdf

Simulated MT564 edge cases

Format: "text": "### Instruction:\n...\n### Input:\n...\n### Response:\n..."

Training Hyperparameters
Parameter	Value
Epochs	3
Batch Size	2
Gradient Accum	4
Learning Rate	3e-5
LoRA r	16
LoRA Alpha	32
Dropout	0.05
Max Length	2048
Quantization	int4
Precision	fp16

Hardware
Environment: Google Colab

GPU: T4

Training Time: ~12 mins

Evaluation
Metrics
Manual evaluation using expected vs. actual anomaly detection

Correctly flagged missing sequences and invalid codes

Environmental Impact
Hardware Type: Google Colab T4

Hours used: ~0.2

Cloud Provider: Google

Carbon Estimate: ~0.02 kgCO₂e (via MLCO2 calculator)

Citation
latex
 
@misc{mt564gemma,
  title={MT564-Gemma-LoRA},
  author={Paresh Mishra},
  year={2025},
  howpublished={\url{https://huggingface.co/pareshmishra/mt564-gemma-lora}},
}
Contact
GitHub: @pareshmishra

Hugging Face: pareshmishra