trjxter/TraceAlchemy-Gemma-4-E4B-Finance-IT-bf16

TraceAlchemy-Gemma-4-E4B-Finance-IT BF16

TraceAlchemy-Gemma-4-E4B-Finance-IT is a finance-focused instruction-tuned Gemma 4 E4B model trained to improve careful financial reasoning, financial table understanding, unit and scale handling, sign and direction checks, and final-answer consistency.

This repository contains the merged BF16 model. The LoRA adapter has already been merged into the base model weights, so this repo can be loaded directly as a full Hugging Face Transformers model.

The model was fine-tuned and merged using Unsloth.

Model Summary

This model was trained as a finance reasoning assistant with a focus on:

• Financial statement reasoning
• Revenue, margin, growth, and ratio calculations
• SEC-style table and excerpt extraction
• Unit and scale conversion, such as thousands to millions
• Sign and direction reasoning
• Multi-step table reasoning
• Final-answer consistency checking
• General finance instruction following

The goal of this run was not to teach the model static finance facts. Instead, the goal was to improve the model’s behavior on finance reasoning workflows. The training data emphasizes showing calculations, checking units, avoiding unsupported assumptions, and producing clear final answers.

Base Model

• Base model: unsloth/gemma-4-E4B-it
• Fine-tuned model name: TraceAlchemy-Gemma-4-E4B-Finance-IT
• Architecture class: Gemma 4 E4B instruction model
• Training method: LoRA supervised fine-tuning
• Final format in this repository: merged BF16 Hugging Face model

Gemma 4 E4B is treated as an effective E4B-class model. During training, the loaded parameter count was approximately 8.1B parameters including embeddings.

What This Repository Contains

This is the merged BF16 model, not just the LoRA adapter.

That means the training flow was:

Base Gemma model
+ trained LoRA adapter
→ merged 16-bit model
→ uploaded as full BF16 model

The important model file is:

model.safetensors

This file contains the merged model weights. You should not expect to see:

adapter_model.safetensors
adapter_config.json

Those files belong in the LoRA adapter repository, not the merged BF16 repository.

Training Configuration

Setting	Value
Base model	unsloth/gemma-4-E4B-it
Training method	LoRA SFT
Base loading during training	8-bit
Merge format	BF16
Max sequence length	16,384
LoRA rank	64
LoRA alpha	64
Learning rate	2e-5
Epochs	1
Per-device batch size	2
Gradient accumulation steps	8
Effective batch size	16
Optimizer	adamw_8bit
Training examples	10,000
Evaluation examples	630
Total training steps	625
Training runtime	~2.93 hours on A100-class GPU

Training Completion

The run completed successfully with the following final training output:

global_step: 625
epoch: 1.0
train_runtime: 10558.7348 seconds
train_samples_per_second: 0.947
train_steps_per_second: 0.059

The final run completed all 625 planned training steps.

Validation and Evaluation During Training

The model was evaluated every 50 training steps on a held-out evaluation set of 630 examples.

Validation loss is reported as language-modeling loss on the held-out evaluation split. It is not the same thing as benchmark accuracy, but it is useful for checking whether the model is improving on examples it is not directly training on.

The validation trend was strong during the run:

Step	Validation Loss
50	0.765309
100	0.409274
150	0.318539
200	0.286068
250	0.271857
300	0.263259
350	0.257434

The validation loss dropped from 0.765309 at step 50 to 0.257434 by step 350, showing steady improvement on held-out finance examples during training.

This suggests the model was not only fitting the training examples, but also improving on the validation set.

More benchmarks soon.

Dataset Mix

The training set used a 10,000-example finance-focused mixture.

The data recipe was built around a real finance-data anchor from previous experimentation, then expanded with targeted synthetic examples designed to address specific finance reasoning failure modes.

Real Finance Data

Source	Train	Eval
FinanceBench	120	30
TAT-QA	2,200	100
ConvFinQA	1,800	100
FinanceReasoning	750	100
Finance-Instruct-500k	400	50
Total real examples	5,270	380

Synthetic Finance Data

The synthetic examples targeted specific reasoning skills:

Synthetic Category	Train	Eval
Hard SEC-style extraction	1,500	50
Final-answer consistency checks	1,200	50
Sign, scale, and direction reasoning	900	50
Multi-step distractor tables	800	50
Hard unit-scale conversion	330	50
Total synthetic examples	4,730	250

Final Dataset Size

Split	Examples
Train	10,000
Eval	630

Dataset Credits

This model was trained using examples derived from or inspired by the following datasets:

• PatronusAI/financebench
• next-tat/TAT-QA
• AdaptLLM/ConvFinQA
• BUPT-Reasoning-Lab/FinanceReasoning
• Josephgflowers/Finance-Instruct-500k

Additional synthetic examples were generated for targeted finance reasoning skills, including SEC-style extraction, scale and unit conversion, sign and direction reasoning, multi-step table reasoning, and final-answer consistency checks.

What Was Tested

During training, the model was tested through validation loss on a held-out evaluation split containing both real and synthetic finance examples.

The evaluation set included:

• FinanceBench-style question answering
• TAT-QA table and text reasoning
• ConvFinQA conversational finance QA
• FinanceReasoning-style calculation and reasoning examples
• Finance-Instruct examples
• Synthetic SEC-style extraction checks
• Synthetic unit and scale conversion checks
• Synthetic sign and direction checks
• Synthetic final-answer consistency checks
• Synthetic multi-step distractor table checks

This validation setup was intended to check whether the model could generalize beyond the exact training examples while staying focused on finance reasoning behavior.

Further external benchmark testing is planned.

More benchmarks soon.

Files in This Repository

The expected main files include:

model.safetensors
config.json
tokenizer.json
tokenizer_config.json
processor_config.json
chat_template.jinja
README.md

The model.safetensors file is the full merged BF16 model.

For an approximately 8.1B parameter model, a BF16 model file around 16 GB is expected because BF16 uses about 2 bytes per parameter.

Loading the Model

Example loading code:

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_id = "trjxter/TraceAlchemy-Gemma-4-E4B-Finance-IT-bf16"

processor = AutoProcessor.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": "A financial table says amounts are in thousands. Revenue is 302,578 and gross profit is 91,842. Express revenue in millions and calculate gross margin.",
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    input_ids=inputs,
    max_new_tokens=256,
    do_sample=False,
)

print(processor.decode(outputs[0], skip_special_tokens=False))

Depending on your local environment and installed Transformers version, Gemma 4 support may require a recent Transformers release.

Intended Use

This model is intended for finance reasoning and educational/research workflows, especially tasks involving:

• Financial statement reasoning
• SEC-style table and excerpt extraction
• Revenue, margin, growth, and ratio calculations
• Unit and scale conversion
• Sign and direction checks
• Final-answer consistency checking
• General finance instruction following

Limitations

This model should not be treated as a source of guaranteed financial truth.

It may still:

• Make calculation mistakes
• Misread financial tables
• Mis-handle units or scales
• Produce unsupported conclusions
• Hallucinate details not present in the prompt
• Fail on complex accounting, legal, or investment questions

For real investment, accounting, legal, or business decisions, verify outputs against original filings, audited statements, and qualified professionals.

Related Artifacts

This BF16 repository is part of the TraceAlchemy Gemma finance run.

Expected related repositories:

• LoRA adapter: trjxter/TraceAlchemy-Gemma-4-E4B-Finance-IT-lora
• Merged BF16 model: trjxter/TraceAlchemy-Gemma-4-E4B-Finance-IT-bf16
• GGUF model: trjxter/TraceAlchemy-Gemma-4-E4B-Finance-IT-gguf

BF16 vs Adapter vs GGUF

This repo is the merged BF16 model.

Repository Type	What It Contains	Use Case
LoRA adapter	Small adapter delta weights	Use with the original base model
Merged BF16	Full merged model weights	Transformers/HF inference or further conversion
GGUF	Quantized merged model files	llama.cpp, LM Studio, Ollama, local inference

This repository does not require loading the adapter separately because the adapter has already been merged into the full BF16 model weights.

Training and Merge Process

This model was fine-tuned and merged using Unsloth.

The merge flow was:

Base Gemma model + LoRA adapter
→ merged 16-bit model
→ saved as full BF16 Hugging Face model
→ Hugging Face upload

Unsloth

This model was trained and merged with Unsloth.