jagrutsharma/gpt-oss-2048-lora

GPT-OSS 2048 LoRA Adapter

A LoRA adapter fine-tuned on the GPT-OSS 20B model using GRPO (Group Relative Policy Optimization) to play the 2048 puzzle game.

Training Details

Parameter	Value
Base Model	unsloth/gpt-oss-20b
Method	QLoRA (4-bit) + GRPO Reinforcement Learning
LoRA Rank	4
Training Steps	10
Hardware	NVIDIA DGX Spark (GB10 Blackwell GPU)
Framework	Unsloth + TRL

What This Model Does

The model was trained with reinforcement learning to play the 2048 sliding puzzle game. Given a board state, it suggests the optimal move (W/A/S/D for up/left/down/right).

Reward Functions

• function_works: Valid move format (W/A/S/D)
• no_cheating: Legal moves only (moves that change the board)
• strategy_succeeds: Moves that increase game score

Training Context

This model was fine-tuned as a learning exercise to understand:

• LoRA/QLoRA parameter-efficient fine-tuning
• GRPO reinforcement learning for LLMs
• Unsloth optimization framework
• DGX Spark infrastructure

Usage

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "jagrutsharma/gpt-oss-2048-lora",
    max_seq_length = 768,
    load_in_4bit = True,
)
FastLanguageModel.for_inference(model)

Limitations

• Only 10 training steps (learning exercise, not production model)
• Would need 500-1000+ steps for significant gameplay improvement

Acknowledgments

• Unsloth for the optimized fine-tuning framework
• NVIDIA DGX Spark for compute
• Tutorial: Fine-tuning LLMs with DGX Spark and Unsloth