gyung commited on
Commit
c413eee
·
verified ·
1 Parent(s): 320d3d5

Update KoHRM terminal lineage with epoch3 score

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -53,6 +53,7 @@ The base model itself is weak on TB2-lite terminal next-action JSON without task
53
  | `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-Top2-Terminal-Tool-Merge-Epoch1` | full fine-tune | 31.59 | 0.3159 | 0.3859 | 0.3415 | 24.8% | 73.3% |
54
  | `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch1` | full fine-tune | 38.56 | 0.3856 | 0.4262 | 0.4341 | 37.0% | 55.1% |
55
  | `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch2` | full fine-tune | 45.90 | 0.4590 | 0.5031 | 0.5098 | 44.9% | 68.3% |
 
56
 
57
  `Score = 100 * avg_command_f1` on the corrected 303-step TB2-lite full replay set.
58
 
@@ -60,7 +61,7 @@ The current best KoHRM terminal checkpoint is:
60
 
61
  https://huggingface.co/LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch2
62
 
63
- Epoch2 is a fine-tuned version of this base model and continues from the Epoch1 LFM25/ToolBench full-SFT checkpoint. It improved over Epoch1 by `+7.34` Score, with Valid JSON `55.1% -> 68.3%`, First Cmd `37.0% -> 44.9%`, Precision `0.4262 -> 0.5031`, and Recall `0.4341 -> 0.5098`.
64
 
65
  Strong Epoch2 areas are `data_querying` (`0.6881` F1), `data_science` (`0.4901`), `debugging` (`0.4857`), `math` (`0.4845`), `software_engineering` (`0.4770`), and `file_operations` (`0.4710`). Remaining weak areas are `swe` (`0.3590`), `data_processing` (`0.4017`), `dependency_management` (`0.4025`), `security` (`0.4220`), and `model_training` (`0.4283`). The main remaining gap to the LFM2.5 top checkpoints is first-action accuracy and late-step command coverage.
66
 
 
53
  | `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-Top2-Terminal-Tool-Merge-Epoch1` | full fine-tune | 31.59 | 0.3159 | 0.3859 | 0.3415 | 24.8% | 73.3% |
54
  | `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch1` | full fine-tune | 38.56 | 0.3856 | 0.4262 | 0.4341 | 37.0% | 55.1% |
55
  | `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch2` | full fine-tune | 45.90 | 0.4590 | 0.5031 | 0.5098 | 44.9% | 68.3% |
56
+ | `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch3` | full fine-tune | 43.57 | 0.4357 | 0.4703 | 0.5003 | 45.5% | 61.7% |
57
 
58
  `Score = 100 * avg_command_f1` on the corrected 303-step TB2-lite full replay set.
59
 
 
61
 
62
  https://huggingface.co/LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch2
63
 
64
+ Epoch2 remains the current best KoHRM terminal checkpoint. Epoch3 was evaluated as a continuation from Epoch2 and scored `43.57`, `-2.33` versus Epoch2. Epoch3 slightly improved First Cmd to `45.5%`, but Cmd F1, precision, recall, and Valid JSON all regressed, so Epoch2 is kept as the representative terminal checkpoint.
65
 
66
  Strong Epoch2 areas are `data_querying` (`0.6881` F1), `data_science` (`0.4901`), `debugging` (`0.4857`), `math` (`0.4845`), `software_engineering` (`0.4770`), and `file_operations` (`0.4710`). Remaining weak areas are `swe` (`0.3590`), `data_processing` (`0.4017`), `dependency_management` (`0.4025`), `security` (`0.4220`), and `model_training` (`0.4283`). The main remaining gap to the LFM2.5 top checkpoints is first-action accuracy and late-step command coverage.
67