Update KoHRM terminal lineage with epoch3 score
Browse files
README.md
CHANGED
|
@@ -53,6 +53,7 @@ The base model itself is weak on TB2-lite terminal next-action JSON without task
|
|
| 53 |
| `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-Top2-Terminal-Tool-Merge-Epoch1` | full fine-tune | 31.59 | 0.3159 | 0.3859 | 0.3415 | 24.8% | 73.3% |
|
| 54 |
| `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch1` | full fine-tune | 38.56 | 0.3856 | 0.4262 | 0.4341 | 37.0% | 55.1% |
|
| 55 |
| `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch2` | full fine-tune | 45.90 | 0.4590 | 0.5031 | 0.5098 | 44.9% | 68.3% |
|
|
|
|
| 56 |
|
| 57 |
`Score = 100 * avg_command_f1` on the corrected 303-step TB2-lite full replay set.
|
| 58 |
|
|
@@ -60,7 +61,7 @@ The current best KoHRM terminal checkpoint is:
|
|
| 60 |
|
| 61 |
https://huggingface.co/LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch2
|
| 62 |
|
| 63 |
-
Epoch2
|
| 64 |
|
| 65 |
Strong Epoch2 areas are `data_querying` (`0.6881` F1), `data_science` (`0.4901`), `debugging` (`0.4857`), `math` (`0.4845`), `software_engineering` (`0.4770`), and `file_operations` (`0.4710`). Remaining weak areas are `swe` (`0.3590`), `data_processing` (`0.4017`), `dependency_management` (`0.4025`), `security` (`0.4220`), and `model_training` (`0.4283`). The main remaining gap to the LFM2.5 top checkpoints is first-action accuracy and late-step command coverage.
|
| 66 |
|
|
|
|
| 53 |
| `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-Top2-Terminal-Tool-Merge-Epoch1` | full fine-tune | 31.59 | 0.3159 | 0.3859 | 0.3415 | 24.8% | 73.3% |
|
| 54 |
| `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch1` | full fine-tune | 38.56 | 0.3856 | 0.4262 | 0.4341 | 37.0% | 55.1% |
|
| 55 |
| `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch2` | full fine-tune | 45.90 | 0.4590 | 0.5031 | 0.5098 | 44.9% | 68.3% |
|
| 56 |
+
| `LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch3` | full fine-tune | 43.57 | 0.4357 | 0.4703 | 0.5003 | 45.5% | 61.7% |
|
| 57 |
|
| 58 |
`Score = 100 * avg_command_f1` on the corrected 303-step TB2-lite full replay set.
|
| 59 |
|
|
|
|
| 61 |
|
| 62 |
https://huggingface.co/LLM-OS-Models/KoHRM-Text-1.4B-FullSFT-LFM25-Terminal-ToolBench-Epoch2
|
| 63 |
|
| 64 |
+
Epoch2 remains the current best KoHRM terminal checkpoint. Epoch3 was evaluated as a continuation from Epoch2 and scored `43.57`, `-2.33` versus Epoch2. Epoch3 slightly improved First Cmd to `45.5%`, but Cmd F1, precision, recall, and Valid JSON all regressed, so Epoch2 is kept as the representative terminal checkpoint.
|
| 65 |
|
| 66 |
Strong Epoch2 areas are `data_querying` (`0.6881` F1), `data_science` (`0.4901`), `debugging` (`0.4857`), `math` (`0.4845`), `software_engineering` (`0.4770`), and `file_operations` (`0.4710`). Remaining weak areas are `swe` (`0.3590`), `data_processing` (`0.4017`), `dependency_management` (`0.4025`), `security` (`0.4220`), and `model_training` (`0.4283`). The main remaining gap to the LFM2.5 top checkpoints is first-action accuracy and late-step command coverage.
|
| 67 |
|