Upload README.md with huggingface_hub

README.md

@@ -260,6 +260,48 @@ We conducted head-to-head live coding tests against a HybridBase variant (same u
 | Feature scope | 8 commands, knowledge tracker | 8 commands, simpler |
 | Token usage | ~80k | Unknown (couldn't complete) |
 
+### One-Shot Project Tests (Live Coding)
+
+Two complete projects were given as one-shot prompts to this model — no follow-up corrections, no multi-turn debugging. The model was expected to produce a fully working project from a single detailed prompt, including virtual environment setup, test scripts, and sample data.
+
+> **Note:** The author believes the sampling parameters used during testing (temp=0.8, top_p=0.95, top_k=20) may not be fully optimal for this model. There is likely room for improved performance with further parameter tuning.
+
+**Test 1: Study Flashcard Engine (one-shot, ~40-50k tokens generated)**
+
+Prompt: Build a complete Python CLI tool with SM-2 spaced repetition, 7 commands, JSON API for agent integration, venv setup, and test suite.
+
+| Aspect | Result |
+|--------|--------|
+| Runs without errors | ✅ Yes |
+| All commands functional | ✅ 7/7 |
+| SM-2 algorithm | ✅ Correctly implemented |
+| Project structure | ✅ Proper folder, setup.sh, test.sh, test_notes.txt |
+| Card quality | ✅ Clean term-definition pairs, proper key term extraction |
+| CSV export | ✅ Real CSV output using csv.writer |
+| Deduplication | ✅ Explicit seen_fronts set prevents duplicates |
+| Code style | Clean functional style, 788 lines |
+| Bugs found | 1 minor (case sensitivity in some edge cases) |
+
+**Test 2: Quiz Generator & Knowledge Tracker (one-shot, ~80k tokens generated)**
+
+Prompt: Build a more complex Python CLI tool with 8 commands, 4 question types (MCQ, T/F, fill-in-blank, short answer), knowledge tracking with concept-level granularity, study suggestions, progress visualization, and scoring with partial credit.
+
+| Aspect | Result |
+|--------|--------|
+| Runs without errors | ✅ Yes |
+| All commands functional | ✅ 6/8 (suggest command produces generic output with fresh data, weak-topics empty when no errors recorded — both technically correct but not useful as demos) |
+| Question generation | ✅ 4 types working (MCQ, T/F, fill, short answer) |
+| Scoring pipeline | ✅ take → score → knowledge tracker update works end-to-end |
+| Knowledge tracking | ✅ Concept-level accuracy tracking, trend detection |
+| Project structure | ⚠️ Files extracted without containing folder |
+| Token usage | ~80k — the model autonomously managed file creation, venv setup, testing, and SSH transfer |
+| Bugs found | 2-3 moderate: MCQ questions include markdown headers in question text, T/F generation has a for/else creating unintentional duplicates, suggest command pads with generic entries |
+
+**Severity assessment of bugs:** All bugs found in Test 2 are **surface-level generation quality issues**, not architectural or logic errors. The scoring pipeline, knowledge tracker, and data persistence all work correctly. The MCQ header parsing and T/F duplicate issues would likely be fixed in a single follow-up prompt ("fix the question text including markdown headers" / "fix the T/F for/else logic"), requiring minimal effort. No bugs required architectural changes.
+
+**For comparison — a rival variant (HybridBase) was given the same Quiz Generator prompt:**
+The HybridBase variant **looped twice** during generation (requiring manual intervention: "you are looping, stop, evaluate then continue"), and the final output **crashed on launch** with a `NameError: name 'sentence' is not defined` — a variable scoping error where the model referenced a loop variable from a different function. After the looping and intervention, the model was unable to complete the task cleanly. This is a fundamental code generation failure that would require significant debugging, not a quick fix.
+
 ### Partial Benchmark Results
 
 **Custom Quick Bench (50 problems: 20 coding, 15 reasoning, 15 math)**