Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+EXAONE-3.5-7.8B-Instruct.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+EXAONE-3.5-7.8B-Instruct.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+EXAONE-3.5-7.8B-Instruct.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+EXAONE-3.5-7.8B-Instruct.F16.gguf filter=lfs diff=lfs merge=lfs -text

EXAONE-3.5-7.8B-Instruct.F16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:061f388d7d4b2d868df255fc242ddf8d62de761c33cbaeed2f6a7958971299de
+size 15641630656

EXAONE-3.5-7.8B-Instruct.Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7e414eb3170959c1621253d5edfc58cff5bca27e2169ef939308383c096c363
+size 4770650048

EXAONE-3.5-7.8B-Instruct.Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37d4b2e27f6e68ffa5cc595bf60d081e730c0b7059b3cc38fd8f416f79bb87f0
+size 5569664960

EXAONE-3.5-7.8B-Instruct.Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:568246549fc32ed784d7790843670943aa505ac25b031deda0cb64fe1ffb357d
+size 8312084416

README.md

@@ -1,5 +1,63 @@
----
-license: other
-license_name: exaone
-license_link: https://huggingface.co/LGAI-EXAONE/EXAONE-Deep-2.4B/blob/main/LICENSE
----
+# Calvin806/EXAONE-3.5-7.8B-Instruct-GGUF
+
+GGUF quantizations for **EXAONE-3.5-7.8B-Instruct**.
+
+## Contents
+This folder typically contains:
+- `EXAONE-3.5-7.8B-Instruct.F16.gguf`
+- `EXAONE-3.5-7.8B-Instruct.Q4_K_M.gguf`
+- `EXAONE-3.5-7.8B-Instruct.Q5_K_M.gguf`
+- `EXAONE-3.5-7.8B-Instruct.Q8_0.gguf` (optional)
+
+---
+
+## 🔧 llama.cpp patch (EXAONE GGUF quantize compatibility)
+
+EXAONE GGUF 변환/양자화 과정에서 일부 모델(예: **2.4B / 7.8B**) 간 **KV key 네이밍 불일치**가 발견되었습니다.
+
+- 어떤 GGUF는 `exaone.attention.layer_norm_epsilon`만 존재
+- 어떤 GGUF는 `exaone.attention.layer_norm_rms_epsilon`만 존재
+
+이 상태에서 vanilla llama.cpp의 `llama-quantize`가 특정 키를 찾지 못해 실패할 수 있어,
+**llama.cpp의 model loader에서 gguf key lookup에 fallback을 추가하는 패치**를 적용했습니다.
+
+### What was patched
+
+`src/llama-model-loader.cpp`에서 `gguf_find_key()` lookup에 다음 fallback을 수행하도록 수정:
+
+- key가 `exaone.attention.layer_norm_epsilon`이고 찾지 못하면 → `exaone.attention.layer_norm_rms_epsilon`로 재시도
+- key가 `exaone.attention.layer_norm_rms_epsilon`이고 찾지 못하면 → `exaone.attention.layer_norm_epsilon`로 재시도
+
+이 패치를 통해 **EXAONE 3.5 / EXAONE-Deep 2.4B, 7.8B, 32B** 계열을 동일 파이프라인으로 GGUF+quantize할 수 있습니다.
+
+### Patch note (minimal diff summary)
+- Added a fallback wrapper/hook for `gguf_find_key()` inside `llama-model-loader.cpp`
+- Ensured all lookups in that translation unit route through the fallback
+
+This repo includes:
+- `exaone-gguf-fallback.patch`
+
+### Tested llama.cpp commit
+- `021cc28bef4dd7d0bf9c91dbbd0803caa6cb15f2`
+
+---
+
+## Build (CUDA)
+```bash
+git clone https://github.com/ggml-org/llama.cpp
+cd llama.cpp
+git apply ../exaone-gguf-fallback.patch
+
+rm -rf build
+cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
+cmake --build build -j
+```
+
+## Convert / Quantize
+```bash
+# Convert HF snapshot -> GGUF(F16)
+python3 llama.cpp/convert_hf_to_gguf.py <LOCAL_SNAPSHOT_DIR> --outtype f16 --outfile model.F16.gguf
+
+# Quantize (example: Q4_K_M)
+llama.cpp/build/bin/llama-quantize model.F16.gguf model.Q4_K_M.gguf Q4_K_M
+```

exaone-gguf-fallback.patch

@@ -0,0 +1,31 @@
+diff --git a/src/llama-model-loader.cpp b/src/llama-model-loader.cpp
+index bd9e6da88..5a120d69a 100644
+--- a/src/llama-model-loader.cpp
++++ b/src/llama-model-loader.cpp
+@@ -1,4 +1,26 @@
+ #include "llama-model-loader.h"
++#include <cstring>
++
++// EXAONE_GGUF_FIND_KEY_MACRO_HOOK
++// Hook gguf_find_key() inside this translation unit to handle EPS/RMS key mismatch for EXAONE.
++static auto gguf_find_key_orig = &gguf_find_key;
++
++static int gguf_find_key_exaone_fallback(const gguf_context * ctx, const char * key) {
++    int kid = gguf_find_key_orig(ctx, key);
++    if (kid >= 0) return kid;
++
++    if (strcmp(key, "exaone.attention.layer_norm_epsilon") == 0) {
++        return gguf_find_key_orig(ctx, "exaone.attention.layer_norm_rms_epsilon");
++    }
++    if (strcmp(key, "exaone.attention.layer_norm_rms_epsilon") == 0) {
++        return gguf_find_key_orig(ctx, "exaone.attention.layer_norm_epsilon");
++    }
++    return -1;
++}
++
++// Redirect all gguf_find_key calls in this file
++#define gguf_find_key(ctx, key) gguf_find_key_exaone_fallback((ctx), (key))
++
+ 
+ #include "ggml.h"
+