mp-juuuns commited on
Commit
38a661e
ยท
verified ยท
1 Parent(s): 0840bf5

Name the Git LFS problem instead of failing inside gzip (the macOS default)

Browse files

macOS git ships without git-lfs, so a plain clone leaves 132-byte pointer files where docker/bundle/runtime_source.tar.gz, the GGUF and model.safetensors should be. Reproduced: extracting the pointer fails with 'gzip: stdin: not in gzip format', which says nothing about LFS.

docker/start.sh checks both LFS-tracked bundle files before unpacking and stops with the fix spelled out (brew/apt install git-lfs, git lfs install, git lfs pull - or use hf download). docker-train's Dockerfile checks that the ADD actually unpacked the archive, because docker copies a pointer verbatim instead of failing. make_base.py checks model.safetensors the same way.

Docs: both cards and all four container READMEs now say how to get real files (hf download, or git-lfs before cloning). The Korean docker-train README still described the old research-repo flow of unpacking a separately supplied base package; it now uses make_base.py like the English one. The Korean docker README pointed at a research-repo path; it now says docker/.

Verified before publishing: the pointer case prints the message and exits 1 in all three places; with real files docker/ reaches ready and classifies, and docker-train runs make_base -> build -> validate -> train -> export (parity 0.0 / 0.0, 0 disagreements) -> serve. Also checked: no case-insensitive path collisions in the repo or inside the archives, and apple-silicon builds are not blocked by x86 flags (docker/ builds llama.cpp with every x86 extension off).

README.md CHANGED
@@ -102,6 +102,18 @@ those are visible in the panels above.
102
  Two stacks, both CPU-only, both with the dedicated no-KV runtime compiled inside the image and
103
  nothing fetched at run time.
104
 
 
 
 
 
 
 
 
 
 
 
 
 
105
  **[`docker/`](docker/) โ€” see it work now.**
106
 
107
  ```bash
 
102
  Two stacks, both CPU-only, both with the dedicated no-KV runtime compiled inside the image and
103
  nothing fetched at run time.
104
 
105
+ **Getting the files.** The weights and the two runtime archives are stored with Git LFS. A
106
+ `git clone` made without git-lfs โ€” the macOS default โ€” leaves ~130-byte pointer files in their
107
+ place, and both containers stop and say so. Either fetch with the Hub CLI, which needs no git-lfs:
108
+
109
+ ```bash
110
+ hf download mp-juuuns/qwen3.5-4l-vocab40k-en-ko-headless --local-dir qwen3.5-4l
111
+ ```
112
+
113
+ or install git-lfs first (macOS `brew install git-lfs`, Debian/Ubuntu `apt install git-lfs`; Git for
114
+ Windows already includes it), run `git lfs install`, and clone. An existing clone is repaired in
115
+ place with `git lfs pull`.
116
+
117
  **[`docker/`](docker/) โ€” see it work now.**
118
 
119
  ```bash
README_KO.md CHANGED
@@ -78,6 +78,18 @@ Qwen3.5-0.8B์—์„œ ์ฆ๋ฅ˜ํ•œ **ํ—ค๋“œ ์—†๋Š” 4๊ณ„์ธต ํ…์ŠคํŠธ ๋ฐฑ๋ณธ**์ด๋ฉฐ,
78
  ๋‘ ๊ฐœ์˜ ์Šคํƒ์ด๋ฉฐ, ๋‘˜ ๋‹ค CPU ์ „์šฉ์ด๊ณ , ๋‘˜ ๋‹ค ์ „์šฉ no-KV ๋Ÿฐํƒ€์ž„์„ ์ด๋ฏธ์ง€ ์•ˆ์—์„œ ์ปดํŒŒ์ผํ•˜๋ฉฐ ์‹คํ–‰
79
  ์‹œ์ ์— ๊ฐ€์ ธ์˜ค๋Š” ๊ฒƒ์€ ์•„๋ฌด๊ฒƒ๋„ ์—†์Šต๋‹ˆ๋‹ค.
80
 
 
 
 
 
 
 
 
 
 
 
 
 
81
  **[`docker/`](docker/) โ€” ์ง€๊ธˆ ๋ฐ”๋กœ ๋Œ์•„๊ฐ€๋Š” ๊ฒƒ์„ ๋ณด๊ธฐ.**
82
 
83
  ```bash
 
78
  ๋‘ ๊ฐœ์˜ ์Šคํƒ์ด๋ฉฐ, ๋‘˜ ๋‹ค CPU ์ „์šฉ์ด๊ณ , ๋‘˜ ๋‹ค ์ „์šฉ no-KV ๋Ÿฐํƒ€์ž„์„ ์ด๋ฏธ์ง€ ์•ˆ์—์„œ ์ปดํŒŒ์ผํ•˜๋ฉฐ ์‹คํ–‰
79
  ์‹œ์ ์— ๊ฐ€์ ธ์˜ค๋Š” ๊ฒƒ์€ ์•„๋ฌด๊ฒƒ๋„ ์—†์Šต๋‹ˆ๋‹ค.
80
 
81
+ **ํŒŒ์ผ ๋ฐ›๊ธฐ.** ๊ฐ€์ค‘์น˜์™€ ๋Ÿฐํƒ€์ž„ ์••์ถ• ํŒŒ์ผ ๋‘ ๊ฐœ๋Š” Git LFS๋กœ ์ €์žฅ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค. git-lfs ์—†์ด
82
+ `git clone`์„ ํ•˜๋ฉด(๋งฅ์˜ ๊ธฐ๋ณธ ์ƒํƒœ๊ฐ€ ์ด๋ ‡์Šต๋‹ˆ๋‹ค) ๊ทธ ์ž๋ฆฌ์— 130๋ฐ”์ดํŠธ ์ •๋„์˜ ํฌ์ธํ„ฐ ํŒŒ์ผ๋งŒ ๋ฐ›์•„์ง€๊ณ ,
83
+ ๋‘ ์ปจํ…Œ์ด๋„ˆ ๋ชจ๋‘ ๊ทธ ์‚ฌ์‹ค์„ ์•Œ๋ฆฌ๋ฉฐ ๋ฉˆ์ถฅ๋‹ˆ๋‹ค. git-lfs๊ฐ€ ํ•„์š” ์—†๋Š” Hub CLI๋กœ ๋ฐ›์œผ์‹œ๊ฑฐ๋‚˜:
84
+
85
+ ```bash
86
+ hf download mp-juuuns/qwen3.5-4l-vocab40k-en-ko-headless --local-dir qwen3.5-4l
87
+ ```
88
+
89
+ git-lfs๋ฅผ ๋จผ์ € ์„ค์น˜ํ•˜์‹  ๋’ค(๋งฅ `brew install git-lfs`, Debian/Ubuntu `apt install git-lfs`, Git for
90
+ Windows์—๋Š” ์ด๋ฏธ ๋“ค์–ด ์žˆ์Šต๋‹ˆ๋‹ค) `git lfs install`์„ ์‹คํ–‰ํ•˜๊ณ  ํด๋ก ํ•˜์‹œ๋ฉด ๋ฉ๋‹ˆ๋‹ค. ์ด๋ฏธ ๋ฐ›์€ ์‚ฌ๋ณธ์€ ๊ทธ
91
+ ์ž๋ฆฌ์—์„œ `git lfs pull`๋กœ ๊ณ ์น  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
92
+
93
  **[`docker/`](docker/) โ€” ์ง€๊ธˆ ๋ฐ”๋กœ ๋Œ์•„๊ฐ€๋Š” ๊ฒƒ์„ ๋ณด๊ธฐ.**
94
 
95
  ```bash
SHA256SUMS CHANGED
@@ -1,6 +1,6 @@
1
  77fd4710def9ec3c0f6225800e0235f15a425abd4a8b03559127fcd782612049 LICENSE
2
- c3918d4b2392d78da6ce775f5776162079eaa15a97f8eb5eb97758addbabc144 README.md
3
- 40b06feb534ec50c43777b838d378feb6755a47e07ce8b63e93cb22961896aea README_KO.md
4
  bf4073de38c3905effd2dcce6f38894f325c8a0a79bedfb2779f0545c86b9c9f benchmark/BENCHMARK_CARD.md
5
  e01bd7ac59a3212e44b77e5fcfbc7c41ec91d74c998306d2438209d6ef1a3aa7 benchmark/arm_summary.csv
6
  2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b benchmark/compression_ladder.csv
@@ -83,9 +83,9 @@ d9b135cf5b0adf4bbfc321200395fbd6112e49fd1a027b30262bcff1c4c0b654 distillation/q
83
  00731ae7838317105f9fff960e3da210b2a55954feb37b3ac793bcfc81fdab2d distillation/qwen35_distill/schema.py
84
  4b341db0a133c1398d0bde0fb881991dacaffd8158f6f593aea30fb6486655df distillation/qwen35_distill/training.py
85
  42d6a7ff85248c4fadbc022c02dc17909b6484dcea34a20c803e00598bac6063 docker-train/.dockerignore
86
- b26681c81ad020cd1e85f4a9faed7b8bae37dbcad7b20d1d05ea16284fd5dd0d docker-train/Dockerfile
87
- 419c3caa21b00da1c2b5cf4a76862e29bbf4559e4c64eafc1ec8f60b83a34991 docker-train/README.md
88
- e9a07d289b8fc3ac593412faa8a4309e2eadcbe148e22a92f8928a065c18cc42 docker-train/README_KO.md
89
  28c72ebaee31a0cb90175a45d07701951905391131698d227b8c5b959eeb536e docker-train/VALIDATION_KO.md
90
  b57039329c8966937707622acab52e791d08777899e901d2248ca95b6e4087b7 docker-train/compose.arm64.yaml
91
  46328308ba5b45d4fe059b864b918dc7725fdb98c753ef916778f766269ce284 docker-train/compose.yaml
@@ -111,7 +111,7 @@ d67ede61740a3af009c500d7ebf28ee48a14cecff22bd281dd94604a0d184508 docker-train/e
111
  56cf9f6386e5b9a034d34ee679532d5cc0f686ad0dc913704efdf8264debe91d docker-train/examples/data/spans-calibration.jsonl
112
  e6086d3cd9dcbb5e5a7b3acf0c2a20349cd7bf911963e24bee559c8b99477803 docker-train/examples/data/spans-train.jsonl
113
  f385666b81fe757fb448aadfcd147d80c0ba4f169b9b3d85fe31aa299a7e75be docker-train/examples/data/train.jsonl
114
- 1ed8e0fb322cea3c1f9ed34003e36166a48b98fa8cd8da4f2d97ede2bd772798 docker-train/make_base.py
115
  666d8b7896a6e347b230314a596b253746c11b180a8903f7abeb09d25634bea5 docker-train/native/gdn_fused.inc
116
  3c34c2b5e1bf34e0520823524b961c5b30a837a987f2d1e3be6f4bbc51d8a1a5 docker-train/native/patch_runtime.py
117
  443e8be62adeb7cbafecab8aca13a60e110a7f35c045e2de19fb464e19d847c8 docker-train/native/q8_kernel.inc
@@ -278,8 +278,8 @@ da920e2c62166ec9f30e407e9fb35ec29fda671310ffbf0f8ce856b62f1c70d7 docker-train/v
278
  7fdb374e511dfe1c0da5b870af83e22bb55c141df4c49f1417cc3b5ca53669f9 docker-train/vendor/runtime_source.tar.gz
279
  d39d33e360727537d02a2c1cec3ef17de494376477b0edd3013fa7f138d28db1 docker/.dockerignore
280
  87b1e2dc98c711c6762cd4e9f40d9fc0228ae2b8f6bd8f301334f6830d0f8d88 docker/Dockerfile
281
- c6e17c6beadaf83d764f4982c4df81a7f83a7ab00c208531c33596c6264718a2 docker/README.md
282
- 016b40c98a2b4a8bd874b8c2b248551c353627c45a3a3da4cabfea71f7bb098e docker/README_KO.md
283
  1bb65784297e7f7319099736eb8d2041899c1af5b3731375d6f578ef0f7e92b1 docker/app.js
284
  3c162659936a2ccc542008e315c0fb5e5778e10808c01bf41e8cbdae7d119b2d docker/benchmark_reference.json
285
  e7ab07e36f17d097fd9832d78f25da608eaee69a66b32f80658bd44cfb711da0 docker/bundle/manifest.json
@@ -293,7 +293,7 @@ e54868871ce4c9fe32e4dab10fb1e1abba4bf7c7c61c4925a351e31ad5cd83b9 docker/compose
293
  f65fac7bd6f881d2ecb0db058db44dbf039b62003171c257bf68a6dd09e70ba7 docker/prepare_bundle.py
294
  c9411e19fe92451c525d7094b7633a1759b6fdd3ef8c06c2516cbc22392463ad docker/requirements.lock
295
  b41d4b7556d3a11f9e2e6c3d0177bbd5dbf0a7afb553087a6dbcb843ca940a82 docker/server.py
296
- 5ed83b7d55bde8ca485e0093bbc22aa975fe6e247efe9593ed8989eb269544f6 docker/start.sh
297
  295a9fff3000e8f719dfb528f2ca12e2ab1ac3fbc3265a74a999733ddd782f97 docker/telemetry.py
298
  a78242281be9d6fe52e381db81eeeafddf8c0c34d1b9a36671ae6ebf0037dbcb docker/test_server.py
299
  1b6e475b9d7d9af52bb8845de020303b1c6e89646d8e4f9be17dbe4ab2910417 docs/RELEASE_CONTRACT.md
 
1
  77fd4710def9ec3c0f6225800e0235f15a425abd4a8b03559127fcd782612049 LICENSE
2
+ b901b5be1af24ece943d174566a9c22e55019eef403081f9fc21672100456f50 README.md
3
+ 9d99d92cf91bcffa39d794ee8f15b7517695f8f2f67311ae081010e37c153dee README_KO.md
4
  bf4073de38c3905effd2dcce6f38894f325c8a0a79bedfb2779f0545c86b9c9f benchmark/BENCHMARK_CARD.md
5
  e01bd7ac59a3212e44b77e5fcfbc7c41ec91d74c998306d2438209d6ef1a3aa7 benchmark/arm_summary.csv
6
  2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b benchmark/compression_ladder.csv
 
83
  00731ae7838317105f9fff960e3da210b2a55954feb37b3ac793bcfc81fdab2d distillation/qwen35_distill/schema.py
84
  4b341db0a133c1398d0bde0fb881991dacaffd8158f6f593aea30fb6486655df distillation/qwen35_distill/training.py
85
  42d6a7ff85248c4fadbc022c02dc17909b6484dcea34a20c803e00598bac6063 docker-train/.dockerignore
86
+ 0e5ee1f54ef418f71307e60374eae14c1897dd515b123d31b7b45b92bde7827c docker-train/Dockerfile
87
+ 08386d1429b06c3e1c2c2de7d5e0effcae646709d4666073a9721573b7306e9e docker-train/README.md
88
+ 016c7a595525d66be1c016925d5571c32a0104a5c4417627747bf9dedd1403d5 docker-train/README_KO.md
89
  28c72ebaee31a0cb90175a45d07701951905391131698d227b8c5b959eeb536e docker-train/VALIDATION_KO.md
90
  b57039329c8966937707622acab52e791d08777899e901d2248ca95b6e4087b7 docker-train/compose.arm64.yaml
91
  46328308ba5b45d4fe059b864b918dc7725fdb98c753ef916778f766269ce284 docker-train/compose.yaml
 
111
  56cf9f6386e5b9a034d34ee679532d5cc0f686ad0dc913704efdf8264debe91d docker-train/examples/data/spans-calibration.jsonl
112
  e6086d3cd9dcbb5e5a7b3acf0c2a20349cd7bf911963e24bee559c8b99477803 docker-train/examples/data/spans-train.jsonl
113
  f385666b81fe757fb448aadfcd147d80c0ba4f169b9b3d85fe31aa299a7e75be docker-train/examples/data/train.jsonl
114
+ af56c89efa4cda482b40d3b5f6a8e6c11a259346fccf81a1da9d3ad18c4c5fee docker-train/make_base.py
115
  666d8b7896a6e347b230314a596b253746c11b180a8903f7abeb09d25634bea5 docker-train/native/gdn_fused.inc
116
  3c34c2b5e1bf34e0520823524b961c5b30a837a987f2d1e3be6f4bbc51d8a1a5 docker-train/native/patch_runtime.py
117
  443e8be62adeb7cbafecab8aca13a60e110a7f35c045e2de19fb464e19d847c8 docker-train/native/q8_kernel.inc
 
278
  7fdb374e511dfe1c0da5b870af83e22bb55c141df4c49f1417cc3b5ca53669f9 docker-train/vendor/runtime_source.tar.gz
279
  d39d33e360727537d02a2c1cec3ef17de494376477b0edd3013fa7f138d28db1 docker/.dockerignore
280
  87b1e2dc98c711c6762cd4e9f40d9fc0228ae2b8f6bd8f301334f6830d0f8d88 docker/Dockerfile
281
+ d0938d7b42be376496bc2235b77fb16a8503ae88b4e6db9fd98b9aff7a4aeb30 docker/README.md
282
+ 19005d0331a7a0a2aac0008a16f9300d16ca77d0b188c5e8facca56a46eeea05 docker/README_KO.md
283
  1bb65784297e7f7319099736eb8d2041899c1af5b3731375d6f578ef0f7e92b1 docker/app.js
284
  3c162659936a2ccc542008e315c0fb5e5778e10808c01bf41e8cbdae7d119b2d docker/benchmark_reference.json
285
  e7ab07e36f17d097fd9832d78f25da608eaee69a66b32f80658bd44cfb711da0 docker/bundle/manifest.json
 
293
  f65fac7bd6f881d2ecb0db058db44dbf039b62003171c257bf68a6dd09e70ba7 docker/prepare_bundle.py
294
  c9411e19fe92451c525d7094b7633a1759b6fdd3ef8c06c2516cbc22392463ad docker/requirements.lock
295
  b41d4b7556d3a11f9e2e6c3d0177bbd5dbf0a7afb553087a6dbcb843ca940a82 docker/server.py
296
+ e0ed7ce704883241b33d6646d18d892208691dd17da1f83032e47512284dd4f7 docker/start.sh
297
  295a9fff3000e8f719dfb528f2ca12e2ab1ac3fbc3265a74a999733ddd782f97 docker/telemetry.py
298
  a78242281be9d6fe52e381db81eeeafddf8c0c34d1b9a36671ae6ebf0037dbcb docker/test_server.py
299
  1b6e475b9d7d9af52bb8845de020303b1c6e89646d8e4f9be17dbe4ab2910417 docs/RELEASE_CONTRACT.md
docker-train/Dockerfile CHANGED
@@ -5,6 +5,7 @@ WORKDIR /src
5
  COPY vendor/manifest.json vendor/manifest.json
6
  # docker unpacks a local tarball on ADD, so the bundled runtime source needs no manual step.
7
  ADD vendor/runtime_source.tar.gz vendor/
 
8
  COPY vendor/native_live_classifier.cpp vendor/native_live_classifier.cpp
9
  COPY native native
10
  COPY tools/build_native.py tools/build_native.py
 
5
  COPY vendor/manifest.json vendor/manifest.json
6
  # docker unpacks a local tarball on ADD, so the bundled runtime source needs no manual step.
7
  ADD vendor/runtime_source.tar.gz vendor/
8
+ RUN test -f vendor/runtime_source/CMakeLists.txt || { echo 'vendor/runtime_source.tar.gz did not unpack: it is a Git LFS pointer, not the archive. Install git-lfs, run git lfs install and git lfs pull, then build again - or fetch the files with hf download.' >&2; exit 1; }
9
  COPY vendor/native_live_classifier.cpp vendor/native_live_classifier.cpp
10
  COPY native native
11
  COPY tools/build_native.py tools/build_native.py
docker-train/README.md CHANGED
@@ -10,6 +10,10 @@ the dependencies and the native binaries. The training, validation and export co
10
 
11
  ## Run it
12
 
 
 
 
 
13
  ```bash
14
  python make_base.py # assembles base/ from this repository's own weights
15
  mkdir -p outputs
 
10
 
11
  ## Run it
12
 
13
+ The root weights are stored with Git LFS, so fetch with `hf download` or clone with git-lfs installed
14
+ (macOS: `brew install git-lfs`, then `git lfs install`). `make_base.py` and the image build both stop
15
+ with a clear message if they find a pointer file instead of the real one.
16
+
17
  ```bash
18
  python make_base.py # assembles base/ from this repository's own weights
19
  mkdir -p outputs
docker-train/README_KO.md CHANGED
@@ -8,10 +8,14 @@
8
 
9
  ## ์ค€๋น„์™€ ์‹คํ–‰
10
 
11
- ๊ฒ€์ฆ๋œ **๋ณ„๋„ ๋ฒ ์ด์Šค ํŒจํ‚ค์ง€**๋ฅผ `base/`์— ํ’€๊ณ , `outputs/` ํด๋”๋ฅผ ๋งŒ๋“ ๋‹ค.
 
 
 
12
  `config/train.yaml`์˜ ๋ผ๋ฒจยทํ•™์Šต/๋ณด์ • ํŒŒ์ผยท์‹คํ–‰ ์ด๋ฆ„์„ ์ˆ˜์ •ํ•œ๋‹ค. ์˜ˆ์ œ ๋ฐ์ดํ„ฐ๋Š” ์ง์ ‘ ๋งŒ๋“  ๊ธฐ๋Šฅ ํ™•์ธ์šฉ ๋ฌธ์žฅ์œผ๋กœ, ๋ชจ๋ธ ํ’ˆ์งˆ ํ‰๊ฐ€์šฉ์ด ์•„๋‹ˆ๋‹ค.
13
 
14
  ```bash
 
15
  mkdir -p outputs
16
  export LOCAL_UID="$(id -u)" LOCAL_GID="$(id -g)" # Linux/macOS only
17
  # ์œˆ๋„์šฐ PowerShell์—์„œ๋Š” ์œ„ export ์ค„์„ ๊ฑด๋„ˆ๋›ด๋‹ค. Docker Desktop์—์„œ๋Š” ๊ธฐ๋ณธ๊ฐ’(1000)์œผ๋กœ ๋™์ž‘ํ•œ๋‹ค.
 
8
 
9
  ## ์ค€๋น„์™€ ์‹คํ–‰
10
 
11
+ `python make_base.py`๋กœ ์ด ์ €์žฅ์†Œ ๋ฃจํŠธ์˜ ๊ฐ€์ค‘์น˜์—์„œ `base/`๋ฅผ ์กฐ๋ฆฝํ•˜๊ณ , `outputs/` ํด๋”๋ฅผ ๋งŒ๋“ ๋‹ค.
12
+ ๋ฃจํŠธ ๊ฐ€์ค‘์น˜๋Š” Git LFS๋กœ ์ €์žฅ๋˜์–ด ์žˆ์œผ๋ฏ€๋กœ `hf download`๋กœ ๋ฐ›๊ฑฐ๋‚˜ git-lfs๋ฅผ ์„ค์น˜ํ•œ ์ƒํƒœ์—์„œ ํด๋ก ํ•œ๋‹ค
13
+ (๋งฅ: `brew install git-lfs` ํ›„ `git lfs install`). ์‹ค์ œ ํŒŒ์ผ ๋Œ€์‹  ํฌ์ธํ„ฐ ํŒŒ์ผ์ด ์žˆ์œผ๋ฉด `make_base.py`์™€
14
+ ์ด๋ฏธ์ง€ ๋นŒ๋“œ๊ฐ€ ๊ทธ๋ ‡๋‹ค๊ณ  ์•Œ๋ฆฌ๋ฉฐ ๋ฉˆ์ถ˜๋‹ค.
15
  `config/train.yaml`์˜ ๋ผ๋ฒจยทํ•™์Šต/๋ณด์ • ํŒŒ์ผยท์‹คํ–‰ ์ด๋ฆ„์„ ์ˆ˜์ •ํ•œ๋‹ค. ์˜ˆ์ œ ๋ฐ์ดํ„ฐ๋Š” ์ง์ ‘ ๋งŒ๋“  ๊ธฐ๋Šฅ ํ™•์ธ์šฉ ๋ฌธ์žฅ์œผ๋กœ, ๋ชจ๋ธ ํ’ˆ์งˆ ํ‰๊ฐ€์šฉ์ด ์•„๋‹ˆ๋‹ค.
16
 
17
  ```bash
18
+ python make_base.py
19
  mkdir -p outputs
20
  export LOCAL_UID="$(id -u)" LOCAL_GID="$(id -g)" # Linux/macOS only
21
  # ์œˆ๋„์šฐ PowerShell์—์„œ๋Š” ์œ„ export ์ค„์„ ๊ฑด๋„ˆ๋›ด๋‹ค. Docker Desktop์—์„œ๋Š” ๊ธฐ๋ณธ๊ฐ’(1000)์œผ๋กœ ๋™์ž‘ํ•œ๋‹ค.
docker-train/make_base.py CHANGED
@@ -33,6 +33,14 @@ def main():
33
  if missing:
34
  raise FileNotFoundError("run this from docker-train/ inside a full clone; missing: " + ", ".join(missing))
35
 
 
 
 
 
 
 
 
 
36
  DEST.mkdir()
37
  for rel in PLAIN + PROVENANCE:
38
  target = DEST / rel
 
33
  if missing:
34
  raise FileNotFoundError("run this from docker-train/ inside a full clone; missing: " + ", ".join(missing))
35
 
36
+ pointers = [f for f in ("model.safetensors",)
37
+ if (ROOT / f).read_bytes()[:23] == b"version https://git-lfs"]
38
+ if pointers:
39
+ raise SystemExit(
40
+ f"{', '.join(pointers)} is a Git LFS pointer, not the real file. Install git-lfs "
41
+ "(macOS: brew install git-lfs), run 'git lfs install' and 'git lfs pull' in the "
42
+ "repository, then run this again - or fetch the files with 'hf download'.")
43
+
44
  DEST.mkdir()
45
  for rel in PLAIN + PROVENANCE:
46
  target = DEST / rel
docker/README.md CHANGED
@@ -6,6 +6,10 @@ time; the weights and the runtime source are in `bundle/`.
6
 
7
  Korean version of this page: [`README_KO.md`](README_KO.md).
8
 
 
 
 
 
9
  ```bash
10
  docker compose up --build
11
  ```
 
6
 
7
  Korean version of this page: [`README_KO.md`](README_KO.md).
8
 
9
+ You need the real files, not Git LFS pointers: fetch with `hf download`, or clone with git-lfs
10
+ installed (macOS: `brew install git-lfs`, then `git lfs install`). A clone made without it is fixed
11
+ in place with `git lfs pull`; the container tells you when this is the problem.
12
+
13
  ```bash
14
  docker compose up --build
15
  ```
docker/README_KO.md CHANGED
@@ -7,6 +7,10 @@
7
 
8
  ## ๋น ๋ฅธ ์‹คํ–‰
9
 
 
 
 
 
10
  ```bash
11
  docker compose up --build
12
  ```
@@ -78,7 +82,7 @@ curl http://127.0.0.1:8787/classify \
78
 
79
  ## ๋‹ค๋ฅธ PC๋กœ ์ด๋™
80
 
81
- `deployment/qwen35-vocab40k/` ํด๋”๋ฅผ `bundle/` ํฌํ•จํ•ด ํ†ต์งธ๋กœ ์˜ฎ๊ธฐ๊ณ 
82
  `docker compose up --build`๋ฅผ ์‹คํ–‰ํ•˜๋ฉด ๋œ๋‹ค. ์›๋ž˜ ํ”„๋กœ์ ํŠธ์˜ ์ ˆ๋Œ€๊ฒฝ๋กœ๋Š” ํ•„์š” ์—†๋‹ค.
83
  bundle์—๋Š” ์ฝ”๋“œยท๋ชจ๋ธยทํ† ํฌ๋‚˜์ด์ €๊ฐ€ ์žˆ๊ณ  ํ•™์Šต/๋ณด์ •/test ๋ฐ์ดํ„ฐ์…‹์€ ์—†๋‹ค.
84
  `bundle/manifest.json`์€ ํŒŒ์ผ ๋ฌด๊ฒฐ์„ฑ ์ž๋ฃŒ์ด์ง€ ์Šน์ธ ๊ธฐ๋ก์ด ์•„๋‹ˆ๋‹ค.
 
7
 
8
  ## ๋น ๋ฅธ ์‹คํ–‰
9
 
10
+ Git LFS ํฌ์ธํ„ฐ๊ฐ€ ์•„๋‹Œ ์‹ค์ œ ํŒŒ์ผ์ด ํ•„์š”ํ•˜๋‹ค. `hf download`๋กœ ๋ฐ›๊ฑฐ๋‚˜, git-lfs๋ฅผ ์„ค์น˜ํ•œ ์ƒํƒœ์—์„œ
11
+ ํด๋ก ํ•œ๋‹ค(๋งฅ: `brew install git-lfs` ํ›„ `git lfs install`). git-lfs ์—†์ด ํด๋ก ํ–ˆ๋‹ค๋ฉด ๊ทธ ์ž๋ฆฌ์—์„œ
12
+ `git lfs pull`๋กœ ๊ณ ์ณ์ง„๋‹ค. ์ด ๋ฌธ์ œ์ผ ๋•Œ๋Š” ์ปจํ…Œ์ด๋„ˆ๊ฐ€ ๊ทธ๋ ‡๋‹ค๊ณ  ์•Œ๋ ค์ค€๋‹ค.
13
+
14
  ```bash
15
  docker compose up --build
16
  ```
 
82
 
83
  ## ๋‹ค๋ฅธ PC๋กœ ์ด๋™
84
 
85
+ ์ด `docker/` ํด๋”๋ฅผ `bundle/` ํฌํ•จํ•ด ํ†ต์งธ๋กœ ์˜ฎ๊ธฐ๊ณ 
86
  `docker compose up --build`๋ฅผ ์‹คํ–‰ํ•˜๋ฉด ๋œ๋‹ค. ์›๋ž˜ ํ”„๋กœ์ ํŠธ์˜ ์ ˆ๋Œ€๊ฒฝ๋กœ๋Š” ํ•„์š” ์—†๋‹ค.
87
  bundle์—๋Š” ์ฝ”๋“œยท๋ชจ๋ธยทํ† ํฌ๋‚˜์ด์ €๊ฐ€ ์žˆ๊ณ  ํ•™์Šต/๋ณด์ •/test ๋ฐ์ดํ„ฐ์…‹์€ ์—†๋‹ค.
88
  `bundle/manifest.json`์€ ํŒŒ์ผ ๋ฌด๊ฒฐ์„ฑ ์ž๋ฃŒ์ด์ง€ ์Šน์ธ ๊ธฐ๋ก์ด ์•„๋‹ˆ๋‹ค.
docker/start.sh CHANGED
@@ -24,6 +24,17 @@ fi
24
  # Unpack it in place before the manifest is verified, so no manual step stands between a fresh
25
  # clone and a working container. Already-unpacked bundles are left alone.
26
  bundle="${MODEL_BUNDLE:-$PWD/bundle}"
 
 
 
 
 
 
 
 
 
 
 
27
  if [[ ! -f "$bundle/runtime_source/CMakeLists.txt" && -f "$bundle/runtime_source.tar.gz" ]]; then
28
  echo 'Unpacking the bundled runtime source...'
29
  tar -xzf "$bundle/runtime_source.tar.gz" -C "$bundle"
 
24
  # Unpack it in place before the manifest is verified, so no manual step stands between a fresh
25
  # clone and a working container. Already-unpacked bundles are left alone.
26
  bundle="${MODEL_BUNDLE:-$PWD/bundle}"
27
+ # A clone made without git-lfs (the macOS default) leaves ~130-byte pointer files where the
28
+ # archive and the weights should be. Say that plainly instead of failing inside gzip.
29
+ for lfs_file in "$bundle/runtime_source.tar.gz" "$bundle/model/classifier-q8_0.gguf"; do
30
+ if [[ -f "$lfs_file" && "$(head -c 23 "$lfs_file" | tr -d '\000')" == "version https://git-lfs" ]]; then
31
+ echo "ERROR: ${lfs_file#"$bundle"/} is a Git LFS pointer, not the real file." >&2
32
+ echo "Install git-lfs (macOS: brew install git-lfs, Debian/Ubuntu: apt install git-lfs)," >&2
33
+ echo "run 'git lfs install' and then 'git lfs pull' in the repository, and start again." >&2
34
+ echo "Or fetch the files with 'hf download', which does not need git-lfs." >&2
35
+ exit 1
36
+ fi
37
+ done
38
  if [[ ! -f "$bundle/runtime_source/CMakeLists.txt" && -f "$bundle/runtime_source.tar.gz" ]]; then
39
  echo 'Unpacking the bundled runtime source...'
40
  tar -xzf "$bundle/runtime_source.tar.gz" -C "$bundle"