ManniX-ITA commited on
Commit
070442f
Β·
verified Β·
1 Parent(s): c9ff9bc

Upload docs/protocols/UPSTREAM_SYNC.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. docs/protocols/UPSTREAM_SYNC.md +103 -5
docs/protocols/UPSTREAM_SYNC.md CHANGED
@@ -7,10 +7,24 @@ syncs cheap.
7
  ## Upstream
8
 
9
  - Tracked upstream: [mann1x/opencode](https://github.com/mann1x/opencode)
10
- (dev fork; opencode-of-record is anomalyco/opencode).
 
11
  - Default upstream branch: `dev`.
12
  - Sync cadence: weekly is the target, more often when upstream is
13
  hot.
 
 
 
 
 
 
 
 
 
 
 
 
 
14
 
15
  ## The rule
16
 
@@ -90,16 +104,75 @@ Add an entry in the same PR you add the hook.
90
  | `tiered-inference` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/tiers": "workspace:*"`) | F1 β€” Tiered Inference Engine | [features/tiered_inference.md](../features/tiered_inference.md) |
91
  | `memory-tools-resolvability` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/memory": "workspace:*"`) | F4 M5 β€” Memory / Embedder | [features/memory_embedder.md](../features/memory_embedder.md) |
92
  | `opencoti-default-plugins` (import) | `packages/opencode/src/config/config.ts` (just below the existing `ConfigPlugin` import) | F4 M5 follow-up β€” auto-wire opencoti plugins | [features/memory_embedder.md](../features/memory_embedder.md) |
93
- | `opencoti-default-plugins` (call site) | `packages/opencode/src/config/config.ts` (in the config-loader Effect.fnUntraced body, just before the `return { config: result, … }`) | F4 M5 follow-up β€” auto-wire opencoti plugins | [features/memory_embedder.md](../features/memory_embedder.md) |
94
- | `tiered-inference-setup` (import) | `packages/opencode/src/index.ts` (just after the existing `isRecord` import) | F1 M5.7 β€” Tier-0 setup CLI | [features/tiered_inference.md](../features/tiered_inference.md) |
95
  | `tiered-inference-setup` (call site) | `packages/opencode/src/index.ts` (`.command(tierCommand)` after `.command(DbCommand)`) | F1 M5.7 β€” Tier-0 setup CLI | [features/tiered_inference.md](../features/tiered_inference.md) |
96
  | `opencoti-server-autostart` (import) | `packages/opencode/src/config/config.ts` (just below the `opencoti-default-plugins` import) | F3 M3 β€” opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
97
- | `opencoti-server-autostart` (schema) | `packages/opencode/src/config/config.ts` (`opencoti: Schema.optional(Schema.Struct({ server: ... }))` block in the main config schema) | F3 M3 β€” opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
98
- | `opencoti-server-autostart` (call site) | `packages/opencode/src/config/config.ts` (in the config-loader Effect.fnUntraced body, just after the `opencoti-default-plugins` call site, before `return { config: result, … }`) | F3 M3 β€” opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
99
  | `opencoti-server-client` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/opencoti-server-client": "workspace:*"`) | F3 M3 β€” opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
100
 
101
  The two TS edits carry `// opencoti-hook: tiered-inference β€” see docs/features/tiered_inference.md` markers; the `package.json` dep has no anchor comment (JSON has no comments) β€” this registry table is its only anchor. On upstream merges, if upstream renames any of these files or restructures the `run` Effect.fn around the call-site marker, the marker comments remain the canonical pointer for re-applying the hook.
102
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
103
  **F3 M4 (2026-05-23): no new registry rows.** M4 (session +
104
  turn events flowing into opencoti-server) ships as a pure
105
  plugin (`@opencoti/session-events-plugin`) that subscribes to
@@ -1718,3 +1791,28 @@ server-context meta field/aggregate-init position; the virtuals are additive.
1718
  - `llamafile/version.h` β€” `OPENCOTI_TAG` default + `OPENCOTI_VERSION_STRING` (bump the default together with `vendors/llamafile/OPENCOTI_TAG` on every packaged cut).
1719
  - `llamafile/main.cpp` β€” `--version` + help banner print `llamafile vX.Y.Z (opencoti-X.Y.Z-<tag>)`; `llamafile vX.Y.Z` token must stay FIRST (scripts/llamafile-version.sh contract).
1720
  - `llamafile/BUILD.mk` β€” `OPENCOTI_TAG_FLAG` injection (make var from build pipeline) + `version.h` dep on `main.o`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  ## Upstream
8
 
9
  - Tracked upstream: [mann1x/opencode](https://github.com/mann1x/opencode)
10
+ (dev fork; opencode-of-record is anomalyco/opencode, fetched as the
11
+ `record` remote).
12
  - Default upstream branch: `dev`.
13
  - Sync cadence: weekly is the target, more often when upstream is
14
  hot.
15
+ - ⚠️ **Sync the fork FIRST.** A sync is only meaningful if
16
+ mann1x/opencode `dev` has itself been updated from anomalyco
17
+ (GitHub "Sync fork", or locally: merge `record/dev` into the fork's
18
+ `dev` and push β€” expect conflicts in any locally-pinned dep files).
19
+ On 2026-07-15 the fork was 2 commits past a 1757-commit-stale base;
20
+ syncing against it would have been a no-op.
21
+ - First full sync completed **2026-07-15**: dev merged
22
+ anomalyco/opencode `dev` @ `05c3e40a` (sync-branch merge
23
+ `540d82d630`, landed on dev as `c896a910f9`). Since that merge the
24
+ histories are properly joined β€” future syncs are a plain
25
+ `git merge upstream/dev` with a real merge-base (no graft needed).
26
+ Full gotcha log: see "2026-07-15 β€” first full opencode sync" below
27
+ the Registry.
28
 
29
  ## The rule
30
 
 
104
  | `tiered-inference` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/tiers": "workspace:*"`) | F1 β€” Tiered Inference Engine | [features/tiered_inference.md](../features/tiered_inference.md) |
105
  | `memory-tools-resolvability` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/memory": "workspace:*"`) | F4 M5 β€” Memory / Embedder | [features/memory_embedder.md](../features/memory_embedder.md) |
106
  | `opencoti-default-plugins` (import) | `packages/opencode/src/config/config.ts` (just below the existing `ConfigPlugin` import) | F4 M5 follow-up β€” auto-wire opencoti plugins | [features/memory_embedder.md](../features/memory_embedder.md) |
107
+ | `opencoti-default-plugins` (call site) | `packages/opencode/src/config/config.ts` (inside `loadInstanceState` Effect.fnUntraced β€” upstream split the single loader into several Effect.fnUntraced functions 2026-07-15; the old `return { config: result, … }` anchor is gone) | F4 M5 follow-up β€” auto-wire opencoti plugins | [features/memory_embedder.md](../features/memory_embedder.md) |
108
+ | `tiered-inference-setup` (import) | `packages/opencode/src/index.ts` (import block, after the `Heap` import β€” the former `isRecord` import anchor was removed upstream 2026-07-15) | F1 M5.7 β€” Tier-0 setup CLI | [features/tiered_inference.md](../features/tiered_inference.md) |
109
  | `tiered-inference-setup` (call site) | `packages/opencode/src/index.ts` (`.command(tierCommand)` after `.command(DbCommand)`) | F1 M5.7 β€” Tier-0 setup CLI | [features/tiered_inference.md](../features/tiered_inference.md) |
110
  | `opencoti-server-autostart` (import) | `packages/opencode/src/config/config.ts` (just below the `opencoti-default-plugins` import) | F3 M3 β€” opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
111
+ | `opencoti-server-autostart` (schema) | **`packages/core/src/v1/config/config.ts`** (`opencoti:` block in `ConfigV1.Info` β€” RELOCATED 2026-07-15: upstream moved the config schema out of packages/opencode into core/v1) | F3 M3 β€” opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
112
+ | `opencoti-server-autostart` (call site) | `packages/opencode/src/config/config.ts` (inside `loadInstanceState` Effect.fnUntraced, just after the `opencoti-default-plugins` call site; since 2026-07-15 upstream's `Log` module is gone β€” the hook `yield*`s `Effect.forkDetach` on a fiber that awaits the autostart promise and logs the outcome via `Effect.logInfo`/`Effect.logWarning`, formatting through `describeAutostartOutcome` exported by the additive package) | F3 M3 β€” opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
113
  | `opencoti-server-client` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/opencoti-server-client": "workspace:*"`) | F3 M3 β€” opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
114
 
115
  The two TS edits carry `// opencoti-hook: tiered-inference β€” see docs/features/tiered_inference.md` markers; the `package.json` dep has no anchor comment (JSON has no comments) β€” this registry table is its only anchor. On upstream merges, if upstream renames any of these files or restructures the `run` Effect.fn around the call-site marker, the marker comments remain the canonical pointer for re-applying the hook.
116
 
117
+ ### 2026-07-15 β€” first full opencode sync (#682): method + gotchas
118
+
119
+ First (and history-joining) sync: dev merged anomalyco/opencode `dev`
120
+ @ `05c3e40a` (+1757 commits past the 2026-05-21 import base
121
+ `dbd709376b`). Executed in an isolated worktree on branch
122
+ `sync/opencode-2026-07-15`; landed on dev as `c896a910f9`.
123
+
124
+ **Method (one-time β€” not needed again):** the bootstrap commit
125
+ `b34a23097f` was a squashed import with no shared history, so a plain
126
+ merge had no merge-base. Fix: `git replace --graft b34a23097f
127
+ dbd709376b` (the empirically-identified import base β€” found by
128
+ blob-matching untouched files against upstream commits), then a
129
+ normal 3-way `git merge record/dev`, then `git replace -d b34a23097f`.
130
+ The merge commit permanently joins the histories.
131
+
132
+ **Conflict surface (10 files, all predicted):** the 5 hook files +
133
+ resurrected identity files (re-deleted) + `packages/extensions` zip
134
+ artifacts + root `package.json`/`bun.lock`. `config/config.ts` and
135
+ `index.ts` were taken upstream-wholesale and the hooks hand-replayed
136
+ (upstream refactor destroyed the anchors β€” see updated Registry rows).
137
+
138
+ **Gotchas (verified this sync β€” check these FIRST next time):**
139
+
140
+ 1. **Corrupt bun store after a crashed install.** A postinstall crash
141
+ (Debian's broken `/usr/bin/node-gyp` on `tree-sitter-powershell`)
142
+ left `node_modules/.bun/<pkg>` half-extracted; a later
143
+ `--ignore-scripts` install did NOT repair it β†’ hundreds of phantom
144
+ typecheck errors (`Cannot find module 'drizzle-orm/sqlite-core'`).
145
+ Rule: after ANY failed `bun install`, `rm -rf node_modules` before
146
+ trusting typecheck. (Root cause fixed 2026-07-15: Debian 11's apt
147
+ node-gyp 7.1.2 is broken under node 22 β€” `npm install -g node-gyp`
148
+ puts a working v13 at `/usr/local/bin`; full `bun install` with
149
+ postinstall scripts now works on solidPC.)
150
+ 2. **`.gitignore` coredump patterns swallowed `packages/core`.** Our
151
+ unanchored `core` / `core.*` ignore patterns hid upstream's new
152
+ package. Fixed by root-anchoring (`/core`, `/core.*`). Any
153
+ unanchored ignore pattern can collide with future upstream names.
154
+ 3. **Bootstrap-zip artifacts create phantom conflicts.** The squashed
155
+ import lost a symlink (`extensions/zed/LICENSE`), an exec bit
156
+ (`script/sync-zed.ts`), and silently dropped a whole file
157
+ (`packages/opencode/script/build-node.ts` β€” which also survived
158
+ UNTRACKED on disk and blocked the dev merge until moved aside).
159
+ Any modify/delete conflict on a file we never touched is probably
160
+ a zip artifact β€” resolve toward upstream.
161
+ 4. **Upstream deleted the `Log` module** (Effect logging is the new
162
+ idiom): any hook passing `log` must drop/adapt it. The config
163
+ schema now lives in `packages/core/src/v1/config/config.ts`
164
+ (`ConfigV1.Info`); `fix-node-pty` postinstall moved to
165
+ `packages/core`.
166
+ 5. **Husky pre-push runs `bun typecheck`** β€” it fails in any scratch
167
+ worktree without node_modules; use `--no-verify` there (content
168
+ already verified in the sync worktree).
169
+ 6. ~~Open follow-ups~~ both CLOSED 2026-07-15: autostart-outcome
170
+ logging restored via `Effect.forkDetach` + `describeAutostartOutcome`
171
+ (see Registry row); system node-gyp fixed via `npm install -g
172
+ node-gyp` (v13 at `/usr/local/bin` shadows Debian's broken 7.1.2),
173
+ so full `bun install` with scripts works on solidPC.
174
+
175
+
176
  **F3 M4 (2026-05-23): no new registry rows.** M4 (session +
177
  turn events flowing into opencoti-server) ships as a pure
178
  plugin (`@opencoti/session-events-plugin`) that subscribes to
 
1791
  - `llamafile/version.h` β€” `OPENCOTI_TAG` default + `OPENCOTI_VERSION_STRING` (bump the default together with `vendors/llamafile/OPENCOTI_TAG` on every packaged cut).
1792
  - `llamafile/main.cpp` β€” `--version` + help banner print `llamafile vX.Y.Z (opencoti-X.Y.Z-<tag>)`; `llamafile vX.Y.Z` token must stay FIRST (scripts/llamafile-version.sh contract).
1793
  - `llamafile/BUILD.mk` β€” `OPENCOTI_TAG_FLAG` injection (make var from build pipeline) + `version.h` dep on `main.o`.
1794
+
1795
+ ## 2026-07-15 β€” 0134 cuda-cross-sbsa (#681, aarch64/DGX Spark lane)
1796
+
1797
+ ### `0134-cuda-cross-sbsa` (#681)
1798
+ Marker: `opencoti-hook: cuda-cross-sbsa` (3 sites). Build-script-only β€” no
1799
+ kernel/host C++ change; native x86_64 builds byte-unaffected (flag default off,
1800
+ `OPENCOTI_HOST_GCC/GXX` default to `gcc`/`g++`).
1801
+ - `llamafile/cuda.sh` β€” `--cross-sbsa` flag: dedicated build dir
1802
+ `~/.cache/llamafile-cuda-sbsa-build` (never mix aarch64 objects into the
1803
+ x86_64 cache), arch set sm_110f (Jetson Thor) + sm_121a (DGX Spark GB10),
1804
+ and `-ccbin aarch64-linux-gnu-g++ -target-dir sbsa-linux` riding in
1805
+ **ARCH_FLAGS** (the LINK receives ARCH_FLAGS β€” putting them in COMMON_FLAGS
1806
+ emits an x86_64 DSO). `-lcuda` resolves against the sbsa driver stub.
1807
+ Tunable `OPENCOTI_SBSA_CCBIN`. Requires CUDA 13.x + the cuda-cross-sbsa
1808
+ target packages (cached at `.opencoti/cross-sbsa/debs/`).
1809
+ - `llamafile/build-functions.sh` β€” `compile_ggml_core` host TUs go through
1810
+ `${OPENCOTI_HOST_GCC:-gcc}` / `${OPENCOTI_HOST_GXX:-g++}` so the cross build
1811
+ doesn't mix host arches at link.
1812
+ Consumers: `build:llamafile:cuda-sbsa` pipeline lane (stages
1813
+ `vendors/dist/llamafile/cuda-sbsa/<ver>/ggml-cuda.so`, NO ~/.llamafile
1814
+ side-load mirror β€” bug-224 cache-poisoning class) and
1815
+ `build:llamafile:package --arch aarch64` (embeds ONLY the sbsa DSO).
1816
+ Sync note: on a future llamafile bump re-anchor the two cuda.sh insertion
1817
+ points (after the BUILD_DIR selection block; after the CUDA-13 arch-selection
1818
+ chain, BEFORE the `--compress` block so `--compress-mode` still appends).