Upload docs/protocols/UPSTREAM_SYNC.md with huggingface_hub
Browse files- docs/protocols/UPSTREAM_SYNC.md +103 -5
docs/protocols/UPSTREAM_SYNC.md
CHANGED
|
@@ -7,10 +7,24 @@ syncs cheap.
|
|
| 7 |
## Upstream
|
| 8 |
|
| 9 |
- Tracked upstream: [mann1x/opencode](https://github.com/mann1x/opencode)
|
| 10 |
-
(dev fork; opencode-of-record is anomalyco/opencode
|
|
|
|
| 11 |
- Default upstream branch: `dev`.
|
| 12 |
- Sync cadence: weekly is the target, more often when upstream is
|
| 13 |
hot.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
## The rule
|
| 16 |
|
|
@@ -90,16 +104,75 @@ Add an entry in the same PR you add the hook.
|
|
| 90 |
| `tiered-inference` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/tiers": "workspace:*"`) | F1 β Tiered Inference Engine | [features/tiered_inference.md](../features/tiered_inference.md) |
|
| 91 |
| `memory-tools-resolvability` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/memory": "workspace:*"`) | F4 M5 β Memory / Embedder | [features/memory_embedder.md](../features/memory_embedder.md) |
|
| 92 |
| `opencoti-default-plugins` (import) | `packages/opencode/src/config/config.ts` (just below the existing `ConfigPlugin` import) | F4 M5 follow-up β auto-wire opencoti plugins | [features/memory_embedder.md](../features/memory_embedder.md) |
|
| 93 |
-
| `opencoti-default-plugins` (call site) | `packages/opencode/src/config/config.ts` (
|
| 94 |
-
| `tiered-inference-setup` (import) | `packages/opencode/src/index.ts` (
|
| 95 |
| `tiered-inference-setup` (call site) | `packages/opencode/src/index.ts` (`.command(tierCommand)` after `.command(DbCommand)`) | F1 M5.7 β Tier-0 setup CLI | [features/tiered_inference.md](../features/tiered_inference.md) |
|
| 96 |
| `opencoti-server-autostart` (import) | `packages/opencode/src/config/config.ts` (just below the `opencoti-default-plugins` import) | F3 M3 β opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
|
| 97 |
-
| `opencoti-server-autostart` (schema) | `packages/
|
| 98 |
-
| `opencoti-server-autostart` (call site) | `packages/opencode/src/config/config.ts` (
|
| 99 |
| `opencoti-server-client` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/opencoti-server-client": "workspace:*"`) | F3 M3 β opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
|
| 100 |
|
| 101 |
The two TS edits carry `// opencoti-hook: tiered-inference β see docs/features/tiered_inference.md` markers; the `package.json` dep has no anchor comment (JSON has no comments) β this registry table is its only anchor. On upstream merges, if upstream renames any of these files or restructures the `run` Effect.fn around the call-site marker, the marker comments remain the canonical pointer for re-applying the hook.
|
| 102 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
**F3 M4 (2026-05-23): no new registry rows.** M4 (session +
|
| 104 |
turn events flowing into opencoti-server) ships as a pure
|
| 105 |
plugin (`@opencoti/session-events-plugin`) that subscribes to
|
|
@@ -1718,3 +1791,28 @@ server-context meta field/aggregate-init position; the virtuals are additive.
|
|
| 1718 |
- `llamafile/version.h` β `OPENCOTI_TAG` default + `OPENCOTI_VERSION_STRING` (bump the default together with `vendors/llamafile/OPENCOTI_TAG` on every packaged cut).
|
| 1719 |
- `llamafile/main.cpp` β `--version` + help banner print `llamafile vX.Y.Z (opencoti-X.Y.Z-<tag>)`; `llamafile vX.Y.Z` token must stay FIRST (scripts/llamafile-version.sh contract).
|
| 1720 |
- `llamafile/BUILD.mk` β `OPENCOTI_TAG_FLAG` injection (make var from build pipeline) + `version.h` dep on `main.o`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
## Upstream
|
| 8 |
|
| 9 |
- Tracked upstream: [mann1x/opencode](https://github.com/mann1x/opencode)
|
| 10 |
+
(dev fork; opencode-of-record is anomalyco/opencode, fetched as the
|
| 11 |
+
`record` remote).
|
| 12 |
- Default upstream branch: `dev`.
|
| 13 |
- Sync cadence: weekly is the target, more often when upstream is
|
| 14 |
hot.
|
| 15 |
+
- β οΈ **Sync the fork FIRST.** A sync is only meaningful if
|
| 16 |
+
mann1x/opencode `dev` has itself been updated from anomalyco
|
| 17 |
+
(GitHub "Sync fork", or locally: merge `record/dev` into the fork's
|
| 18 |
+
`dev` and push β expect conflicts in any locally-pinned dep files).
|
| 19 |
+
On 2026-07-15 the fork was 2 commits past a 1757-commit-stale base;
|
| 20 |
+
syncing against it would have been a no-op.
|
| 21 |
+
- First full sync completed **2026-07-15**: dev merged
|
| 22 |
+
anomalyco/opencode `dev` @ `05c3e40a` (sync-branch merge
|
| 23 |
+
`540d82d630`, landed on dev as `c896a910f9`). Since that merge the
|
| 24 |
+
histories are properly joined β future syncs are a plain
|
| 25 |
+
`git merge upstream/dev` with a real merge-base (no graft needed).
|
| 26 |
+
Full gotcha log: see "2026-07-15 β first full opencode sync" below
|
| 27 |
+
the Registry.
|
| 28 |
|
| 29 |
## The rule
|
| 30 |
|
|
|
|
| 104 |
| `tiered-inference` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/tiers": "workspace:*"`) | F1 β Tiered Inference Engine | [features/tiered_inference.md](../features/tiered_inference.md) |
|
| 105 |
| `memory-tools-resolvability` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/memory": "workspace:*"`) | F4 M5 β Memory / Embedder | [features/memory_embedder.md](../features/memory_embedder.md) |
|
| 106 |
| `opencoti-default-plugins` (import) | `packages/opencode/src/config/config.ts` (just below the existing `ConfigPlugin` import) | F4 M5 follow-up β auto-wire opencoti plugins | [features/memory_embedder.md](../features/memory_embedder.md) |
|
| 107 |
+
| `opencoti-default-plugins` (call site) | `packages/opencode/src/config/config.ts` (inside `loadInstanceState` Effect.fnUntraced β upstream split the single loader into several Effect.fnUntraced functions 2026-07-15; the old `return { config: result, β¦ }` anchor is gone) | F4 M5 follow-up β auto-wire opencoti plugins | [features/memory_embedder.md](../features/memory_embedder.md) |
|
| 108 |
+
| `tiered-inference-setup` (import) | `packages/opencode/src/index.ts` (import block, after the `Heap` import β the former `isRecord` import anchor was removed upstream 2026-07-15) | F1 M5.7 β Tier-0 setup CLI | [features/tiered_inference.md](../features/tiered_inference.md) |
|
| 109 |
| `tiered-inference-setup` (call site) | `packages/opencode/src/index.ts` (`.command(tierCommand)` after `.command(DbCommand)`) | F1 M5.7 β Tier-0 setup CLI | [features/tiered_inference.md](../features/tiered_inference.md) |
|
| 110 |
| `opencoti-server-autostart` (import) | `packages/opencode/src/config/config.ts` (just below the `opencoti-default-plugins` import) | F3 M3 β opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
|
| 111 |
+
| `opencoti-server-autostart` (schema) | **`packages/core/src/v1/config/config.ts`** (`opencoti:` block in `ConfigV1.Info` β RELOCATED 2026-07-15: upstream moved the config schema out of packages/opencode into core/v1) | F3 M3 β opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
|
| 112 |
+
| `opencoti-server-autostart` (call site) | `packages/opencode/src/config/config.ts` (inside `loadInstanceState` Effect.fnUntraced, just after the `opencoti-default-plugins` call site; since 2026-07-15 upstream's `Log` module is gone β the hook `yield*`s `Effect.forkDetach` on a fiber that awaits the autostart promise and logs the outcome via `Effect.logInfo`/`Effect.logWarning`, formatting through `describeAutostartOutcome` exported by the additive package) | F3 M3 β opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
|
| 113 |
| `opencoti-server-client` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/opencoti-server-client": "workspace:*"`) | F3 M3 β opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
|
| 114 |
|
| 115 |
The two TS edits carry `// opencoti-hook: tiered-inference β see docs/features/tiered_inference.md` markers; the `package.json` dep has no anchor comment (JSON has no comments) β this registry table is its only anchor. On upstream merges, if upstream renames any of these files or restructures the `run` Effect.fn around the call-site marker, the marker comments remain the canonical pointer for re-applying the hook.
|
| 116 |
|
| 117 |
+
### 2026-07-15 β first full opencode sync (#682): method + gotchas
|
| 118 |
+
|
| 119 |
+
First (and history-joining) sync: dev merged anomalyco/opencode `dev`
|
| 120 |
+
@ `05c3e40a` (+1757 commits past the 2026-05-21 import base
|
| 121 |
+
`dbd709376b`). Executed in an isolated worktree on branch
|
| 122 |
+
`sync/opencode-2026-07-15`; landed on dev as `c896a910f9`.
|
| 123 |
+
|
| 124 |
+
**Method (one-time β not needed again):** the bootstrap commit
|
| 125 |
+
`b34a23097f` was a squashed import with no shared history, so a plain
|
| 126 |
+
merge had no merge-base. Fix: `git replace --graft b34a23097f
|
| 127 |
+
dbd709376b` (the empirically-identified import base β found by
|
| 128 |
+
blob-matching untouched files against upstream commits), then a
|
| 129 |
+
normal 3-way `git merge record/dev`, then `git replace -d b34a23097f`.
|
| 130 |
+
The merge commit permanently joins the histories.
|
| 131 |
+
|
| 132 |
+
**Conflict surface (10 files, all predicted):** the 5 hook files +
|
| 133 |
+
resurrected identity files (re-deleted) + `packages/extensions` zip
|
| 134 |
+
artifacts + root `package.json`/`bun.lock`. `config/config.ts` and
|
| 135 |
+
`index.ts` were taken upstream-wholesale and the hooks hand-replayed
|
| 136 |
+
(upstream refactor destroyed the anchors β see updated Registry rows).
|
| 137 |
+
|
| 138 |
+
**Gotchas (verified this sync β check these FIRST next time):**
|
| 139 |
+
|
| 140 |
+
1. **Corrupt bun store after a crashed install.** A postinstall crash
|
| 141 |
+
(Debian's broken `/usr/bin/node-gyp` on `tree-sitter-powershell`)
|
| 142 |
+
left `node_modules/.bun/<pkg>` half-extracted; a later
|
| 143 |
+
`--ignore-scripts` install did NOT repair it β hundreds of phantom
|
| 144 |
+
typecheck errors (`Cannot find module 'drizzle-orm/sqlite-core'`).
|
| 145 |
+
Rule: after ANY failed `bun install`, `rm -rf node_modules` before
|
| 146 |
+
trusting typecheck. (Root cause fixed 2026-07-15: Debian 11's apt
|
| 147 |
+
node-gyp 7.1.2 is broken under node 22 β `npm install -g node-gyp`
|
| 148 |
+
puts a working v13 at `/usr/local/bin`; full `bun install` with
|
| 149 |
+
postinstall scripts now works on solidPC.)
|
| 150 |
+
2. **`.gitignore` coredump patterns swallowed `packages/core`.** Our
|
| 151 |
+
unanchored `core` / `core.*` ignore patterns hid upstream's new
|
| 152 |
+
package. Fixed by root-anchoring (`/core`, `/core.*`). Any
|
| 153 |
+
unanchored ignore pattern can collide with future upstream names.
|
| 154 |
+
3. **Bootstrap-zip artifacts create phantom conflicts.** The squashed
|
| 155 |
+
import lost a symlink (`extensions/zed/LICENSE`), an exec bit
|
| 156 |
+
(`script/sync-zed.ts`), and silently dropped a whole file
|
| 157 |
+
(`packages/opencode/script/build-node.ts` β which also survived
|
| 158 |
+
UNTRACKED on disk and blocked the dev merge until moved aside).
|
| 159 |
+
Any modify/delete conflict on a file we never touched is probably
|
| 160 |
+
a zip artifact β resolve toward upstream.
|
| 161 |
+
4. **Upstream deleted the `Log` module** (Effect logging is the new
|
| 162 |
+
idiom): any hook passing `log` must drop/adapt it. The config
|
| 163 |
+
schema now lives in `packages/core/src/v1/config/config.ts`
|
| 164 |
+
(`ConfigV1.Info`); `fix-node-pty` postinstall moved to
|
| 165 |
+
`packages/core`.
|
| 166 |
+
5. **Husky pre-push runs `bun typecheck`** β it fails in any scratch
|
| 167 |
+
worktree without node_modules; use `--no-verify` there (content
|
| 168 |
+
already verified in the sync worktree).
|
| 169 |
+
6. ~~Open follow-ups~~ both CLOSED 2026-07-15: autostart-outcome
|
| 170 |
+
logging restored via `Effect.forkDetach` + `describeAutostartOutcome`
|
| 171 |
+
(see Registry row); system node-gyp fixed via `npm install -g
|
| 172 |
+
node-gyp` (v13 at `/usr/local/bin` shadows Debian's broken 7.1.2),
|
| 173 |
+
so full `bun install` with scripts works on solidPC.
|
| 174 |
+
|
| 175 |
+
|
| 176 |
**F3 M4 (2026-05-23): no new registry rows.** M4 (session +
|
| 177 |
turn events flowing into opencoti-server) ships as a pure
|
| 178 |
plugin (`@opencoti/session-events-plugin`) that subscribes to
|
|
|
|
| 1791 |
- `llamafile/version.h` β `OPENCOTI_TAG` default + `OPENCOTI_VERSION_STRING` (bump the default together with `vendors/llamafile/OPENCOTI_TAG` on every packaged cut).
|
| 1792 |
- `llamafile/main.cpp` β `--version` + help banner print `llamafile vX.Y.Z (opencoti-X.Y.Z-<tag>)`; `llamafile vX.Y.Z` token must stay FIRST (scripts/llamafile-version.sh contract).
|
| 1793 |
- `llamafile/BUILD.mk` β `OPENCOTI_TAG_FLAG` injection (make var from build pipeline) + `version.h` dep on `main.o`.
|
| 1794 |
+
|
| 1795 |
+
## 2026-07-15 β 0134 cuda-cross-sbsa (#681, aarch64/DGX Spark lane)
|
| 1796 |
+
|
| 1797 |
+
### `0134-cuda-cross-sbsa` (#681)
|
| 1798 |
+
Marker: `opencoti-hook: cuda-cross-sbsa` (3 sites). Build-script-only β no
|
| 1799 |
+
kernel/host C++ change; native x86_64 builds byte-unaffected (flag default off,
|
| 1800 |
+
`OPENCOTI_HOST_GCC/GXX` default to `gcc`/`g++`).
|
| 1801 |
+
- `llamafile/cuda.sh` β `--cross-sbsa` flag: dedicated build dir
|
| 1802 |
+
`~/.cache/llamafile-cuda-sbsa-build` (never mix aarch64 objects into the
|
| 1803 |
+
x86_64 cache), arch set sm_110f (Jetson Thor) + sm_121a (DGX Spark GB10),
|
| 1804 |
+
and `-ccbin aarch64-linux-gnu-g++ -target-dir sbsa-linux` riding in
|
| 1805 |
+
**ARCH_FLAGS** (the LINK receives ARCH_FLAGS β putting them in COMMON_FLAGS
|
| 1806 |
+
emits an x86_64 DSO). `-lcuda` resolves against the sbsa driver stub.
|
| 1807 |
+
Tunable `OPENCOTI_SBSA_CCBIN`. Requires CUDA 13.x + the cuda-cross-sbsa
|
| 1808 |
+
target packages (cached at `.opencoti/cross-sbsa/debs/`).
|
| 1809 |
+
- `llamafile/build-functions.sh` β `compile_ggml_core` host TUs go through
|
| 1810 |
+
`${OPENCOTI_HOST_GCC:-gcc}` / `${OPENCOTI_HOST_GXX:-g++}` so the cross build
|
| 1811 |
+
doesn't mix host arches at link.
|
| 1812 |
+
Consumers: `build:llamafile:cuda-sbsa` pipeline lane (stages
|
| 1813 |
+
`vendors/dist/llamafile/cuda-sbsa/<ver>/ggml-cuda.so`, NO ~/.llamafile
|
| 1814 |
+
side-load mirror β bug-224 cache-poisoning class) and
|
| 1815 |
+
`build:llamafile:package --arch aarch64` (embeds ONLY the sbsa DSO).
|
| 1816 |
+
Sync note: on a future llamafile bump re-anchor the two cuda.sh insertion
|
| 1817 |
+
points (after the BUILD_DIR selection block; after the CUDA-13 arch-selection
|
| 1818 |
+
chain, BEFORE the `--compress` block so `--compress-mode` still appends).
|