Developers

Development environment gotchas

Development environment gotchas

Two setups where a plain npm install or a plain test run does not work: locked-down containers, and machines without a Vulkan driver. The everyday setup is AGENTS.md § Prerequisites.

TypeScript compiler roles

Normal repository builds and type-checks use the TypeScript 7 native compiler. Compiler watch commands use TypeScript 6 because TypeScript 7.0.2 can miss filesystem changes on Linux and in containers (upstream issue). The repository also installs TypeScript 6 as the compatibility package for tools that use the TypeScript compiler API, including Jest and documentation tooling. The shared launcher keeps these roles explicit and avoids npm bin link order deciding which compiler a script runs.

To select a compiler for a command, set NODETOOL_TSC_VERSION:

NODETOOL_TSC_VERSION=7 npm run typecheck
NODETOOL_TSC_VERSION=6 npm run typecheck

The named :ts6 and :ts7 package commands are the preferred rollback and comparison entry points. NODETOOL_TSC_HEAP_MB controls the V8 heap only for the TypeScript 6 JavaScript compiler. The TypeScript 7 native compiler does not receive that Node heap option.

If a compiler-role check fails after install, run npm run check:typescript before attempting a build. It reports missing or mismatched compiler packages, native platform packages, and direct compiler-path drift.

Local test and build memory

Local Turbo scripts run one task at a time, and local Jest and Vitest suites use two workers. This leaves memory for macOS and the editor while checking the same code. CI keeps its existing parallelism. To tune a local run, set NODETOOL_TURBO_CONCURRENCY for Turbo tasks or NODETOOL_TEST_WORKERS for test workers, for example:

NODETOOL_TEST_WORKERS=1 npm run test:affected
NODETOOL_TURBO_CONCURRENCY=2 npm run build:packages

TypeScript 7 is a native process, so Node’s heap flags do not limit its memory. The root npm run typecheck checks web, Electron, and mobile sequentially.

libc in the lockfile

npm selects a prebuilt native package by the os, cpu and libc recorded in package-lock.json. npm 10 reads libc from the registry but never writes it, which makes npm ci reify both the glibc and the musl build of every such package — npm install filters them, npm ci does not, so it lands in CI, the Docker image and every fresh clone rather than on a developer’s machine.

npm run fix:lockfile-libc backfills the field from the registry, and npm run check:lockfile-libc (part of npm run check) fails when it goes missing again. Rerun the fix after any npm install under npm 10 that rewrites the lock, and commit the result.

Worktrees with cloned dependencies (macOS)

A full npm install in a new worktree takes minutes and several GB. On APFS, clone the dependency and build folders from an existing checkout instead. cp -c makes a copy-on-write clone: it takes seconds, uses almost no disk, and a later write in one tree does not change the other.

Run scripts/setup-agent-env.sh inside a new worktree to do this automatically. It finds the main checkout through git, clones with cp -cR on macOS or cp -R --reflink=always on Linux, skips worktrees nested in the main checkout, and runs npm install when the file system cannot clone. The manual steps:

MAIN=~/workspace/nodetool                  # checkout with installed deps
WT=~/workspace/nodetool-wt/<name>
git -C "$MAIN" fetch origin
git -C "$MAIN" worktree add -b <branch> "$WT" origin/main
cd "$MAIN"
{ find . -name node_modules -type d -prune | grep -v '/node_modules/.*node_modules'
  find . -path ./node_modules -prune -o -name dist -type d -prune -print | grep -v node_modules
} | sort -u | while read -r d; do
  mkdir -p "$WT/$(dirname "$d")" && cp -cR "$d" "$WT/$d"
done
cd "$WT" && npm run build:packages        # the cloned dist/ is from the source branch
  • The workspace links in node_modules/@nodetool-ai/ are relative, so they resolve to the new worktree’s packages/.
  • The cloned dist/ folders match the source checkout’s branch. Run npm run build:packages before you trust a test or harness result.
  • du counts a clone at full size. Measure the space you get back with df.
  • On Linux, cp --reflink=auto does the same on btrfs and XFS. On other file systems it makes a full copy, so run npm ci instead.

Change dependencies in a worktree

An install in a worktree writes only to that worktree’s clones. The source checkout does not change.

  • A failed root postinstall deletes the better-sqlite3 binary it tried to rebuild. Clone it back with cp -cR "$MAIN/node_modules/better-sqlite3/build" node_modules/better-sqlite3/, or install with --ignore-scripts and run npm run rebuild:native.
  • After npm install changes a version range, run npm explain <package>. npm can place the new version under packages/<name>/node_modules/ instead of the root. A package that the runtime loads by name, such as node-llama-cpp, must stay at the root. Record the root placement in the lockfile. A later npm install keeps a placement that already satisfies the range.
  • npm 10 removes libc on every install that rewrites the lockfile. Run npm run fix:lockfile-libc last, and confirm with npm run check:lockfile-libc.
  • mobile/node_modules is cloned with the rest but can be incomplete. Run npm --prefix mobile ci before mobile typecheck or tests.

Remove worktrees

git worktree remove --force deletes uncommitted work. First list the work that only the worktree holds:

git -C "$WT" status --porcelain                        # uncommitted
git -C "$WT" rev-list --count HEAD --not --remotes     # unpushed commits

Commit anything you keep to a branch, then run git worktree remove --force "$WT" and git worktree prune.

Install in sandboxed / proxied environments

Three postinstall steps break npm install in locked-down containers (CI sandboxes, Claude Code on the web, proxied networks). A failed postinstall makes npm roll back the entire node_modules tree, so one bad package means no dependencies at all — including ESLint and the design-lint gate.

  1. keytar needs libsecret headers on Linux. Without them node-gyp fails with Package libsecret-1 was not found. Fix first: apt-get install -y libsecret-1-dev.
  2. electron downloads its binary in postinstall. Proxies that block the download (HTTP 403) fail the install. Skip it when you don’t need to launch Electron: ELECTRON_SKIP_BINARY_DOWNLOAD=1 npm install.
  3. onnxruntime-node downloads CUDA binaries from GitHub releases in postinstall (it assumes CUDA when nvcc is absent). Same proxy failure mode, and there is no skip env var in our pinned version.

When only the JS toolchain matters (typecheck, lint, unit tests that don’t hit native modules), bypass all of the above in one step:

npm install --ignore-scripts --no-audit --no-fund

This skips every postinstall — including the root better-sqlite3 rebuild — so anything touching the database needs npm run rebuild:native afterwards (which will still require the downloads above to have succeeded).

WebGPU on a headless machine

The image nodes are shader-backed: every lib.image.* generator and every nodetool.image transform reaches WebGPU through Dawn. On a machine with no Vulkan driver they fail with:

No WebGPU adapter available (Node/Dawn). On headless Linux this usually means
no Vulkan driver (ICD) is installed — Dawn has no software fallback of its own.

This is an environment gap, not a broken test and not an unsupported platform. CI already solves it: the test-packages-* shards of .github/workflows/quality-checks.yml and the browser job in test.yml both install mesa-vulkan-drivers, which ships lavapipe — a CPU Vulkan ICD. Do not conclude from this error that shader-backed nodes cannot be tested, and do not skip a test because your box hits it; the same test passes in CI.

With root:

apt-get install -y mesa-vulkan-drivers

Without root (sandboxes, dev containers), extract the driver and point the Vulkan loader at it. libvulkan1 — the loader — is usually already present; only the ICD is missing:

apt-get download mesa-vulkan-drivers
dpkg-deb -x mesa-vulkan-drivers_*.deb /tmp/vk
# The shipped manifest names the library relatively, so rewrite it absolutely:
python3 - <<'PY'
import json
p = "/tmp/vk/usr/share/vulkan/icd.d/lvp_icd.json"
d = json.load(open(p))
d["ICD"]["library_path"] = "/tmp/vk/usr/lib/x86_64-linux-gnu/libvulkan_lvp.so"
json.dump(d, open("/tmp/vk/lvp_icd.json", "w"))
PY
export VK_DRIVER_FILES=/tmp/vk/lvp_icd.json

Then run the tests as usual. Lavapipe is a software rasterizer, so it is slow but exact — pixel comparisons (nodetool.compare.CompareImages) are reproducible under it, which is what packages/base-nodes/tests/image-examples-run.test.ts relies on.

Edit this page on GitHub