# NodeTool Documentation — Full Text > Local-first visual environment for building and running AI workflows. Build agents visually, deploy anywhere, privacy by design. This file concatenates the NodeTool guide documentation for LLM ingestion. The full per-node reference (900+ nodes) is omitted for size — browse it at https://docs.nodetool.ai/nodes/ or search at https://docs.nodetool.ai/search.json.--- # index.md Source: https://docs.nodetool.ai/ The open creative AI workspace Every model. Your keys. Your canvas. NodeTool replaces the chatbox with a node canvas where image, video, audio, and LLM models run side by side. Bring your keys, or run everything locally. Open source, AGPL-3.0. Get started Examples Cookbook Featured use cases Three flagship workflows, end to end. Each starts from a few inputs and builds a finished result on one canvas you can re-run, restyle, and re-point at your own story. See all use cases → Film Movie Trailer Generator One logline becomes a treatment, a shot list, key art, and a cut teaser. Marketing Product Video Generator A brief and one product photo become a cinematic 16:9 clip. Design Movie Poster Generator Title, genre, and audience become a batch of theatrical poster concepts. What you can do Mix models from every vendor — Wire Flux next to GPT-5 next to ElevenLabs in one graph. Pick the best model per step, not per project. Run frontier models locally — Ollama, MLX, and GGUF on your hardware. Works offline. Files never leave your disk. Bring your own keys — Pay OpenAI, Anthropic, Gemini, Replicate, FAL, and ElevenLabs directly. No credit markup, no provider tax. Ship a workflow as a Mini-App — Hide the graph, expose just inputs and outputs. Share a link, no install required. Build agents that drive workflows — Multi-step planning, tool calling, streaming. Drop them into any pipeline. Chat with your documents — Local SQLite-vec, embeddings, RAG. Your data never leaves the machine. Iterate visually, not via prompts — Click a node, change a value, re-run. Watch data flow through every edge. Studio or Cloud Same code, same workflows. Both AGPL-3.0. NodeTool Studio — desktop Mac, Windows, Linux. Local inference via Ollama, MLX, and GGUF. Works offline. Prompts and outputs stay on disk. BYOK for cloud providers when you want them. Download Studio → NodeTool Cloud — browser Hosted, no install. Same canvas, same nodes. BYOK for every cloud provider — OpenAI, Anthropic, Gemini, Replicate, FAL, ElevenLabs, HuggingFace. No local models. Open Cloud → No credit markup. Cloud hosts the same code in this repo. Self-host the Docker images any time. You pay providers directly. What you can build Image and video Flux, Qwen, Wan, Seedance, Sora, Veo, Kling on one canvas. Movie Posters → Story to video Prompt to storyboard to narration to animation to score. Movie Trailer Generator → Sound and voice Music, sound design, narration. ElevenLabs, MusicGen, Whisper in the graph. Image to Audio Story → Agents Multi-step agents that plan, call tools, and drive pipelines. Fetch Papers → More patterns — pipelines, data, RAG, email — in the Cookbook. Get started Download NodeTool for macOS, Windows, or Linux. Open a template, press Run. Edit, re-run, ship as a Mini-App. Explore New here: Getting Started · Key Concepts · UI Building: Chat & Agents · Cookbook · Examples Self-hosting: Deployment · Configuration · API Extending: Developer Guide · Custom Nodes · CLI Open source AGPL-3.0. Discord · GitHub.--- # Kernel Parity Gaps Source: https://docs.nodetool.ai/KERNEL_PARITY_GAPS.html Kernel Parity Gaps Tracked differences between the TypeScript kernel (packages/kernel) and the reference Python runtime. Each gap has regression tests that assert the current TypeScript behavior so a future fix is an explicit, reviewed change rather than a silent one. The desired end-state is recorded here instead of as bare // TODO comments scattered through the tests. Tests: packages/kernel/tests/parity-output-edge-gaps.test.ts. Gap #9 — Output node handling (consecutive deduplication) Python’s process_output_node() consumes a stream item-by-item and emits one output_update per value, deduplicating consecutive identical values. Current TS behavior: the output node is buffered and fires once with the last value, so three yields of same, same, different produce a single output_update carrying different. Desired: when the output node consumes the stream item-by-item, consecutive dedup should reduce same, same, different to two output_update messages (same once, then different). Asserted at: the should ... deduplicate consecutive test (current behavior: outputMsgs.length === 1). output_update message fields (node_id, node_name, output_name, value, output_type, metadata) are already populated — that part of gap #9 is fixed. Gap #15 — Edge counter updates for input-originating edges Python emits edge_update at several lifecycle points, including "drained" during drain_active_edges() and "completed" during _send_EOS(). _drainActiveEdges() now exists in TS and emits "drained" for edges whose target handle is still open at completion. Current TS behavior: input nodes are not actors, so _sendEOS is never called for edges that originate at an input node. Those edges receive an "active" update from _dispatchInputs() but no terminal "completed" / "drained". Desired: input-originating edges should also receive a terminal "completed" (or "drained") update at the end of the run. Asserted at: the two edge-lifecycle tests (current behavior: completedMsgs.length === 0 for input edges). Edges between non-input nodes already get both "active" and "completed".--- # Kernel & WorkflowRunner review — bugs, issues, design flaws Source: https://docs.nodetool.ai/KERNEL_RUNNER_ANALYSIS.html Kernel & WorkflowRunner review — bugs, issues, design flaws Review of packages/kernel/ (runner, actor, inbox, graph, io, correlation analysis, triggers, durable inbox) as of 2026-07-04. Findings are ordered by severity within each section. Line references are against the source at the time of review (commit 21230c7). Fix status (this PR): findings #1, #2, #3, #4, #6, #9, #13, #16, #17 are fixed; #7, #11, #12 now emit warnings instead of failing silently (see the follow-up commits on this branch). Still open, pending a design decision: #5 (control-response attribution needs an event id echoed through the actor), #8 (RunResult.outputs accumulation semantics), #10 (backpressure defaults), #14, #15, and the smaller issues. Bugs 1. Edge status regresses from completed back to active at run end _incrementEdgeCounter throttles edge_update emissions and marks throttled edges dirty (runner.ts:1623-1625). _sendEOS emits the terminal status: "completed" update (runner.ts:1473-1479) but never clears the edge from _edgeCounterDirty. At run end, _drainActiveEdges calls _flushEdgeCounters (runner.ts:1643-1654), which re-emits status: "active" for every dirty edge — including edges already reported completed. Any edge that received a message within the last EDGE_UPDATE_MIN_INTERVAL_MS before its source finished ends the run with completed followed by active as its final message. Fix: clear the edge from _edgeCounterDirty (and stamp _edgeCounterLastEmitMs) inside _sendEOS, or have _flushEdgeCounters skip edges whose EOS was already emitted (_eosSentEdges is available). 2. Behavior-flag hydration is OR-based, so stale saved true flags can never be corrected Graph.loadFromDict computes is_streaming_input: descriptorDefaults.is_streaming_input || node.is_streaming_input || false (graph.ts:437-457), same for is_streaming_output, is_controlled, is_join_node. The comments claim the registry is the source of truth and saved JSON is “a cache that gets overwritten”, and input_mode / output_correlation really are overwritten — but the boolean flags are ORed. If a node type migrates from streaming to buffered (or stops being a join node), every saved workflow keeps the stale true and runs the wrong execution mode, with no way to fix it short of re-saving the graph. The actor trusts these flags to pick run() vs process() vs _runControlled, so this silently changes execution semantics. Fix: treat the registry value as authoritative when the resolver returned a descriptor (descriptorDefaults.is_streaming_input ?? node.is_streaming_input ?? false is still wrong; it should be descriptorDefaults.is_streaming_input ?? false when the registry resolved, falling back to the saved value only for unresolved/dynamic types). 3. pushInputValue bypasses the input node’s process() _dispatchInputs runs the input node’s executor so transforming inputs (StringInput max_length, DocumentFileInput building a DocumentRef, MessageDeconstructor splitting) emit their declared per-handle outputs (runner.ts:938-951). pushInputValue (runner.ts:337-367) puts the raw value straight onto outgoing edges. The same input node behaves differently depending on whether the value arrived as a start param or was streamed in — streamed values skip validation/transformation entirely and leak the raw value to every edge regardless of sourceHandle semantics. It also stamps a different source_edge_id (external:... vs the real edge id), which will not match the per-edge lineage-done/closed bookkeeping keyed by real edge ids. 4. Controlled-node wait can deadlock the whole run _runControlled refuses to process any control event until every data handle has produced at least one value (actor.ts:1301-1303, _waitForDataInputs at actor.ts:1341-1377). The wait loop only exits when all handles are satisfied or the inbox is fully drained — and “fully drained” includes __control__, which stays open while the controller is alive. Two realistic shapes hang forever: An upstream node completes without emitting on the controlled node’s data handle (filter/conditional output). The handle closes but pending never empties, control events are held aside indefinitely, and a controller awaiting sendControlEvent never resolves → _processGraph’s Promise.all never settles. No timeout exists anywhere on this path. An agent both controls the node and (directly or transitively) produces its data input: the agent awaits the control response before emitting data; the controlled node awaits data before answering. The runner already rejects pending control responses when the controlled actor finishes, but here the controlled actor never finishes. At minimum the wait should give up when all data handles are closed (drop pending handles whose upstream count reached zero), mirroring how _runCorrelatedImpl treats closed empty-scope handles as “use defaults”. 5. sendControlEvent responses are matched to the wrong output when event sources mix _sendMessages resolves the oldest pending sendControlEvent waiter on any output emission from the target node (runner.ts:1357-1364). The FIFO assumption holds only when sendControlEvent is the sole source of control events. If the same node is also driven by a control edge (__control_output__ routing) or by dispatchControlEvent, an edge-driven firing’s outputs resolve a sendControlEvent waiter that belongs to a different event. The agent then receives another invocation’s result. The response should be correlated to the event (e.g. stamp an event id on the ControlEvent and echo it through the actor), not inferred from global output order. 6. _dispatchInputs falls back to leaking the raw value on missing handles handleValue = nodeOutputs[edge.sourceHandle] !== undefined ? ... : value (runner.ts:958-961). The fallback exists for test doubles returning {}, but it applies per-edge: a real input node with several declared outputs that legitimately omits one (e.g. “no attachments”) gets the raw input value delivered on that edge instead of nothing. Downstream receives a value of the wrong type/shape. The fallback should trigger only when process() returned an entirely empty record, not per-handle. 7. Streaming-input node without run() silently ignores all its inputs The legacy fallback (actor.ts:388-399) calls process() once with only the node’s own properties. Data queued on its inbox is never read; it is reported only as a post-run “pending inbox work” warning. A misdeclared node (registry says is_streaming_input, implementation has no run) drops all upstream data while the run still reports completed. This should be a node error, not a silent fallback. 8. RunResult.outputs keeps only the last firing of multi-fire terminal nodes Output collection stores result.outputs, which is the actor’s _latestResult — the last invocation’s record (runner.ts:1099-1110, actor.ts:473). A terminal node fired once per streamed item contributes only its final values to RunResult.outputs; earlier firings survive only as output_update messages. The Record<string, unknown[]> shape suggests accumulation was intended. Also, Object.values(result.outputs) flattens multiple output handles into one array, losing handle names, and _isOutputNode is “no outgoing data edges” (runner.ts:1717-1721), so a controller agent with only a control edge out also lands in workflow outputs, and two terminal nodes sharing a name merge into one bucket. 9. TriggerWakeupService defeats DurableInbox’s append serialization DurableInbox.append chains appends per instance because getMaxSeq→save is a read-modify-write (durable-inbox.ts:196-222). But deliverTriggerInput constructs a fresh DurableInbox per call over the shared store (trigger-wakeup.ts:88-89), so concurrent deliveries for the same (run, node) each get their own no-op chain, read the same maxSeq, and persist duplicate seq values — corrupting the ordering that findPending sorts by. The service should cache one DurableInbox per (runId, nodeId). Design flaws / risks 10. No backpressure in production, and the correlated scheduler can’t be backpressured at all No production caller passes bufferLimit, so every inbox is unbounded (runner.ts:771). Independently, _runCorrelatedImpl eagerly drains the inbox into actor-local buckets (maxBuckets, sticky maps), so even with a bufferLimit, buffered nodes release inbox backpressure immediately and buffering just moves into the actor. The only guards are maxPendingKeys/maxPendingMessagesPerKey (10k × 10k envelopes). A fast producer feeding a slow consumer grows the heap without bound; audio-rate streams make this practical, not theoretical. 11. Unmigrated streaming producers silently collapse to last-value-wins downstream The analyzer defaults missing output_correlation to single with repeats: false (correlation-analysis.ts:480). A custom/legacy node that emits many values on one slot (without declaring chunk/iteration) feeds a downstream buffered handle classified empty-scope sticky (actor.ts:568-579), where each arrival overwrites the previous (actor.ts:813-816) and the node fires once at close with only the last value. Pre-correlation semantics fired per message. Correctness of every third-party node now hinges on correlation metadata being declared; the failure mode is silent data loss. There is no warning when multiple envelopes overwrite an empty-scope sticky slot — emitting one would make the migration gap visible. 12. Non-driver duplicate keys are silently dropped inside the actor In _runCorrelatedImpl with no repeating driver, a key fires once (fired set, actor.ts:727-738); a second envelope arriving for the same key on a max-scope handle stays in the actor-local bucket forever. Unlike inbox-stranded data, _checkPendingInboxWork cannot see it (the envelope was already popped), so the loss is completely silent. 13. Graph mutates caller-owned node descriptors _detectControlledNodes writes is_controlled = true onto the node objects passed in (graph.ts:515-525), and the runner passes the caller’s graphData.nodes through (bypass rewrite preserves object identity). A WorkflowRunner.run() therefore mutates the caller’s graph data despite the readonly typing. Harmless today, but it leaks state across runs and violates the descriptor contract. 14. Control context exposes every property value to the LLM _buildControlContext embeds all non-underscore property values of controlled nodes as tool-schema defaults (runner.ts:1810-1900). If a controlled node carries a sensitive string property (token, endpoint, prompt containing PII), it is shipped into the controller LLM’s tool definitions. Similarly, generation_complete rides all scalar resolved inputs into persisted asset metadata (actor.ts:149-164). Both need a notion of secret/sensitive properties to redact. 15. Trigger watchdog restarts forever with no backoff _watchdogCheck restarts failed/completed jobs every tick (trigger-manager.ts:337-374). A workflow that fails at startup restarts every 30 s indefinitely — no backoff, no retry cap, no circuit breaker. getInstance also silently ignores differing opts on subsequent calls, and only the latest watchdog check is awaited by shutdown (_watchdogCheckInFlight is overwritten per tick), so an older, still-running check can theoretically revive a job after shutdown’s await. 16. Two job ids per run edge_update messages use this.jobId from the constructor while job_update uses request.job_id (runner.ts:1629-1635 vs 473-478). Callers that pass different values get edge updates that clients cannot associate with the job. One of the two should win (or the constructor id should be dropped). 17. Runner instance lifecycle races _resetRunState() runs at the top of _runImpl, so cancel() called between construction and run() is silently lost (_cancelled reset, fresh AbortController). A second run() on the same instance while the first is in flight clobbers all shared state. Cheap fix: latch a “cancelled before start” flag and reject concurrent run(). Smaller issues Inbox arrival queue is O(n²) — _arrival is a plain array; _removeFromArrival does indexOf+splice per consumed envelope and prepend does unshift (inbox.ts:113, 579-585). With deep backlogs (audio-rate streams, list aggregation) this degrades quadratically. A per-handle counter or linked list removes the scans. _notifyWaiters / _notifyPutWaiters wake every waiter while their doc comments say “wake one” (inbox.ts:552-566) — thundering herd on hot streams; each put wakes all consumers and all blocked producers. put() on a closed inbox silently drops data and returns normally (inbox.ts:202); fine for cancellation, but producers cannot distinguish delivered from dropped. _emitNodeStatus("running") fires at actor start, before any input arrives (actor.ts:303), so every node in the graph shows “running” immediately; there is no “waiting for inputs” state. getInputSchema/getOutputSchema classify nodes by type.includes("Input") (graph.ts:772-783) while the runner uses type.startsWith("nodetool.input.") — the two predicates disagree for types like ...InputSanitizer, and every schema property is marked required even when a default exists. A node that is both is_streaming_input and is_controlled takes the streaming branch and its control events are never consumed (actor.ts:310-401 branch order); combinations of behavior flags are not validated. _runControlled ignores _listInputHandles and lineage — multi-edge list inputs to controlled nodes keep only the latest value (actor.ts:1384-1395). Duplicate __control_output__ routing is possible by edge shape: the runner routes __control_output__ to all control edges, then the edge loop routes outputs[edge.sourceHandle] again (runner.ts:1154-1227). Convention (sourceHandle: "__control__") prevents overlap today, but an edge saved with sourceHandle: "__control_output__" would deliver the event twice; nothing validates control-edge source handles. MemoryDurableInboxStore is O(n) per operation over one flat array — acceptable for tests, but it is also the default store used by TriggerWakeupService in production paths, and TriggerInput records accumulate until an explicit cleanupProcessed call. What is solid Cycle handling is sound end-to-end (data cycles rejected by the correlation topo sort, self-loops by validateEdgeEndpoints, control cycles by DFS). The double-EOS guard (_eosSentEdges), per-unique-controller __control__ counting, the FIFO queue for control responses replacing the old single-slot resolver, the held-control-events fix in _waitForDataInputs, and the pending-start dedup in TriggerWorkflowManager all show earlier races were found and fixed with regression tests. The message-retention cap and audio-chunk exclusion in _emit are well reasoned. The Stryker annotations indicate real mutation coverage. Suggested priority Flag hydration OR-logic (#2) — silent wrong execution mode on saved graphs. Controlled-node deadlock (#4) and control-response attribution (#5) — hangs and wrong agent results at runtime. Streaming-migration last-value collapse (#11) + duplicate-key silent drop (#12) — silent data loss; at minimum emit warnings. pushInputValue transformation bypass (#3) and dispatch fallback leak (#6). Edge status regression (#1) — small, contained UI fix. Backpressure story (#10) — decide on a default bufferLimit and a bucket-level cap that actually applies to buffered nodes.--- # Agent CLI Source: https://docs.nodetool.ai/agent-cli.html The nodetool agent command runs autonomous AI agents defined in YAML configuration files. Agents use the planning agent architecture to break an objective into steps, execute them with tools, and return a final result. Every run streams a live trace of planning, tool calls, and step results to stderr. Overview The agent CLI enables: Autonomous execution: Run agents that plan and execute tasks independently YAML configuration: Define agent behavior, model, and tools in a config file Planning agent integration: Objectives are decomposed into a step DAG and executed Tool orchestration: Agents use multiple tools to accomplish complex goals Workspace: Each agent run operates against a workspace directory for file operations Subcommands The nodetool agent command has three subcommands: nodetool agent run <yaml-file> # Run an agent from a YAML config nodetool agent test <yaml-file> # Validate a config (provider, model, tools) nodetool agent list <dir> # List YAML agent configs in a directory nodetool agent run Runs an agent. The objective comes from one of three sources, in priority order: The -o, --objective "..." flag Piped stdin The objective: field in the YAML file # Objective via flag nodetool agent run research-agent.yaml --objective "Research the latest AI trends" # Objective via stdin echo "Research the latest AI trends" | nodetool agent run research-agent.yaml # Objective from the YAML file's `objective:` field nodetool agent run research-agent.yaml Options: -o, --objective <text> — Objective for the agent (overrides stdin and the YAML default) -p, --provider <id> — Override the provider from the YAML -m, --model <id> — Override the model from the YAML -w, --workspace <path> — Override the workspace directory from the YAML --json — Emit each agent event as a JSON line on stderr (the final result still goes to stdout) -v, --verbose — Include low-level chunk and other events in the trace The final result is written to stdout; the live trace is written to stderr. This lets you capture the result cleanly: nodetool agent run research-agent.yaml -o "Summarize NodeTool" > result.txt nodetool agent test Validates a config without running it. Checks that model.provider and model.id are present, resolves the tool list (warning about any unknown tools), and tries to instantiate the provider. nodetool agent test research-agent.yaml nodetool agent list Lists the YAML agent configs in a directory, showing each file’s provider/model and description. nodetool agent list examples/agents/ Agent Configuration YAML Structure # Agent identification name: agent-name description: Agent purpose and capabilities # Core agent prompt system_prompt: | Detailed instructions for agent behavior. Multiple lines supported. # Default objective (used if --objective and stdin are both absent) objective: Research the latest AI trends # Model configuration model: provider: openai # AI provider id: gpt-4o # Model identifier name: GPT-4o # Display name (optional) # Planning agent (optional) planning_agent: enabled: true # set false to plan with the main model instead model: provider: openai id: gpt-4o-mini # Available tools tools: - google_search - browser - write_file - read_file # Execution parameters max_tokens: 128000 # per-step context token budget (default 128000) max_steps: 10 # maximum number of steps in the task # Model preferences (optional) preferred_providers: - anthropic - openai preferred_models: image: black-forest-labs/flux-schnell # Workspace workspace: path: ~/agent-workspace auto_create: true Configuration Fields name — Identifier for the agent (used in the trace header). description — Human-readable description (shown by agent list / agent test). system_prompt — Instructions defining agent behavior. objective — Default objective, used when neither --objective nor stdin supplies one. model — Primary model: provider and id are required; name is optional. The provider can be overridden with --provider, the model with --model. planning_agent — Optional. When enabled: false, planning uses the main model instead of a separate planning model. When set with a model, that model is used for the planning phase. There is no requirement that planning be enabled. tools — List of tool names (see below). Unknown names are ignored with a warning. max_steps — Maximum number of steps in the planned task. preferred_providers — Provider ids to prefer when find_model ranks results. The first entry becomes the default provider hint when the LLM omits one. Also surfaced in the system prompt. preferred_models — Map of capability (e.g. image, tts) to preferred model id(s). Injected as the model_hint for matching find_model calls. workspace — path (tilde ~ is expanded) and auto_create (default behavior creates the directory unless set to false). Defaults to the current working directory. Model Configuration model: provider: openai # Provider name id: gpt-4o # Model ID name: GPT-4o # Display name (optional) Supported providers include: openai — OpenAI models anthropic — Anthropic Claude models gemini — Google Gemini models (the aliases google and googleai are normalized to gemini) ollama — Local models via Ollama Other registry providers as configured in NodeTool Available Tools Tool names in the tools: list must match the agent’s tool registry. Common tools: File Operations: write_file — Write content to files read_file — Read file contents edit_file — Edit an existing file list_directory — List directory contents glob — Match files by glob pattern grep — Search within files Web Research: google_search — Search the web (also google_news, google_images) browser — Browse and extract web content download_file, http_request Code Execution: run_code — Run code in a sandbox Media Generation: generate_image, edit_image, animate_image generate_speech, transcribe_audio generate_video Other: find_model, calculator, statistics, geometry, conversion extract_pdf_text, convert_pdf_to_markdown, convert_document NodeTool MCP tools (workflows, nodes, jobs, assets, models) Example tool configuration: tools: - google_search - browser - write_file - read_file - run_code Workspace Configuration workspace: path: ~/agent-workspace # Workspace location (tilde is expanded) auto_create: true # Create if it doesn't exist (default behavior) If workspace.path is not specified (and --workspace is not passed), the workspace defaults to the current working directory. Example Configurations Research Agent See examples/agents/research-agent.yaml for a complete research agent configuration. nodetool agent run examples/agents/research-agent.yaml \ --objective "Research the impact of AI on software development" Code Assistant See examples/agents/code-assistant.yaml for a coding agent. nodetool agent run examples/agents/code-assistant.yaml \ --objective "Create a Python script to analyze log files" Content Creator See examples/agents/content-creator.yaml for a content generation agent. nodetool agent run examples/agents/content-creator.yaml \ --objective "Write a blog post about sustainable technology" Use Cases Automated Research nodetool agent run research-agent.yaml \ --objective "Research competitors in the AI workflow space" > research-report.md The agent will: Plan a research strategy Search for relevant information Browse competitor websites Extract and organize findings Generate a report Code Generation and Refactoring nodetool agent run code-assistant.yaml \ --objective "Refactor utils.py to improve performance" Chaining Agent Runs # Research phase nodetool agent run research-agent.yaml \ --objective "Research topic X" > research.md # Writing phase nodetool agent run content-creator.yaml \ --objective "Write an article based on: $(cat research.md)" Automation Scripts #!/bin/bash # Daily research automation for topic in "AI" "ML" "Web3"; do nodetool agent run research-agent.yaml \ --objective "Find latest news about $topic" \ > "reports/${topic}-$(date +%Y%m%d).md" done Environment Variables export OPENAI_API_KEY=sk-... export ANTHROPIC_API_KEY=sk-ant-... nodetool agent run agent.yaml --objective "Task" Output By default, the run streams a human-readable trace to stderr (planning updates, tool calls, step results) and writes the final result to stdout. Pass --json to emit each event as a JSON line on stderr; the final result is still written to stdout. # Capture the result, discard the trace nodetool agent run agent.yaml --objective "Task" 2>/dev/null > result.txt # Machine-readable event stream on stderr nodetool agent run agent.yaml --objective "Task" --json 2>events.jsonl Best Practices System Prompt Design system_prompt: | You are a [role]. Your responsibilities: 1. [Task 1] 2. [Task 2] Workflow: 1. [Step 1] 2. [Step 2] Model Selection Planning model: Use a fast, cost-effective model (gpt-4o-mini, a small Claude model) Main model: Use a stronger model for the actual reasoning Code tasks: Models with large context windows Tool Configuration Start minimal and add tools as needed: tools: - read_file - write_file - google_search # add when research is needed - browser # add when detailed web content is needed Troubleshooting Validate the config first nodetool agent test agent.yaml This reports missing model.provider / model.id, unknown tool names, and provider instantiation failures. Use a faster model for planning planning_agent: enabled: true model: provider: openai id: gpt-4o-mini Use a local model to avoid rate limits model: provider: ollama id: llama3.2:3b Related Documentation Chat & Agents — Agent system overview Chat CLI — Interactive chat interface NodeTool CLI — Complete CLI reference Agent Configuration Schema — YAML configuration reference Agent Configuration Examples — Sample configurations--- # Agent Configuration YAML Schema Source: https://docs.nodetool.ai/agent-config-schema.html Agent Configuration YAML Schema This document describes the complete YAML schema for NodeTool agent configurations used with the nodetool agent command. Schema Overview # REQUIRED FIELDS model: ModelConfig # Primary model: provider + id required # OPTIONAL FIELDS name: string # Agent identifier description: string # Human-readable description system_prompt: string # Agent behavior instructions objective: string # Default objective (if no --objective/stdin) planning_agent: PlanningConfig # Planning agent (enabled: false → use main model) tools: list[string] # Available tool names max_tokens: integer # Per-step context token budget (default 128000) max_steps: integer # Maximum number of steps in the task preferred_providers: list[string] # Provider ids to prefer for find_model preferred_models: map[string, string|list] # capability → preferred model id(s) workspace: WorkspaceConfig # Workspace configuration The runner accepts only the fields above. context_window, temperature, and max_iterations are not used — earlier versions of this doc listed them, but they are ignored. Use max_steps to bound the number of steps and max_tokens for the per-step context budget. provider, model, and objective can be supplied or overridden from the command line (--provider, --model, --objective); the objective also falls back to piped stdin. Field Definitions name (optional) Type: string Description: Identifier for the agent. Used in the trace header and by agent list / agent test. Examples: name: research-assistant name: code_helper name: content-creator-v2 description (optional) Type: string Description: Human-readable description of the agent’s purpose and capabilities. Examples: description: Autonomous research agent for web information gathering description: AI coding assistant for Python development description: Content creation and copywriting agent objective (optional) Type: string Description: Default objective for the agent. Used only when neither the --objective flag nor piped stdin supplies one. The priority order is: --objective flag → stdin → this objective: field. Example: objective: Research the latest developments in quantum computing system_prompt (optional) Type: string (supports multiline) Description: Core instructions defining agent behavior, workflow, and guidelines. This is the primary way to shape agent behavior. Best practices: Use YAML multiline format (|) Define the agent’s role clearly Include specific workflow steps Provide tool usage instructions Set output format expectations Include quality guidelines Example: system_prompt: | You are a professional research assistant. Your responsibilities: - Conduct thorough research on assigned topics - Verify information across multiple sources - Organize findings systematically - Provide well-structured summaries Workflow: 1. Break down research objective into specific queries 2. Use google_search to find relevant sources 3. Use browser to extract detailed content 4. Save findings using write_file 5. Synthesize results into report Guidelines: - Prioritize authoritative sources - Cross-reference information - Cite sources clearly - Note uncertainties or conflicts model (required) Type: ModelConfig object Description: Configuration for the primary AI model used by the agent. Structure: model: provider: string # REQUIRED: AI provider name id: string # REQUIRED: Model identifier name: string # OPTIONAL: Display name Supported providers: openai — OpenAI models anthropic — Anthropic Claude models gemini — Google Gemini models (the aliases google and googleai are normalized to gemini) ollama — Local Ollama models Other registry providers as configured Examples: # OpenAI model: provider: openai id: gpt-4o name: GPT-4o # Anthropic model: provider: anthropic id: claude-sonnet-4-6 name: Claude Sonnet # Gemini (google / googleai are aliases for gemini) model: provider: gemini id: gemini-3.5-flash name: Gemini 3.5 Flash # Local Ollama model: provider: ollama id: llama3.2:3b planning_agent (optional) Type: PlanningConfig object Description: Configures the model used for the planning phase. Optional — when omitted, planning uses the main model. When enabled: false, planning also falls back to the main model. There is no requirement that planning be enabled. Structure: planning_agent: enabled: boolean # false → plan with the main model model: ModelConfig # model used for the planning phase Best practices: Use fast, cost-effective models for planning The planning model can differ from the main model Recommended: gpt-4o-mini, a small Claude model, or a Gemini Flash model Examples: # Use GPT-4o Mini for cost-effective planning planning_agent: enabled: true model: provider: openai id: gpt-4o-mini # Disable the separate planning model; plan with the main model planning_agent: enabled: false tools (optional) Type: list[string] Description: List of tool names available to the agent. Tools extend agent capabilities beyond pure language model responses. Default: No tools (empty list). Available tools (the name in tools: must match the registry key): File Operations: write_file — Write content to files in the workspace read_file — Read file contents from the workspace edit_file — Edit an existing file list_directory — List directory contents glob — Match files by glob pattern grep — Search for patterns within files Web Research: google_search — Search the web (also google_news, google_images) browser — Browse URLs and extract web content download_file, http_request Code Execution: run_code — Run code in a sandbox Media Generation: generate_image, edit_image, animate_image generate_speech, transcribe_audio generate_video Other: find_model, calculator, statistics, geometry, conversion extract_pdf_text, convert_pdf_to_markdown, convert_document NodeTool MCP tools (workflows, nodes, jobs, assets, models) There is no delete_file or terminal tool. Unknown tool names are ignored at run time with a warning; nodetool agent test <file> reports them. Examples: # Minimal file operations tools: - read_file - write_file # Research agent tools tools: - google_search - browser - write_file - read_file # Code assistant tools tools: - read_file - write_file - edit_file - run_code - grep max_steps (optional) Type: integer Description: Maximum number of steps in the planned task. Examples: max_steps: 5 # Quick tasks max_steps: 10 # Standard max_steps: 20 # Complex tasks preferred_providers (optional) Type: list[string] Description: Provider ids to prefer when the find_model tool ranks results. The first entry becomes the default provider hint when the LLM omits one. These preferences are also surfaced in the system prompt. preferred_providers: - anthropic - openai preferred_models (optional) Type: map[string, string | list[string]] Description: Map of capability to preferred model id(s). When a find_model call matches a capability, the value is injected as the model_hint. preferred_models: image: black-forest-labs/flux-schnell tts: - openai/tts-1 - openai/tts-1-hd workspace (optional) Type: WorkspaceConfig object Description: Configuration for the agent’s file workspace. The workspace is a sandboxed directory where the agent can read and write files. Structure: workspace: path: string # OPTIONAL: Workspace directory path auto_create: bool # OPTIONAL: Create if doesn't exist Default: path: the current working directory (or --workspace if passed) auto_create: the directory is created unless auto_create: false Examples: # Use default workspace workspace: auto_create: true # Custom workspace path (tilde is expanded) workspace: path: ~/my-projects/agent-workspace auto_create: true # Absolute path, do not auto-create workspace: path: /tmp/research auto_create: false Complete Example # Research Assistant Agent name: research-assistant description: Research agent for information gathering system_prompt: | You are a professional research assistant specializing in thorough, accurate research. Responsibilities: - Conduct research on assigned topics - Gather information from multiple credible sources - Verify facts and cross-reference data - Organize findings in structured format - Provide citations and source references Workflow: 1. Analyze the research objective 2. Break down into specific research queries 3. Use google_search to find relevant sources 4. Use browser to extract detailed content from promising URLs 5. Save important findings using write_file 6. Synthesize all information into report 7. Review and verify accuracy Tools Available: - google_search: Find web resources - browser: Extract content from URLs - write_file: Save findings and reports - read_file: Review previous findings - list_directory: Check saved files Output Guidelines: - Structure reports with clear sections - Include executive summary - Cite all sources with URLs - Note any conflicting information - Highlight key findings and insights - Use markdown formatting model: provider: openai id: gpt-4o name: GPT-4o planning_agent: enabled: true model: provider: openai id: gpt-4o-mini name: GPT-4o Mini tools: - google_search - browser - write_file - read_file - list_directory max_tokens: 128000 max_steps: 15 workspace: path: ~/research-workspace auto_create: true Validating a Config There is no separate validation step or required-field enforcement at parse time — only model.provider and model.id are needed to run, and objective must come from the YAML, --objective, or stdin. Use the built-in test subcommand to check a config before running it: nodetool agent test research-assistant.yaml It reports a missing model.provider or model.id, lists the resolved tools, warns about unknown tool names, and tries to instantiate the provider. Unknown tool names do not abort a run — they are simply ignored with a warning. Path Expansion workspace.path supports leading-tilde (~) expansion to the home directory. There is no ${VAR} / ${VAR:-default} environment-variable interpolation inside the YAML — set provider/model via the YAML fields or the --provider / --model flags instead. Migration from Old Format If upgrading from older configuration formats: Old format: agent: name: my-agent model: gpt-4o tools: [search, browser] New format: name: my-agent model: provider: openai id: gpt-4o planning_agent: enabled: true model: provider: openai id: gpt-4o-mini tools: - google_search - browser See Also Agent CLI Documentation — Complete CLI reference Agent Examples — Sample configurations Chat & Agents — Agent system overview--- # Agent Memory System Source: https://docs.nodetool.ai/agent-memory Navigation: Chat & Agents → Agent Memory Long-Term Memory Not the same as long-term memory. This page describes per-run scratch space (context.memory) shared between steps inside one workflow. For durable, cross-session user memory injected into chat prompts, see Long-Term Memory. The agent memory system is the single source of truth for everything that flows between agents, tasks, steps, sub-agents, and tools during a workflow run. One AgentMemory instance lives on every ProcessingContext as context.memory. All executors read from and write to it through a single namespaced API, and every agent accesses it through three auto-attached tools: memory_list — discover what’s available (metadata only) memory_read — fetch full values for specific keys memory_write — publish a value under the shared: namespace This page is the full reference. For a quick orientation, jump to Quick Reference or Examples. Why This Exists Earlier versions of the agent system kept results in three uncoordinated stores (context._variables, ParallelTaskExecutor.taskResults, TaskBoard.task.result) and had each agent type deliver upstream results to the LLM differently. Plan mode replaced the default execution prompt with a “dependency context” block that stripped the finish_step discipline, so downstream tasks routinely lost results. The fix is one store, one API, and one access pattern — progressive disclosure via tool calls: Auto-injection is wasteful: dumping all upstream results into every prompt bloats context with data the step rarely needs. Tool-mediated access is selective: the agent sees a tiny “what’s available” hint in the system prompt and pulls only the values it actually needs. Specific declared dependencies still get a nudge: if the planner declared task.dependsOn or step.dependsOn, those exact memory keys appear in the user message as a hint — but the values are not included. Architecture ┌──────────────────────────────────┐ │ ProcessingContext.memory │ │ │ │ Map<string, MemoryEntry> │ │ │ │ step:<id> step_result │ │ task:<id> task_result │ │ input:<key> input │ │ shared:<k> shared │ └────────────┬─────────────────────┘ │ ┌─────────────────────────┴─────────────────────────┐ │ │ ▼ ▼ StepExecutor ParallelTaskExecutor (writes step:, task: on finish-task (passes task.dependsOn IDs as steps; auto-attaches memory_list / upstream key hints to memory_read / memory_write tools) TaskExecutor) Every executor writes results into context.memory. Every step has the three memory tools available automatically, and the system prompt instructs the model when to use them. Key Namespaces Namespace Helper Written By Used For step:<id> memoryKeys.step(id) StepExecutor, TaskExecutor (process mode) Per-step results task:<id> memoryKeys.task(id) StepExecutor (finish-task steps), ParallelTaskExecutor Per-task results input:<key> memoryKeys.input(key) TaskExecutor, ParallelTaskExecutor Caller-supplied inputs shared:<key> memoryKeys.shared(key) memory_write tool Cross-agent communication, scratch space Always use the helper functions when constructing keys — they prevent typos and make grep-able call sites. import { memoryKeys } from "@nodetool-ai/runtime"; context.memory.has(memoryKeys.task("research_phase")); context.memory.getValue(memoryKeys.step("step_1")); Memory Entry Shape export interface MemoryEntry { /** Globally unique key (use memoryKeys.*). */ key: string; /** Categorization for filtering and rendering. */ kind: "task_result" | "step_result" | "input" | "shared"; /** Stored value (any JSON-serializable structure). */ value: unknown; /** Optional ID of the producer (task / step / agent / tool). */ source?: string; /** Optional human-readable title shown in `memory_list`. */ title?: string; /** Optional brief description. */ description?: string; /** Wall-clock ms when the entry was first written. */ createdAt: number; } title and description flow through to memory_list output, so set them when you want the LLM to see a friendly label rather than a UUID. Memory Tools (the LLM-facing API) Three tools are auto-attached to every StepExecutor. Their schemas are documented at the top of every default execution system prompt so the model knows when to call them. memory_list Discover available entries without paying for their values. // args { "kind": ["task_result", "shared"], // optional filter "key_prefix": "task:", // optional filter "sources": ["research", "summary"] // optional filter } // result { "total": 4, "returned": 4, "truncated": false, "entries": [ { "key": "task:research", "kind": "task_result", "title": "Research findings", "description": "Top three sources from the web search step.", "source": "research", "valueBytes": 142, "createdAt": "2026-05-07T14:00:01.234Z" } // ... ] } valueBytes is the size of the JSON-serialized value — useful for the model to budget reads. The result is hard-capped at 200 entries; older entries are truncated and reported via truncated: true. memory_read Fetch full values for one or more keys. // args { "keys": ["task:research", "step:summary"] } // result { "entries": { "task:research": { "key": "task:research", "kind": "task_result", "value": { "findings": ["alpha", "beta"] }, "source": "research", "title": "Research findings", "createdAt": 1700000000000 } }, "missing": ["step:summary"] } Missing keys are reported in missing so the model can decide whether to retry, list again, or proceed without them. memory_write Publish a value under the shared: namespace so other agents and steps can discover it via memory_list. // args { "key": "top_source", // suffix; stored as "shared:top_source" "value": "https://example.com", "title": "Top source URL", "description": "Picked by the researcher agent." } // result { "ok": true, "key": "shared:top_source", "kind": "shared", "createdAt": "..." } Writes are restricted to the shared: namespace to prevent agents from spoofing step / task / input results. Direct API Reference The AgentMemory class lives in packages/runtime/src/agent-memory.ts and is re-exported from @nodetool-ai/runtime. Tools and executors use this API directly; agents reach it only through the three memory tools above. Writing context.memory.set({ key: memoryKeys.task("research"), kind: "task_result", value: { findings: ["alpha", "beta"] }, source: "research", title: "Research findings", description: "Top three sources from the web search step." }); set returns the persisted MemoryEntry. Repeated writes for the same key overwrite the value but preserve the original createdAt. Reading context.memory.get(memoryKeys.task("research")); // MemoryEntry | undefined context.memory.getValue<T>(memoryKeys.task("research")); // T | undefined context.memory.has(memoryKeys.task("research")); // boolean context.memory.snapshot(); // MemoryEntry[] Listing & Filtering context.memory.list(); // all context.memory.list({ kind: "task_result" }); context.memory.list({ kind: ["task_result", "input"] }); context.memory.list({ keys: ["task:research", "task:report"] }); context.memory.list({ keyPrefix: "step:" }); context.memory.list({ sources: ["research"] }); Subscriptions const unsubscribe = context.memory.subscribe((entry) => { console.log(`memory write: ${entry.key} (${entry.kind})`); }); // later... unsubscribe(); UIs can use this to render a live memory side panel. Clearing context.memory.clear(); // wipe everything context.memory.clear({ kind: "step_result" }); // selective context.memory.clear({ keyPrefix: "input:" }); // by prefix How Each Agent Type Uses Memory StepExecutor (packages/agents/src/step-executor.ts) The execution engine for a single step. Writes: Trigger Key Kind Always step:<step.id> step_result Last step of a task (finish-task) task:<task.id> task_result Step exhausted iterations step:<step.id> (with { error }) step_result LLM access: The default execution system prompts (DEFAULT_EXECUTION_SYSTEM_PROMPT, DEFAULT_FINISH_TASK_SYSTEM_PROMPT, DEFAULT_UNSTRUCTURED_SYSTEM_PROMPT) include a ## Memory Tools (progressive disclosure) section that explains how to use the tools. The user message includes only specific declared upstream keys as a hint: step:<id> for every entry of the step’s dependsOn (intra-task deps). any key supplied via StepExecutorOptions.upstreamMemoryKeys (typically task:<id> from the parent task’s dependsOn). Values are not included; the agent calls memory_read to fetch them. Tool attachment: getMemoryTools() is auto-pushed into the step’s tool list at construction time, alongside any caller-supplied tools and (when the step has a schema) finish_step. The conclusion stage strips everything except finish_step. Custom prompts are preambles, not replacements: A caller-supplied systemPrompt is layered before the default execution prompt, so the contract — including the memory-tool documentation and finish_step discipline — is non-bypassable. TaskExecutor (packages/agents/src/task-executor.ts) Walks the step DAG of a single task. On startup it seeds caller inputs: for (const [key, value] of Object.entries(this.inputs)) { this.context.memory.set({ key: memoryKeys.input(key), kind: "input", value, title: key }); } In process mode (fan-out over a discover step’s list), it reads the discover result via memoryKeys.step(discoverStepId) and writes the aggregated array back under memoryKeys.step(processStepId) after collecting per-item results. TaskExecutor accepts an optional upstreamMemoryKeys array (e.g. task:<id> keys from the parent plan). It forwards this verbatim to every StepExecutor it creates. ParallelTaskExecutor (packages/agents/src/parallel-task-executor.ts) Runs a TaskPlan of multiple tasks as a DAG. It owns no private result map — everything lives in context.memory. Operation Memory Action Startup: seed inputs set each as input:<key> For each task: derive upstream keys task.dependsOn.map(memoryKeys.task) → forwarded as upstreamMemoryKeys to TaskExecutor After task executor completes If no is_task_result was emitted, fall back to step:<lastStepId> Idempotent task write set task:<task.id> only if not already present Read final result getFinalResult() returns getValue(task:<lastTaskId>) Read all results getAllResults() lists all task_result entries Read specific task getTaskResult(id) Downstream tasks see their declared upstream task keys as hints in the step user message and pull values via memory_read when needed. Agent (packages/agents/src/agent.ts) The top-level entry point. Memory itself is execution-mode-independent: Agent delegates to the executors above, which all use context.memory. Propagation Flow This is the canonical end-to-end flow for a multi-task plan: 1. Caller: agent.execute(context) 2. ParallelTaskExecutor.execute() ├─ Seed inputs: context.memory.set({ kind: "input", ... }) └─ For each executable task: └─ TaskExecutor.executeTasks() └─ For each step: └─ StepExecutor.execute() ├─ buildSystemPrompt() → default execution prompt │ (includes "Memory Tools" section) ├─ buildUserMessage() → instructions │ + "Required upstream memory" hint listing │ declared dependency keys (no values) ├─ LLM streams → may emit: │ - memory_list → returns metadata │ - memory_read → returns values for chosen keys │ - memory_write → publishes shared facts │ - other tools / finish_step ├─ finish_step received → storeCompletionResult() │ ├─ context.memory.set({ key: "step:<id>", ... }) │ └─ if useFinishTask: │ context.memory.set({ key: "task:<id>", ... }) └─ yield step_result 3. ParallelTaskExecutor: ensure task: entry exists (idempotent) 4. Mark task.completed = true → unblocks downstream tasks 5. Next iteration: downstream tasks now executable. Their step user messages name the upstream task keys; agents call memory_read when they actually need the values. Examples Inspect memory at the end of an agent run import { Agent } from "@nodetool-ai/agents"; import { memoryKeys } from "@nodetool-ai/runtime"; const agent = new Agent({ /* ... */ }); for await (const _msg of agent.execute(context)) { /* drain */ } console.log("Final result:", agent.getResults()); console.log("All task results:", context.memory.list({ kind: "task_result" })); console.log( "Specific task:", context.memory.getValue(memoryKeys.task("research_phase")) ); What the LLM sees (system prompt extract) The default execution system prompt includes: ## Memory Tools (progressive disclosure) - Shared agent memory holds results from prior steps and tasks, original inputs, and facts published by other agents. - Memory contents are NOT auto-included in your prompt. If you need upstream context, discover it on demand: 1. Call `memory_list` to see what's available (returns metadata only — keys, titles, kinds, byte sizes). 2. Call `memory_read` with the specific keys you actually need; it returns full values. 3. Call `memory_write` to publish a value under `shared:<key>` so other agents can find it via `memory_list`. - Pull only what you need — don't fetch every entry by reflex. And the user message for a step that depends on task:research_phase looks like: Write a report from the upstream findings. # Required upstream memory (call `memory_read` with these keys): - task:research_phase — Research findings The model then chooses whether to call memory_read or proceed. Pre-populate memory before running Useful for tests or for restoring state from a checkpoint: context.memory.set({ key: memoryKeys.task("prior_research"), kind: "task_result", value: cachedFindings, source: "prior_research", title: "Cached prior research", description: "Findings from a previous run; reuse instead of re-researching." }); const agent = new Agent({ name: "follow-up", objective: "Build on the prior research findings.", /* ... */ }); // The agent's first memory_list call will surface this entry. Tool that publishes to shared memory directly (without an LLM round-trip) For deterministic publish-from-code use cases, write to the API directly: context.memory.set({ key: memoryKeys.shared("top_source"), kind: "shared", value: "https://example.com/article", source: "data_pipeline", title: "Top source URL" }); Subsequent agents will see this via memory_list and can fetch it via memory_read. Subscribe in a host application (UI sidebar) const unsubscribe = context.memory.subscribe((entry) => { ui.appendMemoryEntry(entry); // render the new entry in a side panel }); Quick Reference import { AgentMemory, memoryKeys, type MemoryEntry } from "@nodetool-ai/runtime"; // Already mounted on every ProcessingContext context.memory; // AgentMemory // Write context.memory.set({ key: memoryKeys.task("t1"), kind: "task_result", value: result, source: "t1", title: "Task One" }); // Read context.memory.get(memoryKeys.task("t1")); // MemoryEntry | undefined context.memory.getValue(memoryKeys.task("t1")); // unknown | undefined context.memory.has(memoryKeys.task("t1")); // boolean // List / filter context.memory.list({ kind: "task_result" }); context.memory.list({ keyPrefix: "step:" }); context.memory.list({ sources: ["research"] }); // Subscribe const off = context.memory.subscribe((entry) => { /* ... */ }); // Clear context.memory.clear({ kind: "step_result" }); LLM-facing tools (auto-attached to every step): Tool Purpose Returns memory_list Discover entries (metadata only) { total, returned, entries: [...] } memory_read Fetch full values for specific keys { entries: { ... }, missing: [...] } memory_write Publish under shared:<key> { ok, key, kind, createdAt } Design Decisions Why progressive disclosure and not auto-injection? Auto-injecting every memory entry into every prompt is wasteful. Most steps need one or two specific upstream values; dumping all of them costs tokens and pollutes attention. Progressive disclosure mirrors how a human researcher works: scan the index, fetch the specific document, ignore the rest. Why expose memory through tools instead of a dedicated channel? Models in 2026 are excellent tool callers — and tools come with rich JSON schemas that make discovery, filtering, and parameter validation trivial. Adding bespoke prompt syntax for memory access would be reinventing function calling. Tools also give us first-class observability: every memory access shows up as a tool_call_update in the message stream. Why is memory_write restricted to shared:? Step results, task results, and inputs are owned by the executors. Letting an agent overwrite a task:<id> entry would let it spoof the result of work it didn’t actually do, breaking the audit trail. Agents publish under shared: and the executor namespaces stay tamper-proof. Why does the user message still mention specific upstream keys? A pure tool-only design would force the agent to call memory_list even when the planner already declared the dependency. That’s an unnecessary round trip for a known-relevant key. The user message names exactly the keys the planner pinned (step.dependsOn plus parent-task dependsOn), so the agent can go straight to memory_read for declared deps and use memory_list only when it needs to discover beyond them. Why a single map and not a relational store? Agent runs are short-lived and the data is small (typically tens of entries). A Map plus structured rendering covers every observed use case without the operational cost of a database. Persistent storage belongs in the asset / vector layer. Why namespaced string keys instead of typed IDs? The LLM has to read keys back from the tool result and pass them to memory_read. Strings round-trip cleanly through prompts and logs without serialization games. Why isn’t memory shared across context.copy()? Copies are designed for isolated sub-runs. If a sub-run should inherit memory, the caller can set entries from the parent before kicking it off. Default isolation is the safer choice. Why does customPrompt no longer replace the default execution prompt? Replacing it stripped the memory-tool documentation and the finish_step discipline. The fix layers any caller preamble before the default prompt rather than replacing it. The execution contract (memory tools, output schema, completion protocol, conclusion-stage rules) is now non-bypassable. Troubleshooting Symptom: A downstream task’s prompt shows the upstream key hint but the agent never calls memory_read. The model may have decided it doesn’t need the value. If you know it should, make the user instructions more explicit (“read the upstream findings via memory_read before writing”). Check the conclusion stage: if the step is at >90% token budget, only finish_step is allowed and memory_read is filtered out. Symptom: Task result key is missing after the step yielded step_result with is_task_result: true. StepExecutor only writes task:<id> for steps where useFinishTask === true. That flag is set by TaskExecutor.isFinishStep() for the last step in the task (or the explicit finalStepId option). Steps in the middle of a task only write step:<id>. For belt-and-suspenders, ParallelTaskExecutor performs an idempotent task-result write after each task, falling back to the last step’s value. Symptom: Tests pass but memory entries seem stale across test runs. A new AgentMemory instance is constructed for every ProcessingContext. If you reuse a context across tests, call context.memory.clear() between them. Symptom: Agent calls memory_list and gets truncated: true. The list response is hard-capped at 200 entries. If you need more, narrow the filter (kind, key_prefix, or sources) — or accept that for very long runs, only the most recent 200 are visible at once. Related Chat & Agents — agents overview packages/runtime/src/agent-memory.ts — AgentMemory implementation packages/agents/src/tools/memory-tools.ts — memory_list / memory_read / memory_write packages/agents/tests/memory-propagation.test.ts — end-to-end propagation tests including the tool round-trip packages/agents/tests/memory-tools.test.ts — unit tests for the memory tools packages/runtime/tests/agent-memory.test.ts — AgentMemory unit tests--- # Agents and Automation Source: https://docs.nodetool.ai/agents/ NodeTool is an open-source visual workflow runtime for AI models, tools, agents, media generation, and data processing. This guide is the entry point for software agents and automated integrations. Start here Install NodeTool and start a server with npm run dev:server. Find node schemas or fetch the node catalog. Create and validate a workflow. Run and inspect a workflow with the CLI or API. Machine-readable resources Documentation index Complete documentation Node catalog Workflow API WebSocket API Verify a workflow Run the cheap structural check before execution: npm run dev:nodetool -- validate workflow.json After a valid result, execute it and inspect its output: npm run dev:nodetool -- debug workflow.json The command prints the workflow status, output values, errors, and job logs. A successful run has a completed status and no execution errors.--- # api.md Source: https://docs.nodetool.ai/api This page has moved. Please go to API Reference.--- # API Reference Source: https://docs.nodetool.ai/api-reference.html Server Architecture NodeTool runs a single Fastify HTTP + WebSocket server (@nodetool-ai/websocket — packages/websocket/src/server.ts). The same process serves: REST routes under /api/* (workflows, jobs, assets, models, settings, storage). OpenAI-compatible /v1/chat/completions and /v1/models. WebSocket endpoints for workflow execution, chat, the browser extension, downloads, and the agent runtime. Health and liveness probes. Start it with nodetool serve (default 127.0.0.1:7777). serve accepts only --host and --port — there is no --mode flag. Endpoint Matrix For detailed schemas, see Chat API and Workflow API. Area Path Method / Protocol Auth Streaming Notes Models /v1/models GET Bearer when AUTH_PROVIDER enforces no OpenAI-compatible model listing Chat /v1/chat/completions POST Bearer when AUTH_PROVIDER enforces SSE when "stream": true OpenAI-compatible chat; SSE or single JSON Workflows /api/workflows GET Depends on AUTH_PROVIDER no List workflows Workflows /api/workflows/{id}/run POST Depends on AUTH_PROVIDER no Run a workflow once, return final outputs as one JSON response Workflow WS /ws WebSocket Bearer header or api_key query when enforced yes Workflow execution, chat, job control, live updates (MessagePack or JSON) Agent WS /ws/agent WebSocket Bearer header or api_key query when enforced yes Agent runtime Extension WS /ws/extension WebSocket Follows global auth settings yes Browser extension channel Download WS /ws/download WebSocket Follows global auth settings yes Model/file downloads Storage /api/storage/* HEAD/GET/PUT/DELETE Depends on AUTH_PROVIDER streaming for GET Asset/temp storage Health /health GET none no JSON: {status, timestamp, uptime, services} (200/503) Health /api/health GET none no JSON: {version, uptime} Liveness /ready GET none no Always 200 with {status:"ok"} When AUTH_PROVIDER is local or none, endpoints accept requests without a token for convenience. When it is static or supabase, include Authorization: Bearer <token> on every request except the health/liveness routes. Authentication and Headers NodeTool uses Bearer token authentication. The behavior depends on your AUTH_PROVIDER setting: AUTH_PROVIDER Token Required? Use Case local / none No Local development, desktop app static Yes — use the configured static token Simple deployments with a shared secret supabase Yes — use a Supabase JWT Production deployments with user management How to include credentials HTTP requests: Authorization: Bearer <token> header on all non-public routes WebSocket: Authorization: Bearer <token> header (preferred) or api_key query parameter SSE streams (/v1/chat/completions): Authorization: Bearer <token> and Accept: text/event-stream Local development: When running locally with the default config (AUTH_PROVIDER=local), no token is needed. You can omit the Authorization header entirely. See Authentication for full token handling rules. Streaming Behavior /v1/chat/completions uses OpenAI-style SSE when stream is true; otherwise it returns a single JSON response. POST /api/workflows/{id}/run does not stream — it runs the workflow to completion and returns one JSON response. The /ws WebSocket streams workflow/job events (job_update, node_update, node_progress, output_update, chunk, …) and chat tokens/tool calls. See the WebSocket API for the full protocol. Storage routes stream file contents for large assets. Headless Mode: Running Workflows via CLI/API NodeTool can run entirely without the UI—perfect for automation, CI/CD pipelines, and programmatic integrations. This section shows how to execute workflows from the command line or via HTTP requests. Quick Start: Run a Workflow via cURL # Run a workflow and get results (non-streaming) curl -X POST "http://localhost:7777/api/workflows/YOUR_WORKFLOW_ID/run" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer YOUR_TOKEN" \ -d '{ "params": { "prompt": "A cyberpunk cityscape at sunset", "style": "photorealistic" } }' Response: { "job_id": "job_abc123", "workflow_id": "YOUR_WORKFLOW_ID", "status": "completed", "outputs": { "image": { "type": "image", "uri": "http://localhost:7777/api/storage/assets/abc123.png" }, "caption": "Generated image of a cyberpunk cityscape..." }, "error": null, "message_count": 12, "background": false } outputs is an object keyed by output-node name. The route does not stream — for real-time progress, run the workflow over the WebSocket API. Chat API (OpenAI-Compatible) NodeTool exposes OpenAI-compatible endpoints, so you can use standard OpenAI clients: # Simple chat completion curl -X POST "http://localhost:7777/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer YOUR_TOKEN" \ -d '{ "model": "gpt-4", "messages": [ {"role": "user", "content": "Explain quantum computing in simple terms"} ] }' Response: { "id": "chatcmpl-abc123", "object": "chat.completion", "created": 1699000000, "model": "gpt-4", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "Quantum computing uses quantum mechanics..." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 10, "completion_tokens": 150, "total_tokens": 160 } } Streaming Chat # Streaming chat (prints tokens as they arrive) curl -X POST "http://localhost:7777/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer YOUR_TOKEN" \ -d '{ "model": "gpt-4", "messages": [ {"role": "user", "content": "Write a haiku about programming"} ], "stream": true }' Streaming response: data: {"id":"chatcmpl-123","choices":[{"delta":{"role":"assistant"},"index":0}]} data: {"id":"chatcmpl-123","choices":[{"delta":{"content":"Code"},"index":0}]} data: {"id":"chatcmpl-123","choices":[{"delta":{"content":" flows"},"index":0}]} data: {"id":"chatcmpl-123","choices":[{"delta":{"content":" like"},"index":0}]} data: [DONE] List Available Models curl "http://localhost:7777/v1/models" \ -H "Authorization: Bearer YOUR_TOKEN" Response: { "object": "list", "data": [ {"id": "gpt-4", "object": "model", "owned_by": "openai"}, {"id": "gpt-3.5-turbo", "object": "model", "owned_by": "openai"}, {"id": "claude-3-opus", "object": "model", "owned_by": "anthropic"}, {"id": "gpt-oss:20b", "object": "model", "owned_by": "ollama"} ] } List Workflows # List all workflows curl "http://localhost:7777/api/workflows" \ -H "Authorization: Bearer YOUR_TOKEN" Health Check # Check if server is running (no auth required) curl "http://localhost:7777/health" Response: { "status": "ok", "timestamp": "2026-06-20T00:00:00.000Z", "uptime": 123, "services": { "database": "ok", "server": "ok" } } CLI Workflow Execution You can also run workflows from the command line. nodetool run executes a TypeScript/JavaScript DSL workflow file: # Run a DSL workflow file nodetool run ./my_workflow.ts # Output results as JSON nodetool run ./my_workflow.ts --json To run a saved workflow by ID (requires a running server), use the workflows subcommands: # List workflows nodetool workflows list # Run a workflow by ID nodetool workflows run workflow_abc123 --params '{"prompt": "test"}' TypeScript / Node.js Client Example const BASE_URL = 'http://localhost:7777'; const TOKEN = 'your_token_here'; // Run a workflow (runs to completion, returns one JSON response) async function runWorkflow(workflowId, params) { const response = await fetch(`${BASE_URL}/api/workflows/${workflowId}/run`, { method: 'POST', headers: { 'Authorization': `Bearer ${TOKEN}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ params }) }); const body = await response.json(); // body: { job_id, workflow_id, status, outputs, error, message_count, background } return body.outputs; } // For real-time progress, run the workflow over the WebSocket endpoint instead. // See the WebSocket API page and examples/workflow_runner/js/workflow-runner.js. // Using OpenAI SDK (works with NodeTool!) import OpenAI from 'openai'; const openai = new OpenAI({ apiKey: TOKEN, baseURL: `${BASE_URL}/v1` }); const completion = await openai.chat.completions.create({ model: 'gpt-4', messages: [{ role: 'user', content: 'Hello!' }] }); console.log(completion.choices[0].message.content); Python Client Example import requests BASE_URL = "http://localhost:7777" TOKEN = "your_token_here" # Not needed for local development HEADERS = { "Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json", } # List workflows workflows = requests.get(f"{BASE_URL}/api/workflows", headers=HEADERS).json() # Run a workflow (runs to completion, returns one JSON response) result = requests.post( f"{BASE_URL}/api/workflows/{workflows[0]['id']}/run", headers=HEADERS, json={"params": {"prompt": "A sunset over mountains"}}, ).json() # result: {"job_id", "workflow_id", "status", "outputs", "error", "message_count", "background"} print("Outputs:", result["outputs"]) # For real-time progress, run the workflow over the WebSocket endpoint instead. # Use with OpenAI Python SDK (works with NodeTool!) from openai import OpenAI client = OpenAI(api_key=TOKEN, base_url=f"{BASE_URL}/v1") completion = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": "Hello!"}], ) print(completion.choices[0].message.content) Finding Your Workflow ID To run a workflow via API, you need its ID. Here’s how to find it: From the UI: Open a workflow in the editor — the ID appears in the browser URL bar From the API: Call GET /api/workflows to list all workflows with their IDs From the CLI: Run nodetool workflows list Error Handling API errors return standard HTTP status codes with JSON error bodies: { "error": { "message": "Workflow not found: invalid_id", "type": "not_found", "code": 404 } } Status Code Meaning Common Causes 400 Bad Request Invalid parameters, malformed JSON 401 Unauthorized Missing or invalid token 403 Forbidden Token lacks permission 404 Not Found Workflow/resource doesn’t exist 422 Validation Error Parameter validation failed 500 Internal Error Server-side error 503 Service Unavailable Server overloaded or starting up Related Guides Chat API — OpenAI-compatible request/response schema and WebSocket usage. Workflow API — Workflow REST paths and execution. API Server Overview — Server architecture and modules. Deployment Guide — How servers are built and exposed. CLI Reference — Commands including serve.--- # API Server Overview Source: https://docs.nodetool.ai/api-server.html NodeTool exposes a single TypeScript HTTP + WebSocket server runtime built on Node.js. The same process serves REST API routes, WebSocket workflow execution endpoints, and OpenAI-compatible /v1 routes. The server is implemented in the @nodetool-ai/websocket package (packages/websocket/src/server.ts). Key Modules server.ts – HTTP/WebSocket server entry point. Registers all API routes and starts listeners. unified-websocket-runner.ts – Handles WebSocket connections for workflow execution and chat. http-api.ts – Core REST API routes (workflows, jobs, assets, etc.). models-api.ts – Model management and provider registration. settings-registry.ts – Settings and configuration handling. storage-api.ts – File upload and asset storage endpoints. mcp-server.ts – Model Context Protocol (MCP) server integration. openai-api.ts – OpenAI-compatible /v1/chat/completions and /v1/models endpoints. Running the Server # Install the CLI globally (once) npm install -g @nodetool-ai/cli # Start the server nodetool serve --host 127.0.0.1 --port 7777 # Or run without installing globally npx --package=@nodetool-ai/cli nodetool serve --host 0.0.0.0 --port 7777 Development (from repo root): npm run build:packages npm run dev:server # tsx --watch packages/websocket/src/server.ts Configuration The server is configured via environment variables: Variable Default Description PORT 7777 HTTP listen port HOST 127.0.0.1 Bind address DB_PATH ~/.local/share/nodetool/nodetool.sqlite3 SQLite database path. Do not set together with DATABASE_URL. DATABASE_URL — PostgreSQL URL (postgres:// / postgresql://) or SQLite URL/path (file: / sqlite:) ANTHROPIC_API_KEY — Anthropic API key OPENAI_API_KEY — OpenAI API key GEMINI_API_KEY — Google Gemini API key OLLAMA_API_URL http://localhost:11434 Ollama server URL Health Check GET /health Returns 200 OK when healthy (503 when degraded) with a JSON body: { "status": "ok", "timestamp": "2026-06-20T00:00:00.000Z", "uptime": 123, "services": { "database": "ok", "server": "ok" } } GET /ready is a simple liveness probe (always 200 with { "status": "ok" }), and GET /api/health returns { version, uptime }. None require authentication. For deployment setup, see Deployment Guide and Authentication.--- # App Builder Source: https://docs.nodetool.ai/app-builder.html App Builder turns a workflow into a custom Mini App. You place widgets on a canvas, bind them to workflow inputs and outputs, and publish the app document onto the workflow. What it does Builds a structured app document with inputs, actions, display widgets, and layout blocks. Binds widgets to existing Input and Output node names. Runs the workflow from buttons or change events. Streams workflow outputs into bound display widgets. Saves the app document with the workflow and serves it in Mini App mode. Where this fits A Mini App is how a workflow reaches people who should not have to read a node graph. App Builder wraps the graph in a form: input widgets bind to Input nodes, buttons run the workflow, and display widgets stream Output nodes back. It is the share end of NodeTool’s loop — the same assets a workflow produces on the canvas, exposed as a usable app. See Key Concepts → How everything fits together for the full loop. Open App Builder Open a workflow. Switch the tab to App mode. Click App Builder in the tab bar. The builder opens at /app-builder/:workflowId. Build an app Add Input nodes and Output nodes to the workflow first. Their name fields are the binding keys. Add input widgets such as Text Input, Number Input, Slider, Switch, or Select. Set each input widget’s binding to the matching Input node name. Add a Button with the Run workflow action. Add display widgets such as Text, Markdown, Image, JSON, or Progress. Set each display widget’s binding to the matching Output node name. Click Publish. Agent-assisted editing Click Ask Agent in App Builder to open the builder agent. It can inspect the workflow, add widgets, set bindings, and update the workflow graph when an app needs new Input, Output, or Variable nodes. Good prompts name the result you want: Build a compact app for this workflow with all inputs on the left, a run button below them, and outputs on the right. Bindings Bindings must match workflow state exactly: Widget kind Bind to Input widgets Input node name Display widgets Output node name State controls Variable node name If a binding does not match a node name, the widget has no data to read or write. Mini App mode Mini App mode renders the App Builder document when one exists. If a workflow has no app document, NodeTool renders the generated input/output form. Related topics Workflow Editor Mini Apps Chat & Agents Key Concepts--- # app-views-noext.md Source: https://docs.nodetool.ai/app-views This page has moved. Please go to Desktop App Views.--- # app-views-html.md Source: https://docs.nodetool.ai/app-views.html This page has moved. Please go to Desktop App Views.--- # Architecture & Lifecycle Source: https://docs.nodetool.ai/architecture.html Two core principles: Streaming-first execution — results stream as they generate; long-running jobs can be cancelled. Unified runtime — the same workflow JSON runs in the desktop app, headless server, RunPod endpoint, or Cloud Run. System Components NodeTool is organized into distinct packages, each responsible for a specific layer of the system: Core Packages Package Purpose @nodetool-ai/kernel DAG execution engine – graph validation, node actors, inbox routing, edge counting @nodetool-ai/runtime Processing context, cache adapters, storage adapters, asset handling @nodetool-ai/protocol Shared message types (JobUpdate, NodeUpdate, EdgeUpdate, TaskUpdate) @nodetool-ai/agents Agent executor, task planner, step executor, multi-mode agent, 20+ tool types @nodetool-ai/dsl TypeScript DSL for building workflows programmatically with type-safe factories @nodetool-ai/config Settings management and environment configuration Infrastructure Packages Package Purpose @nodetool-ai/deploy Deployment automation for self-hosted, RunPod, and GCP Cloud Run @nodetool-ai/storage Asset storage backends (local filesystem, S3, Supabase) @nodetool-ai/vectorstore Vector database integration (SQLite-vec, Pinecone, Supabase pgvector) for RAG workflows @nodetool-ai/cli Command-line interface for workflow execution, deployment, and package management @nodetool-ai/base-nodes Core node implementations and dynamic node generation Frontend Package Purpose web React application – workflow editor, asset explorer, model manager, global chat Execution Engine WorkflowRunner The WorkflowRunner is the DAG orchestrator that executes workflow graphs. It handles: Graph validation – Ensures all connections are valid and the graph is acyclic Node actor spawning – Creates a NodeActor for each node in the graph Input dispatch – Routes initial data to the correct input nodes Edge counting and EOS propagation – Tracks when nodes have received all inputs and propagates End-Of-Stream signals through the graph Concurrent execution – Runs independent nodes in parallel when their inputs are ready NodeActor Each node in a workflow runs as a NodeActor with one of four execution modes: Mode Behavior When to Use Buffered Collects all inputs before processing Default for most nodes. Use when you need all data before you can produce output (e.g., image resize, text formatting). Streaming input Processes inputs as they arrive, one at a time Use for nodes that handle items in a stream (e.g., filtering, transforming individual items). Streaming output Produces outputs incrementally as they become available Use for LLMs and generators that emit tokens/chunks over time (e.g., Agent, ListGenerator). Controlled Manages its own execution lifecycle with cached input replay Use for nodes that need custom control over when and how they process (e.g., loops, conditional retry). Correlation-Aware Scheduling There is no sync_mode setting (the old zip_all/on_any/sticky modes were removed). Instead, the scheduler is correlation-aware: every value carries a correlation lineage describing which iteration/branch it came from, and the buffered actor path (_runCorrelated in NodeActor) fires a node once per matched set of correlated inputs. Inputs that share a correlation token are matched and processed together — this is what the old zip_all mode approximated, but it is now driven by the actual lineage of each value rather than a static flag. Outputs declare how they relate to their inputs via outputCorrelation (forward, iteration, aggregate, single), which tells the scheduler whether an output is a passthrough, a new per-item iteration, a collapse of a stream, or a one-shot value. Join nodes like Zip and Cross pair values from independent iteration sources within their common parent scope. See correlation-design.md for the full model. ProcessingContext The ProcessingContext provides the runtime environment for node execution: Message queue – Collects ProcessingMessage events for streaming to clients Cache interface – Pluggable cache adapters (memory, disk) for intermediate results Asset storage – StorageAdapter interface supporting local filesystem, S3, or Supabase Asset output modes – data_uri, temp_url, storage_url, workspace, raw User context – Authentication tokens, user data, workspace information Job Lifecycle (run, stream, reconnect, cancel) Workflow execution uses the actor model: WorkflowRunner validates the graph, spawns one NodeActor per node, and routes values between actors over inboxes. Each actor runs in one of the four modes described above (Buffered, Streaming input, Streaming output, Controlled). There is no separate JobExecutionManager class or pluggable threaded/subprocess/docker “execution strategy” — actors run in-process and stream their results out. sequenceDiagram participant Client participant API as API Server participant Runner as WorkflowRunner participant Actor as NodeActor (per node) participant Msg as Messaging/WS Client->>API: POST /api/workflows/{id}/run (stream=true) API->>Runner: Validate graph + spawn actors Runner->>Actor: Dispatch inputs, run per execution mode Actor->>Msg: Emit streaming events (node/edge updates) Msg-->>Client: token/output events Client-->>API: reconnect with thread/job id API-->>Msg: resume stream Client->>API: DELETE /api/workflows/{id}/run (cancel) API->>Runner: cancel run Runner-->>Actor: teardown and cleanup Runner-->>Msg: end event Msg-->>Client: completion / cancelled status Message Types The protocol layer defines several message types for tracking execution state: Message Purpose JobUpdate Overall job status (queued, running, completed, failed, cancelled) NodeUpdate Per-node progress (started, output produced, completed, errored) EdgeUpdate Data flowing through connections between nodes TaskUpdate Agent task lifecycle (created, step started/completed/failed, task completed) Agent System NodeTool includes a full agent execution framework for autonomous task completion: Components Agent – Top-level entry point. Takes an objective (+ optional pre-built task) and orchestrates planning + execution. TaskPlanner – Breaks complex goals into ordered subtasks with dependencies TaskExecutor – Manages the execution of a complete task plan StepExecutor – Runs individual steps within a task, including tool calls CompilerAgent – Final synthesis pass that reads accumulated memory and produces the deliverable Available Tools (20+) Agents can use a wide range of tools during execution: Category Tools Web Browser, HTTP requests, web search, Google APIs Files Filesystem operations, workspace management, asset tools Code JavaScript sandbox execution, code analysis Data Calculator, math operations, vector DB queries Documents PDF processing, email integration AI MCP (Model Context Protocol) tools for external service integration Providers NodeTool supports 20+ AI model providers through a unified provider interface: Provider Types OpenAI Text, image, audio, embeddings Anthropic Text (Claude models) Google Gemini Text, image, video, audio Ollama Local LLMs LM Studio Local LLMs Hugging Face All model types Replicate Image, video, audio FAL Image generation Groq Fast text inference Mistral Text generation Together Text, embeddings Cerebras Fast text inference GMI Cloud Open-weight text inference OpenRouter Multi-provider routing vLLM Self-hosted inference Each provider implements a base interface that handles authentication, model listing, and inference calls. A built-in cost calculator tracks token usage across providers. Storage Architecture NodeTool uses a pluggable storage system with three backends: Backend Use Case Pros Cons Local filesystem Desktop app, development Zero config, fast, private Single machine only S3-compatible Production (AWS, MinIO) Scalable, durable, multi-region Requires cloud account, network latency Supabase Storage Supabase deployments Integrated auth + storage, managed Requires Supabase project The storage adapter is selected automatically based on environment configuration. Assets are stored in two buckets: assets (permanent) and assets-temp (intermediate results, auto-cleaned). See Storage for configuration details. Python Worker Bridge Python nodes and Python-only local providers run in a separate worker process. The TS backend spawns the worker with python -m nodetool.worker --stdio and communicates over a local stdio protocol: binary-safe MessagePack payloads 4-byte big-endian length framing in-band discovery, execution, status, progress, chunk, and error messages structured load_errors so import failures are visible without parsing logs See Python Bridge Protocol for the full wire protocol and lifecycle. Notes All endpoints and examples use http://127.0.0.1:7777 by default; update host/port when deploying. Messaging emits both JSON and optional MessagePack; see Chat Server for protocol details. Execution strategies are detailed in Execution Strategies. Related Key Concepts – High-level overview of workflows, nodes, and models API Reference – REST and WebSocket API documentation Python Bridge Protocol – TS ↔ Python worker transport and message schemas Developer Guide – Building custom nodes and extensions Deployment Guide – Running NodeTool in production--- # Asset Management Source: https://docs.nodetool.ai/asset-management.html NodeTool lets you store and organize files used in your workflows. Assets can be images, audio clips, videos, PDFs, 3D models, or any other resources referenced by nodes. Where this fits Assets are the shared material between NodeTool’s surfaces. Workflows read assets and write new ones; sketches open an asset and render an edited one back; timelines import assets as clips and export a finished video asset. The Asset Explorer is the one store all of them read from and write to, so a file produced in any surface is immediately usable in the others. See Key Concepts → How everything fits together for the full loop. Asset Explorer The Asset Explorer is the central hub for managing your files. Open it from the left sidebar. Views Grid view – Thumbnails with previews, best for visual assets like images and videos List view – Compact rows with metadata columns, best for large libraries Switch between views using the toggle in the toolbar. Navigation Folder tree – Browse your directory hierarchy in the left panel Search – Full-text search across file names, tags, and metadata Infinite scrolling – Large libraries load progressively for performance Uploading Files Drag and drop files directly into the Asset Explorer Use the Upload button in the toolbar Drop files onto a workflow node to create an input node automatically Working with Assets in Workflows Drag any asset from the Asset Explorer directly into the workflow canvas. NodeTool automatically creates the appropriate input node based on the file type (image input, audio input, etc.). Asset Viewers NodeTool includes specialized viewers for common file types: File Type Viewer Features Images Zoom, pan, pixel inspection Audio Waveform display, playback controls, scrubbing Video Frame-by-frame playback, timeline scrubbing PDF Page navigation, text selection, zoom Text Syntax highlighting, line numbers, copy excerpts 3D Models Interactive 3D preview with rotation and zoom Open any asset in its viewer by double-clicking it in the Asset Explorer. Use the toolbar within the viewer to zoom, scrub, or copy content. Collections Collections group related assets for use in RAG (Retrieval-Augmented Generation) workflows and organized project management. Creating Collections Open the Collections panel from the left sidebar Click New Collection and give it a name Drag assets from the Asset Explorer into the collection Using Collections in Workflows Collections integrate with document indexing and vector search nodes. When you connect a collection to an indexing node, all assets in the collection are processed and made searchable. Metadata and Tags Assets automatically track metadata: file size, type, dimensions (for images/video), duration (for audio/video) Tag files to make searching easier as projects grow Tags are searchable from the Asset Explorer search bar Document Indexing Text-based assets (PDFs, text files, documents) can be indexed for vector search: Add documents to a Collection Connect the collection to an Index node in your workflow Use Vector Search nodes to query the indexed documents Combine with language models for Retrieval-Augmented Generation (RAG) See Indexing for detailed setup instructions. Storage Assets are stored locally under the NodeTool data directory (~/.local/share/nodetool/assets on macOS/Linux; %APPDATA%\nodetool\assets on Windows) by default. You keep full control of your data. For deployed instances, assets can be stored in: S3-compatible storage – AWS S3, MinIO, or compatible services Supabase Storage – Integrated with Supabase auth See Storage for backend configuration. Next Steps Indexing – Set up document indexing for RAG workflows Storage – Configure storage backends Workflow Editor – Use assets in your workflows Example: Chat with Docs – Build a RAG workflow with your documents--- # main.scss Source: https://docs.nodetool.ai/assets/css/main.css /* ============================================ NODETOOL FUTURISTIC DARK THEME A cyberpunk-inspired documentation theme ============================================ */ /* === CSS Variables === / :root { / Colors - Matching nodetool-web paletteDark.ts / –color-bg-primary: #08090A; / background.default / –color-bg-secondary: #101113; / background.paper / c_tabs_header / –color-bg-tertiary: #1B1D21; / grey 800 / c_node_bg / –color-bg-elevated: #17181B; / Paper.overlay / c_node_menu */ –color-text-primary: #F7F8F8; –color-text-secondary: #D4D6DB; –color-text-muted: #8A8F98; –color-text-link: #93C5FD; /* Gradients - using nodetool primary/secondary palette */ –gradient-primary: linear-gradient(137deg, #6690d4 0%, #E879F9); –gradient-secondary: linear-gradient(137deg, #22D3EE 0%, #50FA7B); –gradient-accent: linear-gradient(137deg, #FFB86C 0%, #E879F9); –gradient-header: linear-gradient(137deg, #6690d4 0%, #E879F9); /* Accents from nodetool palette */ –color-accent-cyan: #22D3EE; –color-accent-green: #50FA7B; –color-accent-magenta: #E879F9; –color-accent-purple: #C4B5FD; –color-accent-blue: #6690d4; –color-border: rgba(255, 255, 255, 0.08); –color-border-glow: rgba(102, 144, 212, 0.2); /* Neon Glows */ –glow-cyan: 0 0 10px rgba(34, 211, 238, 0.3), 0 0 20px rgba(34, 211, 238, 0.15), 0 0 30px rgba(34, 211, 238, 0.05); –glow-magenta: 0 0 10px rgba(232, 121, 249, 0.3), 0 0 20px rgba(232, 121, 249, 0.15), 0 0 30px rgba(232, 121, 249, 0.05); –glow-purple: 0 0 10px rgba(196, 181, 253, 0.3), 0 0 20px rgba(196, 181, 253, 0.15); /* Typography */ –font-heading: “Inter”, system-ui, -apple-system, BlinkMacSystemFont, “Segoe UI”, sans-serif; –font-body: “Inter”, system-ui, -apple-system, BlinkMacSystemFont, “Segoe UI”, sans-serif; –font-mono: “JetBrains Mono”, “Courier New”, monospace; /* Spacing */ –spacing-xs: 0.25rem; –spacing-sm: 0.5rem; –spacing-md: 1rem; –spacing-lg: 1.5rem; –spacing-xl: 2rem; –spacing-2xl: 3rem; –spacing-3xl: 4rem; /* Layout */ –max-width: 1400px; –sidebar-width: 280px; –header-height: 70px; /* Transitions */ –transition-fast: 150ms cubic-bezier(0.4, 0, 0.2, 1); –transition-base: 250ms cubic-bezier(0.4, 0, 0.2, 1); –transition-slow: 350ms cubic-bezier(0.4, 0, 0.2, 1); } /* === Global Resets & Base === */ { margin: 0; padding: 0; box-sizing: border-box; } html { font-size: 16px; scroll-behavior: smooth; } body { font-family: var(–font-body); font-size: 1rem; line-height: 1.7; color: var(–color-text-primary); background: var(–color-bg-primary); overflow-x: hidden; position: relative; } /* === Cyberpunk Background Effects === */ .cyber-grid { position: fixed; top: 0; left: 0; width: 100%; height: 100%; background-image: linear-gradient(rgba(102, 144, 212, 0.03) 1px, transparent 1px), linear-gradient(90deg, rgba(102, 144, 212, 0.03) 1px, transparent 1px); background-size: 50px 50px; pointer-events: none; z-index: -1; opacity: 0.4; } @keyframes gridPulse { 0%, 100% { opacity: 0.3; } 50% { opacity: 0.6; } } /* === Typography === */ h1, h2, h3, h4, h5, h6 { font-family: var(–font-heading); font-weight: 700; line-height: 1.3; margin-top: var(–spacing-xl); margin-bottom: var(–spacing-md); color: var(–color-text-primary); letter-spacing: 0.5px; } h1 { font-size: 2.5rem; color: var(–color-text-primary); margin-top: 0; } h2 { font-size: 1.75rem; color: var(–color-text-primary); border-bottom: 1px solid var(–color-border); padding-top: var(–spacing-md); padding-bottom: var(–spacing-sm); position: relative; display: inline-block; width: 100%; } h2::after { content: “”; position: absolute; bottom: -1px; left: 0; width: 48px; height: 2px; background: var(–color-accent-blue); } h3 { font-size: 1.25rem; color: var(–color-text-primary); } h4 { font-size: 1rem; color: var(–color-text-secondary); } p { margin-bottom: var(–spacing-md); } a { color: var(–color-text-link); text-decoration: none; transition: color var(–transition-fast), text-shadow var(–transition-fast), transform var(–transition-fast); position: relative; } /* Scope the magenta glow hover to body text links only. */ .page-content a:hover, .persona-tile a:hover, .pattern-card a:hover, .mode-card a:hover, .link-grid a:hover { color: var(–color-accent-magenta); text-shadow: var(–glow-magenta); } /* Keyboard focus styling — visible ring without disturbing mouse users */ a:focus-visible, button:focus-visible, summary:focus-visible, input:focus-visible, [tabindex]:focus-visible { outline: 2px solid var(–color-accent-cyan); outline-offset: 2px; border-radius: 4px; } /* Skip-to-content link (shown when focused) */ .skip-link { position: absolute; left: 0; top: -48px; z-index: 500; background: var(–color-accent-cyan); color: var(–color-bg-primary); padding: 0.55rem 1.1rem; font-weight: 700; border-radius: 0 0 8px 0; box-shadow: var(–glow-cyan); transition: top var(–transition-fast); } .skip-link:focus, .skip-link:focus-visible { top: 0; outline: none; } /* Reading progress bar */ .reading-progress { position: fixed; top: 0; left: 0; width: 100%; height: 3px; background: transparent; z-index: 120; pointer-events: none; } .reading-progress-bar { display: block; height: 100%; width: 0; background: linear-gradient(90deg, var(–color-accent-cyan), var(–color-accent-magenta)); box-shadow: 0 0 10px rgba(102, 144, 212, 0.6); transition: width 80ms linear; } code { font-family: var(–font-mono); font-size: 0.9em; background: var(–color-bg-elevated); padding: 0.2em 0.4em; border-radius: 3px; border: 1px solid var(–color-border); color: var(–color-accent-green); } pre { background: var(–color-bg-secondary); border: 1px solid var(–color-border); border-left: 3px solid var(–color-accent-cyan); border-radius: 6px; padding: var(–spacing-md); overflow-x: auto; margin: var(–spacing-lg) 0; box-shadow: 0 4px 12px rgba(0, 0, 0, 0.3); } pre code { background: transparent; border: none; padding: 0; color: var(–color-text-primary); } blockquote { border-left: 4px solid var(–color-accent-purple); padding-left: var(–spacing-lg); margin: var(–spacing-lg) 0; font-style: italic; color: var(–color-text-secondary); background: var(–color-bg-secondary); padding: var(–spacing-md) var(–spacing-lg); border-radius: 0 6px 6px 0; box-shadow: var(–glow-purple); } .persona-callout { border: 1px solid var(–color-border-glow); border-left: 4px solid var(–color-accent-cyan); border-radius: 10px; margin: var(–spacing-lg) 0; padding: var(–spacing-md) var(–spacing-lg); background: linear-gradient(135deg, rgba(6, 147, 227, 0.08), rgba(0, 255, 163, 0.04)); box-shadow: 0 8px 30px rgba(0, 0, 0, 0.35); font-style: normal; } .persona-callout h4 { margin-top: 0; margin-bottom: var(–spacing-sm); color: var(–color-text-primary); } .persona-callout p { margin-bottom: var(–spacing-sm); color: var(–color-text-secondary); } .persona-callout ul { margin: 0; padding-left: 1.2rem; } .persona-callout li { margin-bottom: 0.25rem; } .persona-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); gap: var(–spacing-lg); margin-bottom: var(–spacing-xl); } .persona-tile { border: 1px solid var(–color-border); border-radius: 12px; padding: var(–spacing-lg); background: var(–color-bg-secondary); box-shadow: 0 8px 25px rgba(0, 0, 0, 0.4); } .persona-tile h4 { margin-top: 0; margin-bottom: var(–spacing-sm); } .persona-tile p { margin-bottom: var(–spacing-sm); color: var(–color-text-secondary); } .persona-tile ul { padding-left: 1.1rem; margin: 0; color: var(–color-text-muted); font-size: 0.95rem; } .pattern-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(260px, 1fr)); gap: var(–spacing-lg); margin-bottom: var(–spacing-xl); } .pattern-card { border: 1px solid var(–color-border); border-radius: 12px; padding: var(–spacing-lg); background: var(–color-bg-secondary); display: flex; flex-direction: column; gap: var(–spacing-sm); } .pattern-card a { font-size: 0.95rem; } /* Hero use-case cards (docs/use-cases.md) */ .usecase-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(280px, 1fr)); gap: var(–spacing-lg); margin: var(–spacing-xl) 0; } .usecase-card { border: 1px solid var(–color-border); border-radius: 12px; background: var(–color-bg-secondary); overflow: hidden; display: flex; flex-direction: column; transition: transform var(–transition-fast), box-shadow var(–transition-fast), border-color var(–transition-fast); } .usecase-card:hover { transform: translateY(-3px); border-color: var(–color-border-glow); box-shadow: 0 12px 30px rgba(0, 0, 0, 0.4); } .usecase-media { display: block; aspect-ratio: 16 / 10; overflow: hidden; background: var(–color-bg-tertiary); } .usecase-media img { width: 100%; height: 100%; object-fit: cover; } .usecase-body { padding: var(–spacing-lg); display: flex; flex-direction: column; gap: var(–spacing-sm); } .usecase-body h3 { margin: 0; font-size: 1.2rem; } .usecase-tag { align-self: flex-start; text-transform: uppercase; letter-spacing: 0.08em; font-size: 0.7rem; font-weight: 600; color: var(–color-accent-cyan); border: 1px solid var(–color-border-glow); border-radius: 999px; padding: 0.15rem 0.6rem; } .usecase-body p { margin: 0; color: var(–color-text-muted); font-size: 0.95rem; } .pipeline-chips { display: flex; flex-wrap: wrap; gap: 0.4rem; margin-top: var(–spacing-xs); } .pipeline-chips span { font-size: 0.75rem; font-family: var(–font-mono, monospace); color: var(–color-text-secondary); background: var(–color-bg-tertiary); border: 1px solid var(–color-border); border-radius: 6px; padding: 0.2rem 0.5rem; } /* Use-case detail pages */ .usecase-eyebrow { text-transform: uppercase; letter-spacing: 0.08em; font-size: 0.8rem; font-weight: 600; color: var(–color-accent-cyan); margin-bottom: var(–spacing-sm); } .usecase-hero { margin: var(–spacing-lg) 0; border: 1px solid var(–color-border); border-radius: 12px; overflow: hidden; background: var(–color-bg-tertiary); } .usecase-hero img, .usecase-hero video { display: block; width: 100%; height: auto; } .usecase-gallery { display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: var(–spacing-md); margin: var(–spacing-lg) 0; } .usecase-gallery.two { grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); } .usecase-gallery figure { margin: 0; border: 1px solid var(–color-border); border-radius: 10px; overflow: hidden; background: var(–color-bg-secondary); } .usecase-gallery img, .usecase-gallery video { display: block; width: 100%; height: auto; } .usecase-gallery figcaption { padding: var(–spacing-sm) var(–spacing-md); font-size: 0.85rem; color: var(–color-text-muted); border-top: 1px solid var(–color-border); } .step-sequence { counter-reset: steps; list-style: none; margin: var(–spacing-lg) 0; padding-left: 0; } .step-sequence li { position: relative; padding-left: 2.5rem; margin-bottom: var(–spacing-md); line-height: 1.5; } .step-sequence li::before { counter-increment: steps; content: counter(steps); position: absolute; left: 0; top: 0; width: 1.75rem; height: 1.75rem; border-radius: 50%; border: 1px solid var(–color-border-glow); display: flex; align-items: center; justify-content: center; background: var(–color-bg-tertiary); color: var(–color-accent-cyan); font-weight: 600; } .step-sequence a { font-weight: 600; color: var(–color-text-primary); } .mode-split { display: grid; grid-template-columns: repeat(auto-fit, minmax(260px, 1fr)); gap: var(–spacing-lg); margin-top: var(–spacing-lg); } .mode-card { border: 1px solid var(–color-border); border-radius: 12px; padding: var(–spacing-lg); background: var(–color-bg-secondary); box-shadow: 0 8px 25px rgba(0, 0, 0, 0.35); } .mode-card ul { padding-left: 1.2rem; } .mode-card a { display: inline-block; margin-top: var(–spacing-sm); font-weight: 600; } .home-hero { text-align: left; padding: var(–spacing-xl) 0 var(–spacing-lg); } .home-hero img, .home-screenshot { display: block; width: 100%; max-width: 100%; height: auto; border-radius: 8px; border: 1px solid var(–color-border); margin: var(–spacing-lg) 0; box-shadow: 0 4px 20px rgba(102, 144, 212, 0.08); } .eyebrow { text-transform: uppercase; letter-spacing: 0.1em; font-size: 0.85rem; color: var(–color-text-muted); margin-bottom: var(–spacing-sm); } .lead { font-size: 1.1rem; color: var(–color-text-secondary); max-width: 720px; margin-bottom: var(–spacing-lg); } .cta-row { display: flex; flex-wrap: wrap; gap: var(–spacing-sm); margin-bottom: var(–spacing-lg); } .cta-button { border: 1px solid var(–color-border); border-radius: 999px; padding: 0.6rem 1.5rem; font-weight: 600; transition: transform var(–transition-fast), box-shadow var(–transition-fast); text-align: center; } .cta-button.primary { background: var(–gradient-primary); border: none; color: #fff; } .cta-button.ghost { border-color: var(–color-text-muted); color: var(–color-text-muted); } .cta-button:hover { transform: translateY(-2px); box-shadow: var(–glow-cyan); } .home-section { margin-bottom: var(–spacing-2xl); } .feature-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); gap: var(–spacing-lg); } .feature-card { border: 1px solid var(–color-border); border-radius: 12px; padding: var(–spacing-lg); background: var(–color-bg-secondary); } .link-grid { list-style: none; display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: var(–spacing-sm); padding-left: 0; } .link-grid li a { display: block; padding: 0.6rem 0.9rem; border: 1px solid var(–color-border); border-radius: 10px; background: var(–color-bg-secondary); } /* === Header === */ .site-header { position: sticky; top: 0; width: 100%; height: var(–header-height); background: rgba(15, 17, 21, 0.95); backdrop-filter: blur(10px); border-bottom: 1px solid var(–color-border); z-index: 100; box-shadow: 0 2px 20px rgba(102, 144, 212, 0.08); } .header-container { max-width: var(–max-width); margin: 0 auto; height: 100%; display: grid; grid-template-columns: auto 1fr auto; align-items: center; padding: 0 var(–spacing-lg); } .logo-section { position: relative; } .logo { font-family: var(–font-heading); font-size: 1.5rem; font-weight: 800; color: var(–color-text-primary); text-decoration: none; display: flex; align-items: center; gap: 0.25rem; transition: all var(–transition-base); } .logo-image { width: 64px; height: 64px; filter: brightness(10); } .logo-text { font-size: 1.25rem; font-weight: 700; letter-spacing: 0.1em; color: transparent; background-image: linear-gradient(to right, #60a5fa, #a78bfa, #fb7185); -webkit-background-clip: text; background-clip: text; } .logo-bracket { color: var(–color-accent-cyan); font-weight: 900; } .logo:hover { transform: translateY(-2px); text-shadow: var(–glow-cyan); } .logo:hover .logo-bracket { text-shadow: var(–glow-cyan); animation: bracket-pulse 0.6s ease-in-out; } @keyframes bracket-pulse { 0%, 100% { transform: scaleX(1); } 50% { transform: scaleX(1.2); } } .main-nav { display: flex; justify-content: center; } .nav-list { display: flex; list-style: none; gap: 0.25rem; align-items: center; padding: 0; } .nav-item { display: block; padding: 0.5rem 1rem; font-size: 0.875rem; white-space: nowrap; font-weight: 500; color: #cbd5e1; text-decoration: none; border-radius: 9999px; transition: all 0.2s; border: 1px solid transparent; } .nav-item:hover { color: #bfdbfe; background-color: rgba(30, 41, 59, 0.6); } .nav-item.active { background-color: rgba(37, 99, 235, 0.25); color: #bfdbfe; border-color: rgba(59, 130, 246, 0.4); } .nav-item-external { color: var(–color-accent-cyan, #67e8f9); border-color: rgba(103, 232, 249, 0.3); } .nav-item-external:hover { background-color: rgba(103, 232, 249, 0.1); color: var(–color-accent-cyan, #67e8f9); } /* Remove old underline effect */ .nav-item::before { display: none; } .header-right { display: flex; align-items: center; } .search-toggle { display: none; align-items: center; justify-content: center; width: 40px; height: 40px; padding: 0; background: transparent; border: 1px solid transparent; border-radius: 10px; color: var(–color-text-secondary); cursor: pointer; transition: background var(–transition-fast), color var(–transition-fast), border-color var(–transition-fast); } .search-toggle:hover, .search-toggle[aria-expanded=”true”] { background: rgba(30, 41, 59, 0.6); color: var(–color-accent-cyan); border-color: rgba(102, 144, 212, 0.4); } .search-container { position: relative; margin-right: var(–spacing-lg); width: 200px; max-width: 100%; transition: width var(–transition-base); } .search-container:focus-within { width: 360px; } .search-input-wrapper { display: grid; grid-template-columns: auto 1fr auto; align-items: center; gap: var(–spacing-sm); background: rgba(15, 23, 42, 0.6); border: 1px solid rgba(30, 41, 59, 0.7); border-radius: 14px; padding: 0.35rem 0.75rem; box-shadow: 0 10px 30px rgba(0, 0, 0, 0.25), inset 0 0 0 1px rgba(255, 255, 255, 0.03); transition: border var(–transition-fast), box-shadow var(–transition-fast), transform var(–transition-fast); } .search-input-wrapper:focus-within { border-color: rgba(102, 144, 212, 0.5); box-shadow: var(–glow-cyan); transform: translateY(-1px); } .search-icon { display: inline-flex; color: var(–color-text-muted); opacity: 0.8; } .search-input-wrapper input { width: 100%; background: transparent; border: none; color: var(–color-text-primary); font-size: 0.95rem; outline: none; padding: 0.35rem 0; } .search-input-wrapper input::placeholder { color: var(–color-text-muted); } .search-shortcut { display: inline-flex; align-items: center; justify-content: center; padding: 0.15rem 0.4rem; border: 1px solid var(–color-border); border-radius: 8px; background: rgba(255, 255, 255, 0.04); color: var(–color-text-muted); font-family: var(–font-mono); font-size: 0.75rem; letter-spacing: 0.03em; } .search-results { position: absolute; top: calc(100% + 10px); right: 0; width: min(520px, 90vw); max-height: 70vh; overflow: hidden; background: rgba(10, 12, 18, 0.98); border: 1px solid rgba(30, 41, 59, 0.9); border-radius: 14px; box-shadow: 0 20px 50px rgba(0, 0, 0, 0.5), var(–glow-purple); backdrop-filter: blur(6px); z-index: 999; } .search-results-header { display: flex; align-items: center; justify-content: space-between; padding: 0.75rem 1rem; border-bottom: 1px solid var(–color-border); color: var(–color-text-muted); font-size: 0.85rem; text-transform: uppercase; letter-spacing: 0.04em; } .search-results-list { list-style: none; max-height: 62vh; overflow-y: auto; padding: 0.35rem 0; } .search-result { padding: 0.65rem 1rem; cursor: pointer; transition: background var(–transition-fast), transform var(–transition-fast); border-left: 2px solid transparent; } .search-result:hover, .search-result.active { background: rgba(102, 144, 212, 0.08); border-left-color: var(–color-accent-cyan); transform: translateX(2px); } .search-result-title { font-weight: 700; color: var(–color-text-primary); display: block; margin-bottom: 0.2rem; } .search-result-meta { display: flex; flex-wrap: wrap; gap: 0.35rem; align-items: center; color: var(–color-text-muted); font-size: 0.85rem; } .search-badge { display: inline-flex; align-items: center; padding: 0.1rem 0.45rem; border-radius: 999px; font-size: 0.75rem; font-weight: 700; letter-spacing: 0.03em; text-transform: uppercase; border: 1px solid transparent; } .search-badge.concept { background: rgba(99, 102, 241, 0.14); border-color: rgba(99, 102, 241, 0.4); color: #c7d2fe; } .search-badge.tutorial { background: rgba(16, 185, 129, 0.14); border-color: rgba(16, 185, 129, 0.4); color: #a7f3d0; } .search-badge.reference { background: rgba(59, 130, 246, 0.12); border-color: rgba(59, 130, 246, 0.35); color: #bfdbfe; } .search-badge.api { background: rgba(236, 72, 153, 0.14); border-color: rgba(236, 72, 153, 0.4); color: #fbcfe8; } .search-badge.node { background: rgba(234, 179, 8, 0.16); border-color: rgba(234, 179, 8, 0.45); color: #fef08a; } .search-snippet { margin-top: 0.2rem; color: var(–color-text-secondary); font-size: 0.9rem; line-height: 1.4; } .search-empty { padding: 0.75rem 1rem; color: var(–color-text-muted); font-size: 0.9rem; } .social-links { display: flex; gap: var(–spacing-sm); } .social-link { padding: 0.25rem; color: var(–color-text-secondary); display: flex; align-items: center; justify-content: center; border-radius: 0.5rem; transition: all var(–transition-base); } .social-link:hover { color: var(–color-text-primary); background: rgba(30, 41, 59, 0.6); transform: none; box-shadow: none; } .mobile-menu-toggle { display: none; background: none; border: none; cursor: pointer; padding: var(–spacing-sm); position: relative; z-index: 250; } .mobile-menu-toggle.active .hamburger { background: transparent; box-shadow: none; } .mobile-menu-toggle.active .hamburger::before { top: 0; transform: rotate(45deg); } .mobile-menu-toggle.active .hamburger::after { bottom: 0; transform: rotate(-45deg); } .hamburger { display: block; width: 25px; height: 2px; background: var(–color-accent-cyan); position: relative; transition: all var(–transition-base); box-shadow: var(–glow-cyan); } .hamburger::before, .hamburger::after { content: “”; position: absolute; width: 25px; height: 2px; background: var(–color-accent-cyan); transition: all var(–transition-base); box-shadow: var(–glow-cyan); } .hamburger::before { top: -8px; } .hamburger::after { bottom: -8px; } /* === Main Container === */ .main-container { max-width: var(–max-width); margin: 0 auto; display: flex; min-height: calc(100vh - var(–header-height) - 200px); } /* === Sidebar === */ .sidebar { width: var(–sidebar-width); padding: var(–spacing-xl) var(–spacing-lg); background: var(–color-bg-secondary); border-right: 1px solid var(–color-border); position: sticky; top: var(–header-height); height: calc(100vh - var(–header-height)); overflow-y: auto; scrollbar-width: thin; scrollbar-color: var(–color-accent-cyan) var(–color-bg-tertiary); } .sidebar::-webkit-scrollbar { width: 6px; } .sidebar::-webkit-scrollbar-track { background: var(–color-bg-tertiary); } .sidebar::-webkit-scrollbar-thumb { background: var(–color-accent-cyan); border-radius: 3px; box-shadow: var(–glow-cyan); } .sidebar-header { margin-bottom: var(–spacing-xl); display: flex; align-items: center; justify-content: space-between; gap: var(–spacing-sm); } .sidebar-title { font-family: var(–font-heading); font-size: 1.1rem; color: var(–color-accent-cyan); display: flex; align-items: center; gap: var(–spacing-sm); margin: 0; } .sidebar-collapse-all { background: transparent; border: 1px solid var(–color-border); color: var(–color-text-muted); font-family: var(–font-mono); font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.08em; padding: 0.3rem 0.55rem; border-radius: 6px; cursor: pointer; transition: color var(–transition-fast), border-color var(–transition-fast), background var(–transition-fast); } .sidebar-collapse-all:hover { color: var(–color-accent-cyan); border-color: var(–color-accent-cyan); background: rgba(102, 144, 212, 0.05); } .title-icon { color: var(–color-accent-magenta); animation: pulse 2s ease-in-out infinite; } @keyframes pulse { 0%, 100% { opacity: 1; } 50% { opacity: 0.5; } } .nav-section { margin-bottom: var(–spacing-lg); border: 1px solid transparent; border-radius: 8px; transition: border-color var(–transition-fast); } .nav-section[open] { border-color: rgba(102, 144, 212, 0.08); } .nav-section-summary { list-style: none; cursor: pointer; display: flex; align-items: center; justify-content: space-between; gap: var(–spacing-sm); padding: 0.45rem 0.55rem; border-radius: 6px; transition: background var(–transition-fast); } .nav-section-summary::-webkit-details-marker { display: none; } .nav-section-summary:hover { background: rgba(102, 144, 212, 0.06); } .nav-section-title { font-family: var(–font-heading); font-size: 0.8rem; text-transform: uppercase; letter-spacing: 1px; color: var(–color-text-muted); font-weight: 600; margin: 0; } .nav-section[open] .nav-section-title { color: var(–color-accent-cyan); } .nav-section-chevron { width: 0; height: 0; border-left: 5px solid var(–color-text-muted); border-top: 4px solid transparent; border-bottom: 4px solid transparent; transition: transform var(–transition-fast); } .nav-section[open] .nav-section-chevron { transform: rotate(90deg); border-left-color: var(–color-accent-cyan); } .nav-section-list { list-style: none; margin-top: 0.3rem; padding: 0; } .sidebar-link { display: block; padding: var(–spacing-sm) var(–spacing-md); color: var(–color-text-secondary); font-size: 0.95rem; border-left: 2px solid transparent; transition: all var(–transition-fast); margin-bottom: 2px; } .sidebar-link:hover { color: var(–color-accent-cyan); background: rgba(102, 144, 212, 0.04); border-left-color: var(–color-accent-cyan); transform: translateX(4px); text-shadow: none; } .sidebar-link.active { color: var(–color-accent-cyan); background: rgba(102, 144, 212, 0.08); border-left-color: var(–color-accent-cyan); font-weight: 600; } /* === Content Area === */ .content { flex: 1; padding: var(–spacing-2xl) var(–spacing-xl); max-width: 900px; margin: 0 auto; width: 100%; } .doc-page { animation: fadeIn 0.5s ease-in; } @keyframes fadeIn { from { opacity: 0; transform: translateY(20px); } to { opacity: 1; transform: translateY(0); } } .page-header { position: relative; margin-bottom: var(–spacing-2xl); padding-bottom: var(–spacing-lg); border-bottom: 1px solid var(–color-border); } .page-title { margin: 0 0 var(–spacing-sm) 0; } .page-description { font-size: 1.1rem; color: var(–color-text-secondary); margin: 0; } .page-content { margin-bottom: var(–spacing-3xl); } .page-content img { max-width: 100%; height: auto; border-radius: 8px; border: 1px solid var(–color-border); margin: var(–spacing-lg) 0; box-shadow: 0 4px 20px rgba(102, 144, 212, 0.08); } .page-content video { display: block; width: 100%; height: auto; border-radius: 8px; border: 1px solid var(–color-border); margin: var(–spacing-lg) 0; background: #0f0f17; box-shadow: 0 4px 20px rgba(102, 144, 212, 0.08); } .page-content ul, .page-content ol { margin-left: var(–spacing-lg); margin-bottom: var(–spacing-md); } @media (max-width: 768px) { .page-content ul, .page-content ol { margin-left: var(–spacing-md); } } .page-content li { margin-bottom: var(–spacing-sm); color: var(–color-text-secondary); } .page-content .icon-list li { display: flex; align-items: flex-start; } .page-content .icon-list li svg { flex-shrink: 0; vertical-align: middle; margin-right: 8px; margin-top: 2px; stroke: var(–color-accent-cyan); } .page-content li::marker { color: var(–color-accent-cyan); } .page-content table { width: 100%; border-collapse: collapse; margin: var(–spacing-lg) 0; background: var(–color-bg-secondary); border: 1px solid var(–color-border); border-radius: 6px; overflow: hidden; } .page-content th, .page-content td { padding: var(–spacing-md); text-align: left; border-bottom: 1px solid var(–color-border); } .page-content th { background: var(–color-bg-tertiary); color: var(–color-accent-cyan); font-family: var(–font-heading); font-weight: 600; text-transform: uppercase; font-size: 0.9rem; letter-spacing: 0.5px; } .page-content tr:hover { background: rgba(102, 144, 212, 0.03); } /* === Page Navigation === */ .page-navigation { display: flex; justify-content: space-between; gap: var(–spacing-md); margin-top: var(–spacing-3xl); padding-top: var(–spacing-xl); border-top: 1px solid var(–color-border); } .nav-link { flex: 1; padding: var(–spacing-lg); background: var(–color-bg-secondary); border: 1px solid var(–color-border); border-radius: 6px; display: flex; align-items: center; gap: var(–spacing-md); transition: all var(–transition-base); text-decoration: none; } .nav-link:hover { background: var(–color-bg-tertiary); border-color: var(–color-accent-cyan); transform: translateY(-2px); box-shadow: 0 4px 12px rgba(102, 144, 212, 0.15); } .nav-link.prev { justify-content: flex-start; } .nav-link.next { justify-content: flex-end; text-align: right; } .nav-arrow { font-size: 1.5rem; color: var(–color-accent-cyan); } .nav-text { font-family: var(–font-heading); font-size: 0.9rem; color: var(–color-text-secondary); } .nav-link:hover .nav-arrow { text-shadow: var(–glow-cyan); } .nav-link:hover .nav-text { color: var(–color-accent-cyan); } /* === Footer === */ .site-footer { background: var(–color-bg-secondary); border-top: 1px solid var(–color-border); padding: var(–spacing-3xl) 0 var(–spacing-xl); margin-top: var(–spacing-3xl); position: relative; overflow: hidden; } .footer-glow { position: absolute; top: 0; left: 50%; transform: translateX(-50%); width: 100%; height: 1px; background: linear-gradient(90deg, transparent, var(–color-accent-cyan), var(–color-accent-magenta), transparent); box-shadow: var(–glow-cyan); } .footer-container { max-width: var(–max-width); margin: 0 auto; padding: 0 var(–spacing-lg); } .footer-grid { display: grid; grid-template-columns: 2fr 1fr 1fr 1fr; gap: var(–spacing-2xl); margin-bottom: var(–spacing-2xl); } .footer-title { font-family: var(–font-heading); font-size: 1.3rem; color: var(–color-accent-cyan); margin-bottom: var(–spacing-sm); } .footer-desc { color: var(–color-text-muted); font-size: 0.95rem; line-height: 1.6; } .footer-marketing-link { margin-top: var(–spacing-md); font-size: 0.95rem; } .footer-marketing-link a { color: var(–color-accent-cyan); text-decoration: none; font-weight: 500; transition: opacity 0.2s; } .footer-marketing-link a:hover { opacity: 0.8; text-decoration: underline; } .footer-heading { font-family: var(–font-heading); font-size: 0.9rem; color: var(–color-text-primary); margin-bottom: var(–spacing-md); text-transform: uppercase; letter-spacing: 1px; } .footer-links { list-style: none; } .footer-links li { margin-bottom: var(–spacing-sm); } .footer-links a { color: var(–color-text-secondary); font-size: 0.95rem; transition: all var(–transition-fast); } .footer-links a:hover { color: var(–color-accent-cyan); transform: translateX(4px); display: inline-block; } .footer-bottom { padding-top: var(–spacing-xl); border-top: 1px solid var(–color-border); text-align: center; } .copyright { color: var(–color-text-muted); font-size: 0.9rem; } .bracket { color: var(–color-accent-magenta); font-weight: bold; } /* === Responsive Design === */ @media (max-width: 1024px) { .footer-grid { grid-template-columns: 1fr 1fr; } .sidebar { width: 250px; } .header-container { grid-template-columns: auto 1fr auto; column-gap: var(–spacing-sm); padding: 0 var(–spacing-lg); } .search-toggle { display: inline-flex; } .search-container { position: absolute; top: 100%; left: 0; right: 0; margin-right: 0; padding: var(–spacing-sm) var(–spacing-lg) var(–spacing-md); width: auto; max-width: none; background: rgba(15, 17, 21, 0.98); backdrop-filter: blur(10px); border-bottom: 1px solid var(–color-border); box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5); z-index: 140; display: none; } .search-container.open { display: block; } .search-container:focus-within { width: auto; } .search-results { position: static; left: auto; right: auto; width: auto; margin-top: var(–spacing-sm); box-shadow: none; } .header-right { justify-content: flex-end; gap: var(–spacing-sm); } } @media (max-width: 768px) { :root { –sidebar-width: 0; } .logo-image { width: 56px; height: 56px; } .logo-text { font-size: 1.5rem; } .sidebar { position: fixed; left: -280px; width: 280px; z-index: 200; transition: left var(–transition-base); } .sidebar.active { left: 0; } .content { padding: var(–spacing-lg); } .main-nav { position: relative; } .nav-list { display: none; position: fixed; top: var(–header-height); left: 0; right: 0; flex-direction: column; background: rgba(15, 17, 21, 0.98); backdrop-filter: blur(10px); border-bottom: 1px solid var(–color-border); padding: var(–spacing-lg); gap: var(–spacing-sm); max-height: calc(100vh - var(–header-height)); overflow-y: auto; z-index: 150; box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5); } .nav-list.active { display: flex; } .mobile-menu-toggle { display: block; } .footer-grid { grid-template-columns: 1fr; gap: var(–spacing-xl); } h1 { font-size: 2rem; } h2 { font-size: 1.5rem; } .page-navigation { flex-direction: column; } .header-container { grid-template-columns: auto 1fr auto; padding: 0 var(–spacing-md); } .search-container { padding: var(–spacing-sm) var(–spacing-md) var(–spacing-md); } .search-container:focus-within { width: auto; } .social-links { gap: var(–spacing-xs); } } /* === Syntax Highlighting (Rouge/Pygments) === */ .highlight { background: var(–color-bg-secondary); border-radius: 6px; border-left: 3px solid var(–color-accent-cyan); } .highlight .c { color: var(–color-text-muted); } /* Comment */ .highlight .k { color: #ff5555; } /* Keyword - using error color */ .highlight .s { color: var(–color-accent-blue); } /* String */ .highlight .mi { color: var(–color-accent-cyan); } /* Number */ .highlight .nf { color: var(–color-accent-purple); } /* Function */ .highlight .nc { color: #ffb86c; } /* Class - using warning color */ .highlight .o { color: #ff5555; } /* Operator - using error color */ /* === Copy Buttons (moved from JS-injected styles) === */ .copy-button { position: absolute; top: 0.5rem; right: 0.5rem; padding: 0.4rem 0.8rem; background: var(–color-bg-elevated); color: var(–color-accent-cyan); border: 1px solid var(–color-border); border-radius: 4px; font-family: var(–font-mono); font-size: 0.8rem; cursor: pointer; transition: all var(–transition-fast); opacity: 0; } pre:hover .copy-button, .copy-button:focus-visible { opacity: 1; } .copy-button:hover { background: var(–color-accent-cyan); color: var(–color-bg-primary); border-color: var(–color-accent-cyan); box-shadow: var(–glow-cyan); } .copy-button.copied { background: var(–color-accent-green); color: var(–color-bg-primary); border-color: var(–color-accent-green); opacity: 1; } .copy-page-button { position: absolute; top: 0; right: 0; display: inline-flex; align-items: center; gap: 0.4rem; padding: 0.4rem 0.8rem; background: var(–color-bg-elevated); color: var(–color-accent-cyan); border: 1px solid var(–color-border); border-radius: 4px; font-family: var(–font-mono); font-size: 0.8rem; cursor: pointer; transition: all var(–transition-fast); } .copy-page-button:hover { background: var(–color-accent-cyan); color: var(–color-bg-primary); border-color: var(–color-accent-cyan); box-shadow: var(–glow-cyan); } .copy-page-button:hover svg { stroke: var(–color-bg-primary); } .copy-page-button.copied { background: var(–color-accent-green); color: var(–color-bg-primary); border-color: var(–color-accent-green); } .copy-page-button.copied svg { stroke: var(–color-bg-primary); } /* === Back to top === */ .back-to-top { position: fixed; bottom: 1.25rem; right: 1.25rem; width: 44px; height: 44px; display: inline-flex; align-items: center; justify-content: center; background: var(–color-bg-elevated); color: var(–color-accent-cyan); border: 1px solid var(–color-border); border-radius: 50%; cursor: pointer; opacity: 0; transform: translateY(8px); transition: opacity var(–transition-base), transform var(–transition-base), box-shadow var(–transition-base), background var(–transition-fast), color var(–transition-fast); z-index: 110; box-shadow: 0 6px 20px rgba(0, 0, 0, 0.35); } .back-to-top.visible { opacity: 1; transform: translateY(0); } .back-to-top:hover { background: var(–color-accent-cyan); color: var(–color-bg-primary); box-shadow: var(–glow-cyan); } @media (max-width: 768px) { .back-to-top { bottom: 0.85rem; right: 0.85rem; width: 40px; height: 40px; } } /* === Page meta (Edit on GitHub, last updated) === */ .page-meta { display: flex; flex-wrap: wrap; align-items: center; justify-content: space-between; gap: var(–spacing-sm); margin-top: var(–spacing-xl); padding-top: var(–spacing-md); border-top: 1px solid var(–color-border); color: var(–color-text-muted); font-size: 0.875rem; } .edit-on-github { display: inline-flex; align-items: center; gap: 0.4rem; color: var(–color-text-muted); border: 1px solid var(–color-border); padding: 0.35rem 0.7rem; border-radius: 6px; font-family: var(–font-mono); transition: color var(–transition-fast), border-color var(–transition-fast), background var(–transition-fast); } .edit-on-github:hover { color: var(–color-accent-cyan); border-color: var(–color-accent-cyan); background: rgba(102, 144, 212, 0.05); text-shadow: none; } .page-updated { font-family: var(–font-mono); font-size: 0.8rem; } /* === Utility Classes === */ .text-center { text-align: center; } .mt-0 { margin-top: 0; } .mb-0 { margin-bottom: 0; } /* === Mermaid Workflow Graphs === / / Restyle Mermaid diagrams to match the NodeTool editor canvas — the same look the workflow graphs have on the marketing site. The theme variables (fills, fonts, edge color) are set in _layouts/default.html; the canvas grid, node chrome, and edge polish live here. / .mermaid { / The editor canvas: dark fill with the dotted grid (NodeCanvas). */ background-color: #08090A; background-image: radial-gradient(circle, #1F2126 1px, transparent 1px); background-size: 16px 16px; border: 1px solid var(–color-border); border-radius: 12px; padding: var(–spacing-xl) var(–spacing-lg); margin: var(–spacing-xl) 0; text-align: center; overflow-x: auto; } .mermaid svg { max-width: 100%; height: auto; } /* Node chrome: dark body, rounded corners, soft drop shadow, faint edge — the NodeTool node card. */ .mermaid .node rect, .mermaid .node polygon, .mermaid .node circle, .mermaid .node ellipse, .mermaid .node path { rx: 8px; ry: 8px; fill: var(–color-bg-tertiary) !important; stroke: rgba(255, 255, 255, 0.10) !important; stroke-width: 1px !important; filter: drop-shadow(0 8px 20px rgba(0, 0, 0, 0.16)); } .mermaid .node .label, .mermaid .nodeLabel, .mermaid .node text { fill: var(–color-text-secondary) !important; color: var(–color-text-secondary) !important; font-family: var(–font-body); font-size: 14px; font-weight: 400; letter-spacing: 0.01em; } /* Edges: smooth accent-blue connectors with rounded caps, like the canvas. */ .mermaid .edgePath .path, .mermaid .flowchart-link { stroke: var(–color-accent-blue) !important; stroke-width: 1.6px !important; stroke-opacity: 0.95; stroke-linecap: round; fill: none !important; } .mermaid .arrowheadPath, .mermaid marker path { fill: var(–color-accent-blue) !important; stroke: none !important; } /* Edge labels sit on a small dark chip so they read over the grid. */ .mermaid .edgeLabel, .mermaid .edgeLabel p { background-color: var(–color-bg-secondary) !important; color: var(–color-text-muted) !important; fill: var(–color-text-muted) !important; font-size: 12px; border-radius: 4px; } /* Subgraph containers, when a graph uses them. */ .mermaid .cluster rect { fill: rgba(16, 17, 19, 0.6) !important; stroke: var(–color-border) !important; rx: 10px; ry: 10px; } .mermaid .cluster .label, .mermaid .cluster text { fill: var(–color-text-muted) !important; color: var(–color-text-muted) !important; } /* === Print Styles === */ @media print { .site-header, .sidebar, .footer, .site-footer, .page-navigation, .back-to-top, .reading-progress, .skip-link, .copy-button, .copy-page-button, .edit-on-github, .page-meta { display: none !important; } .content { max-width: 100%; } .cyber-grid { display: none; } body { background: #fff; color: #000; } h1, h2, h3, h4, h5, h6 { color: #000 !important; background: none !important; -webkit-text-fill-color: initial !important; } a { color: #0066cc !important; } }--- # NodeTool Server Authentication Source: https://docs.nodetool.ai/authentication.html See API Reference for a matrix of endpoints, auth requirements, and streaming behavior. The NodeTool server uses token-based authentication to secure all endpoints when deployed. For environment variable defaults and precedence, see the Configuration Guide. Quick Start The running server enforces authentication when it is in Supabase mode (both SUPABASE_URL and SUPABASE_KEY set). Without those, it runs in Local mode: loopback connections are trusted and run as user "1", while non-loopback requests are rejected. # Start the server (Local mode — loopback trusted) nodetool serve # Enable Supabase mode to enforce auth on every request export SUPABASE_URL=https://your-project.supabase.co export SUPABASE_KEY=your-service-role-key nodetool serve The SERVER_AUTH_TOKEN / deployment.yaml token described below is generated and consumed by the @nodetool-ai/deploy tooling; see Authentication Modes for how the server itself selects its mode. How It Works Token Generation Priority The server looks for a token in this order: Environment variable SERVER_AUTH_TOKEN Config file ~/.config/nodetool/deployment.yaml Auto-generate a new token and save it to the config file Auto-Generation On first run, if no token is found: A cryptographically secure 32-byte token is generated (crypto.randomBytes) The token is saved to ~/.config/nodetool/deployment.yaml File permissions are set to 0600 (owner read/write only) The full token is displayed in the console output Token Location Config file path: Linux/macOS: ~/.config/nodetool/deployment.yaml Windows: %APPDATA%\nodetool\deployment.yaml File format: server_auth_token: your-auto-generated-token-here Using the Token All API requests (except the public health endpoints /health, /ready, and /api/health) require authentication when the server is running in Supabase mode — that is, when both SUPABASE_URL and SUPABASE_KEY are set. When those are not set, the server runs in Local mode and accepts loopback connections without a token for convenience (see Localhost Trust); non-loopback requests are rejected with 401. When authentication is enforced, send the Supabase JWT in the header: # Get token from config file TOKEN=$(cat ~/.config/nodetool/deployment.yaml | grep server_auth_token | cut -d' ' -f2) # Use in requests curl -H "Authorization: Bearer $TOKEN" http://localhost:7777/v1/models Example Requests Chat Completion: curl -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -X POST http://localhost:7777/v1/chat/completions \ -d '{ "model": "llama3.2:latest", "messages": [{"role": "user", "content": "Hello"}] }' List Collections: curl -H "Authorization: Bearer $TOKEN" \ http://localhost:7777/admin/collections Upload File: curl -H "Authorization: Bearer $TOKEN" \ -X PUT http://localhost:7777/admin/storage/assets/image.png \ --data-binary @image.png Authentication Modes The running server does not read an AUTH_PROVIDER variable to choose a strategy. It selects its mode from the presence of Supabase credentials: Supabase mode — enabled when both SUPABASE_URL and SUPABASE_KEY are set. The server validates a Supabase JWT on every non-public request and enforces auth. Local mode — the default when those are not set. Loopback connections bypass auth and run as user "1" (gated by NODETOOL_TRUST_LOCALHOST); non-loopback requests are rejected with 401. AUTH_PROVIDER is only written into deployed-container environments by the @nodetool-ai/deploy tooling. The server itself never branches on it. To enable Supabase mode: export SUPABASE_URL=https://your-project.supabase.co export SUPABASE_KEY=your-service-role-key Behavior: Both HTTP and WebSocket endpoints enforce auth in Supabase mode. Clients send Authorization: Bearer <supabase_jwt>. In Local mode, loopback connections default to user "1" and no token is required. Supabase token verification results are cached in-process. The cache defaults to a 60-second TTL and at most 2000 entries; these are constructor options on the provider and are not configurable via environment variables. WebSockets use the same Authorization header semantics as HTTP when auth is enforced. A ?api_key=<token> query parameter is also accepted as a fallback for WebSocket connections. Architecture The authentication system is split across two packages: @nodetool-ai/auth – defines the abstract AuthProvider base class, token extraction from HTTP and WebSocket headers, and the AuthResult / TokenType types. @nodetool-ai/deploy – provides the concrete token management functions: getServerAuthToken(), generateSecureToken(), verifyServerToken(), and config file I/O. // @nodetool-ai/auth abstract class AuthProvider { extractTokenFromHeaders(headers: Record<string, string> | Headers): string | null; extractTokenFromWs(headers, queryParams?): string | null; abstract verifyToken(token: string): Promise<AuthResult>; } // @nodetool-ai/deploy function getServerAuthToken(): string; // resolve token (env → config → generate) function generateSecureToken(): string; // crypto.randomBytes(32).toString("base64url") function verifyServerToken(auth: string): Promise<"authenticated">; // timing-safe compare function loadAuthConfig(): Record<string, unknown>; function saveAuthConfig(config: Record<string, unknown>): void; Token verification uses crypto.timingSafeEqual to prevent timing attacks. Note: @nodetool-ai/auth also ships a StaticTokenProvider that reads STATIC_AUTH_TOKEN / STATIC_AUTH_TOKENS. This is a distinct mechanism from the deploy-package SERVER_AUTH_TOKEN flow, and the StaticTokenProvider is not wired into the running websocket server — the server uses only the Supabase / Local selection described in Authentication Modes. Environment Variable Override You can override the auto-generated token by setting the environment variable: # Set your own token export SERVER_AUTH_TOKEN="my-custom-secure-token" # Start server (will use this token instead) nodetool serve This is useful for: Docker deployments with secrets CI/CD pipelines Multiple server instances with shared token Docker Deployment Using Auto-Generated Token Mount the config directory as a volume: docker run -v ~/.config/nodetool:/root/.config/nodetool \ -p 7777:7777 \ nodetool-server The token persists across container restarts. Using Environment Variable # Generate token TOKEN=$(openssl rand -base64 32) # Run with token docker run -e SERVER_AUTH_TOKEN="$TOKEN" \ -p 7777:7777 \ nodetool-server Docker Compose version: '3.8' services: server: image: nodetool-server ports: - "7777:7777" volumes: # Mount config to persist token - ./config:/root/.config/nodetool # OR use environment variable environment: - SERVER_AUTH_TOKEN=${SERVER_AUTH_TOKEN} Retrieving Your Token From Console Output The full token is displayed when the server starts: ====================================================================== AUTHENTICATION ====================================================================== Status: ENABLED (all endpoints require authentication) Token: abc12345...xyz9 Source: Auto-generated and saved to /home/user/.config/nodetool/deployment.yaml Authorization: Bearer abc12345-full-token-here-xyz9 ====================================================================== From Config File # Linux/macOS cat ~/.config/nodetool/deployment.yaml # Windows PowerShell Get-Content $env:APPDATA\nodetool\deployment.yaml Programmatically import { getServerAuthToken, getTokenSource } from "@nodetool-ai/deploy"; const token = getServerAuthToken(); const source = getTokenSource(); // "environment" | "config" | "generated" console.log(`Token (${source}): ${token}`); Security Best Practices 1. Protect the Config File The config file contains your authentication token. It’s automatically set to 0600 permissions, but ensure it’s not: Committed to version control Shared publicly Accessible by other users # Verify permissions (should be -rw-------) ls -la ~/.config/nodetool/deployment.yaml # Fix if needed chmod 600 ~/.config/nodetool/deployment.yaml 2. Rotate Tokens Regularly # Delete the config file rm ~/.config/nodetool/deployment.yaml # Restart server (new token will be generated) nodetool serve Or set a new environment variable: export SERVER_AUTH_TOKEN="$(openssl rand -base64 32)" 3. Use Different Tokens per Environment # Development export SERVER_AUTH_TOKEN="dev-token" # Staging export SERVER_AUTH_TOKEN="staging-token" # Production export SERVER_AUTH_TOKEN="$(openssl rand -base64 32)" 4. Use HTTPS in Production Always deploy with TLS/SSL: server { listen 443 ssl http2; server_name api.example.com; ssl_certificate /path/to/cert.pem; ssl_certificate_key /path/to/key.pem; location / { proxy_pass http://localhost:7777; } } Public Endpoints These endpoints do not require authentication: GET /health - Health check GET /ready - Readiness check GET /api/health - API health check This allows load balancers and monitoring systems to check service status without authentication. Error Responses 401 Unauthorized In Supabase mode, requests without a valid token receive a 401 with an error field: Missing token: { "error": "Unauthorized" } Invalid / rejected token: { "error": "<reason from the auth provider>" } Non-loopback request when running in Local mode: { "error": "Remote access requires authentication" } Troubleshooting Token Not Working # Start server and check token source in output nodetool serve # Look for "Source: ..." in the output # Verify token in config cat ~/.config/nodetool/deployment.yaml # Check environment variable echo $SERVER_AUTH_TOKEN Can’t Find Config File # Check default location ls -la ~/.config/nodetool/ # Create directory if missing mkdir -p ~/.config/nodetool # Restart server to generate token nodetool serve Token Not Persisting (Docker) # Mount config directory as volume docker run -v nodetool-config:/root/.config/nodetool \ -p 7777:7777 \ nodetool-server # Or use environment variable docker run -e SERVER_AUTH_TOKEN="your-token" \ -p 7777:7777 \ nodetool-server API Client Examples TypeScript / JavaScript const TOKEN = "your-token-here"; const BASE_URL = "http://localhost:7777"; const headers = { "Authorization": `Bearer ${TOKEN}`, "Content-Type": "application/json", }; // List models const models = await fetch(`${BASE_URL}/v1/models`, { headers }); console.log(await models.json()); // Chat completion const chat = await fetch(`${BASE_URL}/v1/chat/completions`, { method: "POST", headers, body: JSON.stringify({ model: "llama3.2:latest", messages: [{ role: "user", content: "Hello!" }], }), }); console.log(await chat.json()); Python import requests TOKEN = "your-token-here" BASE_URL = "http://localhost:7777" headers = {"Authorization": f"Bearer {TOKEN}"} # List models response = requests.get(f"{BASE_URL}/v1/models", headers=headers) print(response.json()) # Chat completion response = requests.post( f"{BASE_URL}/v1/chat/completions", headers={**headers, "Content-Type": "application/json"}, json={ "model": "llama3.2:latest", "messages": [{"role": "user", "content": "Hello!"}] } ) print(response.json()) Shell Script #!/bin/bash TOKEN=$(cat ~/.config/nodetool/deployment.yaml | grep server_auth_token | awk '{print $2}') BASE_URL="http://localhost:7777" # List models curl -H "Authorization: Bearer $TOKEN" \ "$BASE_URL/v1/models" # Chat completion curl -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -X POST "$BASE_URL/v1/chat/completions" \ -d '{ "model": "llama3.2:latest", "messages": [{"role": "user", "content": "Hello!"}] }' Advanced Configuration Custom Config Location import { loadAuthConfig, saveAuthConfig, generateSecureToken } from "@nodetool-ai/deploy"; // Load current config const config = loadAuthConfig(); // Generate and save a new token config["server_auth_token"] = generateSecureToken(); saveAuthConfig(config); Localhost Trust and Reverse Proxies By default the server bypasses authentication for connections from loopback (127.0.0.1/::1) and runs them as user 1. This is convenient for the desktop app and local development, but dangerous behind a reverse proxy, container network, or SSH tunnel, where the proxy itself connects from loopback — a blanket bypass would silently disable auth in exactly the deployment that needs it. To make this safe: The loopback bypass is gated by NODETOOL_TRUST_LOCALHOST. It defaults off whenever auth is enforced (Supabase mode) and on otherwise. Set it explicitly (true/false) to override. X-Forwarded-For is trusted only for proxies listed in NODETOOL_TRUSTED_PROXIES (comma-separated IPs/CIDRs). When unset, the header is ignored and the unspoofable socket peer address is used to identify the client, so a remote client cannot forge a loopback origin. # Behind an nginx reverse proxy on the same host, enforcing Supabase auth: export SUPABASE_URL=... SUPABASE_KEY=... export NODETOOL_TRUSTED_PROXIES=127.0.0.1 # trust the local proxy's XFF # NODETOOL_TRUST_LOCALHOST stays off — clients must present a valid token. Local mode in Docker Docker NATs a published-port connection to the bridge gateway (e.g. 172.17.0.1), so in Local mode the request never arrives from loopback and the NODETOOL_TRUST_LOCALHOST bypass can’t fire — the UI loads (static assets and /api/config are public) but every API/WebSocket call returns 401 "Remote access requires authentication". NODETOOL_TRUST_LOCAL_NETWORKS (comma-separated source CIDRs) restores the single-local-user model by trusting those sources as user "1" without a token. It is honored only in Local mode — in Supabase mode it is ignored so every request must present a valid token. ⚠️ This bypasses authentication Every source IP in NODETOOL_TRUST_LOCAL_NETWORKS is trusted as admin user "1" with no password — full access to your workflows, files, stored secrets, and API keys. Anyone who can reach the published port from a listed range gets that access. 172.16.0.0/12 — the Docker bridge range. Host and LAN clients reach the app through the port mapping, but nothing off the bridge is trusted. Use this. 0.0.0.0/0 — trusts every source, the whole internet if the port is reachable. Never use this on a public IP. Only on a network you fully control (private LAN / VPN), ideally with the port firewalled. Exposing NodeTool to the internet or untrusted users? Do not widen this list — enable Supabase auth so every request needs a real login. # Docker self-host, single user, no login (safe on a private LAN): NODETOOL_TRUST_LOCAL_NETWORKS=172.16.0.0/12 # the Docker bridge range Security Hardening Production: enable Supabase mode by setting SUPABASE_URL and SUPABASE_KEY, terminate TLS in front of all non-public endpoints, and rotate Supabase service-role keys via your secrets manager. Localhost trust: keep NODETOOL_TRUST_LOCALHOST off in any reverse-proxied or containerized deployment, and list only your real proxies in NODETOOL_TRUSTED_PROXIES. Staging: keep asset buckets private or signed, and run workflows in subprocess or Docker isolation. Development: restrict Local mode to isolated machines and avoid storing real secrets in .env.development. See Security Hardening for detailed checklists across dev, staging, and production.--- # Chain Editor Source: https://docs.nodetool.ai/chain-editor.html The Chain Editor is a linear alternative to the node graph. Instead of placing nodes on an infinite canvas, you compose an ordered chain of cards — “transcribe → summarize → email”, “fetch → parse → index”. Prefer the node graph? The full Workflow Editor is always one click away. Both editors read and write the same workflow format. When to Use the Chain Editor Use the Chain Editor when… Use the Workflow Editor when… Pipeline is mostly linear Pipeline branches or has loops You’re guiding a non-technical user You need fine-grained connection control You want one output per step You need multi-output nodes You’re sketching a workflow quickly You’re building a production agent Opening the Chain Editor Navigate to /chain/:workflowId in the app, or click the Chain layout toggle from a workflow’s context menu. If no workflow is provided, a fresh chain starts. Anatomy of a Chain A chain has three elements: Cards — each card represents a node (the step’s computation). Input mapping — a selector that chooses which output of the previous card becomes this card’s input. Output selector — a selector that decides which output is surfaced to the next card. Adding Steps Click the Add Node button at the end of the chain. The Node Picker Dialog opens — it’s filtered to show only nodes compatible with the current output type. Pick a node and it’s inserted at the end. The editor automatically wires a single input from the previous step. Editing Card Properties Click any card to expand its properties. You get the same inspector fields as the graph editor — string inputs, model pickers, sliders, asset selectors, and so on. Changes are saved the moment you commit a value (blur or press Enter). Reordering and Removing Cards Drag a card’s handle to reorder. Connections are re-mapped automatically. Delete with the trash icon on a card, or press Delete when a card is focused. Incompatible connections are highlighted in red and the editor suggests a compatible replacement. Input / Output Mapping Each card has: Input Mapping Selector — pick which output of the previous card becomes this card’s input. This is useful when a previous step returns multiple values (e.g., text, metadata). Output Selector — if this card has multiple outputs, pick the one that should flow into the next step. Both selectors show the data type, so you can spot mismatches quickly. Running a Chain Press Run at the top of the chain editor. Cards execute in order: A card running turns blue. A completed card gets a green checkmark and its output snippet appears. A failed card turns red and the chain halts. Streaming output appears inline on the card in real time. Switching Between Editors Chains are just a presentation of the underlying workflow graph — you can freely switch: From Chain → Graph: click the graph icon in the toolbar. Any non-linear parts (branches, multiple outputs) are preserved. From Graph → Chain: the chain view only renders the longest linear path; branches become inline cards. No data is lost when switching views. Mobile Chain Editor The chain layout is the default on the mobile app. See the screenshot and docs under Mobile App → Mobile Graph Editor. Next Steps Workflow Editor — the full graph editor Cookbook — linear patterns that work great in the chain editor Mobile App — running chains on iOS and Android--- # Chat API Source: https://docs.nodetool.ai/chat-api.html NodeTool provides both OpenAI-compatible HTTP endpoints and a WebSocket endpoint for chat interactions: The single server (nodetool serve, defaults to port 7777) exposes OpenAI-compatible HTTP endpoints (/v1/chat/completions, /v1/models) for remote clients. The same server exposes a WebSocket endpoint at /ws that carries both chat and workflow messages. See the canonical matrix in API Reference for methods, auth requirements, and streaming behavior. Port Reference: Both development and production deployments use port 7777 by default. Replace localhost with your server hostname for remote connections. OpenAI-Compatible HTTP API NodeTool exposes OpenAI-compatible endpoints that allow you to use standard OpenAI client libraries and tools. When AUTH_PROVIDER is static or supabase, send Authorization: Bearer <token>; in local/none modes the token is optional for development. Chat Completions: POST /v1/chat/completions URL: http://localhost:7777/v1/chat/completions Headers: Content-Type: application/json Authorization: Bearer YOUR_TOKEN Request Body: { "model": "gpt-4", "messages": [ {"role": "user", "content": "Hello, how are you?"} ], "stream": true } Example using OpenAI Python client: import openai # For local development: base_url="http://localhost:7777/v1" # For production/server: base_url="http://localhost:7777/v1" or your server URL client = openai.OpenAI( api_key="YOUR_TOKEN", base_url="http://localhost:7777/v1" ) response = client.chat.completions.create( model="gpt-4", messages=[ {"role": "user", "content": "Hello, how are you?"} ], stream=True ) for chunk in response: if chunk.choices[0].delta.content is not None: print(chunk.choices[0].delta.content, end="") Models: GET /v1/models URL: http://localhost:7777/v1/models List all available models for the configured provider. # For local development and production: http://localhost:7777/v1/models curl http://localhost:7777/v1/models \ -H "Authorization: Bearer YOUR_TOKEN" WebSocket API NodeTool also exposes a /ws WebSocket endpoint for real time conversations. The server side is implemented by UnifiedWebSocketRunner which handles message parsing, tool execution and streaming responses. The connection supports both binary (MessagePack) and text (JSON) messages. Authentication can be provided via Authorization: Bearer <token> headers or an api_key query parameter. Note: The WebSocket endpoint is served on port 7777 (development) or your configured server port. WebSocket Example usage // For local development: ws://localhost:7777/ws // For production deployments, use your server URL const socket = new WebSocket("ws://localhost:7777/ws?api_key=YOUR_KEY"); // Send a chat message const message = { role: "user", content: "Hello world", model: "gpt-3.5-turbo" // or any supported model }; socket.onmessage = async (event) => { const data = msgpack.decode(new Uint8Array(await event.data.arrayBuffer())); if (data.type === "chunk") { console.log(data.content); } }; socket.onopen = () => { socket.send(msgpack.encode(message)); }; Server responses Responses from the server may include: chunk – streamed text from the model tool_call – a request to execute a tool tool_result – the result of a tool execution job_update – status updates when running a workflow error – error messages The runner also supports workflow execution by sending a message with a workflow_id. In that case the WebSocket will stream job updates in addition to regular chat responses.--- # Chat CLI Source: https://docs.nodetool.ai/chat-cli.html The nodetool chat command starts an interactive terminal interface for conversing with language models and running tools. Every chat session runs the unified agent loop — the assistant can call tools and decompose work on its own; there is no separate mode to toggle. Starting the Interface Run nodetool chat from your shell. The current provider and model are shown in the status bar at the bottom. Type /help at any time to toggle the list of available commands. # Start with saved / auto-detected settings nodetool chat # Override provider and model nodetool chat --provider anthropic --model claude-sonnet-4-6 # Set the workspace directory (defaults to the current directory) nodetool chat --workspace /path/to/project # Connect to a running server instead of a local provider nodetool chat --url ws://localhost:7777/ws The --agent flag is deprecated and has no effect — the unified chat agent always runs. Slash Commands Commands use a / prefix. Tab completes commands and arguments (provider names, model ids). /help — Toggle the list of available commands. /new — Start a new chat session (clears history and starts a fresh thread). /clear — Clear the conversation history. /compact [instructions] — Summarize the conversation into a retained context message. Optional instructions focus the summary. /model <model-id> — Set the model. /provider <name> — Set the provider (switches to that provider’s default model). /tools — List the enabled tools. /exit — Exit the chat session. /quit — Exit the chat session. Press ↑/↓ to navigate input history, Tab to complete, and Esc or Ctrl+C to cancel a streaming response (or exit when idle). Tools The assistant runs with a set of enabled tools (file operations, web search, browser, code execution, NodeTool MCP tools, and more). Tools are auto-enabled based on the API keys available in your environment or the encrypted secret store. Use --tools to override the set explicitly, or /tools to see what is currently enabled. nodetool chat --tools read_file,write_file,grep,run_code Workspace File operations run against a workspace directory. The workspace defaults to the current working directory; override it with --workspace <path>. Settings and Logs Persisted settings (provider, model, enabled tools) are stored in ~/.nodetool/chat-settings.json. A log file is written to ~/.nodetool/chat.log so logging does not interfere with the terminal UI. Your provider and model choices are saved automatically as you change them, so they carry over between runs.--- # Serving Chat (OpenAI-Compatible API) Source: https://docs.nodetool.ai/chat-server.html There is no separate chat-server command. To serve chat over HTTP, run the NodeTool server with nodetool serve. It listens on 127.0.0.1:7777 by default and exposes an OpenAI-compatible chat API alongside the WebSocket endpoint. Quick Start # Start the server (default 127.0.0.1:7777) nodetool serve # Bind all interfaces on a custom port nodetool serve --host 0.0.0.0 --port 8080 The server provides: POST /v1/chat/completions — OpenAI-compatible chat completions (streaming and non-streaming). GET /v1/models — OpenAI-compatible model list. /ws — the WebSocket endpoint used by the editor and nodetool chat --url. Chat Completions: POST /v1/chat/completions URL: http://localhost:7777/v1/chat/completions Request: curl http://localhost:7777/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}] }' Set "stream": true to receive Server-Sent Events with OpenAI-compatible chat.completion.chunk payloads, terminated by data: [DONE]. Models: GET /v1/models curl http://localhost:7777/v1/models Integration Example (JavaScript) const response = await fetch("http://localhost:7777/v1/chat/completions", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ model: "gpt-4o", messages: [{ role: "user", content: "Hello, AI!" }], stream: true }) }); Because the endpoint is OpenAI-compatible, any OpenAI client SDK works by pointing its base URL at http://localhost:7777/v1. See Also Chat API — Full request/response schemas, streaming, and tool calls. NodeTool CLI — nodetool serve options. Deployment Guide — Running the server in production.--- # Chat CLI Overview Source: https://docs.nodetool.ai/chat.html Interactive CLI for chatting with models. Works with OpenAI, Anthropic, Gemini, Ollama, and others. The assistant can call tools during a session. Interactive REPL with history and completion Sandboxed workspace for file operations Provider-specific models Unified agent loop — the assistant calls tools and decomposes work on its own; no separate mode to toggle See the Chat CLI and Chat API.--- # Chrome Extension Source: https://docs.nodetool.ai/chrome-extension.html Let NodeTool workflows control your actual Chrome browser — the one you’re already logged into — instead of a fresh, anonymous, headless one. The NodeTool Chrome Extension is a thin relay that hands the Chrome DevTools Protocol (CDP) from your browser to your NodeTool server, so a workflow node can click, type, scroll, and screenshot inside a real tab with your cookies, sessions, and 2FA already in place. New here? Start with Getting Started. For headless (no-login-needed) browsing, see the regular Browser node instead — you only need the extension when a site requires your logged-in session. Overview Feature Notes What it is A Manifest V3 extension that proxies CDP commands between your server and chrome.debugger What it is not Not an automation engine itself — it originates no commands, it only relays them Why you’d use it Sites that block headless/server-launched browsers, or require your existing login (OAuth, 2FA, CAPTCHAs already solved) Transport Dedicated WebSocket at /ws/extension on your NodeTool server, JSON frames Scope One tab at a time, attached explicitly by clicking a button — never automatic Use Case Many AI product sites — Midjourney, Sora, Runway, ElevenLabs, and similar generation tools — either block headless Chrome outright or require a logged-in account with 2FA and session cookies that a fresh, server-launched browser doesn’t have. Re-implementing every such site as a dedicated API integration is slow and brittle to UI changes. The Chrome Extension solves this by reusing the browser you already use every day. You attach the extension to a tab where you’re already signed in, and a workflow’s browser-automation node drives that tab through the same action loop (click, type, scroll, extract, screenshot) it would use against a headless browser — just over a different transport. Typical scenario: You’re signed into Midjourney in a regular Chrome tab. You attach the extension to that tab, then run a workflow that submits a prompt, waits for the image grid to render, and downloads the results — all inside your real session, with no API key or cookie-jar wrangling. Recipe: Generate an Image on a Logged-In Site Install the extension (see Installing below) and pin it to your toolbar. Sign in to the target site (e.g. Midjourney) in a normal Chrome tab, as you would manually. Start your NodeTool server: nodetool serve --port 7777 (the extension talks to it over ws://localhost:7777/ws/extension by default). Open the extension popup on the signed-in tab and click Attach to this tab. Chrome shows its standard “Nodetool is debugging this browser” banner while attached. Build a workflow using a live-browser node configured to reuse the extension transport (set NODETOOL_BROWSER_TRANSPORT=extension on the server, or point it at your extension via NODETOOL_EXTENSION_WS_URL — see Transport Selection). Run the workflow. The node’s action loop drives your attached tab: it types the prompt, submits the form, waits for the result, and can capture a screenshot or download the generated asset. Detach from the popup (or just close the tab) when you’re done — attaching is never automatic and only lasts for the current browser session. Because the site sees your real, logged-in browser rather than a bot-like headless process, you avoid the CAPTCHAs, bot-detection blocks, and re-authentication flows that a server-launched browser would hit. Installing The extension is not published to the Chrome Web Store — build and load it unpacked from source. git clone https://github.com/nodetool-ai/nodetool.git cd nodetool/chrome-extension npm install # standalone install — the extension lives outside the root npm workspace npm run build # vite build -> dist/ Then load it into Chrome: Open chrome://extensions. Enable Developer mode (top right). Click Load unpacked and select chrome-extension/dist. The extension icon appears in your toolbar. Click it to open the popup, which shows a connection-status dot (disconnected / connecting / connected / error), an editable server WebSocket URL, and Attach / Detach buttons. Standalone package: chrome-extension/ is intentionally outside the root npm workspace, Turbo pipeline, and CI — it has its own package.json and is built on demand. npm run build:packages at the repo root does not build it. Development commands cd chrome-extension npm run typecheck # tsc --noEmit npm run dev # vite build --watch, for iterating on the extension itself npm run clean # remove dist/ Configuring the server URL By default the extension connects to ws://localhost:7777/ws/extension — your local NodeTool server. To point it at a different server (a remote deployment, a different port), open the popup, edit the Server URL field, and click Save. The value persists in the extension’s local storage across restarts. How It Works The extension is deliberately “dumb” — a pure conduit with no CDP logic of its own. All browser-automation semantics (which elements to click, how to wait for a page to settle, the action loop) live on the NodeTool server; the extension just carries the bytes. NodeTool server (browser-automation node) │ CDP commands/events, JSON frames ▼ ws://<server>/ws/extension ▲ │ chrome.debugger.sendCommand / onEvent Chrome Extension (service worker) │ ▼ Your real, logged-in Chrome tab Background service worker (src/background/service-worker.ts) owns the relay. It restarts the relay on chrome.runtime.onInstalled/onStartup because Manifest V3 service workers get evicted and need to reconnect, and it uses a chrome.alarms keepalive (roughly every 24 seconds) to stop the worker idling out while a debugger session is attached. CDP relay (src/lib/cdp-relay.ts) maintains the WebSocket to your server with exponential backoff (1–30s) if the connection drops, and answers server heartbeat pings (ping/pong, ~15s). Explicit attach only: chrome.debugger.attach is only ever called from a user click in the popup — never automatically on page load or on a server request. Attaching is mutually exclusive with having Chrome DevTools open on the same tab. Wire protocol: JSON text frames (not the MsgPack used by NodeTool’s main /ws chat/workflow channel), with frame kinds cdp / cdp_result / cdp_event (command/response/event relay), attach / attached / detach (session lifecycle), ping / pong (heartbeat), error (fatal — e.g. the user closed the tab or DevTools banner), asset_chunk (server → extension, for injecting a file upload), and media_chunk / media_end (extension → server, for chrome.downloads-based capture when a site blocks direct clipboard/screenshot access). Transport Selection Browser-automation nodes on the server choose between a headless local browser and the extension relay via an environment variable: # Use the extension-attached browser instead of a headless one NODETOOL_BROWSER_TRANSPORT=extension nodetool serve # Or implicitly, by pointing at a specific extension WebSocket URL NODETOOL_EXTENSION_WS_URL=ws://localhost:7777/ws/extension nodetool serve When the extension transport is selected but no extension is currently attached, the node fails fast with a clear message rather than hanging: “No browser extension is connected to /ws/extension — attach will time out. Install the extension and click ‘Attach to this tab’.” Under the hood, the same action loop, element indexing, and screenshot logic used by the headless browser nodes runs unchanged against the extension transport — only the underlying CDP client differs, so workflows built for headless browsing don’t need to change their node logic to benefit from the logged-in session. Server Requirements A running NodeTool server: nodetool serve --port 7777 (or your deployed URL). The /ws/extension route is part of the standard NodeTool WebSocket server — no extra server-side setup is required beyond running the server. The extension and the server must be able to reach each other over the configured WebSocket URL (typically localhost for local development). Limitations One tab at a time. The extension proxies a single chrome.debugger session; attach a different tab and the previous one is implicitly detached. Not published to the Chrome Web Store. Install as an unpacked extension from source. Mutually exclusive with DevTools. You can’t have Chrome DevTools open on a tab while the extension is attached to it (both use chrome.debugger). Session-only. Attaching does not persist across Chrome restarts — reattach after restarting your browser. Manual build, not CI-gated. The chrome-extension/ package isn’t part of npm run build:packages or the repository’s CI checks; keep the two protocol definitions (chrome-extension/src/lib/protocol.ts and packages/automation-nodes/src/lib/extension-protocol.ts) in sync by hand if you change the wire format. Troubleshooting Popup shows “disconnected” Confirm your NodeTool server is running and the Server URL in the popup matches it (default ws://localhost:7777/ws/extension). Check that nothing else is bound to port 7777, and that firewall rules allow the local WebSocket connection. “No browser extension is connected” error from a workflow The extension must be attached to a tab before running a workflow that uses the extension transport — attaching is never automatic. Open the popup and click Attach to this tab. Verify NODETOOL_BROWSER_TRANSPORT=extension (or NODETOOL_EXTENSION_WS_URL) is set on the server process running the workflow. Chrome shows “Nodetool is debugging this browser” and I can’t open DevTools This is expected while attached — chrome.debugger sessions and DevTools are mutually exclusive on the same tab. Detach from the popup, or use a different tab for DevTools. Attach fails or the tab appears frozen Close any open DevTools panel on that tab first, then retry Attach to this tab. If the tab was closed or navigated away unexpectedly, the relay reports a fatal error and tears down the session — reopen the target page and reattach. Related Topics Getting Started — Desktop setup and first workflow Developer: Node Reference — Browser-automation node reference API Reference — Server API and WebSocket documentation WebSocket API — General WebSocket protocol details--- # NodeTool CLI Source: https://docs.nodetool.ai/cli.html The nodetool CLI is the TypeScript command-line interface for the NodeTool platform. It manages servers, workflows, jobs, assets, and secrets. Run nodetool --help to see the top-level command list. Every sub-command exposes its own --help flag with detailed usage. Installation Install globally from npm to get the nodetool and nodetool-chat commands: npm install -g @nodetool-ai/cli Or run a single command without installing: npx --package=@nodetool-ai/cli nodetool --help npx --package=@nodetool-ai/cli nodetool-chat --agent Requires Node.js 22.x. Check with node --version; install via nvm if needed. Getting Help nodetool --help — list all top-level commands. nodetool <command> --help — show command-specific options (e.g. nodetool serve --help). nodetool <group> --help — list sub-commands for grouped tooling (e.g. nodetool workflows --help). Global Options These flags work on any nodetool command and control OpenTelemetry tracing: --trace-file <path> — append every LLM/agent/workflow span to <path> as JSONL (analyzer-friendly). --trace-stdout [format] — stream spans to stdout: pretty (default) or json. --no-trace-stdout — disable stdout span output (overrides NODETOOL_TRACE_STDOUT). nodetool --trace-file trace.jsonl run workflow.ts nodetool --trace-stdout pretty workflows run <id> Core Commands nodetool info Display system and environment information including Node.js version, platform, and API key configuration. Options: --json — output as JSON. Example: nodetool info nodetool info --json nodetool serve Starts the TypeScript WebSocket + HTTP backend server. This serves the REST API, WebSocket endpoints, and static assets. Options: --host (default 127.0.0.1) — bind address (use 0.0.0.0 for all interfaces). --port (default 7777) — listen port. Examples: # Start the server on the default port nodetool serve # Bind to all interfaces on a custom port nodetool serve --host 0.0.0.0 --port 8080 You can also set the bind address and port via environment variables: PORT=8080 HOST=0.0.0.0 nodetool serve nodetool workflows run <workflow_id_or_file> Executes a workflow by ID (from the local database), JSON file, or TypeScript DSL file. Arguments: <workflow_id_or_file> — workflow ID, path to a .json workflow file, or path to a .ts DSL file. Options: --params <json> — JSON string of workflow parameters. --json — output result as JSON. Examples: # Run workflow by ID nodetool workflows run workflow_abc123 # Run workflow from JSON file nodetool workflows run ./my_workflow.json # Run workflow from TypeScript DSL nodetool workflows run ./my_workflow.ts # Run with parameters as JSON nodetool workflows run workflow_abc123 --params '{"input": "hello"}' # JSON output for automation nodetool workflows run ./my_workflow.json --json nodetool workflows export-dsl <workflow_id_or_file> Exports a workflow as a TypeScript DSL file. Arguments: <workflow_id_or_file> — workflow ID or path to a .json workflow file. Options: -o, --output <file> — write to file instead of stdout. Examples: # Print DSL to stdout nodetool workflows export-dsl workflow_abc123 # Write to file nodetool workflows export-dsl workflow_abc123 -o workflow.ts # Export from JSON file nodetool workflows export-dsl ./my_workflow.json nodetool run <dsl-file> Shorthand for running a TypeScript DSL workflow file directly. Options: --json — output results as JSON. Examples: nodetool run workflow.ts nodetool run workflow.ts --json Database Migrations nodetool db migrate Applies NodeTool migrations to a PostgreSQL/Supabase database. For Supabase, use the direct connection URL from Settings → Database (port 5432), not the transaction pooler URL. Options: --direct-url <url> — Supabase/PostgreSQL direct connection URL. --database-url <url> — connection URL; defaults to DIRECT_URL or DATABASE_URL. --target <version> — stop after a specific migration version. --dry-run — show pending migrations without applying them. --skip-checksums — skip checksum validation. --json — output as JSON. Examples: DIRECT_URL="postgresql://postgres:[password]@db.[project].supabase.co:5432/postgres" \ nodetool db migrate nodetool db status --direct-url "$DIRECT_URL" nodetool db migrate --direct-url "$DIRECT_URL" --dry-run Other migration commands: nodetool db status --direct-url "$DIRECT_URL" nodetool db baseline --direct-url "$DIRECT_URL" # for existing DBs nodetool db rollback --direct-url "$DIRECT_URL" --steps 1 Chat nodetool chat Starts an interactive TUI chat session. Options: -p, --provider <provider> — LLM provider (e.g., anthropic, openai, ollama). -m, --model <model> — model ID. -a, --agent — deprecated, no-op. Every chat session runs the unified agent loop; this flag has no effect. -u, --url <url> — WebSocket server URL (default: uses a local provider). -w, --workspace <path> — workspace directory for file operations (default: current directory). --tools <tools> — comma-separated list of enabled tools. Examples: # Start interactive chat nodetool chat # Chat with a specific provider and model nodetool chat --provider anthropic --model claude-sonnet-4-6 # Connect to a running server nodetool chat --url ws://localhost:7777/ws Workflow Management nodetool workflows Manage workflows via the API. Subcommands: list, get, run, export-dsl, export-example, export-bundle, import-bundle # List all workflows nodetool workflows list nodetool workflows list --api-url http://localhost:7777 --json # Get a workflow by ID nodetool workflows get <workflow_id> # Run a workflow (see above for full options) nodetool workflows run <workflow_id_or_file> # Export as a TypeScript DSL file (see above) nodetool workflows export-dsl <workflow_id_or_file> nodetool workflows export-example <workflow_id_or_file> Export a workflow as a shipped template: materialize its referenced assets into the package’s constant asset directory (rewriting refs to package://<pkg>/<file>) and write the example JSON. Options: --package <name> — owning package (default nodetool-base). -o, --output <file> — write the example JSON to this exact path. --include-remote — also materialize http(s) and local-file refs. nodetool workflows export-example <workflow_id> nodetool workflows export-example <id> --package nodetool-base nodetool workflows export-example workflow.json -o example.json nodetool workflows export-bundle <workflow_id_or_file...> Export one or more workflows as a portable .nodetool bundle (a zip containing the graphs plus the bytes of every asset they reference), sharable as a single file. Options: -o, --output <file> — output path (default <name>.nodetool). --include-remote — also embed http(s) and local-file refs. nodetool workflows export-bundle <id> [<id2> ...] -o my-pack.nodetool nodetool workflows import-bundle <bundle_file> Import a .nodetool bundle into the local library: store its assets and create the workflows with refs rewritten to the imported assets. nodetool workflows import-bundle my-pack.nodetool Job Management nodetool jobs Query job status and results. Subcommands: list, get Options: --api-url <url> — API base URL (default: http://localhost:7777). --workflow-id <id> — filter by workflow ID (for list). --limit <n> — max results (default: 100). --json — output as JSON. Examples: # List all jobs nodetool jobs list # Filter by workflow nodetool jobs list --workflow-id workflow_abc123 # Get a specific job nodetool jobs get <job_id> Asset Management nodetool assets Manage uploaded files and workflow assets. Subcommands: list, get Options: --api-url <url> — API base URL (default: http://localhost:7777). --query <q> — search query (for list). --content-type <type> — filter by content type (for list). --limit <n> — max results (default: 100). --json — output as JSON. Examples: # List assets nodetool assets list # Search assets nodetool assets list --query "landscape" # Get a specific asset nodetool assets get <asset_id> Secrets Management nodetool secrets Manage encrypted secrets stored in the local database with per-user encryption. Subcommands: list, store, get Examples: # List stored secret keys nodetool secrets list # Store a secret (prompts for value) nodetool secrets store OPENAI_API_KEY # Retrieve a secret value nodetool secrets get OPENAI_API_KEY Settings nodetool settings show Display current settings from environment variables. Options: --json — output as JSON. Example: nodetool settings show nodetool settings show --json Model Management nodetool models List models and providers via the API. Subcommands: list — list all models (recommended + provider + HuggingFace cached). providers — list configured providers and their capabilities. recommended — list recommended models. ollama — list Ollama models. huggingface — list HuggingFace cached models (--query, --type to filter). by-provider <provider> — list models for a provider; --kind one of llm, image, tts, asr, video, embedding (default llm). Examples: nodetool models list nodetool models providers nodetool models ollama nodetool models by-provider openai --kind image MCP Integration nodetool mcp Install, remove, or inspect the NodeTool MCP server configuration for AI coding assistants (Claude Code, Codex, OpenCode). Subcommands: install, uninstall, status Examples: # Install for all detected assistants (default URL http://127.0.0.1:7777/mcp) nodetool mcp install # Install for Claude Code only, with a custom URL nodetool mcp install --claude --url http://127.0.0.1:7777/mcp # Show installation status nodetool mcp status # Remove from all assistants nodetool mcp uninstall The NodeTool server must be running (nodetool serve) for MCP to work. Agents nodetool agent Run YAML-defined autonomous agents from the command line. Subcommands: run, test, list # Run an agent with an objective nodetool agent run agent.yaml --objective "Research AI trends" # Validate a config nodetool agent test agent.yaml # List configs in a directory nodetool agent list examples/agents/ See the Agent CLI reference for full details. The nodetool db group (migrate, status, baseline, rollback) is documented under Database Migrations above. Tips Use --json flags for machine-readable output suitable for scripting. Set NODETOOL_API_URL environment variable to avoid specifying --api-url on every command. Use nodetool serve to start the local backend server before running API commands. See Environment Variables for a complete list of configurable variables.--- # Collections Source: https://docs.nodetool.ai/collections.html Collections bundle related documents into a single searchable unit. Connect a collection to an index node and any file in it — PDFs, Markdown notes, HTML, transcripts — becomes queryable from your workflows. Opening Collections Navigate to Collections from the left sidebar, or go directly to /collections. The explorer shows every collection you’ve created with counts of documents and the last time they were indexed. Creating a Collection Click New Collection in the top-right. Give it a name and an optional description. (Optional) Associate a default embedding model. Drag documents into the collection — or import a folder from the Asset Explorer. Collections accept any file type, but only text-extractable formats are indexed: Format Extracted PDF Text + layout (per page) DOCX / DOC Body text Markdown / TXT Raw text HTML Stripped body text CSV / TSV Rows as records EPUB Chapter text Unsupported formats are stored but not indexed — still handy for reference inside a collection. Managing Documents Click a collection tile to open its details. You can: Add documents by drag-and-drop or the Upload button. Remove documents from the collection (doesn’t delete the underlying asset). Re-index after adding new documents or changing the embedding model. Preview any document inline with the built-in viewer. Using Collections in Workflows Collections shine in RAG pipelines: Add an IndexDocuments or HybridSearch node to your workflow. Connect the collection to its collection input — the node menu will suggest the selector. Run the workflow. The first run indexes the collection (embeddings + keyword index); subsequent runs reuse the index. See the full pattern in the Cookbook → RAG. Indexing Options By default a collection uses the embedding model set in Settings → Default Models. Override per-collection from its settings: Embedding model — any embedding model from HuggingFace, OpenAI, Gemini, Cohere. Chunk size — tokens per passage (default: 512). Overlap — tokens shared between chunks (default: 64). Hybrid — enable BM25 alongside vector search for better recall on proper nouns. See Indexing for deeper tuning notes. Storage Index data is stored alongside your workflow database in SQLite (via sqlite-vec). Nothing leaves your machine unless you’ve opted into a cloud provider for the embedding model. For multi-user deployments, Supabase-backed collections are an option — see Supabase Deployment. Related Docs Indexing — advanced chunking, hybrid search, maintenance Asset Management — add documents to the underlying asset library Cookbook → RAG — wire a collection into a workflow Chat with Docs example — end-to-end RAG workflow--- # Why NodeTool: Comparisons vs ComfyUI, n8n & More Source: https://docs.nodetool.ai/comparisons.html NodeTool is the open creative AI workspace — every major model, your keys, one canvas. NodeTool replaces the chatbox with a node canvas where image, video, audio, and LLM models run side by side. Bring your keys, or run everything locally. Open source, AGPL-3.0. Head-to-head comparisons Full write-ups against specific tools: NodeTool vs ComfyUI — a diffusion-only node editor vs one canvas for image, video, audio, and text, with editing tools built in. NodeTool vs Dify — a text-first LLM app platform vs the same agent and RAG ground plus native image, video, and music generation. NodeTool vs Flowise — the fastest path to a LangChain RAG chatbot vs that chatbot plus native media generation on the same canvas. NodeTool vs Langflow — a low-code builder for chat, RAG, and agents vs the same ground plus native image, video, and music generation. NodeTool vs n8n — app-to-app automation vs workflows built to generate media and run agents. NodeTool vs Weavy — SaaS credits and a curated model roster vs open source, BYOK, no lock-in. NodeTool vs Figma Weave — Weavy’s new life inside Figma: hosted, credit-billed, ecosystem-bound vs open source and BYOK. The problem Creative AI is a tab graveyard: Every model lives behind its own UI. Switching between Flux, ElevenLabs, Sora, and a chatbot means losing context every time. Outputs don’t compose. SaaS canvases tax every token. Hosted creative tools mark up provider credits 2–5x. You pay for the same OpenAI call twice. Local-only tools are model-narrow. ComfyUI nails diffusion internals but treats LLMs, agents, and cloud APIs as second-class. Workflows aren’t portable. Prompts in chat history, settings in screenshots, pipelines in your head. Nothing is reproducible. Privacy is a yes/no toggle. Either everything goes to a vendor, or nothing leaves your machine. No mixed mode. How NodeTool solves this One canvas, every model, your keys: One canvas for every modality. Wire Flux to GPT-5 to ElevenLabs to Wan in a single graph. Outputs flow as typed edges — image, audio, text, embeddings — not pasted strings. BYOK to every provider. OpenAI, Anthropic, Gemini, Replicate, FAL, Kie, ElevenLabs, MiniMax, HuggingFace. Pay them directly. No credit markup, no provider lock-in. Local + cloud, mixed. Run Llama on MLX, route image gen to FAL, send audio to ElevenLabs — in one workflow. Toggle per node. Workflows as files. Save, share, version. Ship a workflow as a Mini-App with a one-click hide-the-graph mode. Agents that drive pipelines. Built-in planning, tool calling, streaming. Drop an agent node into any graph. Open source, no markup. AGPL-3.0. Cloud edition hosts the same code in this repo. Self-host the Docker images any time. Feature Comparison Feature NodeTool Figma Weave (formerly Weavy) ComfyUI Category Open creative AI workspace Closed SaaS creative canvas Diffusion-focused node editor License AGPL-3.0 (open source) Proprietary SaaS GPL-3.0 (open source) Runs on your machine ✅ Mac, Windows, Linux desktop ❌ Browser-only, hosted ✅ Local-first Bring your own keys (BYOK) ✅ FAL, Kie, OpenAI, Anthropic, Gemini, Replicate, Atlas, ElevenLabs, MiniMax ❌ Credits only, provider markup ⚠️ Via custom nodes for cloud APIs Pricing model Pay providers directly, no markup Proprietary credits Free (you pay your own GPU/API) Model coverage Image, video, audio, text, TTS, ASR — local + cloud Image, video, audio — cloud only Diffusion (image/video) — local Image generation Local: FLUX, Qwen Image · API: FAL, Kie, Replicate, OpenAI, Gemini Cloud: FLUX, Seedance, Ideogram, etc. Deep control over diffusion internals Video generation Local: Wan · API: FAL, Kie, Sora, Veo, Kling Cloud: Kling, Veo, Runway, etc. Local diffusion video (AnimateDiff, etc.) Audio & music Local: MusicGen, AudioLDM, Stable Audio · API: Kie, ElevenLabs, MiniMax Cloud: Suno, ElevenLabs, etc. ⚠️ Via custom nodes TTS / ASR Local: Kokoro, Sesame, Whisper · API: OpenAI, ElevenLabs Cloud only ⚠️ Via custom nodes LLMs & agents Built-in agent nodes, tool calling, streaming, Ollama, MLX Limited LLM nodes ⚠️ Via custom nodes Diffusion control Standard parameters ❌ Hidden behind presets ✅ Latents, VAE, samplers, ControlNet RAG / vector search ✅ Local SQLite-vec, plus Pinecone & Supabase pgvector ❌ ❌ Mini-apps from workflows ✅ Turn a graph into a simple UI ⚠️ Share-as-template ❌ Real-time streaming ✅ Token-by-token, live progress ✅ Live preview ❌ Queue-based execution Source available ✅ Full source on GitHub ❌ ✅ Full source on GitHub When to pick each NodeTool — every modality, every provider, on one canvas. Local, cloud, or mixed. Figma Weave (formerly Weavy) — hosted SaaS if you want a managed product with credits inside the Figma ecosystem and don’t need BYOK, local execution, or open source. ComfyUI — deep diffusion control: samplers, VAE, ControlNet, latents. Next steps Quick Start — install and run your first workflow in minutes. Models & Providers — every model NodeTool wires up.--- # Configuration Guide Source: https://docs.nodetool.ai/configuration.html NodeTool uses a layered configuration system so local development, automated deployments, and production environments can share sensible defaults with minimal duplication. Settings come from environment variables and .env files, while secrets are stored encrypted at rest in a local SQLite database (managed via the CLI). The configuration helpers live in the @nodetool-ai/config package (environment.ts). They are plain functions — loadEnvironment(), getEnv(), and requireEnv() — there is no Environment class and no settings.yaml/secrets.yaml loading. Configuration Layers loadEnvironment() loads .env files in this order, with later files overriding earlier ones; system environment variables always win over file values: .env .env.<NODE_ENV> .env.<NODE_ENV>.local NODE_ENV defaults to development when unset. Use getEnv("KEY") to read a value (returns undefined when unset) and requireEnv("KEY") to read a value that must be present (throws a descriptive error otherwise). This hierarchy allows committed defaults, per-environment overrides, and developer-specific overrides to co-exist without file conflicts. Managing Settings & Secrets The in-app Settings dialog is the easiest way to manage everything. It has a sidebar with subsections: Section What it covers General Theme, startup behavior, language Providers API keys for OpenAI, Anthropic, Google, etc. Default Models Pick the default LLM, image model, and embedding model Folders Workspace, cache, and asset directories Secrets Encrypted provider tokens and credentials Remote Point the app at a remote NodeTool server About Version and build info From the command line: nodetool settings show [--json] – print the resolved environment configuration. nodetool secrets list – list stored secret keys (values are never shown). nodetool secrets store <key> – store or update a secret (prompts for the value). nodetool secrets get <key> – print a stored secret value. Secrets are encrypted and persisted in a local SQLite database, not in YAML files. There is no nodetool settings edit command and no --secrets option. Secret Storage and Master Key Secrets saved through the CLI are encrypted with AES-256-GCM, using a per-user key derived from the master key via PBKDF2-SHA256 (100,000 iterations). initMasterKey() in @nodetool-ai/security resolves the master key in this order: SECRETS_MASTER_KEY environment variable. Local system keyring (macOS Keychain, Windows Credential Manager, or Secret Service via keytar). Generates a new key and persists it to the keyring. For shared deployments you must pre-provision the master key (via the SECRETS_MASTER_KEY environment variable) so every server can decrypt secrets generated locally. On a headless host with no keychain and no provisioned key, master-key initialization (and therefore startup) fails because there is no place to persist a generated key. Migrating Secrets to a Server Export the master key once and set it on every server instance using the value from your deployment pipeline or secrets manager: export SECRETS_MASTER_KEY="<your-base64-master-key>" The nodetool deploy apply command automatically synchronizes all secrets from your local database to the target server right after a successful deploy. If you ever need to do it manually, POST the encrypted payload to the admin endpoint using the worker bearer token: curl -H "Authorization: Bearer $NODETOOL_WORKER_TOKEN" \ -H "Content-Type: application/json" \ -X POST https://your-server.example.com/admin/secrets/import \ --data-binary @secrets-export.json The server stores the ciphertext verbatim, so both sides must share the same master key. Storage Backend Selection The storage backend is selected explicitly by NODETOOL_STORAGE_BACKEND (one of file, s3, or supabase; defaults to file). It is not auto-detected from the presence of S3 or Supabase credentials. file (default) — assets are written to the local assets directory. s3 — requires ASSET_BUCKET (and, for the temp store, TEMP_BUCKET); reads S3_REGION and S3_ENDPOINT as needed. supabase — requires SUPABASE_URL, SUPABASE_KEY, and the relevant bucket var. The asset store uses ASSET_BUCKET; the temp store uses TEMP_BUCKET. See storage-config.ts in @nodetool-ai/config. Using Environment Variables in Code When adding a feature that reads configuration: Read the variable with getEnv("YOUR_ENV_VAR") from @nodetool-ai/config so .env load order is respected. If the value is required, use requireEnv("YOUR_ENV_VAR") so a missing key raises a descriptive error. Document the new entry in .env.example. Recommended Workflow cp .env.example .env.development.local vim .env.development.local # add OpenAI/Anthropic/HF tokens, S3 credentials, etc. nodetool secrets store OPENAI_API_KEY # encrypt a provider key into the secrets DB (optional) nodetool settings show # verify resolved configuration Use .env.<env>.local for machine-specific overrides and keep secrets out of version control. When deploying, provide environment variables via your orchestrator or the deployment.yaml env section—NodeTool will merge them automatically at runtime. Supabase Settings NodeTool integrates with Supabase for both user authentication and asset storage. Set the following to enable Supabase: NODETOOL_STORAGE_BACKEND=supabase – select the Supabase storage backend SUPABASE_URL – your project URL, e.g. https://<ref>.supabase.co SUPABASE_KEY – a service role key (server-side only) ASSET_BUCKET – Supabase Storage bucket used for assets (e.g. assets) TEMP_BUCKET – bucket for temporary assets (e.g. assets-temp) Use a separate Supabase for user-provided nodes: NODE_SUPABASE_URL – user/node project URL (kept distinct from core SUPABASE_URL) NODE_SUPABASE_KEY – service role key for user/node data (kept distinct from core SUPABASE_KEY) NODE_SUPABASE_SCHEMA – optional schema for node tables (defaults to public) NODE_SUPABASE_TABLE_PREFIX – optional prefix applied to node tables to avoid collisions with core tables Behavior: Storage is used for Supabase buckets only when NODETOOL_STORAGE_BACKEND=supabase; the backend is chosen explicitly, not auto-detected from the presence of credentials. Authentication enters Supabase mode (enforced auth) when both SUPABASE_URL and SUPABASE_KEY are set — storage and auth are configured independently. See Authentication. Security notes: Use the service role key only in server environments. Do not expose it to clients. Public buckets make generated URLs directly accessible. For private buckets, add a signing step. Environment Variables Index Variable Purpose Secret Notes NODE_ENV Environment name (development, test, production) no Defaults to development; controls .env file load order SUPABASE_URL / SUPABASE_KEY Enable Supabase auth mode (both required) SUPABASE_KEY When both are set, the server enforces auth and validates Supabase JWTs. See Authentication SERVER_AUTH_TOKEN Deploy-tooling bearer token (@nodetool-ai/deploy) yes Generated automatically if unset; not used by the websocket server’s auth mode selection NODETOOL_TRUST_LOCALHOST Allow loopback connections to bypass auth as user 1 no Defaults off when auth is enforced (Supabase), on otherwise. Leave off behind a reverse proxy/SSH tunnel where the proxy connects from loopback. NODETOOL_TRUST_LOCAL_NETWORKS ⚠️ Source CIDRs trusted as user 1 without a password (Local mode only) no Comma-separated IPs/CIDRs; ignored in Supabase mode. Needed so Docker’s NAT’d bridge traffic isn’t rejected — scope to the bridge (172.16.0.0/12), never 0.0.0.0/0 on a public IP. See Authentication → Local mode in Docker. NODETOOL_TRUSTED_PROXIES Reverse proxies whose X-Forwarded-For is trusted no Comma-separated IPs/CIDRs. When unset, X-Forwarded-For is ignored and the socket peer address is used. OPENAI_API_KEY / ANTHROPIC_API_KEY / GEMINI_API_KEY Provider access yes Set only the providers you use HF_TOKEN / FAL_API_KEY / REPLICATE_API_TOKEN HuggingFace-family providers yes Optional per workflow OLLAMA_API_URL Local Ollama base URL no Default http://127.0.0.1:11434 DB_PATH / DATABASE_URL Database connection no Set only one. DB_PATH configures SQLite; DATABASE_URL supports PostgreSQL (postgres://, postgresql://) and SQLite (file:, sqlite:) NODETOOL_STORAGE_BACKEND Storage backend (file, s3, supabase) no Default file. Selected explicitly — not auto-detected from credentials S3_* S3-compatible storage settings yes Includes access keys and region ASSET_BUCKET / TEMP_BUCKET Asset and temp buckets (s3 / supabase backends) no Use signed URLs for private buckets NODETOOL_VECTOR_PROVIDER / VECTORSTORE_DB_PATH Vector store config no Default backend is local SQLite-vec; switch to pinecone or supabase for remote. See Indexing. NODE_SUPABASE_URL / NODE_SUPABASE_KEY / NODE_SUPABASE_SCHEMA / NODE_SUPABASE_TABLE_PREFIX User/node Supabase config NODE_SUPABASE_KEY Kept separate from core Supabase credentials and tables NODETOOL_RATE_LIMIT_DISABLED Disable per-IP HTTP rate limiting no Limiter is on by default; localhost is always exempt NODETOOL_RATE_LIMIT_MAX Max HTTP requests per window per IP no Default 1000 NODETOOL_RATE_LIMIT_WINDOW_MS Rate-limit window length (ms) no Default 60000 (1 minute) NODETOOL_RATE_LIMIT_TRUST_PROXY Key the limiter by X-Forwarded-For (req.ip) instead of the socket address no Enable only behind a trusted proxy that sets the header NODETOOL_WS_RATE_LIMIT_DISABLED Disable the per-connection WebSocket inbound message cap no Cap is on by default NODETOOL_WS_RATE_LIMIT_MAX Max inbound WS messages per window per connection no Default 200; over-cap clients are closed with code 1008 NODETOOL_WS_RATE_LIMIT_WINDOW_MS WebSocket rate-limit window length (ms) no Default 1000 (1 second) LOG_LEVEL / NODETOOL_LOG_LEVEL Logging level no Defaults to info (NODETOOL_LOG_LEVEL takes precedence) SECRETS_MASTER_KEY Master key for secret encryption yes See Secret Storage and Master Key RUNPOD_API_KEY RunPod deployments yes Used by CLI and providers NODETOOL_WORKER_TOKEN Worker bearer token for admin endpoints yes Rotate regularly Use nodetool settings show to view resolved values and verify the merge order. Related Documentation Storage Guide – how asset storage backends are selected. Deployment Guide – passing environment variables in deployment.yaml. CLI Reference – settings-related commands.--- # NodeTool Workflow Cookbook Source: https://docs.nodetool.ai/cookbook.html Reusable patterns for building on the NodeTool canvas. Start with a pattern that matches what you want to build, then adapt one of the worked examples. Cookbook Sections Core Concepts & Architecture — how graphs, typed edges, streaming, and node types work. Workflow Patterns — 13 patterns with diagrams and the nodes each one uses. Workflow Gallery — step-by-step example workflows you can open and run. New to the canvas? Read Core Concepts first, then follow a beginner example like Creative Story Ideas. Choose Your Pattern I want to… Pattern Key nodes Transform one input step by step 1 · Simple Pipeline ImageInput, UnsharpMask, AutoContrast Generate content with an LLM 2 · Agent-Driven Generation Agent, ListGenerator, TextToSpeech Show progress during a long run 3 · Streaming with Previews Agent, ListGenerator, Preview Answer questions about documents 4 · RAG HybridSearch, FormatText, Agent Store data persistently 5 · Database Persistence CreateTable, Insert, Query Process emails or web content 6 · Email & Web Integration GmailSearch, Template, Summarizer Convert between media types 7 · Multi-Modal Workflows Whisper, StableDiffusion, TextToSpeech Transform images with AI 8 · Advanced Image Processing StableDiffusionV3MediumImageToImage, ImageToText, Canny Generate video from text 9 · Text-to-Video KlingVideoV16ProTextToVideo, Sora2TextToVideo Animate an image into video 10 · Image-to-Video KlingVideoV16StandardImageToVideo, SeeDanceV15ProImageToVideo Create a lip-synced avatar 11 · Talking Avatar KlingVideoAiAvatarV2Pro, Infinitalk Upscale and clean up images 12 · Image Enhancement TopazUpscaleImage Turn keyframes into a video 13 · Storyboard to Video Sora2ImageToVideoPro Worked Examples Full walkthroughs with diagrams live in the Workflow Gallery. A few to start with: Creative Story Ideas — beginner tutorial for agent generation. Image Enhance — a simple sharpen-and-contrast pipeline. Movie Posters — multi-stage generation with strategy and previews. Image to Audio Story — picture in, narrated story out. Chat with Docs — RAG document Q&A. Build Any Example Press Space to open the node menu and add a node. Drag from an output handle to an input handle to connect nodes. Press Ctrl/⌘ + Enter to run. Add Preview nodes to inspect intermediate results.--- # Core Concepts & Architecture Source: https://docs.nodetool.ai/cookbook/core-concepts.html Core Concepts What is a NodeTool Workflow? A NodeTool workflow is a Graph where: Nodes represent operations (processing, generation, transformation) Edges represent data flow between nodes Execution follows dependency order automatically Input → Process → Transform → Output Key Principles Data flows through typed edges — connections enforce type compatibility (image → image, text → text). You cannot connect an image output to a text input. Dependency-driven execution — nodes execute automatically when all their inputs are ready. You never need to specify execution order manually. Streaming by default — many nodes produce output incrementally (token by token, frame by frame), enabling real-time feedback. Parallel when possible — independent branches of a workflow execute concurrently. NodeTool analyzes the graph to maximize parallelism. Node Types Type Purpose Examples Input Nodes Accept parameters StringInput, ImageInput, AudioInput Processing Nodes Transform data Resize, Filter, ExtractText Agent Nodes LLM-powered logic Agent, Summarizer, ListGenerator Output Nodes Return results Output, Preview Control Nodes Flow control Collect, FormatText Storage Nodes Persistence CreateTable, Insert, Query Data Types NodeTool enforces type safety on all connections. Here are the common data types: Type Description Example Nodes String Text data StringInput, FormatText, Agent Image Image data (PNG, JPEG, etc.) TextToImage, Resize, ImageInput Audio Audio data (WAV, MP3, etc.) AudioInput Video Video data Sora2TextToVideo, KlingVideoV16StandardImageToVideo List[T] A list of items of type T ListGenerator, Split, Collect Dict Key-value pairs ParseJSON, SaveJSON Number Integer or float IntegerInput, FloatInput Boolean True/false BooleanInput When connecting nodes, NodeTool validates types automatically. Some conversions happen implicitly (e.g., Image formats), while others require explicit conversion nodes. Streaming Architecture Why Streaming? NodeTool workflows support streaming execution for: Real-time feedback — see results as they’re generated Lower latency — start processing before all data arrives Better UX — progress indicators and incremental results Efficient memory — process large data in chunks Streaming Nodes Nodes that support streaming output: Node What It Streams Agent LLM responses token by token ListGenerator List items as they’re generated RealtimeAgent (openai.agents.RealtimeAgent) Audio + text responses simultaneously RealtimeTranscription (openai.agents.RealtimeTranscription) Transcription as audio arrives RealtimeAudioInput Audio from an input source Data Flow Patterns Pattern 1: Sequential Pipeline graph LR A[Input] --> B[Process] --> C[Transform] --> D[Output] Each node waits for the previous node to complete before starting. Pattern 2: Parallel Branches graph LR A[Input] --> B[Process A] A --> C[Process B] B --> D[Output A] C --> E[Output B] Independent branches execute in parallel, improving throughput. Pattern 3: Streaming Pipeline graph LR A[Input] --> B[Streaming Agent] B -->|yields chunks| C[Collect] C --> D[Output] Data flows in chunks, enabling real-time updates. The Collect node gathers all chunks into a single output. Pattern 4: Fan-In Pattern graph LR A[Source A] --> C[Combine] B[Source B] --> C C --> D[Process] --> E[Output] Multiple inputs combine before processing. The Combine node waits for all sources. Error Handling When a node fails during execution: The node turns red — indicating an error occurred Downstream nodes are skipped — they won’t receive data from the failed node Other branches continue — independent parts of the workflow still execute Error details are available — click the failed node to see the full error message Common strategies for handling errors in workflows: Add Preview nodes before and after suspect nodes to inspect data Use default values on inputs to handle missing data gracefully Check model availability before running — ensure required models are installed Test incrementally — build and test workflows one node at a time--- # Workflow Patterns Source: https://docs.nodetool.ai/cookbook/patterns.html To build any example: – press Space to add nodes – drag connections – press Ctrl/⌘+Enter to run – add Preview nodes to inspect intermediate results Pattern 1: Simple Pipeline Use Case: Transform input → process → output Example: Image Enhancement {% mermaid %} graph TD output[“Output”] image_input[“ImageInput”] sharpen[“UnsharpMask”] auto_contrast[“AutoContrast”] image_input –> sharpen sharpen –> auto_contrast auto_contrast –> output {% endmermaid %} When to Use: Simple data transformations Single input, single output No conditional logic needed Pattern 2: Agent-Driven Generation Use Case: LLM generates content based on input Example: Image to Story {% mermaid %} graph TD image_input[“Image”] agent_story[“Agent (Story Generator)”] preview_audio[“Preview (Audio)”] text_to_speech[“TextToSpeech”] image_input –> agent_story agent_story –> text_to_speech text_to_speech –> preview_audio {% endmermaid %} When to Use: Creative generation tasks Multimodal transformations (image→text→audio) Need semantic understanding Key Nodes: Agent: General-purpose LLM agent with streaming Summarizer: Specialized for text summarization ListGenerator: Streams list of items Pattern 3: Streaming with Multiple Previews Use Case: Show intermediate results during generation Example: Movie Poster Generator {% mermaid %} graph TD strategy_llm[“Agent (Strategy)”] strategy_template_prompt[“String (Strategy Template)”] strategy_preview[“Preview (Strategy)”] strategy_prompt_formatter[“FormatText (Strategy)”] movie_title_input[“StringInput (Title)”] genre_input[“StringInput (Genre)”] audience_input[“StringInput (Audience)”] image_preview[“Preview (Image)”] prompt_list_generator[“ListGenerator”] designer_instructions_prompt[“String (Designer Instructions)”] preview_prompts[“Preview (Prompts)”] text_to_image[“TextToImage”] strategy_llm –> strategy_preview strategy_template_prompt –> strategy_prompt_formatter strategy_prompt_formatter –> strategy_llm strategy_llm –> prompt_list_generator designer_instructions_prompt –> prompt_list_generator audience_input –> strategy_prompt_formatter movie_title_input –> strategy_prompt_formatter genre_input –> strategy_prompt_formatter prompt_list_generator –> preview_prompts prompt_list_generator –> text_to_image text_to_image –> image_preview {% endmermaid %} When to Use: Complex multi-stage generation User needs to see progress Agent planning + execution workflow Key Concepts: Strategy Phase: Agent plans approach Preview Nodes: Show intermediate results ListGenerator: Streams generated prompts Image Generation: Final output Pattern 4: RAG (Retrieval-Augmented Generation) Use Case: Answer questions using documents as context Example: Chat with Docs {% mermaid %} graph TD chat_input[“StringInput”] output[“Output (Answer)”] format_text[“FormatText”] hybrid_search[“HybridSearch”] agent[“Agent”] chat_input –> format_text chat_input –> hybrid_search hybrid_search –> format_text format_text –> agent agent –> output {% endmermaid %} When to Use: Question-answering over documents Need factual accuracy from specific sources Reduce LLM hallucinations Key Components: Search: Query vector database for relevant documents Format: Inject retrieved context into prompt Generate: Stream LLM response with context Pattern 5: Database Persistence Use Case: Store generated data for later retrieval Example: AI Flashcard Generator with SQLite {% mermaid %} graph TD topic_input[“StringInput (Topic)”] create_table[“CreateTable”] format_prompt[“FormatText”] generate_flashcards[“DataGenerator”] insert_flashcard[“Insert”] query_all[“Query”] display_result[“Preview”] topic_input –> format_prompt format_prompt –> generate_flashcards generate_flashcards –> insert_flashcard create_table –> query_all create_table –> insert_flashcard query_all –> display_result {% endmermaid %} When to Use: Need persistent storage Building apps with memory Agent workflows that need to recall past interactions Key Nodes: CreateTable: Initialize database schema Insert: Add records Query: Retrieve records Update: Modify records Delete: Remove records Database Flow: Create table structure Generate data with agent Insert into database Query and display results Pattern 6: Email & Web Integration Use Case: Process emails or web content Example: Summarize Newsletters {% mermaid %} graph TD gmail_search[“GmailSearch”] email_fields[“Template”] summarizer_streaming[“Summarizer”] preview_summary[“Preview (Summary)”] preview_body[“Preview (Body)”] gmail_search –> email_fields email_fields –> summarizer_streaming summarizer_streaming –> preview_summary email_fields –> preview_body {% endmermaid %} When to Use: Automate email processing Monitor RSS feeds Extract web content Key Nodes: GmailSearch (lib.mail.GmailSearch): Search Gmail with queries Template: Format email fields into text (lib.mail also provides AddLabel, MoveToArchive, SendEmail) FetchRSSFeed: Get RSS feed entries GetRequest: Fetch web content Pattern 7: Multi-Modal Workflows Use Case: Convert between different media types Example: Audio to Image {% mermaid %} graph TD stable_diffusion[“StableDiffusion”] whisper[“Whisper”] audio_input[“AudioInput”] output[“Output”] whisper –> stable_diffusion audio_input –> whisper stable_diffusion –> output {% endmermaid %} When to Use: Converting between media types Creating rich multimedia experiences Accessibility applications Common Chains: Audio → Text → Image Image → Text → Audio Video → Audio → Text → Summary Pattern 8: Advanced Image Processing Use Case: AI-powered image transformations Example: Style Transfer {% mermaid %} graph TD sd_img2img[“StableDiffusionV3MediumImageToImage”] image_input_1[“ImageInput”] image_input_2[“ImageInput”] output[“Output”] image_to_text[“ImageToText”] fit_1[“Fit”] fit_2[“Fit”] sd_img2img –> output image_to_text –> sd_img2img image_input_2 –> fit_1 image_input_1 –> fit_2 fit_2 –> image_to_text image_input_2 –> sd_img2img fit_2 –> sd_img2img {% endmermaid %} When to Use: Style transfer between images Controlled image generation Preserving structure while changing style Key Techniques: Img2Img: Transform while maintaining composition (StableDiffusionV3MediumImageToImage, or StableDiffusion / StableDiffusionXL for text-to-image) Image-to-Text: Generate descriptions (ImageToText) Canny: Edge detection preprocessing Pattern 9: Text-to-Video Generation Use Case: Generate videos from text descriptions Example: Cinematic Video from Prompt {% mermaid %} graph TD string_input[“StringInput (Prompt)”] output[“Output”] kling_text_to_video[“KlingVideoV16ProTextToVideo”] string_input –> kling_text_to_video kling_text_to_video –> output {% endmermaid %} When to Use: Create videos from text descriptions Generate concept videos and storyboards Produce cinematic content from prompts Rapid video prototyping Key Nodes: KlingVideoV16ProTextToVideo: High-quality text-to-video (Kling 1.6 Pro) MinimaxHailuo23ProTextToVideo: Professional quality (Hailuo 2.3) Sora2TextToVideo: OpenAI Sora 2 model WanProTextToVideo: Alibaba Wan Configuration Tips: Duration: 5-10 seconds for most models Resolution: 768P for faster generation, 1080P for quality Aspect ratios: 16:9 (landscape), 9:16 (portrait), 1:1 (square) Pattern 10: Image-to-Video Generation Use Case: Animate images into videos Example: Bring Images to Life {% mermaid %} graph TD image_input[“ImageInput”] string_input[“StringInput (Motion Guide)”] output[“Output”] kling_image_to_video[“KlingVideoV16StandardImageToVideo”] image_input –> kling_image_to_video string_input –> kling_image_to_video kling_image_to_video –> output {% endmermaid %} When to Use: Animate static images Create motion from photographs Multi-image video generation Product showcases from images Key Nodes: KlingVideoV16StandardImageToVideo: Kling 1.6 Standard image-to-video MinimaxHailuo02ProImageToVideo: High-quality animation (Hailuo 02 Pro) SeeDanceV15ProImageToVideo: ByteDance SeeDance 1.5 Pro WanV225bImageToVideo: Alibaba Wan 2.2 Advanced Pattern: Multi-Image Animation {% mermaid %} graph TD image1[“ImageInput (Frame 1)”] image2[“ImageInput (Frame 2)”] image3[“ImageInput (Frame 3)”] motion_prompt[“StringInput (Motion)”] output[“Output”] kling_i2v[“KlingVideoV16StandardImageToVideo”] kling_i2v –> output image1 –> kling_i2v image2 –> kling_i2v image3 –> kling_i2v motion_prompt –> kling_i2v {% endmermaid %} Pattern 11: Talking Avatar Generation Use Case: Create lip-synced avatar videos Example: Virtual Presenter {% mermaid %} graph TD image_input[“ImageInput (Face Photo)”] audio_input[“AudioInput (Speech)”] output[“Output”] kling_avatar[“KlingVideoAiAvatarV2Pro”] image_input –> kling_avatar audio_input –> kling_avatar kling_avatar –> output {% endmermaid %} When to Use: Create virtual presenters Generate lip-synced avatar videos Educational content with AI speakers Virtual influencers and spokespersons Key Nodes: KlingVideoAiAvatarV2Pro: Pro-quality avatar generation KlingVideoAiAvatarV2Standard: Standard mode for faster generation Infinitalk: Audio-driven video generation Workflow: Audio + Image → Talking Avatar Photo Input: Front-facing portrait image Audio Track: Speech recording or TTS output Optional Prompt: Guide emotions and expressions Mode Selection: Standard (faster) or Pro (higher quality) Pattern 12: Image Enhancement & Upscaling Use Case: Improve image quality and resolution Example: HD Image Upscaling {% mermaid %} graph TD image_input[“ImageInput (Low Res)”] output[“Output (High Res)”] topaz_upscale[“TopazUpscaleImage”] image_input –> topaz_upscale topaz_upscale –> output {% endmermaid %} When to Use: Upscale low-resolution images Remove noise and artifacts Enhance image quality Prepare images for high-resolution displays Key Nodes: TopazUpscaleImage: AI-powered upscaling to higher resolutions Denoise option: Reduces artifacts during upscaling Configuration: Higher target resolutions for crisp output Denoise: Enable for noisy input images Best for: Enhancing old photos, smartphone images, web graphics Pattern 13: Storyboard to Video Use Case: Convert image sequences to coherent videos Example: Visual Story Generation {% mermaid %} graph TD story_prompt[“StringInput (Story)”] image1[“ImageInput (Scene 1)”] image2[“ImageInput (Scene 2)”] image3[“ImageInput (Scene 3)”] output[“Output”] sora_storyboard[“Sora2ImageToVideoPro”] sora_storyboard –> output story_prompt –> sora_storyboard image1 –> sora_storyboard image2 –> sora_storyboard image3 –> sora_storyboard {% endmermaid %} When to Use: Create narrative videos from storyboards Combine multiple scenes into one video Professional video pre-visualization Animated story creation Key Nodes: Sora2ImageToVideoPro: OpenAI Sora 2 Pro keyframe-driven generation Supports: 1-3 keyframe images Output: Smooth transitions between scenes Workflow: Story Prompt: Describe the narrative arc Keyframes: Provide 1-3 scene images Generation: Sora creates smooth transitions Duration: 1-60 frames configurable--- # Creative Agent Source: https://docs.nodetool.ai/creative-agent NodeTool turns a one-line brief into a finished film through a pipeline you can inspect and steer at every step: a Director agent writes a typed screenplay, a storyboard renders cheap stills you pick from before video spend, each shot becomes a clip you can revise in place, and one click assembles the cut into the timeline editor for finishing and export. The whole chain is drivable three ways: from the UI, from chat (the agent uses the same ui_* tools), or from an external agent such as Claude Code over MCP (nodetool mcp serve). The workflow, end to end graph LR brief["Brief"] direct["Direct(Director agent)"] stills["Stills(cents)"] pick["Pick a still"] clips["Clips(dollars)"] revise["Revise(video-to-video)"] assemble["Assembletimeline"] nle["Timeline editor(trim, mix, captions)"] export["Export"] brief --> direct --> stills --> pick --> clips --> assemble --> nle --> export clips --> revise --> clips revise -. "round-trips into the cut" .-> nle Cheap stages come first. A still costs cents; a clip costs dollars; you pick the still you like before paying for the clip. Nothing renders twice unless you ask for it, and a revision made after assembly lands in the persisted cut automatically. Direct: brief in, screenplay out Open a storyboard (New → New storyboard), write a brief and a visual style, pick a shot count, and press Direct. The nodetool.creative.Director node returns a typed screenplay — title, logline, style bible, narration, music direction, and one structured shot per card: action, camera (framing, lens, angle, movement), motion, and duration. Two sibling nodes make the screenplay usable inside any workflow graph: nodetool.creative.ScreenplayShots streams each shot with a composed image-generation prompt, and nodetool.creative.ApplyEntities injects entity descriptors for consistency (below). The storyboard: plan, pick a still, then spend Each card walks one shot through its lifecycle (Planned → Still ready → Rendering → Rendered) with the actions that fit its state: generate stills until one looks right, click the one to use, generate the clip from it, revise the clip with a text instruction (“make it darker, add rain”). Revision runs video-to-video on the existing clip and swaps the result in place, so fixing shot 3 never means re-rolling shots 1–5. Agents drive the same surface through nine ui_storyboard_* tools: get_state, set_screenplay, add_shot, update_shot, generate_keyframe, generate_clip, revise_shot, assemble_timeline, and select_shot. Assemble: from storyboard to timeline Assemble timeline turns the rendered shots into a persisted timeline sequence and opens the editor: shot clips laid end to end on a video track, the screenplay’s narration and music as draft text-to-audio clips on their own tracks, the script panel carrying the screenplay text. From here it’s a normal edit — trim, transitions, audio mix, captions — and export. Every assembled clip stays linked to its shot. Revise a shot on the storyboard afterward and the new render replaces the clip in the persisted cut. The storyboard plans; the timeline finishes; revisions flow forward. Entities: reusable ingredients Characters, locations, styles, and props are named, reusable objects. Tag any image asset with a kind, a name, and a canonical descriptor — the exact sentence pasted into every prompt that names the entity. That verbatim descriptor is what holds a character’s look steady across shots. Entities live in the asset library (a metadata marker, no migration), appear under Entities in the app menu, and reach prompts through the ApplyEntities node or the ui_entity_apply tool. On a storyboard, the Entities field pins a cast to the board: styles and locations season every shot’s still and clip prompt, while characters and props activate on the shots that mention them by name. Each shot card shows which entities its prompt will use as chips — click one to include or exclude it for that shot. The Direct run also hands the cast to the screenplay model so shots reference entities by their exact names. Entities are also one @ away wherever prompts are written: the mention picker in the chat composer and the workflow editor’s Prompt node lists them first, and a picked entity carries its descriptor and reference image into the generation. Cost governance The estimate panel in the workflow editor prices a graph before it runs, aggregated from per-node fal and kie unit pricing, with unpriced nodes surfaced as unknown rather than hidden. A budget cap with over-budget warnings and a live spend ticker in chat round it out. Current limits: estimates count each node once (fan-out is not multiplied in), and the draft-mode toggle stores intent without routing models yet. Graph templates The same pipeline ships as workflow templates for batch runs with no surface interaction: Script to Screen — brief → direction document → style-frame-anchored keyframes → per-shot animation → cut with voiceover and score (Concat assembly). Directed Film to Timeline — the same direction assembled onto the timeline with nodetool.timeline.AddClips → RenderTimeline. For continuity across cuts, nodetool.creative.ShotChain generates clips sequentially, extracting each clip’s last frame to seed the next shot. Driving it from outside nodetool mcp serve exposes the workflow and creative tools over MCP (stdio), so an external agent can search nodes, build and run workflows, and generate media; nodetool mcp install writes the config for Claude Code, Codex, or OpenCode. Inside NodeTool, the chat agent reaches every surface through the same frontend tools the buttons use. Where this is heading The build plan, the market context it responds to, and the honest list of current limits live in the Creative Agent Roadmap.--- # Creative Agent Roadmap Source: https://docs.nodetool.ai/creative-agent-roadmap Creative Agent Roadmap Snapshot: July 2026. Market facts below carry dates; re-verify before building on them. What has to happen for NodeTool to be the strongest creative agent available: an agent that takes a brief and delivers a finished multi-shot video, with the plan inspectable and editable at every step. This doc records the market context, audits what the codebase already has, and lays out the work in priority order. For the user-facing walkthrough of the shipped workflow, see Creative Agent. The first deliverable already exists: the P0 pipeline is shipped as the Script to Screen template (packages/base-nodes/nodetool/examples/nodetool-base/Script to Screen.json), built entirely from existing nodes. Everything in P0 hardens that pipeline into product features; P1 and P2 extend it. Market snapshot (mid-2026) The video-generation market moved in three phases: single-clip models (2024 to early 2025), pipeline products with storyboards and reference consistency (late 2025), and agents that plan and drive production (2026). Every major player now ships a named agent: Google Flow + Flow Agent (I/O, May 2026): Gemini-powered agent with whole-project memory; brainstorms, edits, batch-operates, and recommends plot and dialogue. Scene builder and “ingredients” (reference images for character/object consistency) were already in place. “Flow Tools” lets users vibe-code creative tools inside Flow. Kling 3.0 Director Mode (Feb 2026): the model generates up to 6 shots / 15 s in one call with characters, lighting, and palette carried across shots; camera angles, durations, and pacing are dictated per shot. The model absorbed part of the assembly job. ByteDance Seedance 2.0 / 2.5 (Feb / Jun 2026): multi-shot output with native stereo audio (music, ambience, synced dialogue); 2.5 does 30 s in one generation. Both on fal. LTX Studio: script → storyboard → shots → render, with “Elements” — every character, location, and object in the script becomes a reusable entity with a persistent profile enforced across shots. OpenAI Sora 2: storyboards (plan scenes/keyframes before generating), cameos, and the Disney licensing deal (Dec 2025) for 200+ characters. Adobe: Project Graph (node-based workflows across Creative Cloud, Nov 2025) plus the Firefly AI Assistant (Apr 2026), a conversational agent orchestrating Photoshop, Premiere, and the rest. Comfy MCP (Jun 2026): the ComfyUI engine exposed over MCP so external agents search models, build workflows, and execute them. Krea Node Agent: a sentence in, the agent reads the existing canvas, plans, wires nodes, and runs. Figma acquired Weavy (Oct 2025). One-click agents (HeyGen Video Agent, InVideo Agent One, Vidu Agent, Hailuo Video Agent): brief in, finished video out. MiniMax markets Hailuo explicitly against node-based workflows. Three conclusions: A node canvas is table stakes. Adobe, Figma, Krea, Freepik, and Runway all have one. The canvas alone no longer differentiates. The differentiator is the agent that reads and writes the canvas. Krea Node Agent and Comfy MCP set the bar. NodeTool’s graph agent mode and ui_* tools already match the shape; the gap is the production layer above them. Users’ loudest complaints about the one-click agents are NodeTool’s opening: character drift across shots, credit burn on failed generations (users coined “CPUS” — cost per usable second), and no way to fix shot 3 without regenerating everything. An open, multi-provider tool with an inspectable plan answers all three. What NodeTool already has Audited July 2026: Models: 400+ video endpoints across fal, replicate, kie, together, and atlascloud manifests (packages/*/src/*-manifest.json) — Veo, Kling, Sora, Seedance, Wan, LTX, Hunyuan, Luma, plus lip sync, TTS (ElevenLabs, MiniMax, OpenAI), and music (Suno via kie, MusicGen, MiniMax). Generation nodes: provider-routed nodetool.video.TextToVideo / ImageToVideo / VideoToVideo / LipSync, nodetool.image.TextToImage / ImageToImage, nodetool.audio.TextToSpeech / TextToMusic (packages/video-nodes, packages/image-nodes, packages/audio-nodes). Editing: ffmpeg-backed Concat, Trim, Overlay, Transition, AddSubtitles, AddAudio, ChromaKey, color tools (packages/video-nodes/src/nodes/video.ts); a full audio effects rack. Timeline: a real NLE — data model in packages/timeline/, render nodes in packages/video-nodes/src/nodes/timeline.ts, editor UI in web/src/components/timeline/, and agent tools (ui_timeline_generate_clip, split/trim/move/bind) in web/src/lib/tools/builtin/timeline.ts. Agent system: GraphPlanner authors runnable workflow graphs from an objective (packages/agents/src/graph-planner.ts); media tools (generate_video, animate_image, generate_speech in packages/agents/src/tools/media-tools.ts); debug_workflow and validate_workflow for self-correction; the chat agent edits the live canvas through ui_* tools. Cost data: per-call cost in OTel spans (gen_ai.usage.cost_usd), a spend dashboard (web/src/components/costs/), and per-node pre-run pricing for fal and kie (FalPricingFooter, KieCreditsFooter). Confirmed absent: a storyboard/shot-list surface, a character-entity system, a direction layer, shot-continuity chaining, whole-pipeline cost estimates, live cost in chat, and (until the Script to Screen template) a shipped script-to-render example. P0 — the production spine Status: shipped (July 2026). All four P0 features and the three P1 items landed as product code on branch claude/nodetool-creative-agent-j96pji. Delivered artifacts and known limits are listed under each item. P2 remains planned. 1. Direction layer Shipped: Screenplay/Shot/CameraDirection/ShotStatus types in packages/protocol/src/creative.ts; three nodes in packages/llm-nodes/src/nodes/director.ts (registered via base-nodes): nodetool.creative.Director (brief → typed Screenplay + narration + music prompt), nodetool.creative.ScreenplayShots (streams each shot + a composed per-shot prompt), nodetool.creative.ApplyEntities (injects entity descriptors + reference images). A structured screenplay artifact — scenes and shots with duration, camera, motion, dialogue, characters, and style references — produced by a Director agent from a brief, and consumed by everything downstream. Define ShotList / Screenplay types in packages/protocol (peer of the timeline types in packages/timeline/src/types.ts). A Director node (or agent tool) that turns a brief into that artifact. The Script to Screen template prototypes this with nodetool.agents.Agent + nodetool.generators.StructuredOutputGenerator; the product version makes the artifact typed instead of free text. Nodes that consume it: shot-list → per-shot generation params; shot-list → timeline clips. 2. Entities (“ingredients”) Reusable Character / Location / Style objects: name, reference images, voice id, optional LoRA. Stored as assets, referenced by shots in the direction artifact, auto-injected into every generation call that names them. The model layer already takes reference images and LoRAs (hundreds of manifest endpoints expose reference_image, first_frame/last_frame, lora); what’s missing is the abstraction, storage, and UI. This is the single most-complained-about gap in competing products. Shipped: Entity/EntityRef types in protocol; entities persist as assets tagged with a nodetool_entity metadata marker (no migration) — web/src/serverState/useEntities.ts; an Entity library page (web/src/components/entities/, opened from the rail menu); ui_entity_list/ui_entity_apply agent tools; and the ApplyEntities node for graph-level injection. Injection is opt-in (wire the node, or call ui_entity_apply); automatic injection into every generation call that names an entity is still open. 3. Storyboard surface A workspace tab (peer of Timeline/Sketch in web/src/components/workspace/) that renders the direction artifact as shot cards: script text, style frame, generated still, then generated clip, with per-shot status, cost, and approve/regenerate. This is the plan-approve-spend gate: stills are cents, clips are dollars. Agent tools (ui_storyboard_*) mirror the existing timeline bridge. Shipped: "storyboard" workspace tab (web/src/components/workspace/StoryboardSurface.tsx, web/src/components/storyboard/), StoryboardStore/StoryboardGenerationStore, useGenerateShot (keyframe via TextToImage, clip via ImageToVideo through the workflow runner), a Direct button that runs the Director node from the board (useDirectScreenplay), a storyboardAgentBridge, and nine ui_storyboard_* tools (get_state, set_screenplay, add/update_shot, generate_keyframe, generate_clip, revise_shot, assemble_timeline, select_shot). The original approve_shot gate was later replaced by still selection: picking a still is the go-ahead for video spend. Opened from the “New storyboard” menu item. Boards are in-memory (no persistence yet), and the per-shot cost chip has no data source until estimates land per shot. Shipped, the timeline handoff: “Assemble timeline” (useAssembleTimeline + buildTimelineDocument) turns the board’s rendered shots into a persisted timeline sequence — asset-backed clips laid end to end, plus draft narration/music text-to-audio clips — and opens the editor. Each clip carries storyboardBoardId/storyboardShotId (typed on TimelineClip and the protocol schema), and a shot revision round-trips into the cut via syncShotClipToTimeline. Verified live end to end (assemble click → persisted document → revise → clip asset swapped). 4. Cost governance Pre-run estimate for a whole workflow or timeline render, aggregated from the existing per-node pricing data (fal-node-type-pricing.json, kie-node-type-pricing.json, cost-calculator.ts). A budget parameter the agent must respect when planning generation. Draft mode: route to cheap/low-res models first, final render on approval. Live cost ticker in Global Chat (the data already flows through OTel; it stops short of the chat UI). Shipped: estimateWorkflowCost in packages/node-sdk/src/cost-estimate.ts (aggregates fal/kie unit pricing across a graph, multiplied by each node’s configured fan-out, surfaces unpriced nodes as “unknown”); WorkflowCostEstimatePanel in the right panel and a CostTicker in Global Chat; useWorkflowCostEstimate/useLiveRunCost hooks. Deferred (removed, out of scope for now): the budget cap and draft-mode toggle, along with the BudgetStore and over-budget warnings — revisit when the agent can act on a cap rather than just warn. Known limits, still open: the ticker reads costs from editor-runner jobs, not chat-initiated ones. P1 — close the loop Flagship pipeline hardening: evolve the Script to Screen template from Concat assembly to timeline assembly (nodetool.timeline.AddClips → RenderTimeline), add a ShotBatch node (structured shot list in, N parallel generations out) and a ShotChain node (auto last-frame → first-frame continuity between clips). Shipped: nodetool.creative.ShotBatch and ShotChain (packages/llm-nodes/src/nodes/shots.ts — sequential clips with ffmpeg last-frame extraction seeding the next shot), plus a new Directed Film to Timeline example that assembles onto the timeline via AddClips → RenderTimeline. Script to Screen keeps its Concat cut (both assembly styles ship as examples), and no shipped example exercises ShotChain yet. Shot-level revision: “fix shot 3” as an agent flow — locate the clip via ui_timeline_get_state, regenerate through video-to-video endpoints (Aleph-class and Ray3-Modify-class models are already in the fal manifest), swap in place. Shipped for the storyboard: ui_storyboard_revise_shot + a Revise button on the shot card regenerate a shot’s clip via nodetool.video.VideoToVideo and swap it in place — and when the board has been assembled, the revision also lands in the persisted timeline clip (syncShotClipToTimeline). Revising a timeline clip that has no storyboard link is still open. NodeTool MCP server: expose the existing internal tools (list_nodes, create_workflow, run_workflow, debug_workflow, media tools) over MCP so external agents (Claude Code, ChatGPT) can drive NodeTool. Comfy MCP and vidu-skills show distribution now flows through general agents. Shipped: 18 workflow/creative tools bridged into the existing MCP server (packages/websocket/src/mcp-agent-tools.ts) and a nodetool mcp CLI (serve over stdio; install/uninstall/status for Claude Code, Codex, and OpenCode). P2 — polish and differentiation Per-shot model routing: extend find_model into a router that picks the model per shot from task, budget, and style (multi-shot models for dialogue scenes, cheap models for B-roll). Semantic timeline editing: “make the intro punchier” — the agent inspects the sequence and adjusts cuts, music, and transitions through existing ui_timeline_* tools; add a beat-detection → auto-cut node for music videos. Production status panel: shots done/pending/failed, spend against budget, approve/retry — replacing chat scrollback as the record of a long production. Project-level agent memory of entities and decisions. Positioning NodeTool’s claim against Flow, Adobe, and Krea: the open, local-first, multi-provider creative agent where the plan is inspectable. The agent writes a graph and a timeline the user can see, edit, and re-run, with real cost control, across every model vendor instead of one. That is exactly the complaint profile (consistency, cost, editability) users voice about the closed one-click agents.--- # End-to-End Deployment Guide Source: https://docs.nodetool.ai/deployment-e2e-guide.html This guide is a practical runbook for deploying NodeTool end-to-end with the unified server entrypoint (nodetool serve). It covers two production scenarios: Local / desktop runtime (no remote auth) Supabase-backed server (multi-user auth) What Runs in Production The server entrypoint is: nodetool serve --host 0.0.0.0 --port 7777 nodetool serve accepts only --host and --port. Use environment variables to control auth and runtime behavior. Important runtime behavior: Auth is enabled automatically when both SUPABASE_URL and SUPABASE_KEY are set; otherwise the server uses a local auth provider. There is no AUTH_PROVIDER switch read by serve. Production mode is selected by NODETOOL_ENV=production (the container also sets NODE_ENV=production). In production, SECRETS_MASTER_KEY is required. /health and /ready require no authentication. Prerequisites NodeTool CLI installed (@nodetool-ai/cli). Container runtime: Docker or Podman Image build for local deployment tests: docker build -t nodetool:local . If you use Podman: podman build -t nodetool:local . 1) Local / Desktop Runtime (no remote auth) Use this for local runtime where the app talks to a local API server. With no SUPABASE_* vars set, the server uses the local auth provider. export DB_PATH=/path/to/workspace/nodetool.db export HF_HOME=/path/to/hf-cache nodetool serve --host 127.0.0.1 --port 7777 Verify: curl -s http://127.0.0.1:7777/health curl -s http://127.0.0.1:7777/ready 2) Supabase-Backed Server Use this for internet-facing deployments where user auth is Supabase-backed. Setting both SUPABASE_URL and SUPABASE_KEY enables and enforces Supabase auth. export NODETOOL_ENV=production export SUPABASE_URL=https://<project>.supabase.co export SUPABASE_KEY=<service-role-or-server-key> export SECRETS_MASTER_KEY=<strong-random-secret> export DB_PATH=/workspace/nodetool.db export HF_HOME=/hf-cache nodetool serve --host 0.0.0.0 --port 7777 Verify: curl -i http://<host>:7777/health curl -i http://<host>:7777/api/workflows Expected: /health returns 200 (no auth required). /api/workflows without auth returns 401/403. Containerized End-to-End Run Docker docker run --rm -p 7777:7777 \ -e NODETOOL_ENV=production \ -e SECRETS_MASTER_KEY=<secret> \ -e DB_PATH=/workspace/nodetool.db \ -e HF_HOME=/hf-cache \ -v $(pwd)/workspace:/workspace \ -v $(pwd)/hf-cache:/hf-cache \ nodetool:local To enable Supabase auth, add -e SUPABASE_URL=… -e SUPABASE_KEY=…. Podman podman run --rm -p 7777:7777 \ -e NODETOOL_ENV=production \ -e SECRETS_MASTER_KEY=<secret> \ -e DB_PATH=/workspace/nodetool.db \ -e HF_HOME=/hf-cache \ -v $(pwd)/workspace:/workspace \ -v $(pwd)/hf-cache:/hf-cache \ nodetool:local Workflow Sync to a Deployed Server Create deployment config (~/.config/nodetool/deployment.yaml) with the target host and auth token, then sync: nodetool deploy workflows sync <deployment-name> <workflow-id> List remote workflows: nodetool deploy workflows list <deployment-name> Run synced workflow via REST: curl -s -X POST http://<host>:7777/api/workflows/<workflow-id>/run \ -H "Authorization: Bearer <token>" \ -H "Content-Type: application/json" \ -d '{}' Production Checklist NODETOOL_ENV=production SECRETS_MASTER_KEY set Supabase auth (SUPABASE_URL + SUPABASE_KEY) configured if you need remote auth /health and /ready are green Unauthorized access to protected endpoints returns 401/403 Workflow sync and workflow run both succeed Troubleshooting Server exits on startup in production Check SECRETS_MASTER_KEY: openssl rand -base64 32 E2E tests skip with “image not found” Validate image in active runtime context: docker image ls | grep nodetool podman image ls | grep nodetool Then rebuild/tag in that same runtime context.--- # Deployment Guide Source: https://docs.nodetool.ai/deployment.html NodeTool has two orthogonal deployment concerns. Don’t conflate them: Server (publish) — long-lived NodeTool infrastructure that humans and the UI connect into. NodeTool self-hosts as a single Docker server. See Self-Hosted Deployment. Worker (attach) — an ephemeral, billing-sensitive GPU box that one NodeTool instance connects out to, runs Python nodes on, and tears down. See Worker Deployment.   Server Worker Direction humans/UI connect in a NodeTool instance connects out Lifetime long-lived, always-on ephemeral — spin up, attach, tear down Identity a URL handed to people a {wsUrl, token} an instance adopts Cost flat bills by the minute — teardown matters How nodetool deploy … (Docker) nodetool worker … (RunPod, Vast) For a walkthrough across desktop, public, private, and Docker/Podman self-hosting, see the End-to-End Deployment Guide. Quick Reference: What do you want to do? I want to… Go to Run the NodeTool server on my own machine/host Self-Hosted Deployment Rent a GPU to run Python nodes (RunPod / Vast) Worker Deployment Tune GPU/memory/volumes for the server container Docker Resource Management Use Supabase for auth/storage Supabase Deployment Integration Set up TLS/HTTPS Self-Hosted Deployment Server: self-host with Docker The quickest path is the reference docker-compose.yml at the repo root — cp .env.example .env && docker compose up -d. See Self-Hosted Deployment › Docker Compose. For a managed flow (remote hosts over SSH, image transfer, workflow sync), the nodetool deploy commands manage a single Docker server target driven by a deployment.yaml file. Pull the base image: docker pull ghcr.io/nodetool-ai/nodetool:latest Initialize and add a target: nodetool deploy init nodetool deploy add my-server --type docker This scaffolds deployment.yaml using the schema in @nodetool-ai/deploy (deployment-config.ts). The single target type is docker; the entry specifies the container image, persistent paths, and environment variables (under container.environment). Review & plan (no remote mutation): nodetool deploy list nodetool deploy show my-server nodetool deploy plan my-server Apply & monitor: nodetool deploy apply my-server nodetool deploy status my-server nodetool deploy logs my-server --follow nodetool deploy destroy my-server See Self-Hosted Deployment for the full server walkthrough, and Supabase Deployment Integration to add hosted auth and storage. Heads up: NodeTool used to ship server-deploy targets for RunPod serverless, Google Cloud Run, Fly, Railway, and HuggingFace Spaces. Those are gone. Server self-hosting is Docker only; GPU access is now a worker concern, not a server target. To run on a GPU, rent a worker (below). Worker: rent a GPU (RunPod, Vast) When a graph needs a GPU you don’t have, provision a remote worker, attach to it, run your Python nodes there, and tear it down: nodetool secrets store RUNPOD_API_KEY nodetool worker profile add hf-a40 --target runpod \ --image ghcr.io/nodetool-ai/nodetool-worker:latest \ --gpu "NVIDIA A40" --idle-timeout 15 nodetool worker create --profile hf-a40 --attach nodetool worker list # what's live, and what it's costing nodetool worker stop --all # tear everything down GPU workers bill by the minute, so the worker subsystem ships a cost guard: real teardown on every stop, idle auto-stop, a hard TTL, and orphan reconcile. See Worker Deployment for profiles, attaching from the UI or CLI, supported targets, and the cost guard in full. Server configuration deployment.yaml accepts these top-level keys per deployment (see DockerDeployment in @nodetool-ai/deploy deployment-config.ts): type – always docker enabled – whether the deployment is active host – Docker host (IP/hostname, or localhost) ssh – SSH connection details for remote hosts (omit for local) paths – workspace and HF cache paths persistent_paths – persistent storage paths inside the container image – container image name/tag/registry container – name, port, GPU, and environment (env vars injected into the container) server_auth_token – auto-generated bearer token for admin/sync calls state – deployment state tracked by the deployer Environment variables live under container.environment. Secrets such as SECRETS_MASTER_KEY are auto-generated into container.environment when missing. Monitoring & health checks # Health endpoint (no auth required) curl http://your-server:7777/health # Expected: {"status": "ok", ...} (or "degraded" when a service is unhealthy) nodetool deploy status <name> nodetool deploy logs <name> --follow Indicator What to watch Action Health endpoint Should return 200 Restart service if unhealthy Memory usage Models consume significant RAM/VRAM Scale up or use smaller models Disk space Model cache and assets grow over time Periodic cleanup or larger volumes Response time First request after cold start is slow (model loading) Warm up via health check Troubleshooting Problem Likely cause Fix Container exits immediately Missing env vars or invalid config Check nodetool deploy logs <name> Health check fails Service still starting (model loading) Increase timeout; large models need 60–120s 503 Service Unavailable Overloaded or out of memory Scale up resources or reduce concurrency Port already in use Another service on the same port Change container.port in deployment.yaml “Image not found” Docker image not present docker pull ghcr.io/nodetool-ai/nodetool:latest Permission denied on volumes Container user lacks access Fix host directory permissions For more, see the Troubleshooting Guide. Upgrading docker pull ghcr.io/nodetool-ai/nodetool:latest nodetool deploy apply <name> nodetool deploy status <name> Workflows, assets, and settings are preserved across upgrades when using persistent volumes. Related Self-Hosted Deployment Worker Deployment Docker Resource Management Supabase Deployment Integration End-to-End Deployment Guide </content>--- # desktop-app.md Source: https://docs.nodetool.ai/desktop-app This page has moved. Please go to Self-Hosted Setup.--- # Developer Guide Source: https://docs.nodetool.ai/developer/ Resources for building custom nodes, extending NodeTool, and integrating workflows programmatically. Quick Start: Custom Nodes Creating custom nodes in TypeScript: import { BaseNode, prop } from "@nodetool-ai/node-sdk"; export class UppercaseTextNode extends BaseNode { static readonly nodeType = "mypackage.text.Upper"; static readonly title = "Uppercase Text"; static readonly description = "Convert text to uppercase."; static readonly metadataOutputTypes = { output: "str" }; @prop({ type: "str", default: "", title: "Input Text" }) declare inputText: string; async process(): Promise<Record<string, unknown>> { return { output: String(this.inputText ?? "").toUpperCase() }; } } Export a register(registry) function from your package and NodeTool discovers the node automatically. -> Full Custom Nodes Guide (TypeScript) – End-to-end walkthrough: packaging, governance, streaming, testing, distribution. Guides Provider & Model Guides Provider Guides – Add models & nodes to any provider. Per-provider runbooks (OpenAI, Anthropic, Gemini, xAI, FAL, Replicate, KIE, Together, AtlasCloud, Topaz, MiniMax, Reve, ElevenLabs, HuggingFace, Ollama, local inference, OpenAI-compatible) with exact files, commands, and how past PRs did it. Written for coding agents and contributors. Custom Node Development (TypeScript) Custom Nodes Guide – Start here! End-to-end guide for authoring, packaging, and distributing TypeScript node packs. TypeScript DSL Guide – Type-safe workflow definitions with auto-generated factory functions. Custom Node Reference (TypeScript) Node Implementation Quick Reference – Templates and common @prop / process() patterns. Node Implementation Patterns – Architectural patterns: multi-output, streaming, stateful, secrets. Python Nodes Node Implementation Examples – Python node examples for the Python bridge. Advanced Suspendable Nodes – Build nodes that can pause and resume workflows. Programmatic Workflows TypeScript DSL Guide – Type-safe workflow definitions with auto-generated factory functions API Integration API Reference – REST API endpoints and authentication Headless Mode – Run workflows via CLI and HTTP API Chat API – OpenAI-compatible chat endpoints Workflow API – Execute workflows programmatically Contributing Contribute to NodeTool on GitHub. Share Custom Nodes Options: Publish as a separate package Contribute to the core node library Share workflow examples on Discord--- # Custom Nodes Guide (TypeScript) Source: https://docs.nodetool.ai/developer/custom-nodes-guide.html This is the full guide to writing TypeScript custom nodes for NodeTool — the in-process counterpart to the Python node guides (Node Examples, Node Patterns, Node Reference). A custom node lives in a standalone npm package — a pack — that the server discovers and loads at startup. There is nothing to register by hand: drop a nodetool field in package.json, install the package, and the loader does the rest. Related: Node Packs (user-facing intro), Package Registry Guide (first-party package conventions), TypeScript DSL Guide (using nodes in code-defined workflows). Contents Quick start Package layout package.json and the pack manifest Trust model and governance tsconfig.json Anatomy of a node The @prop decorator reference The type system Declaring outputs ProcessingContext — the runtime surface Streaming nodes (genProcess) Lifecycle hooks Registering nodes Building the pack Testing nodes Installing and running the pack Versioning the pack API Common pitfalls 1. Quick start Scaffold a pack, write one node, install it, run the server: mkdir nodetool-mypack && cd nodetool-mypack npm init -y npm install --save @nodetool-ai/node-sdk npm install --save-dev typescript @types/node vitest src/nodes/reverse.ts: import { BaseNode, prop } from "@nodetool-ai/node-sdk"; export class ReverseTextNode extends BaseNode { static readonly nodeType = "mypack.text.Reverse"; static readonly title = "Reverse Text"; static readonly description = "Reverse a string character by character."; static readonly metadataOutputTypes = { output: "str" }; @prop({ type: "str", default: "", title: "Text" }) declare text: string; async process(): Promise<Record<string, unknown>> { return { output: [...(this.text ?? "")].reverse().join("") }; } } src/index.ts: import type { NodeRegistry } from "@nodetool-ai/node-sdk"; import { ReverseTextNode } from "./nodes/reverse.js"; export function register(registry: NodeRegistry): void { registry.register(ReverseTextNode); } Add a nodetool field to package.json (see §3), build with tsc, then npm link (or npm install) into your NodeTool workspace and restart the server. The node appears in the menu as Reverse Text under mypack.text. 2. Package layout A pack is a normal npm package. Keep one node (or a small group) per file: nodetool-mypack/ ├── package.json ├── tsconfig.json ├── README.md ├── src/ │ ├── index.ts # entry — exports `register(registry)` │ └── nodes/ │ ├── reverse.ts │ └── math.ts └── tests/ └── reverse.test.ts Keep the public surface in src/index.ts minimal: re-export node classes and the register function. Everything else stays internal. 3. package.json and the pack manifest { "name": "@acme/cool-nodes", "version": "0.1.0", "description": "Cool custom nodes for NodeTool", "type": "module", "main": "dist/index.js", "types": "dist/index.d.ts", "exports": { ".": { "import": "./dist/index.js", "types": "./dist/index.d.ts" } }, "scripts": { "build": "tsc", "test": "vitest run", "lint": "tsc --noEmit" }, "dependencies": { "@nodetool-ai/node-sdk": "latest" }, "devDependencies": { "@types/node": "^22.0.0", "typescript": "^5.4.0", "vitest": "^1.6.1" }, "nodetool": { "apiVersion": 1, "register": "register" } } The nodetool field is what makes the package a pack. At startup the loader scans installed packages, picks up any with this field, imports the resolved entry, and calls the named export with the registry. Field Default Meaning apiVersion 1 Pack API version you built against. Packs declaring a version newer than the host supports are skipped with a warning. register "register" Named export the loader calls with the registry. Can be async. The loader uses the "." entry of exports (the import condition, falling back to default), or main, or index.js — in that order. 4. Trust model and governance Custom nodes run in the server process as the server user. They have full filesystem, network, secret-store, and process.env access, with no sandbox. A pack is exactly as trusted as any dependency you npm install. Only install packs you trust. To avoid silently running whatever happens to be in node_modules in production, the loader is gated: Allowlist — a list of trusted pack names; "*" allows everything. Set via: The env var NODETOOL_PACKS_ALLOWLIST (comma-separated names), or The allow field of ~/.config/nodetool/packs.json (path overridable with the NODETOOL_PACKS_CONFIG env var). allowUnlisted — whether packs not on the allowlist load anyway. Defaults to true in development (so installing a pack just works) and false in production (NODETOOL_ENV=production). Override via the config file. Two further guards protect the registry regardless of trust: Reserved namespaces — packs cannot register node types under first-party namespaces (nodetool., lib., provider names like openai., replicate., …). Such nodes are skipped with a warning. Collision protection — a pack cannot shadow an already-registered node type (a built-in, or a node registered earlier by another pack). The conflicting node is skipped with a warning; the original wins. Example allowlist file: { "allow": ["@acme/cool-nodes", "@other-org/audio-pack"], "allowUnlisted": false } 5. tsconfig.json The SDK uses legacy (experimental) decorators without runtime metadata emission. Match those flags or builds will produce unusable output: { "compilerOptions": { "target": "ES2022", "module": "Node16", "moduleResolution": "Node16", "lib": ["ES2022"], "strict": true, "esModuleInterop": true, "skipLibCheck": true, "declaration": true, "sourceMap": true, "outDir": "dist", "rootDir": "src", "experimentalDecorators": true, "emitDecoratorMetadata": false }, "include": ["src"] } Do not enable "useDefineForClassFields" or "emitDecoratorMetadata" — they conflict with the SDK’s decorator protocol. Property declarations must use declare (see §6). 6. Anatomy of a node Every node extends BaseNode and declares its inputs with @prop. The runtime assigns properties on the instance before calling process() — your method reads inputs from this.<field>, not from a parameter. import { BaseNode, prop } from "@nodetool-ai/node-sdk"; import type { ProcessingContext } from "@nodetool-ai/runtime"; export class AddOffsetNode extends BaseNode { // ── Identity ──────────────────────────────────────────────── static readonly nodeType = "mypack.math.AddOffset"; static readonly title = "Add Offset"; static readonly description = "Add a constant offset to a number."; // ── Output type ──────────────────────────────────────────── static readonly metadataOutputTypes = { output: "float" }; // ── Inputs ───────────────────────────────────────────────── @prop({ type: "float", default: 0.0, title: "Value" }) declare value: number; @prop({ type: "float", default: 1.0, title: "Offset" }) declare offset: number; // ── Execution ────────────────────────────────────────────── async process(_context?: ProcessingContext): Promise<Record<string, unknown>> { return { output: (this.value ?? 0) + (this.offset ?? 1) }; } } Required static members Member Type Purpose nodeType string Unique dotted identifier. By convention: <namespace>.<category>.<Name>. title string Display name in the node menu. description string One-line tooltip. Common optional static members Member Type Purpose metadataOutputTypes { [name]: typeString } Maps output handle name → NodeTool type string. Required if outputs are anything other than the default output: any. isStreamingInput boolean Marks the node as consuming a stream via the run(...) hook (pair with inputMode = "stream"). supportsDynamicInputs boolean Allows users to add/remove input handles in the UI; read/write them with getDynamic / setDynamic. inputMode "buffered" \| "stream" \| "controlled" How inputs are consumed. Default (undefined) is buffered. outputCorrelation { [output]: OutputCorrelation } Controls per-iteration / per-chunk fan-out semantics. Streaming output is inferred from this (e.g. forward/iteration kinds) — there is no isStreamingOutput flag. The process method abstract process(context?: ProcessingContext): Promise<Record<string, unknown>> Returns an object whose keys are output handle names declared in metadataOutputTypes. Reads inputs from this.<field>. The runtime has already populated them via assign(). The context parameter is optional. Many pure-compute nodes ignore it. Anything that touches secrets, storage, HTTP, or providers will use it — see §10. Throwing an Error fails the node and surfaces the message to the UI. Throw Error objects, not strings. Do not add an inputs parameter — that pattern is from an older draft of this guide and does not match the runtime contract. Property assignment happens before process() runs. 7. The @prop decorator reference @prop is the only decorator pack authors call directly. Outputs are declared via static class members, not decorators. @prop(options: PropOptions) Option Type Purpose type string (required) NodeTool type string — see §8. default unknown Default value when no upstream input is connected. title string Display name in the UI. Defaults to the field name. description string Tooltip text. min / max number Bounds for numeric types — used by sliders and validation. required boolean Whether the input must be connected or set. values (string \| number)[] Allowed values; renders as a dropdown / enum selector. json_schema_extra Record<string, unknown> Custom UI metadata (renderer hints, layout, etc.). Always pair @prop with a declare field — the runtime owns assignment, so emitted initializers would just be overwritten. @prop({ type: "str", default: "", title: "Prompt", description: "What to generate" }) declare prompt: string; @prop({ type: "float", default: 0.7, min: 0, max: 2, title: "Temperature" }) declare temperature: number; @prop({ type: "str", default: "auto", values: ["auto", "fast", "best"], title: "Mode" }) declare mode: string; 8. The type system The type string in @prop and the values in metadataOutputTypes come from the same vocabulary — the one shared with Python nodes. Scalars String TS type "str" string "int" number "float" number "bool" boolean "json" unknown "any" unknown Collections String TS type "list[T]" T[] (e.g. list[str], list[any]) "dict[str, any]" Record<string, unknown> "dict[str, str]" Record<string, string> Media references Media flows through the graph as small reference objects, not as raw bytes. Import the types from @nodetool-ai/node-sdk: @nodetool-ai/node-sdk re-exports only ImageRef, AudioRef, VideoRef, TextRef, and DataframeRef. DocumentRef and Model3DRef are not re-exported by the SDK — import those from @nodetool-ai/protocol: import type { ImageRef, AudioRef, VideoRef, TextRef, DataframeRef } from "@nodetool-ai/node-sdk"; import type { DocumentRef, Model3DRef } from "@nodetool-ai/protocol"; String TS type Use for "image" ImageRef Images (URI or inline bytes) "audio" AudioRef Audio clips "video" VideoRef Video files "document" DocumentRef PDFs, Word, plain-text files "text" TextRef Large text blobs by reference "dataframe" DataframeRef Tabular data "model_3d" Model3DRef 3D meshes / glTF Model selectors These render as model pickers in the UI: String Meaning "language_model" An LLM (provider + model id pair). "image_model" An image-generation model. "video_model" A video-generation model. "tts_model" A text-to-speech model. "asr_model" A speech-recognition model. "embedding_model" An embedding model. Domain types "date", "datetime", "image_size", "enum" — render with specialised inputs. 9. Declaring outputs Single output, default name output: static readonly metadataOutputTypes = { output: "str" }; async process(): Promise<Record<string, unknown>> { return { output: "hello" }; } Multiple outputs: static readonly metadataOutputTypes = { text: "str", tokenCount: "int" }; async process(): Promise<Record<string, unknown>> { const text = this.input ?? ""; return { text, tokenCount: text.split(/\s+/).length }; } Output keys returned by process() must match keys in metadataOutputTypes — extra keys are dropped, missing keys produce undefined on the wire. 10. ProcessingContext — the runtime surface The context passed to process() and genProcess() is your gateway to everything the runtime owns: secrets, storage, cache, HTTP, providers, messages. You only need it when a node has side effects. import type { ProcessingContext } from "@nodetool-ai/runtime"; Identity context.jobId; // string — unique per workflow run context.workflowId; // string | null context.userId; // string context.workspaceDir; // string | null — per-job scratch dir Secrets const apiKey = await context.getSecret("OPENAI_API_KEY"); // string | null const required = await context.getSecretRequired("STRIPE_KEY"); // throws if missing Always prefer getSecret over process.env — secrets are user-scoped and may come from an encrypted store, not the process environment. HTTP helpers const resp = await context.httpGet("https://api.example.com/data", { headers: { Authorization: `Bearer ${apiKey}` } }); await context.httpPost(url, { json: { foo: 1 } }); // Also: httpPut, httpPatch, httpDelete, httpHead Cache The per-job cache is exposed at context.cache. Both methods are async; set takes an optional TTL in seconds, and get takes only the key (it returns undefined on a miss — there is no default-value argument): // CacheAdapter signatures: // get(key: string): Promise<unknown | undefined> // set(key: string, value: unknown, ttlSeconds?: number): Promise<void> const hit = await context.cache.get("my-key"); if (hit !== undefined) return hit as Record<string, unknown>; const result = await expensive(); await context.cache.set("my-key", result, 3600); // expires after 1 hour For node-result memoization, the convenience helpers wrap the same cache: const cached = await context.getCachedResult(this.nodeType, this.serialize()); if (cached) return cached; const result = await expensive(); await context.cacheResult(this.nodeType, this.serialize(), result, 3600); return result; Storage and workspace files const uri = await context.storage.store("output.png", bytes, "image/png"); const data = await context.storage.retrieve(uri); await context.workspaceStorage.store("notes.txt", "hello", "text/plain"); LLM providers if (await context.isProviderConfigured("openai")) { const provider = await context.getProvider("openai"); const stream = provider.streamChat({ messages, model: "gpt-5.4-mini" }); } Variables (per-job key/value scratch) context.set("counter", 0); const n = context.get<number>("counter", 0); Messages context.emit(msg) posts a ProcessingMessage (log line, status update, tool call, …) onto the run’s event stream. The UI uses these to surface progress. 11. Streaming nodes (genProcess) For nodes that produce results incrementally, override genProcess and declare an iteration outputCorrelation. The base class detects that the subclass overrides genProcess and iterates it — there is no isStreamingOutput flag to set: import { BaseNode, prop } from "@nodetool-ai/node-sdk"; import type { ProcessingContext } from "@nodetool-ai/runtime"; import type { OutputCorrelation } from "@nodetool-ai/protocol"; export class WordStreamNode extends BaseNode { static readonly nodeType = "mypack.text.WordStream"; static readonly title = "Word Stream"; static readonly description = "Emit each word of the input as a separate event."; static readonly metadataOutputTypes = { word: "str" }; static readonly outputCorrelation: Record<string, OutputCorrelation> = { word: { kind: "iteration", source: "__execution__", group: "items" }, }; @prop({ type: "str", default: "", title: "Text" }) declare text: string; async process(): Promise<Record<string, unknown>> { // Fallback for non-streaming consumers. return { word: this.text }; } async *genProcess( _context?: ProcessingContext ): AsyncGenerator<Record<string, unknown>> { for (const word of String(this.text ?? "").split(/\s+/)) { if (word) yield { word }; } } } Output correlation When a streaming node mixes per-item outputs with aggregate ones, declare outputCorrelation so the runtime knows how to fan downstream nodes: static readonly outputCorrelation = { word: { kind: "iteration", source: "__execution__", group: "items" }, count: { kind: "single", source: "__execution__" } }; kind: "iteration" means each yielded value advances downstream consumers; kind: "single" means the value is the run’s final aggregate. See the OutputCorrelation type in @nodetool-ai/protocol for the full vocabulary. Default behaviour If you don’t override genProcess, the base implementation yields the single result of process(). So a non-streaming node needs nothing extra; only streaming nodes override genProcess. 12. Lifecycle hooks Override these on your class to run setup and teardown: async initialize(): Promise<void> { /* once when the node enters the run */ } async preProcess(): Promise<void> { /* before every process()/genProcess() */ } async finalize(): Promise<void> { /* once when the node leaves the run */ } Use initialize for expensive one-time setup (e.g. opening a connection) and finalize to release it. preProcess is called per execution and is rarely needed. 13. Registering nodes Your pack’s register function — the export named in the manifest — receives the registry and adds each node class: import type { NodeRegistry } from "@nodetool-ai/node-sdk"; import { AddOffsetNode } from "./nodes/math.js"; import { ReverseTextNode } from "./nodes/reverse.js"; import { WordStreamNode } from "./nodes/stream.js"; const ALL_NODES = [AddOffsetNode, ReverseTextNode, WordStreamNode] as const; export function register(registry: NodeRegistry): void { for (const cls of ALL_NODES) { registry.register(cls); } } Things the loader will refuse — silently dropping the offending node and warning in the log: A nodeType under a reserved namespace. A nodeType already registered by a built-in or an earlier pack (no shadowing). The register function may be async — useful if you build node classes from a manifest at load time. What NodeRegistry exposes Pack authors only need: Method Purpose register(nodeClass, opts?) Add a node class. has(nodeType) Already registered? list() All registered node types. getClass(nodeType) The class for a node type. listMetadata() UI metadata for every registered node. Everything else on the registry is for the runtime. 14. Building the pack npm run build # tsc → dist/ npm run lint # tsc --noEmit npm run test # vitest run Ship the dist/ directory plus package.json and a README.md. Don’t ship src/ — it won’t be loaded. Recommended package.json “files” allowlist "files": ["dist", "README.md", "LICENSE"] 15. Testing nodes Use Vitest (same harness the SDK uses internally). The pattern for an end-to-end test: import { describe, it, expect } from "vitest"; import { NodeRegistry } from "@nodetool-ai/node-sdk"; import { register } from "../src/index.js"; import { ReverseTextNode } from "../src/nodes/reverse.js"; describe("ReverseTextNode", () => { it("registers under its nodeType", () => { const registry = new NodeRegistry(); register(registry); expect(registry.has(ReverseTextNode.nodeType)).toBe(true); }); it("reverses a string", async () => { const node = new ReverseTextNode({ text: "hello" }); const result = await node.process(); expect(result).toEqual({ output: "olleh" }); }); }); For nodes that need a ProcessingContext, build a minimal one with the constructor options shown in §10 or use a test double that stubs only the methods your node calls. Testing streaming nodes it("yields one event per word", async () => { const node = new WordStreamNode({ text: "foo bar baz" }); const chunks: unknown[] = []; for await (const chunk of node.genProcess()) chunks.push(chunk); expect(chunks).toEqual([{ word: "foo" }, { word: "bar" }, { word: "baz" }]); }); 16. Installing and running the pack For a published pack: cd /path/to/nodetool npm install @acme/cool-nodes # in production, allowlist it: echo '{ "allow": ["@acme/cool-nodes"] }' > ~/.config/nodetool/packs.json npm run dev:server For local development, npm link the pack so edits land without re-publishing: cd nodetool-mypack && npm run build && npm link cd /path/to/nodetool && npm link @acme/cool-nodes npm run dev:server The server logs which packs were discovered: Loaded node pack @acme/cool-nodes@0.1.0 (3 node(s)) Skipped node pack @other/blocked@1.0.0: not on pack allowlist Pack @evil/shadowy: skipped node nodetool.text.Override (reserved-namespace) 17. Versioning the pack API The pack API is versioned with a single integer (apiVersion, currently 1). Forward compatibility: If your pack declares apiVersion: 2 and runs on a host that only knows 1, the host skips it cleanly rather than crashing. If the host knows 2 and your pack still declares 1, everything keeps working — the host is responsible for backward compatibility. When the host bumps the version, the changelog will name exactly what changed and what (if anything) authors need to migrate. 18. Common pitfalls Property declarations without declare. TypeScript will emit initializers that overwrite the values the runtime just assigned. Always: @prop({ type: "str", default: "" }) declare text: string; // good @prop({ type: "str", default: "" }) text: string = ""; // bad — runs after assign() ESM .js extensions in imports. With module: "Node16", internal imports must use the .js extension (matching the compiled output), even from .ts source: import { ReverseTextNode } from "./nodes/reverse.js"; // good import { ReverseTextNode } from "./nodes/reverse"; // bad emitDecoratorMetadata. Leave this false. The SDK does not consume runtime metadata and enabling it can produce conflicting decorator output. Loading from src/ instead of dist/. The loader resolves the package’s exports/main, which points at dist/. Forgetting to build before installing means the pack loads stale code (or nothing). Reserved-namespace nodeTypes. A nodeType of nodetool.foo.MyNode will be silently rejected. Use your own namespace (mypack.foo.MyNode, @acme.foo.MyNode, …). Collision with built-ins. If a built-in already owns the nodeType you picked, your node is dropped. Pick a unique name or run nodetool workflows list --json and grep to be sure. Throwing strings instead of Error. The runtime wraps errors and surfaces .message; a thrown string becomes undefined in the log. Long-running process blocking the event loop. If your work is CPU-bound, do it in a Worker or via code-runners. The server is single-threaded. Reading from process.env for user-scoped secrets. Use context.getSecret(key) — it consults the user’s secret store, which may be encrypted and is not the process environment. Related documentation Node Packs — user-facing intro to packs. Package Registry Guide — first-party package conventions. Node Implementation Examples (Python) — annotated Python node examples. Node Patterns (Python) — design patterns shared across languages. Node Reference (Python) — exhaustive Python API reference. TypeScript DSL Guide — use your nodes in code-defined workflows. Suspendable Nodes — long-running and resumable patterns.--- # developer-dsl-guide.md Source: https://docs.nodetool.ai/developer/dsl-guide This page has moved. Please go to TypeScript DSL Guide.--- # Node Graph Editor Performance Audit Source: https://docs.nodetool.ai/developer/node-editor-performance-audit.html Node Graph Editor Performance Audit Audited: 2026-07-02. Sources verified against web/src at commit 8ff1421. ReactFlow (@xyflow/react) 12.10.2, React 19, Zustand 5. Scope: the canvas render path — ReactFlowWrapper, BaseNode and everything rendered per node, the Zustand stores the editor subscribes to, edge/handle/ pointer interaction paths, and the editor CSS. All 27 existing performance guard tests pass; the issues below live in paths those tests do not cover. The costs concentrate on three hot paths: Node drag / pan — per-frame work that is O(all nodes), not O(moved nodes). Workflow execution — per-websocket-message work that sweeps the whole canvas, including other tabs’ runs. Paint — CSS filter chains on every node (and, in light mode, every edge and handle) that force filter-pipeline rasterization during drags. High severity H1. onNodesChange clones every node on every change batch NodeStore.ts:58-65, 504-508: const rawNodes = applyNodeChanges(filteredChanges, currentNodes); const nodes = syncAllReactFlowNodeChromeClass(rawNodes ?? currentNodes); // = nodes.map((node) => ({ ...node, className: reactFlowNodeChromeClassName(node.data) })) applyNodeChanges preserves object identity for untouched nodes so ReactFlow can skip them; the unconditional .map() + spread then gives every node a fresh identity on every change batch — every drag pointermove frame, every selection click, every dimension change. Verified against the installed @xyflow/system (adoptUserNodes): RF12 skips re-adoption only when userNode === internals.userNode, so every frame re-runs position clamping, handle parsing, and z-calculation for all N nodes, and every NodeWrapper re-renders (the inner BaseNode is saved only by its memo comparator). It also poisons every downstream cache keyed on node identity. Fix (small): only clone when the class actually changed, and return the original array when nothing did: const syncReactFlowNodeChromeClass = (node) => { const cls = reactFlowNodeChromeClassName(node.data); return cls === node.className ? node : { ...node, className: cls }; }; bypassed/collapsed only change in toggleBypass/updateNodeData, which already set className on the touched node — the hot path can be fully identity-preserving. H2. useViewport() re-renders the whole canvas wrapper every pan/zoom frame ReactFlowWrapper.tsx:362, consumed only at :718-733 to derive a zoomed-out class. useViewport fires on every transform change (x, y, and zoom — every pointermove of a pan). The wrapper is the heaviest component in the editor (~10 store subscriptions, ~15 memos, a ~50-prop <ReactFlow> element) and re-runs per frame to compute a boolean that only changes when zoom crosses ZOOMED_OUT. Fix: useStore((s) => s.transform[2] <= ZOOMED_OUT) — re-renders only on threshold crossings. Same fix for ViewportStatusIndicator.tsx:39 (full MUI toolbar re-render per pan frame; it only needs zoom) and NodeInfoPanel.tsx:201 (re-renders per frame even with nothing inspected). AxisMarker already does this right (useOnViewportChange + ref writes). H3. The memoized node chrome components are dead code — every consumer imports the unmemoized export NodeHeader.tsx, NodeErrors.tsx, NodeInputs.tsx, NodeOutputs.tsx each export a memo(..., isEqual) default export (NodeHeader.tsx:394, NodeErrors.tsx:223, NodeInputs.tsx:321, NodeOutputs.tsx:102) — and every canvas consumer (BaseNode.tsx:30-31, NodeContent.tsx:3-4, all bespoke node types) imports the named, unmemoized component. Verified by grep: the memo wrappers have zero canvas imports. BaseNode re-renders on every streaming chunk, status flip, and hover enter/leave; each re-render re-runs NodeHeader (~350 lines, a large emotion css object, an always-mounted logs dialog) and NodeErrors (nodeErrorToDisplayString runs twice per render) for every affected node. Fix: memoize the named exports themselves (or switch imports to the defaults), and give NodeHeader a comparator narrower than deep-equal on the whole data blob — it only reads data.bypassed. H4. findNode is O(N), and per-node selectors call it on every store write → O(N²) per drag frame NodeStore.ts:724-725 (get().nodes.find(...)). Callers that run per node on every NodeStore update: useNodeGenerations.ts:40-47 — two subscriptions, mounted by every BaseNode via useNodeArtifacts (BaseNode.tsx:534), plus PreviewNode. NodeOutputs.tsx:58-61, PreviewNode.tsx:268-282. NodeOutput.tsx connect-drag fallback (per output handle). Zustand runs all listeners on every set(); dragging updates nodes at ~60fps. At 300 nodes that is ~270k id comparisons per frame in selector work alone, before React renders anything. Same cost per property keystroke. Fix: maintain a nodesById index in NodeStore (updated wherever nodes is set) and make findNode O(1) — one change fixes all call sites. For useNodeGenerations specifically, selected_generation(s) are already on the data prop RF passes to the node — thread them in as parameters. H5. The canvas subscribes to the global status maps — every execution message re-renders it ReactFlowWrapper.tsx:589-590: const edgeStatuses = useResultsStore((state) => state.edges); const nodeStatuses = useStatusStore((state) => state.statuses); Both maps are rebuilt with a full spread on every write (StatusStore.ts:121, ResultsStore.ts:335-341) and are keyed globally (workflowId:jobId:nodeId). Every node_update/edge_update from any workflow and any job — including runs in other tabs — re-renders the wrapper and re-runs the useProcessedEdges status pass, an O(E) map that produces fresh edge objects for active edges each time (useProcessedEdges.ts: 387-424), so RF re-diffs edges per message while anything is running. Fix: key the stores per run (statusesByJob[jobKey][nodeId]) so the canvas selects only its focused run’s sub-map (identity stable when other jobs write), and make the status pass return prior edge objects when that edge’s status/counter did not change. H6. CSS filter chains on every node — and on every edge and handle in light mode styles/nodes.states.css:6-27: every node carries a filter: drop-shadow(...) at all times; selected nodes get two stacked drop-shadows (one with 24px blur) on top of the box-shadow the node body already draws (selectionStyles.ts:97-101). Since clicking a node selects it, every drag is a selected-node drag: the filter region re-rasterizes per frame. nodes.base.css also transitions filter on all nodes. The code already knows filters are costly (CRISP_NO_BLUR_STYLES sets filter: none on the inner body) — the outer rule re-adds the cost. styles/handle_edge_tooltip.css:310-317: light mode applies filter: saturate(...) brightness(...) to every handle and every edge path, permanently. Filters on SVG paths disable cheap stroke repaint; node drags repaint every attached edge through the filter pipeline. CustomEdge.tsx:88-101: selected edges get an inline SVG drop-shadow while their path changes every drag frame (node-driven edge selection makes this the common case). handle_edge_tooltip.css:656-665: the run-time edgeFlow dash animation adds filter: drop-shadow(...) per animated edge and will-change: stroke-dashoffset (buys nothing — dash offset is main-thread paint), running for offscreen edges too. Fix: replace node drop-shadows with the existing body box-shadow; bake the light-mode desaturation into the palette instead of filtering at paint time; approximate the selected-edge glow with a second wider translucent stroke; drop filter/will-change from the animated edge rule. Medium severity M1. Streaming amplification: per-token store writes, whole-node re-renders, O(k²) accumulation ResultsStore.addChunk (:738-749) spreads the full chunks map per token; setOutputResult with append (:606-654) copies the whole accumulated array per item — O(k²) over a k-item stream. Audio already has coalescing (200ms flush, workflowUpdates.ts:49-93); text and other streams do not. BaseNode.tsx:534 subscribes to the accumulated chunk string at the top of the node, so each token re-renders the entire node (amplified by H3), and ChunkDisplay.tsx:78 re-parses the full markdown per token — O(n²) per stream. NodeLogsDialog is mounted inside every NodeHeader (NodeHeader.tsx:384-389) and its memos re-slice/join the full log history on every appended line even while closed (NodeLogs.tsx:64-91); NodeHeader itself subscribes to the log array just to show a count. workflowUpdates.ts:587-599 JSON.stringifys every non-string output into a log line — a multi-MB image ref serializes before the 20k-char truncation runs. Fix: reuse the audio coalescing for text chunks/appends; move chunk/terminal subscriptions into leaf components (NodeProgress already models this); mount the logs dialog only when open and select logsByNode[key]?.length for the count; summarize rich types before stringifying. M2. nodeTypes identity is tied to MetadataStore — map replacement remounts all nodes ReactFlowWrapper.tsx:399-422 memoizes on useMetadataStore((s) => s.nodeTypes), whose identity changes on every setNodeTypes/addNodeType. Loading a workflow with unknown types fires one addNodeType per type (NodeStore.ts:257-261); each replaces the map, and a changed nodeTypes identity makes ReactFlow remount every node — full DOM teardown and handle re-measure, possibly several times in a row. Related: NodeStore’s metadata subscriber (NodeStore.ts:267-281) re-runs sanitizeGraph and replaces the edges array unconditionally on every MetadataStore write (including fal/kie pricing merges), invalidating the expensive structural edge pass in every open tab. Fix: unknown types already fall through to default: PlaceholderNode, so the per-type placeholder registration can likely be dropped; otherwise batch into one store write and bail when keys are unchanged. Make the sanitize subscriber a no-op when the sanitized edges are unchanged. M3. Every Ctrl/Meta press re-renders every property editor on the canvas PropertyField.tsx:94-99 subscribes each field to isKeyPressed("Control")/isKeyPressed("Meta") and threads the result into PropertyInput’s memo comparator. Every modifier press and release (copy/paste, shortcuts, Cmd-click) re-renders hundreds of property editors. The only consumer is a mouse handler that already receives event.ctrlKey/event.metaKey (PropertyInput.tsx:392). Fix: delete both subscriptions; read the event modifiers (or useKeyPressedStore.getState() inside the handler). M4. Per-frame O(N) bookkeeping in the wrapper, doubled by a redundant second store write The layout-fingerprint effect (ReactFlowWrapper.tsx:295-359) rebuilds a Map of signature strings for all nodes on every nodes change — every drag frame. nodesStructureKey (useProcessedEdges.ts:76-94) joins an O(N) string per frame (multi-KB allocation on large graphs). setWorkflowDirty(true) is called unconditionally after graph mutations (NodeStore.ts:511-513 and ~15 other sites) with no change guard, so every drag frame performs two set() calls — every mounted selector in the graph runs twice per frame. Fix: guard setWorkflowDirty on actual change (one line); skip the fingerprint/structure-key passes while a drag is in progress (the isSelecting trick already exists for marquee drags), or key them on a structural version counter bumped by the store. M5. updateNodeInternals over all nodes on edge changes, six refresh passes per layout change ReactFlowWrapper.tsx:276-290 refreshes handle bounds for all node ids on every edge add/remove — O(N × handles) getBoundingClientRect calls for a change touching two nodes. scheduleNodeInternalsRefresh.ts then runs each refresh six times over ~160ms (raf, raf, timeout 0/24/72/160). Fix: refresh only the affected edge’s endpoints (the withEdgeNeighborNodeIds helper already exists for this); trim the scheduler to raf-raf plus one late fallback. M6. Connection-drag costs: per-pointermove BFS and a whole-graph handle sweep isValidConnection (useConnectionEvents.ts:30-40 → graphCycle.ts) rebuilds a full adjacency map over all edges and BFSes per call, and RF calls it from pointermove whenever the wire is near a handle. Grabbing a wire re-renders every handle in the graph (they subscribe to ConnectionStore connect state); each handle re-runs hashType (typeFilterUtils.ts:6-13 — recursive string building, uncached) and NodeOutput falls back to O(N) findNode per output handle. Cost lands twice per wire drag (grab and release) — a visible hitch at 100+ nodes. Fix: precompute reachable-set once in onConnectStart (O(1) checks per move); cache hashType per TypeMetadata in a WeakMap; both shrink further once H4’s nodesById index exists. M7. Undo history: full-graph equality scan per tracked write, one entry per keystroke, and phantom entries from runtime echoes zundo is configured with equality: customEquality (NodeStore.ts:1423-1431), which iterates every node with three shallow() compares each (customEquality.ts:31-63) — on every tracked set(). Property typing is undebounced (updateNodeProperties per keystroke), so each keystroke pays the scan and pushes a history entry (undo steps back one character). Drags are correctly pause()/resume()d. Worse: node_update websocket echoes call updateNodeData while tracking is active (workflowUpdates.ts:1146-1173) — running a workflow creates undo entries the user never authored and marks the workflow dirty (autosave version churn). Related: BaseNode’s overlay re-measure (BaseNode.tsx:767-778) fires updateNode per node at run completion, each dirtying the workflow. Fix: pause temporal (and skip the dirty flag) around runtime echoes and re-measures; group/debounce property-edit history. M8. onlyRenderVisibleElements={false} — every cost above scales with total nodes, not visible nodes ReactFlowWrapper.tsx:770. All nodes, property editors, handles, and tooltips stay mounted regardless of viewport; collapsed nodes also keep their full subtree mounted (CSS-only collapse via display: none). RF12’s visibility culling is much cheaper than v11’s. Worth a benchmark, at least above a node-count threshold, and worth skipping field-editor rendering for collapsed nodes (handles must stay mounted for edge anchoring — HandleColumn/ HandleOnlyField already provide that). Low severity (grouped) Stale-edge correctness bug in PreviewNode: PreviewNode.tsx:237-243 memoizes the incoming edge on [getEdges, props.id] — both stable — so it computes once at mount and never updates when the preview is re-wired. Also uses fast-deep-equal as a subscriber equality over potentially huge property values (:268-282); property values are replaced immutably, so reference equality suffices. FindInWorkflowDialog subscribes to nodes while closed (useFindInWorkflow.ts:85) — re-renders per drag frame to return null. Latent footgun: useEdgeHandlers.ts:77-121 (onEdgeMouseEnter/Leave) mutates store edges in place and would trigger a full O(E) structural reprocess per hover — currently unwired; delete or rewrite before wiring. Every mounted OutputRenderer keeps a permanent document-level mousemove listener (OutputRenderer.tsx:325-336); attach on mousedown instead. transition: all on every handle (handle_edge_tooltip.css:298) plus layout-affecting width/height transitions when is-connecting flips — violates the repo’s own MOTION rules; list the animated properties. :has() selectors in per-node CSS (nodes.states.css:30, nodes.zoomed.css) make style invalidation walk node subtrees during runs; set a class from JS instead. getNodeCategoryColor rebuilds its color table per node per minimap frame (ColorUtils.ts:151-165); hoist to module scope. panOnDrag array recreated per render (useDragHandlers.ts:291-294); whole-settings subscription in the same hook; inline 30-line onNodeDoubleClick arrow (ReactFlowWrapper.tsx:823-851). NodeEditor subscribes to the whole upload store (NodeEditor.tsx:65) — re-renders per upload progress tick. Marquee-end edge hit-testing does one document.querySelector per edge (useSelectionEvents.ts:139-156); use one querySelectorAll. PropertyInput.tsx:228 subscribes to the entire metadata map for a reset-to-default handler; read getState() in the callback. KieCreditsFooter subscribes every node to two hot stores before its “not a KIE node” early return (KieCreditsFooter.tsx:96). useAutoEnableNodePacks allocates an O(N) types array on every store update (useAutoEnableNodePacks.ts:31-33); cache by nodes identity. What is already done well Preserve these patterns when fixing the above: useNodes defaults to shallow equality (NodeContext.tsx:64); useShallow is used consistently for multi-value selections; no whole-store no-selector subscriptions in the canvas path. Per-node execution state is correctly isolated: status/progress/error/ duration lookups are O(1) keyed maps returning stable references — a progress tick on node A does not re-render node B, and NodeProgress is a leaf subscriber mounted only while running. useProcessedEdges two-stage split (heavy structural pass keyed by a structure fingerprint; light status pass with whole-result identity bail), O(1) lookup maps, and the incomingByTargetHandle index that removed a prior O(E²). Identity-cached selectors: selectedNodeIdsSelector (ReactFlowWrapper.tsx:629-651), getSelectedNodeCount/getSelection closures, BaseNode’s hasConnectedInput/hasControlEdge, GroupNode’s children-status selector. History is paused during drags, slider scrubs, and group operations; partialize keeps viewport/hover out of history; selection changes don’t create undo entries. Audio streaming is fully coalesced (200ms batches, single-copy append, rolling caps, React-free worklet fast path) — the template for text streams. Connection drag writes no store state per pointermove; the connectability matrix is precomputed at metadata load; EdgeGradientDefinitions renders one gradient per active type pair with stabilized keys; CustomEdge/ControlEdge have explicit memo comparators. AxisMarker and GhostNode handle per-frame updates via refs/rAF with zero React re-renders; viewport is persisted only on onMoveEnd; drag intersection checks are throttled to 50ms. LogStore is pre-keyed by node with caps and truncation; autosave is interval-based, not per-change; no structuredClone/JSON round-trips on hot paths. Suggested fix order # Fix Effort Pays off on 1 H1: identity-preserving syncReactFlowNodeChromeClass tiny every drag frame 2 H2: boolean zoom selector instead of useViewport (3 components) tiny every pan/zoom frame 3 H3: use the memoized node chrome exports; narrow NodeHeader comparator small every streaming chunk/hover 4 M3: drop Ctrl/Meta subscriptions in PropertyField tiny every shortcut press 5 M4: setWorkflowDirty change guard tiny every graph mutation 6 H4: nodesById index for findNode small drag frames, keystrokes, connect drags 7 H5: per-run keying for status/results maps medium execution messages 8 H6: remove node/edge/handle CSS filters small paint during drag, light mode 9 M1: coalesce text-stream writes; leaf-subscribe chunks; lazy logs dialog medium streaming runs 10 M2: stabilize nodeTypes; conditional sanitize small workflow load 11 M8: benchmark onlyRenderVisibleElements on large graphs experiment large graphs Test coverage gaps The existing suites (web/src/__tests__/performance/) guard LogStore bucket identity, selection-count memoization, and selector patterns — none cover the top findings. Worth adding once fixed: untouched node objects keep identity through onNodesChange (locks in H1); ReactFlowWrapper does not re-render on another job’s setStatus (H5); a streamed chunk re-renders the chunk display leaf, not BaseNode (M1); modifier keypress does not re-render PropertyInput (M3). The pattern-level assertions in componentPerformance.test.tsx are gated behind PERF_TESTS=true and do not enforce in CI.--- # Node Implementation Examples Source: https://docs.nodetool.ai/developer/node-examples.html Real Examples from Codebase This document provides annotated examples of real nodes from the NodeTool codebase. Each example highlights specific patterns and implementation details to help you build your own nodes. 1. Simple Processing Nodes These nodes take one or more inputs, perform a calculation, and return a single result. They are the building blocks of most workflows. Example: Text Concatenation Pattern: BaseNode with simple inputs and a single return value. class Concat(BaseNode): """ Concatenates two text inputs into a single output. text, concatenation, combine, + """ # Define inputs using Pydantic fields a: str = Field(default="") b: str = Field(default="") @classmethod def get_title(cls): return "Concatenate Text" # The return type annotation (str) tells the UI what this node outputs async def process(self, context: ProcessingContext) -> str: return self.a + self.b Key Takeaways: Use Field(default="") to define inputs. The process method must be async. The return type annotation is crucial for the UI to validate connections. 2. Structured Output Nodes Sometimes a node needs to return multiple values (e.g., an audio transcription that returns both the text and the detected language). Example: Automatic Speech Recognition Pattern: Using TypedDict to define multiple named outputs. class AutomaticSpeechRecognition(BaseNode): """ Automatic speech recognition node. audio, speech, recognition """ # Define the output structure class OutputType(TypedDict): text: str language: str model: ASRModel = Field(...) audio: AudioRef = Field(...) async def process(self, context: ProcessingContext) -> OutputType: # ... logic to transcribe audio ... return { "text": "Hello world", "language": "en" } Key Takeaways: Define a TypedDict named OutputType inside your class. Set the return annotation of process to OutputType. Return a dictionary matching the structure. 3. Streaming Nodes For operations that take a long time or produce results incrementally (like reading a large folder of images), use a generator. Example: Load Image Folder Pattern: gen_process with AsyncGenerator. class LoadImageFolder(BaseNode): """ Load all images from a folder. image, load, folder """ folder: str = Field(default="") class OutputType(TypedDict): image: ImageRef path: str # Use gen_process instead of process async def gen_process( self, context: ProcessingContext ) -> AsyncGenerator[OutputType, None]: # Iterate over files and yield results one by one for path in self.iter_files(self.folder): image = await context.image_from_bytes(...) yield {"image": image, "path": path} Key Takeaways: Use gen_process instead of process. Return type is AsyncGenerator[OutputType, None]. yield results as they become available. This allows downstream nodes to start processing immediately. 4. Dynamic Nodes Some nodes need to adapt their inputs based on user configuration. For example, a template node might need different inputs depending on the variables in the template string. Example: Format Text (Jinja2) Pattern: _is_dynamic flag and _dynamic_properties. {% raw %} class FormatText(BaseNode): """ Replaces placeholders in a string with dynamic inputs. """ # 1. Flag this node as dynamic _is_dynamic: ClassVar[bool] = True template: str = Field(default="Hello {{ name }}!") async def process(self, context: ProcessingContext) -> str: # 2. Access the dynamic inputs provided by the user # These are inputs that were added to the node at runtime dynamic_inputs = self.get_dynamic_properties() # Render the template using these inputs return self.render_template(self.template, **dynamic_inputs) {% endraw %} Key Takeaways: Set _is_dynamic = True. The UI will allow users to add arbitrary inputs to this node. Access these inputs via self.get_dynamic_properties() or self._dynamic_properties. 5. Working with Assets Nodes often need to load or save heavy assets like images or audio. Example: Save Text to File Pattern: Using ProcessingContext to create assets. class SaveText(BaseNode): """ Saves input text to a file. """ text: str = Field(default="") filename: str = Field(default="output.txt") async def process(self, context: ProcessingContext) -> TextRef: # 1. Create the asset using the context asset = await context.create_asset( filename=self.filename, mime_type="text/plain", data=self.text.encode("utf-8") ) # 2. Create a reference to return result = TextRef(uri=asset.uri, asset_id=asset.id) # 3. Notify the UI that a file was saved (optional but recommended) context.post_message(SaveUpdate( node_id=self.id, name=self.filename, value=result, output_type="text" )) return result Key Takeaways: Never write directly to disk if you can avoid it. Use context.create_asset. Return a Ref object (like TextRef, ImageRef) so other nodes can use the asset. Use SaveUpdate to show the saved file in the UI’s “Outputs” tab. 6. Control Flow Nodes can control the execution flow of the graph. Example: If Node Pattern: Conditional logic in gen_process. class If(BaseNode): """ Conditionally executes branches. """ condition: bool = Field(default=False) value: Any = Field(default=None) class OutputType(TypedDict): if_true: Any if_false: Any async def gen_process(self, context: Any) -> AsyncGenerator[OutputType, None]: if self.condition: # Only emit to the 'if_true' output yield {"if_true": self.value, "if_false": None} else: # Only emit to the 'if_false' output yield {"if_true": None, "if_false": self.value} Key Takeaways: Control flow nodes often use gen_process to conditionally yield results. By yielding None for a specific output, you effectively stop execution on that branch (downstream nodes won’t trigger). 7. Model Nodes (HuggingFace / model-backed) Nodes that load a machine-learning model (HuggingFace, diffusers, …) extend HuggingFacePipelineNode — a BaseNode subclass with extra hooks for loading, device placement, and surfacing downloadable checkpoints. This is the pattern the nodetool-huggingface package uses for every node. Example: A text-to-audio model node class MyAudioGen(HuggingFacePipelineNode): """ One-line description shown in the UI. audio, generation, text-to-audio """ model: HFTextToAudio = Field( default=HFTextToAudio(repo_id="org/my-model"), description="The model to use.", ) prompt: str = Field(default="", description="Text prompt.") seed: int = Field(default=-1, ge=-1) _pipeline: Any = None @classmethod def get_title(cls) -> str: return "My Audio Gen" @classmethod def get_basic_fields(cls) -> list[str]: # Fields rendered on the node body; the rest live in the expanded view. return ["model", "prompt"] @classmethod def get_recommended_models(cls) -> list[HuggingFaceModel]: # Checkpoints offered for one-click download in the Model Manager. return [ HFTextToAudio( repo_id="org/my-model", allow_patterns=["**/*.safetensors", "**/*.json", "**/*.txt"], ), ] def get_model_id(self) -> str: return self.model.repo_id async def preload_model(self, context: ProcessingContext): # Called once before process(); load the model here. from diffusers import MyPipeline self._pipeline = await self.load_model( context=context, model_class=MyPipeline, model_id=self.get_model_id(), torch_dtype=available_torch_dtype(), device="cpu", # the runner moves it to the GPU/MPS via move_to_device ) async def move_to_device(self, device: str): if self._pipeline is not None: self._pipeline.to(device) async def process(self, context: ProcessingContext) -> AudioRef: # run_pipeline_in_thread keeps the asyncio event loop responsive. output = await self.run_pipeline_in_thread(prompt=self.prompt) return await context.audio_from_numpy(output.audios[0], 24000) Key Takeaways: Extend HuggingFacePipelineNode and declare a model: field of the matching HF type (HFTextToImage, HFTextToVideo, HFTextToAudio, HFTextGeneration, …). get_recommended_models() lists checkpoints the Model Manager can download; get_basic_fields() selects which fields show on the node body; get_title() sets the display name. preload_model() loads the model, move_to_device() places it on the accelerator, and process() (or gen_process()) runs inference. Use run_pipeline_in_thread() so long-running inference doesn’t block the event loop. After adding or changing a node, regenerate the package metadata so the app can discover it: nodetool package scan To make a model usable outside the node graph (e.g. via an agent or the generation API), implement the matching method on the package’s local provider — e.g. text_to_audio() for the TEXT_TO_AUDIO capability.--- # Node Implementation Patterns Source: https://docs.nodetool.ai/developer/node-patterns.html Overview This guide covers the key implementation patterns you will encounter when building custom nodes for NodeTool. Every node extends BaseNode from @nodetool-ai/node-sdk, declares its inputs with the @prop decorator, and implements a process() or genProcess() method that reads input values from this.<field> and returns an outputs record. The optional argument is a ProcessingContext from @nodetool-ai/runtime. Simple Single-Output The most common pattern. The node declares one or more @prop inputs and returns a single keyed output. Use metadataOutputTypes to tell the UI the output’s type. This pattern is taken from ConstantStringNode in packages/core-nodes/src/nodes/constant.ts: import { BaseNode, prop } from "@nodetool-ai/node-sdk"; export class ConstantStringNode extends BaseNode { static readonly nodeType = "nodetool.constant.String"; static readonly title = "String"; static readonly description = "Represents a string constant in the workflow.\n text, string, characters"; static readonly metadataOutputTypes = { output: "str", }; @prop({ type: "str", default: "", title: "Value" }) declare value: any; async process(): Promise<Record<string, unknown>> { return { output: this.value ?? "" }; } } Key points: metadataOutputTypes maps each output key to its type string ("str", "int", "float", "bool", "image", "audio", etc.). The engine assigns connected input values onto the instance before calling process(), so read them as this.field. The return value is always Record<string, unknown> – a plain object whose keys match the output names. Multi-Output When a node produces more than one output, list every key in metadataOutputTypes. Each key becomes a separate output connector in the UI. This pattern is taken from IfNode in packages/core-nodes/src/nodes/control.ts. It reads its inputs as a buffered set (inputMode = "buffered") and routes the value input to whichever branch matches the condition. The outputCorrelation map declares that both outputs forward the value input — so each output carries the same correlation lineage as its source, which is how the scheduler knows the branch is a passthrough rather than a new value: import type { InputMode, OutputCorrelation } from "@nodetool-ai/protocol"; export class IfNode extends BaseNode { static readonly nodeType = "nodetool.control.If"; static readonly title = "If"; static readonly description = "Conditionally executes one of two branches based on a condition.\n" + " control, flow, condition, logic"; static readonly metadataOutputTypes = { if_true: "any", if_false: "any", }; static readonly inputMode: InputMode = "buffered"; static readonly outputCorrelation: Record<string, OutputCorrelation> = { if_true: { kind: "forward", source: "value" }, if_false: { kind: "forward", source: "value" }, }; @prop({ type: "bool", default: false, title: "Condition" }) declare condition: any; @prop({ type: "any", default: [], title: "Value" }) declare value: any; async process(): Promise<Record<string, unknown>> { const condition = Boolean(this.condition ?? false); const value = this.value ?? null; if (condition) { return { if_true: value, if_false: null }; } return { if_true: null, if_false: value }; } } Key points: Every key declared in metadataOutputTypes must appear in the returned object. inputMode is one of "buffered" (collect a matched set of inputs, then call process() once), "stream" (consume inputs as an async stream via run()), or "controlled". Most nodes leave it undefined (the default buffered behavior). outputCorrelation maps each output to how it relates to its source input. A { kind: "forward", source: "<input>" } entry means the output forwards that input’s correlation token unchanged — there is no separate isStreamingOutput flag; streaming behavior is inferred from forward/iteration correlation. Streaming with genProcess() For nodes that emit multiple results over time, implement genProcess() as an async generator. The engine calls genProcess() instead of process() when it is defined. This pattern is taken from ForEachNode in packages/core-nodes/src/nodes/control.ts. It buffers its input list, then emits each element as a separate iteration — outputCorrelation declares the outputs as kind: "iteration", which mints a fresh correlation token per emitted item so downstream nodes treat each one as a distinct logical value: import type { InputMode, OutputCorrelation } from "@nodetool-ai/protocol"; export class ForEachNode extends BaseNode { static readonly nodeType = "nodetool.control.ForEach"; static readonly title = "For Each"; static readonly description = "Iterate over a list and emit each item sequentially.\n" + " iterator, loop, list, sequence"; static readonly metadataOutputTypes = { output: "any", index: "int", }; static readonly inputMode: InputMode = "buffered"; static readonly outputCorrelation: Record<string, OutputCorrelation> = { output: { kind: "iteration", source: "__execution__", group: "items" }, index: { kind: "iteration", source: "__execution__", group: "items" }, }; @prop({ type: "list[any]", default: [], title: "Input List" }) declare input_list: any; async process(): Promise<Record<string, unknown>> { return {}; } async *genProcess(): AsyncGenerator<Record<string, unknown>> { const values = (this.input_list ?? []) as unknown[]; const list = Array.isArray(values) ? values : [values]; for (const [index, item] of list.entries()) { yield { output: item, index }; } } } Key points: You must still provide a process() stub (it can return {}) — the base class requires it. Each yield sends one batch of outputs to downstream nodes. genProcess() is detected automatically (the base class checks whether the subclass overrides it) — there is no isStreamingOutput flag to set. The outputCorrelation iteration kind tells the scheduler each yielded value is a new correlated item. Stateful Collector Some nodes accumulate values across multiple invocations within a single workflow run. Hold the state in a private instance field and use initialize() to reset it at the start of each run. This pattern is taken from CollectTextNode in packages/text-nodes/src/nodes/text-extra.ts: export class CollectTextNode extends BaseNode { static readonly nodeType = "nodetool.text.Collect"; static readonly title = "Collect Text"; static readonly description = "Collects streaming text inputs into a single concatenated string.\n" + " text, collect, stream, aggregate"; static readonly metadataOutputTypes = { output: "str", }; private _items: string[] = []; @prop({ type: "str", default: "", title: "Input Item" }) declare input_item: any; @prop({ type: "str", default: "", title: "Separator" }) declare separator: any; async initialize(): Promise<void> { this._items = []; } async process(): Promise<Record<string, unknown>> { this._items.push(String(this.input_item ?? "")); const sep = String(this.separator ?? ""); return { output: this._items.join(sep) }; } } Key points: Private instance fields (like _items) hold state between invocations within a single run. initialize() runs once at the start of each workflow execution – use it to clear accumulated state. The node instance is reused across the items it receives, so process() is called once per incoming value and appends to _items. Media Refs Images, audio, and video are passed between nodes as ref objects – plain objects with uri, data, and metadata fields. Nodes load bytes from the ref and return new refs after processing. @nodetool-ai/runtime provides a shared loadMediaRefBytes(ref, context?) helper. The pattern below shows the manual equivalent for reference: import sharp from "sharp"; // Helper: extract bytes from an image ref async function imageBytesAsync(image: unknown): Promise<Uint8Array> { if (!image || typeof image !== "object") return new Uint8Array(); const ref = image as { uri?: string; data?: Uint8Array | string }; if (ref.data) { return ref.data instanceof Uint8Array ? ref.data : Uint8Array.from(Buffer.from(ref.data, "base64")); } if (typeof ref.uri === "string" && ref.uri) { const response = await fetch(ref.uri); return new Uint8Array(await response.arrayBuffer()); } return new Uint8Array(); } export class ResizeNode extends BaseNode { // ... @prop({ type: "image", default: { type: "image", uri: "", data: null }, title: "Image" }) declare image: any; @prop({ type: "int", default: 512, title: "Width", min: 0, max: 4096 }) declare width: any; @prop({ type: "int", default: 512, title: "Height", min: 0, max: 4096 }) declare height: any; async process(): Promise<Record<string, unknown>> { const bytes = await imageBytesAsync(this.image ?? {}); const width = Number(this.width ?? 512); const height = Number(this.height ?? 512); const outputBytes = await sharp(bytes).resize(width, height).toBuffer(); return { output: { data: Buffer.from(outputBytes).toString("base64"), width, height, }, }; } } The same approach applies to audio refs (see audioBytesAsync in audio.ts). The key insight is that media data travels as base64-encoded data or as a uri that the node fetches on demand. Enum Inputs Use @prop with type: "enum" and a values array to present a dropdown in the UI. This pattern is taken from LengthTextNode in packages/text-nodes/src/nodes/text-extra.ts: @prop({ type: "enum", default: "characters", title: "Measure", description: "Choose whether to count characters, words, or lines", values: ["characters", "words", "lines"], }) declare measure: any; In your process() method, read and cast the value from this: const measure = String(this.measure ?? "characters"); Secret Access Nodes that call external APIs declare the keys they need in requiredSettings. Before calling process(), the base class resolves each key from the runtime’s secret store via ProcessingContext.getSecret() and exposes them as this._secrets. This pattern is adapted from the OpenAI nodes in packages/llm-nodes/src/nodes/openai.ts (e.g. openai.text.Embedding, whose output type is "list"): export class EmbeddingNode extends BaseNode { static readonly nodeType = "openai.text.Embedding"; static readonly title = "Embedding"; static readonly description = "Generate vector representations of text."; static readonly requiredSettings = ["OPENAI_API_KEY"]; @prop({ type: "str", default: "", title: "Input" }) declare input: any; static readonly metadataOutputTypes = { output: "list[float]" }; async process(): Promise<Record<string, unknown>> { const apiKey = this._secrets.OPENAI_API_KEY || process.env.OPENAI_API_KEY || ""; if (!apiKey) throw new Error("OPENAI_API_KEY is not configured"); const response = await fetch("https://api.openai.com/v1/embeddings", { method: "POST", headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json", }, body: JSON.stringify({ input: this.input, model: "text-embedding-3-small" }), }); const data = await response.json(); return { output: data.data[0].embedding }; } } Key points: requiredSettings is a static string array of secret key names. The base class resolves each key from the active ProcessingContext and assigns them to this._secrets before process() runs. Always fall back to process.env for local development. Best Practices Input Resolution Property values for @prop fields are populated on the instance before process() is called. Resolve with: const value = this.field ?? defaultValue; The engine handles connected edges, manually set values, and declared defaults — by the time process() runs, this.field already holds the effective value. Async / Await All process() and genProcess() methods are async. Use await for any I/O – file reads, HTTP calls, or image processing – to keep the workflow engine responsive. metadataOutputTypes Always declare metadataOutputTypes so the UI can render the correct output connectors and validate connections between nodes. If you omit it, the node will have no visible outputs. Error Handling Let exceptions propagate or throw Error with a clear message. The engine catches them and displays the error on the node in the UI. Docstring Format The description static field serves double duty: the first line is the node’s description in the UI; subsequent indented lines are search keywords. static readonly description = "Short description of what the node does.\n" + " keyword1, keyword2, keyword3"; File Structure Nodes are organized by category across the node packages — packages/core-nodes/ (constants, control flow, math), packages/text-nodes/, packages/image-nodes/, packages/llm-nodes/, packages/automation-nodes/, packages/integration-nodes/, and others. (The old packages/base-nodes/src/nodes/ directory no longer holds individual node sources — it now only re-exports/aggregates.) Each file exports an array of node classes (e.g., CONTROL_NODES, TEXT_NODES) and the package index registers them all. The system automatically discovers all registered classes inheriting from BaseNode.--- # Node Implementation Quick Reference Source: https://docs.nodetool.ai/developer/node-reference.html Essential Node Templates import { BaseNode, prop } from "@nodetool-ai/node-sdk"; import type { InputMode, OutputCorrelation } from "@nodetool-ai/protocol"; // SIMPLE PROCESSING NODE export class SimpleNode extends BaseNode { static readonly nodeType = "mypackage.example.Simple"; static readonly title = "Simple Node"; static readonly description = "Clear description of what this node does.\n" + " keyword1, keyword2, keyword3"; static readonly metadataOutputTypes = { output: "str", }; @prop({ type: "str", default: "", title: "Input Value", description: "Help text" }) declare input_value: any; @prop({ type: "int", default: 100, title: "Threshold", min: 0, max: 255 }) declare threshold: any; async process(): Promise<Record<string, unknown>> { return { output: `Result: ${String(this.input_value ?? "")}` }; } } // MULTI-OUTPUT NODE export class MultiOutputNode extends BaseNode { static readonly nodeType = "mypackage.example.MultiOutput"; static readonly title = "Multi Output"; static readonly description = "Produces multiple outputs.\n multi, output"; static readonly metadataOutputTypes = { if_true: "any", if_false: "any", }; // Forward correlation: both outputs carry the `value` input's correlation // token unchanged. Streaming behavior is inferred from this — there is no // isStreamingOutput flag. static readonly inputMode: InputMode = "buffered"; static readonly outputCorrelation: Record<string, OutputCorrelation> = { if_true: { kind: "forward", source: "value" }, if_false: { kind: "forward", source: "value" }, }; @prop({ type: "bool", default: false, title: "Condition" }) declare condition: any; @prop({ type: "any", default: [], title: "Value" }) declare value: any; async process(): Promise<Record<string, unknown>> { if (this.condition) { return { if_true: this.value, if_false: null }; } return { if_true: null, if_false: this.value }; } } // STREAMING / GENERATOR NODE export class StreamingNode extends BaseNode { static readonly nodeType = "mypackage.example.Streaming"; static readonly title = "Streaming Node"; static readonly description = "Emit multiple items.\n stream, iterate"; static readonly metadataOutputTypes = { output: "any", index: "int", }; // Each yielded item is a new correlated value (iteration). genProcess() // is detected automatically — no isStreamingOutput flag. static readonly inputMode: InputMode = "buffered"; static readonly outputCorrelation: Record<string, OutputCorrelation> = { output: { kind: "iteration", source: "__execution__", group: "items" }, index: { kind: "iteration", source: "__execution__", group: "items" }, }; @prop({ type: "list[any]", default: [], title: "Input List" }) declare input_list: any; async process(): Promise<Record<string, unknown>> { return {}; } async *genProcess(): AsyncGenerator<Record<string, unknown>> { const values = (this.input_list ?? []) as unknown[]; const list = Array.isArray(values) ? values : [values]; for (const [index, item] of list.entries()) { yield { output: item, index }; } } } // STATEFUL COLLECTOR NODE export class CollectorNode extends BaseNode { static readonly nodeType = "mypackage.example.Collector"; static readonly title = "Collector"; static readonly description = "Collect streamed items.\n collect, aggregate"; static readonly metadataOutputTypes = { output: "list[any]", }; private _items: unknown[] = []; @prop({ type: "any", default: [], title: "Input Item" }) declare input_item: any; async initialize(): Promise<void> { this._items = []; } async process(): Promise<Record<string, unknown>> { this._items.push(this.input_item); return { output: [...this._items] }; } } Common @prop Patterns // Text input @prop({ type: "str", default: "", title: "Text" }) declare text: any; @prop({ type: "str", default: "", title: "Text", description: "Help text" }) declare text: any; // Number with constraints @prop({ type: "int", default: 0, title: "Count", min: 0, max: 100 }) declare count: any; @prop({ type: "float", default: 0.5, title: "Threshold", min: 0.0, max: 1.0 }) declare threshold: any; // Boolean @prop({ type: "bool", default: false, title: "Enabled" }) declare enabled: any; // Optional (nullable) @prop({ type: "str", default: null, title: "Label" }) declare label: any; // List @prop({ type: "list[str]", default: [], title: "Tags", description: "List of tags" }) declare tags: any; @prop({ type: "list[any]", default: [], title: "Items" }) declare items: any; // Enum choices (dropdown in UI) @prop({ type: "enum", default: "option_a", title: "Choice", values: ["option_a", "option_b", "option_c"], }) declare choice: any; // Model selections @prop({ type: "language_model", default: null, title: "Model", required: true }) declare model: any; @prop({ type: "image_model", default: null, title: "Image Model", required: true }) declare image_model: any; @prop({ type: "tts_model", default: null, title: "TTS Model", required: true }) declare tts_model: any; // Asset references @prop({ type: "image", default: { type: "image", uri: "", data: null }, title: "Image" }) declare image: any; @prop({ type: "audio", default: { type: "audio", uri: "", data: null }, title: "Audio" }) declare audio: any; @prop({ type: "video", default: { type: "video", uri: "", data: null }, title: "Video" }) declare video: any; @prop({ type: "document", default: { type: "document", uri: "", data: null }, title: "Document" }) declare document: any; @prop({ type: "folder", default: { type: "folder", uri: "" }, title: "Folder" }) declare folder: any; // Data structures @prop({ type: "dataframe", default: { type: "dataframe", uri: "", data: null }, title: "Data" }) declare dataframe: any; @prop({ type: "dict[str, any]", default: {}, title: "Config" }) declare config: any; ProcessingContext Essentials The optional argument to process() and genProcess() is a ProcessingContext from @nodetool-ai/runtime. It provides access to provider predictions, secrets, and runtime services. Property values for declared @prop fields are assigned to this before process() is called — read them directly from this.<field>. import type { ProcessingContext } from "@nodetool-ai/runtime"; async process(context?: ProcessingContext): Promise<Record<string, unknown>> { // Access injected secrets (requires static requiredSettings). // The base class resolves keys from context and exposes them on this._secrets. const apiKey = this._secrets.MY_API_KEY ?? process.env.MY_API_KEY ?? ""; // Run a provider prediction (image generation, TTS, etc.) if (context && typeof context.runProviderPrediction === "function") { const output = await context.runProviderPrediction({ provider: "openai", capability: "text_to_image", model: "gpt-image-1", params: { prompt: "a cat" }, }); } // Stream a provider prediction (e.g., TTS chunks) if (context && typeof context.streamProviderPrediction === "function") { for await (const chunk of context.streamProviderPrediction({ provider: "openai", capability: "text_to_speech", model: "tts-1", params: { text: "hello" }, })) { // process each chunk } } // Resolve a secret manually (bypasses requiredSettings) if (context && typeof context.getSecret === "function") { const secret = await context.getSecret("SOME_KEY"); } return { output: "result" }; } Working with Media Bytes Nodes handle media as ref objects. Extract bytes, process them, and return a new ref: // Image: load bytes from ref async function imageBytesAsync(image: unknown): Promise<Uint8Array> { if (!image || typeof image !== "object") return new Uint8Array(); const ref = image as { uri?: string; data?: Uint8Array | string }; if (ref.data) { return ref.data instanceof Uint8Array ? ref.data : Uint8Array.from(Buffer.from(ref.data as string, "base64")); } if (ref.uri) { const res = await fetch(ref.uri); return new Uint8Array(await res.arrayBuffer()); } return new Uint8Array(); } // Image: create ref from bytes function imageRef(data: Uint8Array, extras: Record<string, unknown> = {}): Record<string, unknown> { return { data: Buffer.from(data).toString("base64"), ...extras }; } // Audio: same pattern function audioRefFromBytes(data: Uint8Array, uri?: string): Record<string, unknown> { return { uri: uri ?? "", data: Buffer.from(data).toString("base64") }; } @nodetool-ai/runtime exports a shared loadMediaRefBytes(ref, context?) helper that handles data, uri, and storage-backed refs in one call. Static Class Properties // Declare output types (required for UI connectors) static readonly metadataOutputTypes = { output: "str", count: "int" }; // Enable dynamic (user-added) input connectors. Read/write extra inputs at // runtime with this.getDynamic(key) / this.setDynamic(key, value). static readonly supportsDynamicInputs = true; // Support dynamic output slots static readonly supportsDynamicOutputs = true; // Field split for the UI: inlineFields render compactly on the node body; // inputFields render as the larger expanded inputs. static readonly inlineFields = ["prompt"]; static readonly inputFields = ["model"]; // Input consumption mode (default is undefined → buffered): // "buffered" — collect a matched set of inputs, call process() once // "stream" — consume inputs as an async stream via run() // "controlled" — node manages its own input/output flow static readonly inputMode = "buffered"; // Accept streaming input (used together with inputMode = "stream") static readonly isStreamingInput = true; // Per-output correlation. Drives scheduling and whether an output streams. // forward — output carries the source input's correlation token unchanged // iteration — each emitted value is a fresh correlated item // aggregate — collapse a stream into one value // single — one value per invocation, not correlated to a source static readonly outputCorrelation = { output: { kind: "forward", source: "value" }, }; // Declare required secrets — injected onto this._secrets static readonly requiredSettings = ["OPENAI_API_KEY"]; There is no isStreamingOutput, syncMode, isDynamic, or basicFields static field. Streaming is inferred from outputCorrelation (and from defining genProcess()/run()); dynamic inputs use supportsDynamicInputs; the field split uses inlineFields/inputFields. Lifecycle Hooks // Called once at the start of a workflow run -- reset state here async initialize(): Promise<void> { this._items = []; } // Called before each process() invocation async preProcess(): Promise<void> {} // Called after all processing is complete async finalize(): Promise<void> {} Return Type Patterns // Single output async process(): Promise<Record<string, unknown>> { return { output: "result" }; } // Multiple outputs async process(): Promise<Record<string, unknown>> { return { text: "hello", score: 0.95 }; } // Streaming (generator) async *genProcess(): AsyncGenerator<Record<string, unknown>> { for (const [i, item] of items.entries()) { yield { output: item, index: i }; } } Input Node Quick List StringInput - Text value IntegerInput - Whole number (min/max) FloatInput - Decimal (min/max) BooleanInput - True/False toggle StringListInput - List of strings LanguageModelInput - Select LLM ImageModelInput - Select image model ImageInput - Image asset reference AudioInput - Audio asset reference VideoInput - Video asset reference DocumentInput - Document asset reference AssetFolderInput - Folder asset reference ColorInput - Color picker CollectionInput - Vector DB collection FolderPathInput - Local folder path FilePathInput - Local file path DocumentFileInput - Load document from file Output Node Quick List Output - Generic output for any data type Docstring Keywords by Category Data Types text, string, number, integer, float, boolean, list, array, dict, object, document, file Operations extract, filter, map, reduce, merge, split, join, sort, group, aggregate, transform, analyze Media image, picture, visual, video, audio, sound, document, file, folder, asset AI/ML model, embedding, classification, clustering, generation, language, agent, tool Control flow, condition, loop, iterator, generator, stream, branch, switch I/O input, output, load, save, read, write, import, export, download, upload Testing Pattern // packages/<your-pkg>/src/nodes/my-nodes.ts export class MyNode extends BaseNode { static readonly nodeType = "mypackage.MyNode"; static readonly title = "My Node"; static readonly description = "My node.\n keywords"; static readonly metadataOutputTypes = { output: "str" }; @prop({ type: "str", default: "", title: "Value" }) declare value: any; async process(): Promise<Record<string, unknown>> { return { output: String(this.value ?? "").toUpperCase() }; } } // tests/my-nodes.test.ts import { describe, it, expect } from "vitest"; import { MyNode } from "../src/nodes/my-nodes.js"; describe("MyNode", () => { it("uppercases input", async () => { const node = new MyNode({ value: "hello" }); const result = await node.process(); expect(result.output).toBe("HELLO"); }); }); Key Reminders All process() and genProcess() methods must be async Always declare metadataOutputTypes – it drives the UI output connectors Use @prop for every input – it provides validation, defaults, and UI hints Read input values from this.<field> – the engine assigns them before process() runs Return a plain object whose keys match the metadataOutputTypes keys Use genProcess() with yield for streaming outputs Use initialize() to reset state in stateful / collector nodes Declare requiredSettings for API keys; read them from this._secrets Node discovery is automatic when classes are registered in the package index Test with vitest – nodes are plain classes, easy to instantiate and call--- # Provider Guides: Add Models & Nodes Source: https://docs.nodetool.ai/developer/providers/ One runbook per provider for adding new models and nodes. Each guide names the exact files, the command to run, a copy-pasteable snippet, and how the same change looked in past PRs. Audience: coding agents and contributors keeping NodeTool current — a new image model shipped, a new endpoint went live, a new provider needs wiring. Start with the provider you’re updating; each guide is self-contained. NodeTool spans many providers but only a few distinct mechanisms. Find your provider’s mechanism first — it tells you whether a new model needs zero code, a manifest entry, or a codegen run. How adding a model works, by mechanism Mechanism What “add a model” means Providers Dynamic fetch Nothing — models appear from the provider’s live /models (or local daemon) endpoint. Code only for capability/cost overrides. OpenAI (chat), Anthropic, Gemini (text), xAI, OpenAI-compatible (Groq, Mistral, DeepSeek, Moonshot, Cerebras, Cohere, OpenRouter), Ollama, Local inference (LM Studio, llama.cpp, vLLM) Static model list A code edit to a hardcoded array in the provider (plus cost entry). OpenAI (image), Gemini (Imagen/Veo) Codegen from upstream schemas Add an endpoint id to a config, run the generator — the manifest is regenerated, never hand-edited. FAL, Replicate, KIE Hand-maintained manifest Add a JSON entry to the package manifest. AtlasCloud, Topaz, Together Static node package Add a model id to the provider/base arrays, or a new BaseNode subclass. MiniMax, Reve, ElevenLabs, HuggingFace All guides OpenAI — chat models auto-fetch; image models (gpt-image-*) are a static list. Anthropic (Claude) — models fetched live from the Anthropic API. Google Gemini — text auto-fetches; Imagen/Veo are static lists. xAI (Grok) — chat, image, and video classified from /v1/models. OpenAI-compatible providers — Groq, Mistral, DeepSeek, Moonshot, Cerebras, Cohere, OpenRouter, and how to add a new one. FAL — nodes generated from FAL OpenAPI schemas via codegen. Replicate — nodes generated from Replicate model schemas via codegen. KIE — nodes generated from per-model configs into a manifest. AtlasCloud — hand-maintained manifest drives nodes and the model picker. Topaz — manifest-driven image/video enhancement nodes. Together AI — generated image/video manifest plus live-fetched chat models. MiniMax — chat, image, video, and audio via provider arrays and a node pack. Reve — image create/edit/remix nodes. ElevenLabs — TTS models and voices as static arrays (decorator package). HuggingFace — Hub-backed model discovery plus a hand-written node pack. Ollama — local models discovered from the running daemon (ollama pull). Local inference — LM Studio, llama.cpp, vLLM over OpenAI-compatible localhost. Before you open a PR Whatever the mechanism, run the full check from the repo root: npm run check # typecheck + lint + test across web, electron, mobile For decorator/codegen packages (FAL, Replicate, KIE, ElevenLabs, and other dist/-loaded packs), run npm run build:packages after regenerating so the runtime picks up the change. Each guide lists its exact verify commands. Contributions are welcome — open a PR on GitHub or say hi on Discord.--- # Anthropic (Claude): Provider Guide Source: https://docs.nodetool.ai/developer/providers/anthropic.html Anthropic provider NodeTool calls Anthropic’s Messages API through AnthropicProvider. The provider supports text, image input, client tools, native web search, streaming, prompt cache accounting, and extended thinking. It uses ANTHROPIC_API_KEY by default through the built-in provider registry. Direct integrations can construct the provider with an auth token, structured SDK credentials, or a custom base URL; those alternatives are not resolved by the built-in registry. Anthropic changes model capabilities independently of the Models API. A model appearing in the selector does not prove that every NodeTool request option works with it. Check the model’s thinking mode, sampling parameters, stop reasons, and tool behavior before making it a default or recommendation. Files Concern Path Provider packages/runtime/src/providers/anthropic-provider.ts Recommended models packages/runtime/src/recommended-models.ts Provider registration packages/runtime/src/providers/index.ts Provider tests packages/runtime/tests/providers/anthropic-provider*.test.ts Catalog tests packages/runtime/tests/recommended-models.test.ts Cost calculation packages/runtime/src/providers/cost-calculator.ts ClaudeAgentProvider is a separate integration backed by the Claude Agent SDK and a local Claude subscription. Changes to AnthropicProvider do not affect it. Model discovery getAvailableLanguageModels() calls /v1/models on the configured API base URL. It follows the Models API pagination cursor and uses each model’s display_name. Discovery retries transient errors, including Anthropic’s 529 overload response, and honors Retry-After as described in Anthropic API errors. There is no static fallback list. If discovery fails, the provider returns no Anthropic models for that fetch. The curated entry in recommended-models.ts is independent of discovery and must name an active model. Model lifecycle Use Anthropic’s model deprecation table before adding or changing a recommended model. Do not recommend a -latest alias: it can move between snapshots and does not communicate a lifecycle date or version. Anthropic’s dateless ids from Claude 4.6 onward are pinned model versions, not moving aliases. When changing the recommendation: Confirm the model is active and generally available. Check that @pydantic/genai-prices recognizes its exact id. Verify text, streaming, tool use, and configured thinking behavior. Update the catalog test anchor in the same change. Compatibility checks Thinking NodeTool uses manual thinking on models that accept budget_tokens: { "thinking": { "type": "enabled", "budget_tokens": 4096 } } This shape is not valid for every current model. The provider uses adaptive thinking for models that require it. Anthropic’s extended thinking guide requires adaptive configuration for Claude Fable 5, Claude Mythos 5 and Preview, Claude Opus 4.8 and 4.7, and Claude Sonnet 5. Sonnet 5 enables adaptive thinking by default; Fable and Mythos do not allow thinking to be disabled. Manual budget_tokens is deprecated but still accepted for Claude Opus 4.6 and Claude Sonnet 4.6. Claude Haiku 4.5 also accepts manual thinking. Manual budgets must be at least 1,024 tokens and less than max_tokens. Tests must cover both streaming and non-streaming request shapes. Thinking with tools must preserve signed thinking blocks unchanged across the tool-result turn. Sampling parameters Anthropic rejects non-default temperature, top_p, and top_k on current adaptive-only/default model families, including Claude Opus 4.7 and later, Sonnet 5, Fable 5, and Mythos. The provider omits incompatible sampling parameters according to Anthropic’s parameter deprecation table. Stop reasons Every successful Messages response has a stop_reason. Anthropic documents end_turn, max_tokens, stop_sequence, tool_use, pause_turn, refusal, and model_context_window_exceeded in its stop-reason guide. pause_turn requires another request with the assistant content preserved; the provider continues these turns internally. A refusal is an HTTP 200 response and may contain no content. Claude Fable 5 also returns stop_details. The provider surfaces refusals and truncated output instead of treating them as a normal completed answer. Tools and content The provider maps NodeTool client tools to Anthropic input_schema definitions without dropping supported schema constraints and maps web_search to web_search_20250305. Anthropic still lists that web search version as GA, alongside newer versions in the tool reference. The adapter also passes strict, input_examples, cache_control, defer_loading, and allowed_callers when configured. Tool execution failures are returned with Anthropic’s is_error marker. Image input supports JPEG, PNG, GIF, and WebP through inline base64 blocks. Document input supports inline PDF data, HTTPS PDF URLs, and plain text, including document titles, context, and citation configuration. See Anthropic’s PDF support for API limits. Uploaded Anthropic Files API ids are not part of NodeTool’s shared message contract. Citation ranges and source metadata remain structured on text content; the provider does not append citation Markdown to Claude’s text. Cost tracking Claude token prices come from @pydantic/genai-prices. Check a new model id before recommending it: node -e "const {calcPrice}=require('@pydantic/genai-prices'); \ console.log(calcPrice({input_tokens:1000,output_tokens:500},'claude-sonnet-5','anthropic'))" Do not add Claude token prices to the local non-token modality table. Update the upstream catalog or bump the dependency when it already contains the model. Anthropic reports uncached input separately from cache reads and writes. The provider adds those values before sending usage to the shared cost calculator. Verify Run the focused catalog and provider tests first: npm test --workspace=packages/runtime -- \ --run tests/recommended-models.test.ts \ tests/providers/anthropic-provider.test.ts \ tests/providers/anthropic-provider-coverage.test.ts Then run the required repository checks: npm run typecheck npm run lint npm run test With ANTHROPIC_API_KEY configured, inspect discovery and run a small request: npm run dev:nodetool -- info --json npm run dev:nodetool -- node run nodetool.llm.Claude \ --props '{"prompt":"hello","model":"claude-sonnet-5","max_tokens":50}'--- # AtlasCloud: Add Models & Nodes Source: https://docs.nodetool.ai/developer/providers/atlascloud.html Navigation: Developer Index Providers Overview → AtlasCloud Audience: contributors and coding agents adding AtlasCloud models or node classes. No familiarity with the runtime internals is assumed; you need to edit JSON and run two commands. TL;DR Add an entry to packages/atlascloud-nodes/src/atlascloud-manifest.json. Run npm run build:packages to copy the manifest into dist/. Run npm run typecheck && npm run test --workspace=packages/atlascloud-nodes. That’s it. No TypeScript code is required for most new models. Skip to Add a new model/node for the exact steps. Where things live What Path Manifest (hand-maintained JSON, single source of truth) packages/atlascloud-nodes/src/atlascloud-manifest.json Manifest copy in dist/ (generated by build) packages/atlascloud-nodes/dist/atlascloud-manifest.json Node class factory (reads manifest at runtime) packages/atlascloud-nodes/src/atlascloud-factory.ts HTTP helpers, poll/submit/download packages/atlascloud-nodes/src/atlascloud-base.ts Package entry point (exports ATLASCLOUD_NODES) packages/atlascloud-nodes/src/index.ts Runtime provider (image/video model lists, textToImage etc.) packages/runtime/src/providers/atlascloud-provider.ts Provider registration packages/runtime/src/providers/index.ts line 222 Manifest-to-model-list helper (shared with KIE, FAL) packages/runtime/src/providers/manifest-models.ts How AtlasCloud models/nodes are defined There is no codegen script. atlascloud-manifest.json is hand-maintained. It is the single file you edit to add a model. Manifest → node class packages/atlascloud-nodes/src/index.ts reads the manifest with readFileSync at module load time and passes it to loadAtlasNodesFromManifest. That function calls createAtlasNodeClass once per manifest entry. createAtlasNodeClass (in atlascloud-factory.ts) produces a BaseNode subclass whose nodeType is atlascloud.<moduleName>.<className>, whose input props match the fields array, and whose process() method calls atlasSubmit → atlasPoll → atlasDownload. The build script in package.json compiles TypeScript then copies src/atlascloud-manifest.json to dist/: "build": "node ../../scripts/build-typescript-workspace.mjs && node -e \"require('node:fs').cpSync('src/atlascloud-manifest.json', 'dist/atlascloud-manifest.json')\"" The manifest export declared in package.json is: "./atlascloud-manifest.json": "./dist/atlascloud-manifest.json" Manifest → model list (provider) AtlasCloudProvider.getAvailableImageModels() and getAvailableVideoModels() call loadImageModels / loadVideoModels from manifest-models.ts, which requires the manifest via @nodetool-ai/atlascloud-nodes/atlascloud-manifest.json. The model-list functions read outputType, modelId, and title from each entry. buildModelMap() in the provider reads the same manifest to know which fields each model accepts for the generic textToImage / imageToVideo etc. methods. Key point: the manifest drives both the node UI (fields, defaults, enum values) and the model picker (image/video model lists). Editing the manifest is sufficient for both. Wire protocol Submit: POST https://api.atlascloud.ai/api/v1/model/generateImage or .../generateVideo Body: flat JSON — { model: "<modelId>", ...fields } (not nested under input) Poll: GET https://api.atlascloud.ai/api/v1/model/prediction/<id> Terminal statuses: completed, succeeded, success, done, complete, canceled, failed, error Submit is never retried — a 5xx may have already created and billed the job upstream. Add a new model/node Step 1: Identify the model From the AtlasCloud docs or API, collect: The model id string, e.g. "bytedance/seedance-3.0/text-to-video" The modality: "image" or "video" The field names and types the endpoint accepts Step 2: Choose a className and moduleName className: PascalCase identifier used as the TypeScript class name and the last segment of the node type. Example: "Seedance3TextToVideo". moduleName: one of "image" or "video" (matches the output type). Used in the node type: atlascloud.video.Seedance3TextToVideo. Step 3: Append an entry to the manifest Open packages/atlascloud-nodes/src/atlascloud-manifest.json. Append an object following this shape (copy the closest existing entry and adjust): { "className": "Seedance3TextToVideo", "moduleName": "video", "modality": "video", "modelId": "bytedance/seedance-3.0/text-to-video", "outputType": "video", "title": "Seedance 3.0 — Text to Video", "description": "AtlasCloud / ByteDance Seedance 3.0 — text-to-video with native audio.", "pollInterval": 5000, "maxAttempts": 600, "fields": [ { "name": "prompt", "type": "str", "default": "", "title": "Prompt", "description": "Text prompt describing the desired video.", "required": true }, { "name": "duration", "type": "int", "default": 5, "title": "Duration (seconds)", "values": [-1, 4, 5, 6, 7, 8, 9, 10] }, { "name": "resolution", "type": "enum", "default": "720p", "title": "Resolution", "values": ["480p", "720p", "1080p"] }, { "name": "ratio", "type": "enum", "default": "16:9", "title": "Aspect Ratio", "values": ["16:9", "9:16", "1:1", "adaptive"] } ] } Field type reference: type Description "str" Text input "int" Integer (coerced from string dropdowns) "float" Float "bool" Checkbox "enum" Dropdown; requires "values" array "image" Single image ref "video" Single video ref "audio" Single audio ref "list[image]" Multiple image refs "list[video]" Multiple video refs "list[audio]" Multiple audio refs Use "list[image]" (not image with "array": true) for multi-image inputs. Use "null" as the default for optional enum fields without a sensible preselected value (e.g. aspect_ratio on Nano Banana models). pollInterval: milliseconds between poll requests. Use 3000 for image models, 5000 for video models. maxAttempts: maximum polls before timing out. 300 is reasonable for image; 600 for long-form video. Step 4: Build npm run build:packages This compiles TypeScript and copies the manifest into dist/. The factory picks it up at module load time — no TypeScript changes needed. Step 5: Smoke-test the node ATLASCLOUD_API_KEY=<your_key> \ npm run dev:nodetool -- node run \ atlascloud.video.Seedance3TextToVideo \ --props '{"prompt":"a red balloon floating upward","duration":5,"resolution":"720p","ratio":"16:9"}' For a hermetic check (no secrets, no API call — verifies the node can be instantiated): npm run dev:nodetool -- node run atlascloud.video.Seedance3TextToVideo \ --props '{"prompt":"test"}' --no-secrets 2>&1 | grep -E "error|ATLASCLOUD" Expected: ATLASCLOUD_API_KEY is not configured (meaning the node loaded and reached the key check). Step 6: Validate (no API call) npm run dev:nodetool -- validate workflow.json If you have a workflow JSON referencing the new node, this verifies the node type is registered and all edges type-check. Verify Run these in order after any manifest change: # 1. Rebuild (copies manifest to dist) npm run build:packages # 2. Type check the whole repo npm run typecheck # 3. Test atlascloud-nodes npm run test --workspace=packages/atlascloud-nodes # 4. Test runtime providers (checks manifest-models and the provider) npm run test --workspace=packages/runtime # 5. Full check (runs all three root-level checks) npm run check None of these make live API calls. Tests use vitest with mocked HTTP. How past PRs did it The atlascloud-nodes package and atlascloud-provider.ts were introduced together in commit d1491abf (“add claude agent package”), which also added the entire file set: packages/atlascloud-nodes/src/atlascloud-manifest.json +1248 lines packages/atlascloud-nodes/src/atlascloud-factory.ts +637 lines packages/atlascloud-nodes/src/atlascloud-base.ts +220 lines packages/atlascloud-nodes/src/index.ts +47 lines packages/atlascloud-nodes/tests/... +1120 lines packages/runtime/src/providers/atlascloud-provider.ts +492 lines The initial manifest shipped with 12 entries covering Seedance 2.0, Seedance 2.0 Fast, GPT Image 2, Nano Banana 2, and Nano Banana Pro. Each subsequent model addition follows the same pattern: one new JSON object in atlascloud-manifest.json, then npm run build:packages. Contributing Source: https://github.com/nodetool-ai/nodetool Discord: https://discord.gg/WmQTWZRcYE Before opening a PR, run npm run check (typecheck + lint + tests). The PR description should include the AtlasCloud model id, the node type string, and a brief note on which fields were included and why any fields from the API docs were omitted.--- # ElevenLabs: Add Models & Nodes Source: https://docs.nodetool.ai/developer/providers/elevenlabs.html Audience: Contributors and coding agents adding ElevenLabs models, voices, or nodes to NodeTool. TL;DR Add the model id to the MODELS array in elevenlabs-provider.ts and the @prop values array in the relevant node file. To add a voice, add one entry to VOICE_ID_MAP in both elevenlabs-base.ts and elevenlabs-provider.ts. To add a node, create packages/elevenlabs-nodes/src/nodes/<name>.ts, export a readonly NodeClass[], and add it to src/index.ts. Run npm run build:packages (required — this package loads from dist/), then npm run check. Where things live What Path Provider (TTS, voice list, model list) packages/runtime/src/providers/elevenlabs-provider.ts Shared voice/key helpers packages/elevenlabs-nodes/src/elevenlabs-base.ts Node package entry packages/elevenlabs-nodes/src/index.ts TTS node packages/elevenlabs-nodes/src/nodes/text-to-speech.ts STT node packages/elevenlabs-nodes/src/nodes/speech-to-text.ts Realtime TTS node (WebSocket) packages/elevenlabs-nodes/src/nodes/realtime-tts.ts Realtime STT node (WebSocket) packages/elevenlabs-nodes/src/nodes/realtime-stt.ts Standard voice picker node packages/elevenlabs-nodes/src/nodes/standard-voice.ts Provider registration packages/runtime/src/providers/index.ts line 219 Node tests packages/elevenlabs-nodes/tests/ How ElevenLabs models and nodes are defined Models Models are static arrays, not fetched at runtime. Two separate places hold them: Provider (elevenlabs-provider.ts): the MODELS array drives what getAvailableTTSModels() returns to the unified TTS picker: const MODELS: Array<{ id: string; name: string }> = [ { id: "eleven_v3", name: "Eleven v3" }, { id: "eleven_multilingual_v2", name: "Multilingual v2" }, // … ]; Node (text-to-speech.ts / realtime-tts.ts): each node’s model_id prop carries its own values enum that controls what appears in the workflow UI: @prop({ type: "enum", default: "eleven_monolingual_v1", title: "Model", values: ["eleven_v3", "eleven_multilingual_v2", "eleven_turbo_v2_5", /* … */] }) declare model_id: any; The two lists are independent. Keep them in sync when adding a model. Voices Voices are also static. The canonical map lives in two files that mirror each other: packages/elevenlabs-nodes/src/elevenlabs-base.ts — used by all nodes (voice ID resolution, the StandardVoice node enum) packages/runtime/src/providers/elevenlabs-provider.ts — used by the provider to surface voices in the unified TTS picker VOICE_ID_MAP maps display name to voice id. VOICE_NAMES = Object.keys(VOICE_ID_MAP) is passed to StandardVoiceNode’s enum values and to getAvailableTTSModels() voice lists. Nodes Each node file exports a readonly NodeClass[] named <CATEGORY>_NODES. src/index.ts spreads them all into ELEVENLABS_NODES and registerElevenLabsNodes() calls registry.register() on each. The package loads from dist/ (see "main": "dist/index.js" in package.json). TypeScript decorators (@prop) require a build step before any changes take effect. Add a new model or node Add a model to an existing node Step 1. Add the model id to MODELS in packages/runtime/src/providers/elevenlabs-provider.ts: const MODELS: Array<{ id: string; name: string }> = [ { id: "eleven_v3", name: "Eleven v3" }, { id: "eleven_multilingual_v3", name: "Multilingual v3" }, // new // … ]; Step 2. Add the same id to the values array of the model_id prop in the relevant node(s). For TextToSpeechNode in packages/elevenlabs-nodes/src/nodes/text-to-speech.ts: @prop({ type: "enum", default: "eleven_monolingual_v1", title: "Model", values: [ "eleven_v3", "eleven_multilingual_v3", // new "eleven_multilingual_v2", // … ] }) declare model_id: any; Repeat for realtime-tts.ts if the model supports streaming. Add a voice Step 1. Add the entry to VOICE_ID_MAP in packages/elevenlabs-nodes/src/elevenlabs-base.ts: export const VOICE_ID_MAP: Record<string, string> = { // existing entries … Matilda: "XrExE9yKIg1WjnnlVkGX", // new }; Step 2. Mirror the same entry in packages/runtime/src/providers/elevenlabs-provider.ts: const VOICE_ID_MAP: Record<string, string> = { // existing entries … Matilda: "XrExE9yKIg1WjnnlVkGX", // new }; VOICE_NAMES derives from Object.keys(VOICE_ID_MAP) in both files, so no other change is needed. Add a node Step 1. Create packages/elevenlabs-nodes/src/nodes/<name>.ts. Follow the shape of an existing node — extend BaseNode, set the required static fields, use @prop for inputs, implement process(): import { BaseNode, prop } from "@nodetool-ai/node-sdk"; import type { NodeClass } from "@nodetool-ai/node-sdk"; import { getElevenLabsApiKey } from "../elevenlabs-base.js"; export class SoundEffectNode extends BaseNode { static readonly nodeType = "elevenlabs.SoundEffect"; static readonly body = "content_card"; static readonly title = "Sound Effect"; static readonly description = "Generate a sound effect from a text prompt."; static readonly metadataOutputTypes = { output: "audio" }; static readonly requiredSettings = ["ELEVENLABS_API_KEY"]; @prop({ type: "str", default: "", title: "Prompt", description: "Text description of the sound to generate." }) declare prompt: any; async process(): Promise<Record<string, unknown>> { const apiKey = getElevenLabsApiKey(this._secrets); const prompt = String(this.prompt ?? ""); if (!prompt) throw new Error("Prompt is required"); const response = await fetch("https://api.elevenlabs.io/v1/sound-generation", { method: "POST", headers: { "xi-api-key": apiKey, "Content-Type": "application/json" }, body: JSON.stringify({ text: prompt }) }); if (!response.ok) { throw new Error(`ElevenLabs API error: ${await response.text()}`); } const data = Buffer.from(await response.arrayBuffer()).toString("base64"); return { output: { type: "audio", data: `data:audio/mpeg;base64,${data}` } }; } } export const SOUND_EFFECT_NODES: readonly NodeClass[] = [SoundEffectNode]; Step 2. Register it in packages/elevenlabs-nodes/src/index.ts: import { SOUND_EFFECT_NODES } from "./nodes/sound-effect.js"; export const ELEVENLABS_NODES: readonly NodeClass[] = [ ...TEXT_TO_SPEECH_NODES, ...SPEECH_TO_TEXT_NODES, ...REALTIME_TTS_NODES, ...REALTIME_STT_NODES, ...STANDARD_VOICE_NODES, ...SOUND_EFFECT_NODES, // new ]; Step 3. Add a test in packages/elevenlabs-nodes/tests/<name>.test.ts and add a registry.has("elevenlabs.SoundEffect") assertion in tests/registration.test.ts. Verify # Build first — required because this package loads from dist/ npm run build:packages # Type check all packages npm run typecheck # Run the elevenlabs-nodes test suite npm run test --workspace=packages/elevenlabs-nodes # Smoke-test a node in isolation (no workflow needed) npm run dev:nodetool -- node run elevenlabs.TextToSpeech \ --props '{"text":"hello","voice_id":"9BWtsMINqrJLrRacOk9x"}' \ --no-secrets # Validate a workflow that uses the node (if you have one) npm run dev:nodetool -- validate workflow.json # Full check before committing npm run check How past PRs did it 00d96654 feat(elevenlabs): add v3 model and standard voice node — added "eleven_v3" to TextToSpeechNode’s model_id enum and introduced StandardVoiceNode as a new node file. Changed files: src/index.ts, src/nodes/standard-voice.ts, src/nodes/text-to-speech.ts, tests/registration.test.ts, tests/standard-voice.test.ts, tests/text-to-speech.test.ts. That’s the canonical pattern: one file per node, export a readonly NodeClass[], add it to index.ts, add a test. 7c7b5157 test(elevenlabs): assert enum values via getDeclaredProperties — shows the correct testing pattern: TextToSpeechNode.getDeclaredProperties().find(p => p.name === "model_id")?.options.values to assert enum contents, rather than inspecting toDescriptor(). Contributing Source and issues: https://github.com/nodetool-ai/nodetool Community discussion: https://discord.gg/WmQTWZRcYE Run npm run check (typecheck + lint + tests) and ensure it passes before opening a PR. Two things catch most mistakes before review: npm run build:packages (catches missing exports, wrong import paths) and npm run test --workspace=packages/elevenlabs-nodes (catches registration gaps and API contract regressions).--- # FAL: Add Models & Nodes Source: https://docs.nodetool.ai/developer/providers/fal.html FAL nodes in NodeTool are generated, not hand-written. A codegen pipeline fetches each endpoint’s OpenAPI schema from fal.ai, converts it to a manifest, and the runtime loads node classes from that manifest. Audience: coding agents and contributors adding new FAL endpoints or fixing generated node behavior. TL;DR Add the endpoint ID + NodeConfig to the right src/configs/<category>.ts file in packages/fal-codegen/. Run npm run generate:fal from the repo root. Run npm run build:packages (fal-nodes loads from dist/). Run npm run lint --workspace=packages/fal-codegen and npm run test --workspace=packages/fal-nodes. Open a PR — never commit manual edits to fal-manifest.json. Where things live Concern Path Endpoint configs (source of truth) packages/fal-codegen/src/configs/<category>.ts Config type definitions packages/fal-codegen/src/types.ts Schema fetcher (caches to .codegen-cache/) packages/fal-codegen/src/schema-fetcher.ts Schema → NodeSpec parser packages/fal-codegen/src/schema-parser.ts Config overrides applier packages/fal-codegen/src/node-generator.ts Codegen entry point packages/fal-codegen/src/generate.ts Generated manifest (do not edit) packages/fal-nodes/src/fal-manifest.json Runtime node class factory packages/fal-nodes/src/fal-factory.ts FAL API call utilities packages/fal-nodes/src/fal-base.ts Package entry point packages/fal-nodes/src/index.ts Pricing bundles (generated) packages/fal-nodes/src/generated/ High-level provider (text→image, TTS, etc.) packages/runtime/src/providers/fal-provider.ts Manifest model loading packages/runtime/src/providers/manifest-models.ts How FAL nodes are generated src/configs/<category>.ts ← you edit this ↓ schema-fetcher.ts fetches https://fal.ai/api/openapi/queue/openapi.json?endpoint_id=... caches each schema in packages/fal-codegen/.codegen-cache/<sha>.json ↓ schema-parser.ts converts OpenAPI → NodeSpec (inputFields, outputType, enums) ↓ node-generator.ts merges NodeConfig overrides (className, fieldOverrides, enumOverrides…) ↓ generate.ts writes packages/fal-nodes/src/fal-manifest.json ↓ fal-factory.ts creates runtime node classes from manifest at startup The root generate:fal script runs in strict mode by default: if any configured endpoint cannot be fetched or parsed, generation fails. Remove stale endpoints from the config or fix them before regenerating. Never edit fal-manifest.json directly. Changes are overwritten on the next npm run generate:fal. Add a new FAL endpoint 1. Find the right config file Pick the file in packages/fal-codegen/src/configs/ that matches the endpoint’s modality: Endpoint type Config file text → image text-to-image.ts image → video image-to-video.ts text → video text-to-video.ts text → speech text-to-speech.ts image → image image-to-image.ts other matching <input>-to-<output>.ts, or unknown.ts 2. Add a NodeConfig entry Open the chosen config file and add a key inside configs whose value is a NodeConfig: // packages/fal-codegen/src/configs/text-to-image.ts import type { ModuleConfig } from "../types.js"; export const config: ModuleConfig = { configs: { // ... existing entries ... "fal-ai/my-new-model": { className: "MyNewModel", docstring: "One sentence describing what the model does.", tags: ["image", "generation", "text-to-image", "txt2img"], useCases: [ "Generate images from text descriptions", "Create concept art from prompts" ], fieldOverrides: { prompt: { description: "The text prompt to generate an image from" }, image_size: { propType: "enum", enumRef: "ImageSizePreset", // reuse a sharedEnum if available default: "landscape_4_3", description: "Output image size preset" }, seed: { propType: "int", default: -1, description: "Seed for reproducible results. Use -1 for random" } } } } }; NodeConfig fields (all optional — only add what needs overriding): Field Type Purpose className string PascalCase class name for the node docstring string Node description shown in the UI tags string[] Used for node search useCases string[] Shown in node detail panel fieldOverrides Record<string, Partial<FieldDef>> Override parsed field properties (type, default, description…) enumOverrides Record<string, string> Rename enums: { ImageSize: "ImageSizePreset" } enumValueOverrides Record<string, Record<string, string>> Rename enum values FieldDef override keys: propType, default, description, enumRef, min, max. Valid propType values: "str" "float" "int" "bool" "image" "video" "audio" "enum" "list[image]" "list[video]" "list[audio]" "dict[str, any]". 3. (Optional) Add a shared enum If the endpoint introduces a new enum that other nodes in the same module will reuse, add it to the sharedEnums block: export const config: ModuleConfig = { sharedEnums: { MyAspectRatio: { name: "MyAspectRatio", values: [ ["SQUARE", "1:1"], ["WIDESCREEN", "16:9"], ["PORTRAIT", "9:16"] ], description: "Aspect ratio for output" } }, configs: { /* ... */ } }; Then reference it via enumRef: "MyAspectRatio" in a fieldOverrides entry. 4. Run codegen # From the repo root: npm run generate:fal This fetches the schema (or reads from .codegen-cache/ if cached), writes packages/fal-nodes/src/fal-manifest.json, and updates pricing bundles under packages/fal-nodes/src/generated/ when FAL_API_KEY is set. To force a fresh schema fetch (bypass cache): npm run generate --workspace=packages/fal-codegen -- --strict --no-cache Verify Run these in order after any config change: # 1. Regenerate the manifest npm run generate:fal # 2. Check for TypeScript errors in codegen npm run lint --workspace=packages/fal-codegen # 3. Run codegen tests npm run test --workspace=packages/fal-codegen # 4. Build fal-nodes (loads manifest from dist/) npm run build:packages # 5. Run fal-nodes tests (exercises the generated node classes) npm run test --workspace=packages/fal-nodes # 6. Smoke-test one generated node (replace type and props as appropriate) npm run dev:nodetool -- node run fal.text_to_image.FluxDev \ --props '{"prompt": "a red apple on a white table"}' \ --no-secrets The node type follows the pattern fal.<module_name>.<ClassName>, where module_name comes from the config file key in src/configs/index.ts (e.g. text_to_image). Before committing, inspect the manifest diff: git diff packages/fal-nodes/src/fal-manifest.json | head -80 Confirm the new entry appears with the expected className, inputFields, and outputType. How past commits did it The entire FAL codegen and node system was introduced in commit d1491abf (“add claude agent package”), which added all src/configs/ files, the codegen pipeline, and the factory. That commit is the canonical reference for the shape of every config file and the runtime loading path. Subsequent behavioral fixes follow a clear split: ff5824a6 — fixed schema-parser.ts to collapse single-asset wrapper structs (list[ImageInput]) to list[image] with a nestedAssetKey hint; the fix lived in codegen, not the manifest. 2997a678 — added FAL billing reconciliation; changes touched fal-base.ts, fal-factory.ts, and fal-billing.ts in fal-nodes/. Both commits follow the rule: parser/codegen bugs go in packages/fal-codegen/; runtime bugs go in packages/fal-nodes/; never patch fal-manifest.json. Fixing a wrong input or output on a generated node Problem Where to fix Field parsed with wrong type packages/fal-codegen/src/schema-parser.ts Field needs a different default or description fieldOverrides in the endpoint’s config entry Enum name conflicts across nodes enumOverrides in the config entry Asset input defaults to "" instead of AssetRef defaultForPropType() in packages/fal-nodes/src/fal-factory.ts Audio/video output missing preview set metadataOutputTypes in the factory, not outputTypes API call behavior (retry, upload, mapping) packages/fal-nodes/src/fal-base.ts Contributing PRs are welcome. Open one at https://github.com/nodetool-ai/nodetool. Before pushing, run: npm run check # typecheck + lint + test across all workspaces Join the discussion on Discord.--- # Google Gemini: Add Models Source: https://docs.nodetool.ai/developer/providers/gemini.html The Gemini provider supports six modalities: text/chat, image generation, video generation, TTS, ASR, and embeddings. Language models are auto-discovered from the API; every other modality uses a hand-maintained array in the provider file. Audience: coding agents and contributors who need to add a new Gemini, Imagen, or Veo model to NodeTool. TL;DR Text/chat models: nothing to do — they are fetched live from GET /v1beta/models. Image models (Imagen + gemini-* image generation): add one entry to getAvailableImageModels() in the provider. Video models (Veo): add one entry to getAvailableVideoModels(). TTS / ASR / embedding models: add one entry to the matching method. Run npm run check before committing. Where things live Concern Path Provider (all Gemini logic) packages/runtime/src/providers/gemini-provider.ts Language model listing (dynamic) GeminiProvider.getAvailableLanguageModels() Image model listing (static) GeminiProvider.getAvailableImageModels() TTS model listing (static) GeminiProvider.getAvailableTTSModels() ASR model listing (static) GeminiProvider.getAvailableASRModels() Video model listing (static) GeminiProvider.getAvailableVideoModels() Embedding model listing (static) GeminiProvider.getAvailableEmbeddingModels() Token/chat cost @pydantic/genai-prices catalog (automatic — no edit needed) Non-token cost tiers packages/runtime/src/providers/cost-calculator.ts — PRICING_TIERS / MODEL_TO_TIER Provider registration packages/runtime/src/providers/index.ts line 211 How Gemini models are defined Language models — dynamic getAvailableLanguageModels() calls GET https://generativelanguage.googleapis.com/v1beta/models?key=<GEMINI_API_KEY>, filters entries whose supportedGenerationMethods includes "generateContent", and maps each to { id, name, provider: "gemini" }. A new text/chat model becomes available when Google adds it to that endpoint. Image models — static array getAvailableImageModels() returns a hardcoded array. Two dispatch paths exist inside textToImage(): IDs starting with "gemini-" — call POST /models/<id>:generateContent with responseModalities: ["IMAGE", "TEXT"]. All other IDs (Imagen: "imagen-*") call POST /models/<id>:predict. Video models — static array getAvailableVideoModels() returns a hardcoded array. Both textToVideo() and imageToVideo() require a veo-* model ID. Veo calls use the async predictLongRunning endpoint with polling. TTS / ASR / Embedding — static arrays Each is a simple array returned from the matching getAvailable* method. TTS models carry a voices field. Embedding models carry a dimensions field. Add a new model 1. Text/chat model Nothing to do. The model appears automatically once Google adds it to the list API. Verify it shows up: curl "https://generativelanguage.googleapis.com/v1beta/models?key=$GEMINI_API_KEY" \ | jq '[.models[] | select(.supportedGenerationMethods[] | contains("generateContent")) | .name]' Token/chat cost is priced through @pydantic/genai-prices. NodeTool maps the "gemini" provider to "google" in GENAI_PROVIDER_MAP. 2. Image model Open packages/runtime/src/providers/gemini-provider.ts and add an entry to getAvailableImageModels(): async getAvailableImageModels(): Promise<ImageModel[]> { return [ { id: "gemini-3.1-flash-image", name: "Gemini 3.1 Flash Image", provider: "gemini" }, { id: "gemini-3-pro-image", name: "Gemini 3 Pro Image", provider: "gemini" } ]; } The "gemini-" prefix uses the native generateContent endpoint. Legacy Imagen IDs use predict. Check Google’s deprecation table before adding an Imagen model. 3. Gemini native image model (gemini-*) Same as above, but use a "gemini-" prefixed ID. The textToImage() dispatcher routes it to generateContent with responseModalities: ["IMAGE", "TEXT"] automatically. Use the exact ID returned by Google. Do not add guessed future IDs. 4. Veo video model Open getAvailableVideoModels() and add an entry: override async getAvailableVideoModels(): Promise<VideoModel[]> { return [ { id: "veo-3.1-generate-preview", name: "Veo 3.1 Preview", provider: "gemini" }, { id: "veo-3.1-fast-generate-preview", name: "Veo 3.1 Fast Preview", provider: "gemini" }, { id: "veo-3.1-lite-generate-preview", name: "Veo 3.1 Lite Preview", provider: "gemini" } ]; } Veo model IDs must start with "veo-". Check supported durations and resolutions for each variant before adding it. 5. ASR model async getAvailableASRModels(): Promise<ASRModel[]> { return [ { id: "gemini-3.5-flash", name: "Gemini 3.5 Flash", provider: "gemini" }, { id: "gemini-3.1-flash-lite", name: "Gemini 3.1 Flash-Lite", provider: "gemini" } ]; } 6. TTS model Add the model and its supported voices: async getAvailableTTSModels(): Promise<TTSModel[]> { const voices = ["Zephyr", "Puck" /*, ... existing voices ... */]; return [ { id: "gemini-3.1-flash-tts-preview", name: "Gemini 3.1 Flash TTS Preview", provider: "gemini", voices } ]; } 7. Embedding model async getAvailableEmbeddingModels(): Promise<EmbeddingModel[]> { return [ { id: "gemini-embedding-2", name: "Gemini Embedding 2", provider: "gemini", dimensions: 3072 } ]; } Verify # 1. Type-check all packages npm run typecheck # 2. Lint npm run lint # 3. Run all tests npm run test # 4. Smoke-test a new image node (requires GEMINI_API_KEY in env or DB) npm run dev:nodetool -- node run nodetool.image.GenerateImage \ --props '{"prompt": "a red apple", "model": {"id": "gemini-3.1-flash-image", "provider": "gemini", "name": "Gemini 3.1 Flash Image"}}' # 5. Smoke-test via chat agent (text model — auto-discovered, no list change needed) npm run dev:chat -- --provider gemini --model gemini-3.5-flash # Combined (typecheck + lint + test): npm run check Contributing Open a PR at https://github.com/nodetool-ai/nodetool. Run npm run check (typecheck + lint + test) before pushing. Join the discussion on Discord.--- # HuggingFace: Add Models & Nodes Source: https://docs.nodetool.ai/developer/providers/huggingface.html Audience: coding agents and contributors who need to add new HuggingFace models or node types to NodeTool. For user-facing docs (setup, authentication, example workflows) see docs/huggingface.md. TL;DR New model in an existing task — nothing to do. Models are discovered live from the HF Hub API; any warm inference model for a given pipeline tag appears automatically. New node type — add a BaseNode subclass to packages/huggingface-nodes/src/nodes/<modality>.ts, export it in the modality’s array and src/index.ts, add its type to registration.test.ts, run npm run build:packages && npm run test --workspace=packages/huggingface-nodes. Where things live Concern Path Provider class (model discovery + API calls) packages/runtime/src/providers/huggingface-provider.ts Provider registration + secret key packages/runtime/src/providers/index.ts (line 236) Provider ID constant packages/protocol/src/api-types.ts — PROVIDER_IDS.HUGGINGFACE = "huggingface" Shared HTTP helpers (fetch, auth, binary handling) packages/huggingface-nodes/src/huggingface-base.ts Node implementations — text/NLP packages/huggingface-nodes/src/nodes/text.ts Node implementations — image/vision packages/huggingface-nodes/src/nodes/image.ts Node implementations — audio packages/huggingface-nodes/src/nodes/audio.ts Node implementations — video packages/huggingface-nodes/src/nodes/video.ts Package entry + registration helper packages/huggingface-nodes/src/index.ts Node tests (unit, registration) packages/huggingface-nodes/tests/ How HuggingFace models and nodes work Two separate concerns NodeTool has two different HuggingFace integration points that serve different purposes. HuggingFaceProvider (in packages/runtime/) is the AI-provider abstraction used by agent nodes, chat, and workflow LLM nodes. It wraps the @huggingface/inference SDK and implements the standard BaseProvider interface — generateMessage, textToImage, textToSpeech, etc. packages/huggingface-nodes/ is a separate node package that talks to the HuggingFace Inference Providers API directly over fetch (no SDK dependency). These nodes cover the full set of HF Inference pipeline tasks, including NLP tasks the provider abstraction does not expose. Both use HF_TOKEN from the NodeTool secrets store. Model discovery (provider side) HuggingFaceProvider does not use a static model list. It queries https://huggingface.co/api/models at runtime, filtered to inference=warm models sorted by likes, one per pipeline tag: getAvailable*() method HF pipeline tag queried getAvailableLanguageModels text-generation getAvailableImageModels text-to-image getAvailableVideoModels text-to-video getAvailableTTSModels text-to-speech getAvailableASRModels automatic-speech-recognition getAvailableEmbeddingModels feature-extraction Results are cached for 10 minutes (CACHE_TTL_MS = 10 * 60 * 1000). Any model that becomes warm on the Hub appears automatically in the next request. Node transport (node-package side) All node API calls go to https://router.huggingface.co: Chat/LLM tasks: POST /v1/chat/completions (OpenAI-compatible) All other pipeline tasks: POST /hf-inference/models/{model} with a { inputs, parameters } body Helpers in huggingface-base.ts: hfChatCompletion(token, body) — chat completions hfPipelineJson<T>(token, model, body) — pipeline tasks returning JSON hfPipelineBinary(token, model, body) — pipeline tasks returning raw media bytes (image/video) refToBase64(ref, context) — resolve an image/audio MediaRef to a base64 string for API input imageRefFromBytes(bytes, mimeType) / videoRefFromBytes(bytes, mimeType) — wrap output bytes into a MediaRef-shaped object cleanParams(params) — strip null/undefined entries from a parameters object Add a new model If the model runs a pipeline task that already has a node (or that the provider already supports), nothing needs changing. The model can be typed into the node’s model field directly, or it will appear in provider model lists once it goes warm on the Hub. If the model needs a new node type (new task or new output shape), follow the steps below. Add a new node 1. Choose the right modality file Task type File Text / NLP packages/huggingface-nodes/src/nodes/text.ts Image / vision packages/huggingface-nodes/src/nodes/image.ts Audio packages/huggingface-nodes/src/nodes/audio.ts Video packages/huggingface-nodes/src/nodes/video.ts 2. Implement the node class Copy the shape of an existing node in the same file. Below is an annotated skeleton for a JSON-output pipeline task: // packages/huggingface-nodes/src/nodes/text.ts (example addition) import { BaseNode, prop } from "@nodetool-ai/node-sdk"; import type { NodeClass } from "@nodetool-ai/node-sdk"; import { cleanParams, getHfToken, hfPipelineJson } from "../huggingface-base.js"; export class MySentenceSimilarityNode extends BaseNode { // Required statics ————————————————————————————————————————————— static readonly nodeType = "huggingface.MySentenceSimilarity"; // unique ID static readonly title = "My Sentence Similarity"; static readonly description = "Compute similarity between a query and a list of sentences.\n" + "text, similarity, sentence-transformers, embedding, huggingface\n\n" + "Use cases:\n" + "- Semantic search\n" + "- Duplicate detection"; static readonly inlineFields = ["source_sentence"]; // shown in node body static readonly requiredSettings = ["HF_TOKEN"]; // always include this static readonly metadataOutputTypes = { output: "list" }; // declare output shape // Properties (one @prop per input) ———————————————————————————— @prop({ type: "str", default: "sentence-transformers/all-MiniLM-L6-v2", title: "Model", description: "Sentence-similarity model repo id." }) declare model: string; @prop({ type: "str", default: "", title: "Source Sentence", description: "Query sentence." }) declare source_sentence: string; @prop({ type: "str", default: "", title: "Sentences", description: "Newline-separated sentences to compare." }) declare sentences: string; // process() calls the HF Inference API ———————————————————————— async process(): Promise<Record<string, unknown>> { const token = getHfToken(this._secrets); const source = String(this.source_sentence ?? ""); if (!source) throw new Error("Source sentence cannot be empty"); const items = String(this.sentences ?? "") .split("\n") .map((s) => s.trim()) .filter(Boolean); const result = await hfPipelineJson<number[]>( token, String(this.model ?? "sentence-transformers/all-MiniLM-L6-v2"), { inputs: { source_sentence: source, sentences: items } } ); return { output: result }; } } For binary output (image or video), use hfPipelineBinary and wrap the result with imageRefFromBytes or videoRefFromBytes. See TextToImageNode in image.ts for the full pattern. For audio input, resolve the MediaRef with refToBase64(ref, context) — the process() method must accept context as its first argument and forward it. See AutomaticSpeechRecognitionNode in audio.ts. @prop type values: "str" "int" "float" "bool" "enum" "image" "audio" "video" "dict" "list" 3. Export the new class In the same modality file, add the class to the exported array: // packages/huggingface-nodes/src/nodes/text.ts — at the bottom export const HUGGINGFACE_TEXT_NODES: readonly NodeClass[] = [ ChatCompletionNode, // ... existing nodes ... MySentenceSimilarityNode // ← add here ]; No changes needed to src/index.ts — it re-exports the modality arrays. 4. Update the registration test Open packages/huggingface-nodes/tests/registration.test.ts and add the new type to EXPECTED_NODE_TYPES: const EXPECTED_NODE_TYPES = [ // ... existing types ... "huggingface.MySentenceSimilarity" // ← add here ]; 5. Add a unit test (recommended) Add a describe block in packages/huggingface-nodes/tests/nodes.test.ts. Mock fetch to return a fixture, construct the node via new MySentenceSimilarityNode({...}), call process(), and assert on the output. See existing tests in that file for the pattern. Verify # 1. Build the package (huggingface-nodes loads from dist/) npm run build:packages # 2. Run the package tests npm run test --workspace=packages/huggingface-nodes # 3. Typecheck + lint across all packages npm run typecheck npm run lint # 4. Smoke-test the new node (replace type and props) npm run dev:nodetool -- node run huggingface.MySentenceSimilarity \ --props '{"source_sentence":"hello","sentences":"world\nhi there"}' \ --no-secrets # 5. Full check npm run check If the smoke-test hits a real HF endpoint (i.e., HF_TOKEN is set), it calls the router at https://router.huggingface.co/hf-inference/models/<model>. For a hermetic run, pass --no-secrets and mock the network in tests. How past commits did it The huggingface-nodes package and HuggingFaceProvider were both introduced in commit d1491abf (the same commit that added the Claude Agent package). The package started with the full node set for all Inference Providers pipeline tasks and the live Hub discovery mechanism. The only subsequent changes have been version bumps (e.g. ca742050 rc.26, f900e7a6 rc.25) and the broader provider registry work. Contributing PRs are welcome at https://github.com/nodetool-ai/nodetool. Join the discussion on Discord.--- # KIE: Add Models & Nodes Source: https://docs.nodetool.ai/developer/providers/kie.html Audience: Coding agents and contributors adding new KIE models or modifying KIE node behavior. This page covers the full cycle from manifest entry to verified node. TL;DR kie-manifest.json is generated — do not edit it directly. The source of truth is packages/kie-codegen/src/configs/{image,audio,video}.ts. Edit those configs, run npm run generate:kie, then build and verify. Where things live Path Purpose packages/kie-codegen/src/configs/image.ts Image-node config source packages/kie-codegen/src/configs/audio.ts Audio-node config source (TTS, music) packages/kie-codegen/src/configs/video.ts Video-node config source packages/kie-codegen/src/generate.ts Reads configs, writes manifest + pricing packages/kie-codegen/src/generate-configs.ts Fetches KIE docs, re-generates configs packages/kie-nodes/src/kie-manifest.json Generated — do not edit packages/kie-nodes/src/kie-factory.ts Creates node classes from manifest at runtime packages/kie-nodes/src/kie-base.ts API submission, polling, upload, result conversion packages/kie-nodes/src/index.ts Loads manifest, exports KIE_NODES registry packages/runtime/src/providers/kie-provider.ts KieProvider — chat + media generation packages/runtime/src/providers/manifest-models.ts Reads manifest to build model lists How KIE nodes and models are defined The manifest packages/kie-nodes/src/kie-manifest.json is an array of KieManifestEntry objects. Each entry becomes one node class at runtime via kie-factory.ts. The node type is kie.<moduleName>.<className>. A minimal image-generation entry looks like this: { "className": "BytedanceSeedream", "moduleName": "image", "modelId": "bytedance/seedream", "title": "Seedream3.0 - Text to Image", "description": "...", "outputType": "image", "pollInterval": 1500, "maxAttempts": 400, "fields": [ { "name": "prompt", "type": "str", "default": "", "title": "Prompt", "description": "...", "required": true }, { "name": "guidance_scale", "type": "float", "default": 2.5, "title": "Guidance Scale", "min": 1, "max": 10 } ], "validation": [ { "field": "prompt", "rule": "not_empty", "message": "Prompt is required" } ] } Field types type value UI / wire type str text input int / float number input bool checkbox enum dropdown; requires values: string[] image single image AssetRef audio / video single audio/video AssetRef list[image] / list[video] / list[audio] list of AssetRefs list[str] list of strings Upload descriptors When a field holds an AssetRef that the KIE API expects as a URL, declare an uploads entry. The factory uploads the asset and injects the URL under the correct API parameter name before submitting the task. "uploads": [ { "field": "images", "kind": "image", "isList": true, "paramName": "image_urls" } ] field — the node property name (must match a field in fields) kind — "image", "audio", or "video" isList — set true when the API receives an array of URLs paramName — API request body key; defaults to <field>_url (single) or <field>_urls (list) groupKey — group multiple fields into one array parameter (e.g., image1 + image2 → image_urls) isVideoClip — builds {url, start, ends} clip payloads instead of plain URLs Conditional fields conditionalFields controls whether a scalar field is included in the request: condition Behavior gte_zero Include only when Number(value) >= 0 truthy Include only when the value is truthy not_default / (anything else) Always include Model surfacing manifest-models.ts reads the manifest to build the provider’s model lists: outputType === "image" → getAvailableImageModels() outputType === "video" → getAvailableVideoModels() outputType === "audio" and name/id contains a TTS signal → getAvailableTTSModels() outputType === "audio" and name/id contains a music signal → getAvailableMusicModels() Task classification (text_to_image, image_to_image, text_to_video, etc.) is inferred from the model id and title. Add supportedTasks to the manifest entry to override inference. Add a new KIE model 1. Find the KIE API model id and endpoint shape Check https://docs.kie.ai or the KIE dashboard. Note: modelId (e.g. vendor/model-name) Output type (image, video, audio) Input fields and their types/defaults/constraints Any media upload fields (where the API wants a URL, not raw bytes) 2. Add the node config to the right config file Open the appropriate config file in packages/kie-codegen/src/configs/. Add a new entry to the nodes array. Match the existing shape exactly: // packages/kie-codegen/src/configs/image.ts { "className": "VendorNewModel", // PascalCase, used as node class name "modelId": "vendor/new-model", // KIE model id "title": "New Model - Text to Image", // Human-readable label "description": "...", // Shown in the node panel "outputType": "image", // "image" | "video" | "audio" "fields": [ { "name": "prompt", "type": "str", "default": "", "title": "Prompt", "description": "...", "required": true }, { "name": "aspect_ratio", "type": "enum", "default": "1:1", "title": "Aspect Ratio", "values": ["1:1", "16:9", "9:16"] } ], "validation": [ { "field": "prompt", "rule": "not_empty", "message": "Prompt is required" } ] } If the model takes image inputs, add an uploads array: "uploads": [ { "field": "input_image", "kind": "image", "paramName": "image_url" } ] The fields entry for input_image must use "type": "image" (not "str"). Never use raw URL strings as field types — the factory uploads AssetRefs. Poll tuning: pollInterval (ms between status checks) and maxAttempts inherit from the module’s defaults (defaultPollInterval, defaultMaxAttempts) unless overridden on the node entry. Image defaults are 1500 ms / 400 attempts; video defaults are 8000 ms / 450 attempts. 3. Regenerate the manifest npm run generate:kie This writes packages/kie-nodes/src/kie-manifest.json and updates the pricing bundles in packages/kie-nodes/src/generated/. If the KIE pricing API is unreachable, pass --no-pricing to write empty bundles and proceed: npm run generate --workspace=packages/kie-codegen -- --all --no-pricing 4. Rebuild kie-nodes loads from dist/, so a build is required before the new node appears at runtime: npm run build:packages Verify # Type-check codegen and runtime packages npm run typecheck # Run kie-nodes and kie-codegen tests npm run test --workspace=packages/kie-nodes npm run test --workspace=packages/kie-codegen # Run a single node in isolation (replace with the new node type) npm run dev:nodetool -- node run kie.image.VendorNewModel \ --props '{"prompt":"a red apple"}' --no-secrets # Static validation (catches unknown field types, missing required props) npm run dev:nodetool -- validate --json Check git diff packages/kie-nodes/src/kie-manifest.json to confirm only the expected entry was added or changed. How past changes were made commit e9e03f42 (fix(kie): map video_list to native list[video]) — changed the Gemini Omni video_list field from a bespoke video_clip_list type to the canonical list[video]. Touched packages/kie-codegen/src/configs/video.ts, kie-factory.ts, kie-manifest.json, and node-sdk/src/field-classification.ts. The factory’s isVideoClip upload flag still builds {url, start, ends} clip payloads internally. commit 6de0ef90 (feat(nodes): make content-card body fully metadata-driven) — the factory gained body: "content_card" for all media-output nodes so the frontend renders them as content cards without hardcoded namespace checks. Any new KIE node with outputType === "image" | "video" | "audio" automatically gets this flag through the factory. Both changes went through the config → generate:kie → build:packages cycle described above. Contributing Source: https://github.com/nodetool-ai/nodetool Discord: https://discord.gg/WmQTWZRcYE Before opening a PR, run npm run check (typecheck + lint + tests). Patches to kie-manifest.json alone will be rejected — changes must come from the config files and codegen pipeline.--- # Local Inference (LM Studio, llama.cpp, vLLM): Add Models Source: https://docs.nodetool.ai/developer/providers/local-inference.html NodeTool ships three providers that talk to a local OpenAI-compatible HTTP server: LM Studio, llama.cpp (llama_cpp), and vLLM. All three follow the same pattern: point NodeTool at the server’s base URL, load a model in the server, and the model appears in the UI automatically via a /v1/models fetch. Audience: coding agents and contributors adding local models to NodeTool, or changing these providers’ code. TL;DR Start the local server and load a model. Set the server’s base URL (env var or Settings → API Keys). Models appear in NodeTool automatically — no code change needed. The only time you touch code is when you are changing provider behavior (URL resolution, message normalization, tool-call handling) — see Provider-level changes. Where things live Concern Path LM Studio provider packages/runtime/src/providers/lmstudio-provider.ts llama.cpp provider packages/runtime/src/providers/llama-provider.ts vLLM provider packages/runtime/src/providers/vllm-provider.ts Default base URLs packages/runtime/src/providers/defaults.ts Provider registration block packages/runtime/src/providers/index.ts lines 241–282 How models are discovered Each provider implements getAvailableLanguageModels(), which calls GET <baseURL>/v1/models at runtime. The response shape is the standard OpenAI list: { "data": [{ "id": "model-name" }, ...] } llama.cpp also accepts a models top-level key as a fallback (payload.data ?? payload.models ?? []). The provider maps each id to a LanguageModel record; no static list exists in the codebase. Loading a new model in the server makes it visible in NodeTool on the next model-list refresh. URL resolution order (all three providers): options.baseURL (constructor arg — tests only) Secret store / env var (LMSTUDIO_API_URL / LLAMA_CPP_URL / VLLM_BASE_URL) Hard-coded default (LM Studio only — http://127.0.0.1:1234; llama.cpp and vLLM throw if unset) Trailing slashes are stripped before /v1 is appended. Default ports: Server Default URL LM Studio http://127.0.0.1:1234 (defined in defaults.ts) llama.cpp none — must be set vLLM none — must be set Registration All three are registered in packages/runtime/src/providers/index.ts inside a guard that skips production: if (_envProcess.env["NODETOOL_ENV"] !== "production") { registerBuiltinProvider( PROVIDER_IDS.LMSTUDIO, LMStudioProvider, {}, { LMSTUDIO_API_URL: LMSTUDIO_DEFAULT_URL, LMSTUDIO_API_KEY: "lm-studio" } ); registerBuiltinProvider(PROVIDER_IDS.LLAMA_CPP, LlamaProvider, { LLAMA_CPP_URL: "" }); registerBuiltinProvider( PROVIDER_IDS.VLLM, VLLMProvider, { VLLM_BASE_URL: "" }, { VLLM_API_KEY: "sk-no-key-required" } ); } The fourth argument is optionalKwargs — settings re-resolved from the secret store on every getProvider() call without blocking isProviderConfigured(). That is why LM Studio works by default (the URL has a fallback); llama.cpp and vLLM are unavailable until their URL is set. Per-server setup LM Studio Env var / settings key: LMSTUDIO_API_URL Default URL: http://127.0.0.1:1234 API key: lm-studio (hard-coded default; override with LMSTUDIO_API_KEY) LM Studio’s OpenAI-compatible server starts when you enable it in LM Studio → Local Server. Once running, NodeTool connects automatically with no URL config needed. To use a different port: # In your shell environment, or via Settings → API Keys export LMSTUDIO_API_URL=http://127.0.0.1:8080 Or in the chat CLI: npm run dev:chat -- --agent --provider lmstudio --model <model-id> Tool calls are always reported as supported (hasToolSupport returns true); whether a specific model actually handles them depends on the model. llama.cpp Env var / settings key: LLAMA_CPP_URL (required — no default) API key: none (sk-no-key-required is sent automatically) Start the llama.cpp HTTP server: ./llama-server --model /path/to/model.gguf --port 8080 --ctx-size 4096 Then set the URL: export LLAMA_CPP_URL=http://127.0.0.1:8080 llama.cpp does not reliably support the OpenAI tool-call wire format. LlamaProvider sets hasToolSupport to false and falls back to text-emulated tool calls: after a stop finish reason, the provider scans the accumulated text for lines matching FunctionName(arg=value, ...) patterns and converts them to ToolCall objects. This parsing is in parseEmulatedToolCalls() / parseKeywordArgs() in llama-provider.ts. Message normalization is also heavier here: system messages are collected and prepended, tool role messages are rewritten as user messages, and strict user/assistant alternation is enforced by inserting empty filler turns. vLLM Env var / settings key: VLLM_BASE_URL (required — no default) API key: optional via VLLM_API_KEY; defaults to sk-no-key-required Start a vLLM server: vllm serve meta-llama/Llama-3-8B-Instruct --port 8000 Then set the URL: export VLLM_BASE_URL=http://127.0.0.1:8000 If your vLLM deployment requires an API key: export VLLM_API_KEY=your-key VLLMProvider extends OpenAIProvider and reports tool support as true. Provider-level changes (dev path) You need to edit source only when changing how a provider works, not when adding models. Change File to edit URL resolution or default port packages/runtime/src/providers/defaults.ts and the constructor in the provider file Tool-call handling lmstudio-provider.ts / llama-provider.ts / vllm-provider.ts Message normalization (llama.cpp) normalizeMessagesForLlama() in llama-provider.ts Registration kwargs (env var name, default value) packages/runtime/src/providers/index.ts registration block Container env propagation getContainerEnv() in the provider class These providers do not load from dist/ (unlike base-nodes, node-sdk), so no build step is needed — changes take effect on the next npm run dev. Verify # 1. Check types and lint npm run typecheck npm run lint # 2. Start the local server (example: llama.cpp on port 8080) ./llama-server --model /path/to/model.gguf --port 8080 # 3. Set the URL and run a single node against it export LLAMA_CPP_URL=http://127.0.0.1:8080 npm run dev:nodetool -- node run nodetool.text.llm.LlamaAgent \ --props '{"prompt": "What is 2+2?", "model": {"provider": "llama_cpp", "model": "your-model-id"}}' # Or use the chat CLI npm run dev:chat -- --agent --provider llama_cpp --model your-model-id # 4. Run all checks npm run check For LM Studio and vLLM, substitute the provider name (lmstudio, vllm) and the corresponding env var. How past commits did it 566441b4 (“refactor: dedupe constants to single canonical definitions”) introduced packages/runtime/src/providers/defaults.ts, consolidating LMSTUDIO_DEFAULT_URL (http://127.0.0.1:1234) and OLLAMA_DEFAULT_URL from scattered literals across runtime, websocket, and cli. It also wired LMSTUDIO_API_URL resolution through the provider constructor so Settings → API Keys changes take effect without a restart. Files changed: defaults.ts (new), lmstudio-provider.ts, index.ts, websocket/src/models-api.ts, websocket/src/openai-api.ts, cli/src/providers.ts. Contributing PRs are welcome at https://github.com/nodetool-ai/nodetool. Before pushing: npm run check # typecheck + lint + test Join the discussion on Discord.--- # MiniMax: Add Models & Nodes Source: https://docs.nodetool.ai/developer/providers/minimax.html Audience: contributors and coding agents adding new MiniMax models or workflow nodes. TL;DR Add a model ID to the relevant catalogue array in minimax-provider.ts (for the generic provider) and to the matching constant in minimax-base.ts (for the node pack’s @prop enum). For a new node type, copy an existing node file in packages/minimax-nodes/src/nodes/, register it in src/index.ts, add a test under tests/. Build: npm run build:packages. Verify: npm run typecheck, then npm run dev:nodetool -- node run minimax.<YourNodeType> --props '{...}'. Where things live What Path Provider class (chat/image/video/TTS/music) packages/runtime/src/providers/minimax-provider.ts Shared node constants and helpers packages/minimax-nodes/src/minimax-base.ts Node implementations packages/minimax-nodes/src/nodes/*.ts Node pack entry point packages/minimax-nodes/src/index.ts Node pack tests packages/minimax-nodes/tests/*.ts Provider ID constant packages/protocol/src/api-types.ts line 872 (MINIMAX: "minimax") Provider registration packages/runtime/src/providers/index.ts line 215 API docs https://platform.minimax.io/docs/api-reference/api-overview How MiniMax models and nodes are defined Provider catalogue (static arrays) MinimaxProvider in minimax-provider.ts overrides six getAvailable* methods. Each returns a hardcoded array — there is no live API discovery. The arrays are the source of truth for the generic nodes (chat, generic TTS, generic video). // packages/runtime/src/providers/minimax-provider.ts override async getAvailableVideoModels(): Promise<VideoModel[]> { return [ { id: "MiniMax-Hailuo-2.3", name: "MiniMax Hailuo 2.3", provider: "minimax", supportedTasks: ["text_to_video", "image_to_video"] }, // ... ]; } Node pack constants (in minimax-base.ts) packages/minimax-nodes/src/minimax-base.ts duplicates the model IDs as plain string arrays. These feed the values field in each node’s @prop decorator, which drives UI drop-downs and validation. // packages/minimax-nodes/src/minimax-base.ts export const MINIMAX_T2V_MODELS: string[] = [ "MiniMax-Hailuo-2.3", "MiniMax-Hailuo-2.3-Fast", "MiniMax-Hailuo-02", "T2V-01-Director" ]; Why two places? Nodes call MiniMax’s REST API directly (not through the provider) so they can expose MiniMax-specific fields (emotion, volume, pitch, camera direction, lyrics) that the generic BaseProvider interface does not carry. Node registration src/index.ts exports MINIMAX_NODES (the full array) and registerMinimaxNodes (the registry helper). Every new node class must be imported and added to both. Add a new model or node Case A: new model ID in an existing node type Add the ID to minimax-provider.ts in the matching getAvailable* method: { id: "MiniMax-Hailuo-3.0", name: "MiniMax Hailuo 3.0", provider: "minimax", supportedTasks: ["text_to_video", "image_to_video"] }, Add the same ID to minimax-base.ts in the matching constant: export const MINIMAX_T2V_MODELS: string[] = [ "MiniMax-Hailuo-3.0", // new "MiniMax-Hailuo-2.3", // ... ]; Do the same for MINIMAX_I2V_MODELS if the model supports image-to-video. The node’s @prop enum picks up the new entry automatically — no change to the node file needed. Case B: new node type Say MiniMax releases a new endpoint, e.g. POST /v1/text_to_3d. Create packages/minimax-nodes/src/nodes/text-to-3d.ts, modelled on an existing node: import { BaseNode, prop } from "@nodetool-ai/node-sdk"; import type { NodeClass } from "@nodetool-ai/node-sdk"; import { assertBaseResp, getMinimaxApiKey, MINIMAX_BASE_URL, minimaxHeaders } from "../minimax-base.js"; const TEXT_TO_3D_MODELS = ["3d-01"]; export class MinimaxTextTo3DNode extends BaseNode { static readonly nodeType = "minimax.TextTo3D"; static readonly body = "content_card"; static readonly title = "MiniMax Text to 3D"; static readonly description = "Generate a 3D model from a text prompt using MiniMax 3D-01.\n" + "3d, generation, text-to-3d, minimax\n\n" + "Use cases:\n" + "- Prototype 3D assets from descriptions\n" + "- Generate objects for scenes"; static readonly metadataOutputTypes = { output: "model3d" }; static readonly inlineFields: string[] = []; static readonly inputFields: string[] = ["prompt"]; static readonly requiredSettings = ["MINIMAX_API_KEY"]; static readonly autoSaveAsset = true; @prop({ type: "enum", default: "3d-01", title: "Model", description: "The MiniMax 3D model to use.", values: TEXT_TO_3D_MODELS }) declare model: any; @prop({ type: "str", default: "A ceramic vase with floral patterns", title: "Prompt", description: "Text prompt describing the 3D object." }) declare prompt: any; async process(): Promise<Record<string, unknown>> { const apiKey = getMinimaxApiKey(this._secrets); const prompt = String(this.prompt ?? ""); if (!prompt) throw new Error("Prompt is required"); const res = await fetch(`${MINIMAX_BASE_URL}/v1/text_to_3d`, { method: "POST", headers: minimaxHeaders(apiKey), body: JSON.stringify({ model: String(this.model ?? "3d-01"), prompt }) }); if (!res.ok) { throw new Error(`MiniMax text_to_3d failed: ${res.status} ${await res.text()}`); } const data = (await res.json()) as Record<string, unknown>; assertBaseResp(data, "text_to_3d"); // parse response and return output ref return { output: { type: "model3d", data: "" } }; } } export const TEXT_TO_3D_NODES: readonly NodeClass[] = [MinimaxTextTo3DNode]; Register in src/index.ts: import { TEXT_TO_3D_NODES } from "./nodes/text-to-3d.js"; export { MinimaxTextTo3DNode } from "./nodes/text-to-3d.js"; export const MINIMAX_NODES: readonly NodeClass[] = [ ...VOICE_NODES, ...TEXT_TO_SPEECH_NODES, ...MUSIC_NODES, ...TEXT_TO_IMAGE_NODES, ...TEXT_TO_VIDEO_NODES, ...IMAGE_TO_VIDEO_NODES, ...TEXT_TO_3D_NODES // add here ]; Add a test in packages/minimax-nodes/tests/text-to-3d.test.ts. Stub fetch with vi.stubGlobal (see any existing test for the pattern). At minimum assert the node calls the right endpoint and returns an output. Update tests/registration.test.ts — add "minimax.TextTo3D" to both registry.has(...) checks and the types array. Verify Run these in order after any change: # 1. Type check all packages npm run typecheck # 2. Build (minimax-nodes loads from dist/) npm run build:packages # 3. Run a single node in isolation (no full server needed) npm run dev:nodetool -- node run minimax.TextToSpeech \ --props '{"text":"hello","voice_id":"English_Trustworth_Man","model":"speech-2.6-hd"}' \ --no-secrets # 4. Run the minimax-nodes test suite npm run test --workspace=packages/minimax-nodes # 5. Validate a workflow that uses the new node type npm run dev:nodetool -- validate workflow.json The --no-secrets flag lets the node runner skip the database. The node will fail at the network call (no real API key), but a type error or missing-field panic surfaces before that. How past commits did it The minimax node pack and provider were introduced before the current git window, but the two most recent commits that touched these files are: ca742050 — version bump to 0.7.0-rc.26 (touched packages/minimax-nodes/package.json along with all other packages) f900e7a6 — version bump to 0.7.0-rc.25 These are housekeeping commits. The substantive minimax work (node pack creation, provider implementation) predates the visible history on this branch, but the pattern matches the elevenlabs-nodes pack (same direct-fetch approach, same minimax-base.ts ↔ minimax-provider.ts split). To find the original introduction commit across all branches: git log --all --oneline -- packages/minimax-nodes/src/index.ts | tail -1 Contributing Source: https://github.com/nodetool-ai/nodetool Discord: https://discord.gg/WmQTWZRcYE Before opening a PR: npm run check # runs typecheck + lint + test for all packages All three must pass. Do not commit if either npm run typecheck or npm run lint fails.--- # Ollama: Add Models Source: https://docs.nodetool.ai/developer/providers/ollama.html The Ollama provider connects NodeTool to a local Ollama daemon. Models are discovered dynamically from the running daemon — no code change is needed to add a new model. Provider-level changes (tool detection, embedding behavior, URL configuration) require editing the provider source. Audience: coding agents and contributors — both users adding local models and devs modifying the provider. TL;DR ollama pull llama3.2 # makes the model appear in NodeTool automatically ollama pull nomic-embed-text # embedding models work the same way No code change. No restart required (NodeTool queries the daemon on every model-list call). If you need to change tool-support detection, the default URL, or keep_alive behavior, see Provider-level changes below. Where things live Concern Path Provider implementation packages/runtime/src/providers/ollama-provider.ts Default daemon URL packages/runtime/src/providers/defaults.ts (OLLAMA_DEFAULT_URL) Registration + optionalKwargs packages/runtime/src/providers/index.ts (lines 248–253) Provider registry resolution logic packages/runtime/src/providers/provider-registry.ts How Ollama models are discovered OllamaProvider.getAvailableLanguageModels() (line 357 of ollama-provider.ts) fetches GET {apiUrl}/api/tags from the local daemon on every call. It maps each entry’s model field (falling back to name) to a LanguageModel record: // packages/runtime/src/providers/ollama-provider.ts:357-372 async getAvailableLanguageModels(): Promise<LanguageModel[]> { const response = await this._fetch(`${this.apiUrl}/api/tags`); if (!response.ok) return []; const payload = (await response.json()) as { models?: Array<{ name?: string; model?: string }>; }; ... } getAvailableEmbeddingModels() (line 374) calls getAvailableLanguageModels() and re-wraps every result as an EmbeddingModel with dimensions: 0. Ollama does not advertise context size or dimensions over /api/tags, so both methods surface all pulled models regardless of type. The caller (RAG pipeline, agent config) is responsible for picking an appropriate model. URL resolution order (highest priority first): Secret store — key OLLAMA_API_URL (set via nodetool secrets store OLLAMA_API_URL or Settings → API Keys) Environment variable — OLLAMA_API_URL Registered default — http://127.0.0.1:11434 (OLLAMA_DEFAULT_URL in defaults.ts) The registration in index.ts uses an optionalKwarg for OLLAMA_API_URL so the registry always considers Ollama “configured” (no key required) while still resolving a user-set URL on every getProvider() call without a restart. // packages/runtime/src/providers/index.ts:248-253 registerBuiltinProvider( PROVIDER_IDS.OLLAMA, OllamaProvider, {}, { OLLAMA_API_URL: OLLAMA_DEFAULT_URL } ); Ollama is registered only when NODETOOL_ENV !== "production" (lines 241–282 of index.ts). In production/cloud deployments the provider is pruned from the registry. The constructor also reads OLLAMA_KEEP_ALIVE from process.env. Default is "10m". Set it to "-1" to keep models resident indefinitely, or "0" to unload immediately after each request. Add a model (user path) Pull the model with the Ollama CLI: # Language models ollama pull llama3.2 ollama pull gemma3:27b ollama pull phi4 ollama pull deepseek-r1:8b # Embedding models ollama pull nomic-embed-text ollama pull mxbai-embed-large # Vision models (multimodal) ollama pull llava:13b ollama pull minicpm-v NodeTool picks up the model on the next call to getAvailableLanguageModels() or getAvailableEmbeddingModels() — no restart needed. The model ID passed to the provider must match the tag exactly as shown by ollama list (e.g. llama3.2:latest, gemma3:27b). To point NodeTool at a non-default Ollama instance (remote host, different port): npm run dev:nodetool -- secrets store OLLAMA_API_URL # Paste the URL when prompted, e.g.: http://192.168.1.10:11434 Or set the environment variable before starting the server: OLLAMA_API_URL=http://192.168.1.10:11434 npm run dev Provider-level changes (dev path) Edit packages/runtime/src/providers/ollama-provider.ts. The main extension points: Tool-support detection hasToolSupport(model) (line 129) queries POST {apiUrl}/api/show and checks for "tools" in the capabilities array. It caches results in _modelInfoCache (a Map<string, Record<string, unknown>>). When the /api/show call fails or the model has no capabilities field, it defaults to true. If a model reports no tool support, generateMessage and generateMessages fall back to _injectToolEmulationPrompt + _parseEmulatedToolCalls, which inject tool descriptions into the system prompt and parse function_name(param='value') patterns from the output. To override detection for a specific model (e.g. a model that advertises tools but calls them incorrectly), add a guard at the top of hasToolSupport: if (model.startsWith("my-broken-model")) return false; Embedding dimensions getAvailableEmbeddingModels() returns dimensions: 0 for every model because Ollama’s /api/tags response does not include that field. If you need accurate dimensions, query POST /api/show for the pulled model and read model_info["embedding_length"]. The _modelInfoCache already stores the full /api/show response, so the call is free on the second access. Keep-alive Change the default (currently "10m") by editing the fallback in the constructor: // packages/runtime/src/providers/ollama-provider.ts:109-110 const keepAlive = process.env.OLLAMA_KEEP_ALIVE?.trim(); this.keepAlive = keepAlive && keepAlive.length > 0 ? keepAlive : "10m"; Default URL Change OLLAMA_DEFAULT_URL in packages/runtime/src/providers/defaults.ts. The constant is re-exported from packages/runtime/src/providers/index.ts so callers import it as @nodetool-ai/runtime. Verify # 1. Confirm the daemon is running and a model is present curl http://127.0.0.1:11434/api/tags # 2. Type-check the provider package npm run typecheck --workspace=packages/runtime # 3. Run a single Ollama-backed node (replace model with one you have pulled) npm run dev:nodetool -- node run nodetool.text.Generate \ --props '{"prompt":"hello","provider":"ollama","model":"llama3.2:latest"}' # 4. Full suite npm run check If the daemon is not running, getAvailableLanguageModels() returns [] silently (the provider guards if (!response.ok) return []). Check that ollama serve (or the Ollama desktop app) is running before debugging further. How past commits did it The provider file was first tracked in commit d1491abf (“add claude agent package”), which established the provider registration pattern, the optionalKwargs mechanic for OLLAMA_API_URL, and the NODETOOL_ENV !== "production" guard for local-only providers. Commit 364620b4 (PR #3969, “Add failure diagnosis, cassette recording, and plan caching”) is the most recent change touching packages/runtime/src/providers/index.ts; the Ollama registration block was unchanged, confirming the zero-required-kwarg pattern is stable. Contributing PRs welcome at https://github.com/nodetool-ai/nodetool. Before pushing: npm run lint # must pass npm run typecheck # must pass Join the discussion on Discord.--- # OpenAI-Compatible Providers: Add a Provider or Models Source: https://docs.nodetool.ai/developer/providers/openai-compatible.html Audience: Coding agents and contributors who want to add a new OpenAI-compatible cloud provider or change how an existing one exposes models. This guide covers seven providers that share one implementation pattern: Groq, Mistral, DeepSeek, Moonshot (Kimi), Cerebras, Cohere, and OpenRouter. TL;DR Models appear automatically: each provider fetches its /models endpoint at runtime, so new upstream models show up in the model picker without a code change. Adding a whole new provider is three small edits: one constant in @nodetool-ai/protocol, one ~90-line file in packages/runtime/src/providers/, one registerBuiltinProvider call in the runtime index. Cohere is the only exception: it subclasses BaseProvider directly (embeddings only, no chat). Moonshot is another exception: it subclasses AnthropicProvider (Kimi exposes an Anthropic-compatible endpoint, not OpenAI). Where things live What Path Provider ID constants packages/protocol/src/api-types.ts (PROVIDER_IDS const, line 858) Provider source files packages/runtime/src/providers/<name>-provider.ts Registration + exports packages/runtime/src/providers/index.ts OpenAI base class packages/runtime/src/providers/openai-provider.ts Base class for all providers packages/runtime/src/providers/base-provider.ts Provider types (LanguageModel, etc.) packages/runtime/src/providers/types.ts The shared pattern DeepSeek is the canonical example. The full 93-line file is annotated below. // packages/runtime/src/providers/deepseek-provider.ts import OpenAI from "openai"; import { OpenAIProvider } from "./openai-provider.js"; // (1) inherit chat, streaming, tools import type { LanguageModel } from "./types.js"; const DEEPSEEK_BASE_URL = "https://api.deepseek.com/v1"; // (2) provider's OpenAI-compatible base interface DeepSeekProviderOptions { client?: OpenAI; clientFactory?: (apiKey: string) => OpenAI; fetchFn?: typeof fetch; // (3) injected for testability } export class DeepSeekProvider extends OpenAIProvider { // (4) requiredSecrets() names the env/DB key the registry resolves at call time static override requiredSecrets(): string[] { return ["DEEPSEEK_API_KEY"]; } private _deepseekFetch: typeof fetch; constructor( secrets: { DEEPSEEK_API_KEY?: string }, options: DeepSeekProviderOptions = {} ) { const apiKey = secrets.DEEPSEEK_API_KEY; if (!apiKey) { throw new Error("DEEPSEEK_API_KEY is required"); } const fetchFn = options.fetchFn ?? globalThis.fetch.bind(globalThis); // (5) pass key as OPENAI_API_KEY; supply a clientFactory that sets baseURL super( { OPENAI_API_KEY: apiKey }, { client: options.client, clientFactory: options.clientFactory ?? ((key) => new OpenAI({ apiKey: key, baseURL: DEEPSEEK_BASE_URL })), fetchFn } ); // (6) override the `provider` field so spans and model objects carry the right id (this as { provider: string }).provider = "deepseek"; this._deepseekFetch = fetchFn; } // (7) export the provider-specific key to Docker/subprocess environments override getContainerEnv(): Record<string, string> { return { DEEPSEEK_API_KEY: this.apiKey }; } // (8) true when the API supports function/tool calling; override per-model if mixed override async hasToolSupport(_model: string): Promise<boolean> { return true; } // (9) fetch the live model list; return [] on failure (graceful degradation) override async getAvailableLanguageModels(): Promise<LanguageModel[]> { const response = await this._deepseekFetch( `${DEEPSEEK_BASE_URL}/models`, { headers: { Authorization: `Bearer ${this.apiKey}` } } ); if (!response.ok) return []; const payload = (await response.json()) as { data?: Array<{ id?: string; name?: string }>; }; const rows = payload.data ?? []; return rows .filter( (row): row is { id: string; name?: string } => typeof row.id === "string" && row.id.length > 0 ) .map((row) => ({ id: row.id, name: row.name ?? row.id, provider: "deepseek" })); } } Groq, Mistral, Cerebras, and OpenRouter follow this pattern exactly. OpenRouter adds a few extras: defaultHeaders on the OpenAI client (HTTP-Referer, X-Title), textToImage, and a hasToolSupport override that returns false for o1/o3 model IDs. Add a brand-new OpenAI-compatible provider Step 1 — Add the provider ID to @nodetool-ai/protocol Open packages/protocol/src/api-types.ts and add a key to PROVIDER_IDS: // packages/protocol/src/api-types.ts (around line 858) export const PROVIDER_IDS = { // ... existing entries ... ACME: "acme", // wire id — used in model objects, spans, settings } as const; The comment above the const (line 855) says: “Adding a provider? Add it here first, then register it in @nodetool-ai/runtime’s provider index.” Follow that order. Build the protocol package so downstream packages see the new constant: cd packages/protocol && npm run build Step 2 — Write acme-provider.ts Create packages/runtime/src/providers/acme-provider.ts. Copy the DeepSeek template above and substitute: Placeholder Replace with DeepSeek Acme DEEPSEEK_API_KEY ACME_API_KEY DEEPSEEK_BASE_URL / https://api.deepseek.com/v1 ACME’s base URL "deepseek" (the wire id string) "acme" this._deepseekFetch this._acmeFetch If the provider’s /models response uses a different shape (not { data: [{ id, name }] }), adjust the getAvailableLanguageModels parser accordingly. If the provider does not support tool calling for some or all models, implement hasToolSupport to return false where appropriate (see the OpenRouter implementation for a model-name heuristic). Step 3 — Export and register in packages/runtime/src/providers/index.ts Two lines: // near the other thin-provider imports import { AcmeProvider } from "./acme-provider.js"; // near the other exports export { AcmeProvider }; And one registerBuiltinProvider call in the registration block (around line 199): registerBuiltinProvider(PROVIDER_IDS.ACME, AcmeProvider, { ACME_API_KEY: "" }); The empty string for ACME_API_KEY is intentional — it forces the registry to resolve the key from the DB or env on every getProvider() call rather than baking a stale value at module load. Adjust an existing provider’s models Chat models are fetched dynamically — no change needed If Groq, DeepSeek, Mistral, Cerebras, or OpenRouter adds a new chat model, it appears in the model picker automatically on the next call to getAvailableLanguageModels(). No code change is required. Tool-support overrides hasToolSupport(model: string) controls whether the UI offers tool/function calling for a given model. All five OpenAI-subclass providers currently return true unconditionally. OpenRouter overrides per-model-id: // packages/runtime/src/providers/openrouter-provider.ts override async hasToolSupport(model: string): Promise<boolean> { const lower = model.toLowerCase(); if (lower.includes("o1") || lower.includes("o3")) return false; return true; } Apply the same pattern to any other provider where some models lack tool support. Embedding models (Mistral) Mistral is the only OpenAI-subclass provider that also exposes embeddings. The model list is static (one entry, mistral-embed) and lives in getAvailableEmbeddingModels() in packages/runtime/src/providers/mistral-provider.ts. To add an embedding model, append to that array. Cohere — embeddings-only, different base class Cohere subclasses BaseProvider directly, not OpenAIProvider. It has no chat support. Its embedding model list is the static COHERE_EMBEDDING_MODELS array at the top of packages/runtime/src/providers/cohere-provider.ts. Append there to add a new Cohere embedding model. Moonshot — Anthropic-compatible endpoint Moonshot subclasses AnthropicProvider and uses a hardcoded knownModels list in getAvailableLanguageModels() (not a live fetch). To add a Kimi model, add its id to that array in packages/runtime/src/providers/moonshot-provider.ts. Verify After any change: 1. Build protocol if you changed api-types.ts. cd packages/protocol && npm run build 2. Type-check. npm run typecheck 3. Confirm the provider returns models (requires the API key in secrets or env). npm run dev:nodetool -- info --json | grep acme Or run a single chat to exercise the full path: npm run dev:chat -- --provider acme --model <model-id> 4. Single-node smoke test (chat node via the provider). npm run dev:nodetool -- node run nodetool.text.TextGenerationNode \ --props '{"provider":"acme","model":"<model-id>","prompt":"hello"}' 5. Full suite. npm run check How past PRs did it Commit 25b0623f (merged as PR #3377) introduced the five OpenAI-subclass providers in one shot. It added: packages/runtime/src/providers/cerebras-provider.ts (85 lines) packages/runtime/src/providers/cohere-provider.ts (162 lines) packages/runtime/src/providers/deepseek-provider.ts (92 lines) packages/runtime/src/providers/groq-provider.ts (85 lines) packages/runtime/src/providers/mistral-provider.ts (115 lines) packages/runtime/src/providers/moonshot-provider.ts (65 lines) packages/runtime/src/providers/openrouter-provider.ts (225 lines, with image-gen extras) All seven were wired into index.ts in that same commit. To see the full diff: git show 25b0623f. Commit 34599547 (“refactor(providers): centralize provider IDs in protocol”) moved provider id strings out of scattered literals and into PROVIDER_IDS in @nodetool-ai/protocol. Any new provider follows the post-refactor convention: constant in protocol, PROVIDER_IDS.X everywhere else. Contributing Repository: https://github.com/nodetool-ai/nodetool Discord: https://discord.gg/WmQTWZRcYE Run npm run check (typecheck + lint + tests) before opening a PR. PRs that break any of the three will not merge. Follow docs/WRITING_STYLE.md for any Markdown you touch.--- # OpenAI: Add Models Source: https://docs.nodetool.ai/developer/providers/openai.html NodeTool’s OpenAI integration lives in one file: packages/runtime/src/providers/openai-provider.ts (1399 lines). The provider is registered under PROVIDER_IDS.OPENAI = "openai" in packages/protocol/src/api-types.ts (line 860). Audience: coding agents and contributors adding new OpenAI models. TL;DR Model type What to do Chat / LLM (e.g. gpt-5, o3) Nothing — the list is fetched live from https://api.openai.com/v1/models Image (e.g. gpt-image-3) Add one entry to getAvailableImageModels() (~line 249) and one pricing entry if quality-based Video (e.g. sora-3) Add one entry to getAvailableVideoModels() (~line 232) TTS / ASR / Embedding Add one entry to the matching static getter Where things live Concern Path Notes Provider class packages/runtime/src/providers/openai-provider.ts All model lists and API calls Provider ID constant packages/protocol/src/api-types.ts line 860 PROVIDER_IDS.OPENAI = "openai" Provider registration packages/runtime/src/providers/index.ts Imports and re-exports OpenAIProvider Non-token pricing tiers packages/runtime/src/providers/cost-calculator.ts lines 56–87 PRICING_TIERS object Per-model tier mapping packages/runtime/src/providers/cost-calculator.ts lines 94–113 MODEL_TO_TIER object Quality-based image cost packages/runtime/src/providers/cost-calculator.ts lines 368–392 calculateImageCost() How OpenAI models are defined Chat / LLM models — dynamic getAvailableLanguageModels() (line 178) makes a live GET request to https://api.openai.com/v1/models using the stored API key, then maps every returned id to a LanguageModel object: // openai-provider.ts lines 178–203 async getAvailableLanguageModels(): Promise<LanguageModel[]> { const response = await this._fetch("https://api.openai.com/v1/models", { headers: { Authorization: `Bearer ${this.apiKey}` } }); if (!response.ok) return []; const payload = await response.json() as { data?: Array<{ id?: string }> }; return (payload.data ?? []) .filter((row): row is { id: string } => typeof row.id === "string") .map((row) => ({ id: row.id, name: row.id, provider: "openai" })); } A new chat model released by OpenAI appears in NodeTool automatically the next time the model list is refreshed — no code change needed. Tool support is controlled by hasToolSupport() (line 174): async hasToolSupport(model: string): Promise<boolean> { return !(model.startsWith("o1") || model.startsWith("o3")); } If a new model needs to opt out of tool use, add its prefix here. Image, video, TTS, ASR, and embedding models — static These model types are returned from hardcoded lists inside the provider because OpenAI’s /v1/models endpoint does not distinguish modalities. Each getter returns an array of typed objects. getAvailableImageModels() (line 249) currently lists four models: gpt-image-2, gpt-image-1.5, gpt-image-1, gpt-image-1-mini. getAvailableVideoModels() (line 232) lists sora-2 and sora-2-pro. getAvailableTTSModels() (line 205), getAvailableASRModels() (line 222), and getAvailableEmbeddingModels() (line 278) follow the same pattern. Add a new image model 1. Add the entry to getAvailableImageModels() Open packages/runtime/src/providers/openai-provider.ts and add to the array returned at line 249. Match the shape of existing entries exactly: // packages/runtime/src/providers/openai-provider.ts ~line 249 async getAvailableImageModels(): Promise<ImageModel[]> { return [ { id: "gpt-image-3", // OpenAI API model ID name: "GPT Image 3", // display name shown in the UI provider: "openai", supportedTasks: ["text_to_image", "image_to_image"] }, // ... existing entries ... ]; } supportedTasks must be a subset of ["text_to_image", "image_to_image", "inpainting"] — use whichever the model actually supports. 2. Add pricing if quality-based calculateImageCost() in cost-calculator.ts (line 376) special-cases any model whose ID contains "gpt-image" (excluding gpt-image-1.5) and routes it through quality tiers defined in PRICING_TIERS (lines 57–60): // cost-calculator.ts lines 57–60 imageGptLow: { costType: CostType.IMAGE_BASED, perImage: 0.011 }, imageGptMedium: { costType: CostType.IMAGE_BASED, perImage: 0.042 }, imageGptHigh: { costType: CostType.IMAGE_BASED, perImage: 0.167 }, If gpt-image-3 uses the same three-tier structure at different prices, add new tiers and extend the qualityMap inside calculateImageCost() (line 377): // cost-calculator.ts PRICING_TIERS — add new tiers imageGpt3Low: { costType: CostType.IMAGE_BASED, perImage: 0.015 }, imageGpt3Medium: { costType: CostType.IMAGE_BASED, perImage: 0.060 }, imageGpt3High: { costType: CostType.IMAGE_BASED, perImage: 0.200 }, Then guard the existing qualityMap lookup to branch on model ID: // cost-calculator.ts calculateImageCost() if (modelId.toLowerCase().includes("gpt-image-3")) { const qualityMap = { low: "imageGpt3Low", medium: "imageGpt3Medium", high: "imageGpt3High" }; // ... } If the model is flat-rate (no quality levels), add it to MODEL_TO_TIER (line 94) instead: // cost-calculator.ts MODEL_TO_TIER "openai:gpt-image-3": "imageGptMedium", // or a new tier 3. Add pricing for non-image modalities (TTS / ASR) New TTS or ASR models follow the same pattern as existing entries in MODEL_TO_TIER (lines 96–101). Add "openai:<model-id>": "<tierName>" and, if needed, a new tier object in PRICING_TIERS. Add a new chat / LLM model Usually you do not need to do anything. The live fetch covers all models in your account. Check the model is available in your tier at https://platform.openai.com/docs/models. A code change is needed only for these cases: Disable tool use for a new reasoning model prefix — extend hasToolSupport() (line 174): async hasToolSupport(model: string): Promise<boolean> { return !( model.startsWith("o1") || model.startsWith("o3") || model.startsWith("o4") // add new reasoning prefix here ); } Token-based pricing — chat models are priced via @pydantic/genai-prices (imported in cost-calculator.ts line 19). That package is community-maintained and tracks OpenAI pricing. If a brand-new model is missing from the catalog, wait for a @pydantic/genai-prices release or pin an interim entry by adding a dummy MODEL_TO_TIER key that maps to an existing tier. Verify Run these in order after any edit: # 1. Type-check the runtime package (and all packages) npm run typecheck # 2. Smoke-test the model list (requires OPENAI_API_KEY in environment) npm run dev:nodetool -- info # 3. Smoke-test a single image node (no secrets needed for type check) npm run dev:nodetool -- node run nodetool.image.generate.OpenAIImageNode \ --props '{"prompt": "a red circle", "model": {"id": "gpt-image-1", "provider": "openai"}}' \ --no-secrets # 4. Full check (typecheck + lint + tests) npm run check All three of npm run typecheck, npm run lint, and npm run test must pass before committing. How past PRs did it The XAI provider addition (commit 69dd6f88, PR #3951, “Add image and video generation support to XAI provider”) is the closest parallel: it shows the exact pattern for adding static image and video model lists to a provider that already handles dynamic language model fetching. The files changed were packages/runtime/src/providers/xai-provider.ts and packages/runtime/src/providers/cost-calculator.ts — the same two files you edit for a new OpenAI image model. The OpenAI provider’s static video list (sora-2, sora-2-pro) was added following the same approach, visible in getAvailableVideoModels() at line 232 of openai-provider.ts. Contributing Open a PR at https://github.com/nodetool-ai/nodetool. Before pushing: npm run check # typecheck + lint + test across all workspaces Join the discussion on Discord.--- # Replicate: Add Models & Nodes Source: https://docs.nodetool.ai/developer/providers/replicate.html Audience: Coding agents and contributors adding Replicate models to NodeTool. This guide assumes familiarity with the monorepo structure described in CLAUDE.md. TL;DR Add the model’s "owner/name" entry to the matching config file in packages/replicate-codegen/src/configs/. Export REPLICATE_API_TOKEN and run npm run generate:replicate. Run npm run build:packages (replicate-nodes loads from dist/). Verify with npm run dev:nodetool -- node run replicate.<module>.<ClassName> --props '{...}'. Do not hand-edit packages/replicate-nodes/src/replicate-manifest.json — it is generated output. Where things live Path Purpose packages/replicate-codegen/src/configs/ One TypeScript file per module (e.g. image-generate.ts). Edit these to add models. packages/replicate-codegen/src/configs/index.ts Imports every module config and exports allConfigs. Register new config files here. packages/replicate-codegen/src/generate.ts CLI entry point for the generator. Fetches Replicate schemas and writes the manifest. packages/replicate-codegen/src/schema-fetcher.ts Fetches/caches OpenAPI schemas from api.replicate.com. Cache lives in .schema-cache/replicate/. packages/replicate-codegen/src/schema-parser.ts Parses the OpenAPI schema into a NodeSpec. packages/replicate-codegen/src/node-generator.ts Applies NodeConfig overrides (className, returnType, fieldOverrides) onto a NodeSpec. packages/replicate-codegen/src/types.ts ModuleConfig, NodeConfig, NodeSpec, FieldDef, EnumDef. packages/replicate-nodes/src/replicate-manifest.json Generated. Node definitions consumed at runtime. Never edit by hand. packages/replicate-nodes/src/replicate-factory.ts Builds NodeClass instances from manifest entries at runtime. packages/replicate-nodes/src/replicate-base.ts Shared Replicate API utilities: replicateSubmit, assetToUrl, output converters. packages/replicate-nodes/src/index.ts Exports REPLICATE_NODES (array of NodeClass) and registerReplicateNodes. packages/runtime/src/providers/replicate-provider.ts ReplicateProvider — chat, image, video, audio, TTS, ASR, embeddings. packages/runtime/src/providers/manifest-models.ts Reads the manifest to surface image/video/music model lists to the UI. packages/runtime/src/providers/index.ts Registers ReplicateProvider under PROVIDER_IDS.REPLICATE. How Replicate nodes are generated The codegen pipeline runs in four steps: Config lookup — generate.ts reads allConfigs from configs/index.ts. Each config maps a Replicate model id ("owner/name") to a NodeConfig (class name, return type, field overrides). Schema fetch — SchemaFetcher calls https://api.replicate.com/v1/models/<owner>/<name> to get the model’s OpenAPI input/output schema. Results are cached under .schema-cache/replicate/ (keyed by SHA-256 of the model id). Pass --no-cache to force a fresh fetch. Parsing and override — SchemaParser converts the OpenAPI schema to a NodeSpec. NodeGenerator.applyConfig() then merges the NodeConfig overrides onto it (renames class, sets outputType, fixes propType for image/audio/video fields that Replicate types as string). Manifest write — The generator writes every NodeSpec as a ManifestEntry to packages/replicate-nodes/src/replicate-manifest.json. At runtime, replicate-factory.ts reads this file and produces NodeClass objects via createReplicateNodeClass(). The node type for a manifest entry follows the pattern replicate.<moduleName>.<className> where moduleName uses dots instead of dashes (e.g. replicate.image.generate.Flux_Schnell). Add a new Replicate model Step 1 — Choose the right config file Pick the config whose module matches the model’s task: File Module key Covers image-generate.ts image.generate Text-to-image, image-to-image image-enhance.ts image.enhance Enhancement, retouching image-upscale.ts image.upscale Upscaling image-background.ts image.background Background removal video-generate.ts video.generate Text/image to video video-enhance.ts video.enhance Video enhancement audio-generate.ts audio.generate Music, sound generation audio-speech.ts audio.speech TTS audio-transcribe.ts audio.transcribe ASR text-generate.ts text.generate Text LLMs via Replicate embedding.ts embedding Embedding models If none fit, create a new file (see Step 1b below). Step 2 — Add the model entry Add one key to the configs object in the chosen file. The key is the Replicate model id exactly as it appears in the URL (replicate.com/<owner>/<name>). Minimal entry (no image inputs): "acme-org/my-flux-variant": { className: "MyFluxVariant", returnType: "image" } Entry with image inputs (Replicate types these as string; NodeTool needs image/video/audio): "acme-org/my-img2img": { className: "MyImg2Img", returnType: "image", fieldOverrides: { image: { propType: "image" }, mask: { propType: "image" } } } Full NodeConfig shape (all fields optional): interface NodeConfig { className?: string; // PascalCase; if omitted the parser derives one docstring?: string; // Override the model description tags?: string[]; useCases?: string[]; returnType?: string; // "image" | "video" | "audio" | "str" fieldOverrides?: Record<string, Partial<FieldDef>>; // propType corrections enumOverrides?: Record<string, string>; // rename enum classes enumValueOverrides?: Record<string, Record<string, string>>; } returnType is required for media-producing models. Without it the factory cannot map the output to the right AssetRef type. Asset input propType rules — Replicate schemas type all URLs as string. Override these manually: Field carries Set propType to Single image "image" Single video "video" Single audio "audio" List of images "list[image]" Step 1b (optional) — Create a new config file If no existing module fits, create packages/replicate-codegen/src/configs/<domain>-<task>.ts: // packages/replicate-codegen/src/configs/image-3d.ts import type { ModuleConfig } from "../types.js"; export const image3dConfig: ModuleConfig = { configs: { "acme-org/mesh-gen": { className: "MeshGen", returnType: "image" } } }; Then register it in packages/replicate-codegen/src/configs/index.ts: import { image3dConfig } from "./image-3d.js"; // ... export const allConfigs: Record<string, ModuleConfig> = { // ...existing entries... "image.3d": image3dConfig, }; The module key ("image.3d") becomes the moduleName in the manifest (stored as "image-3d") and the middle segment of every node’s type string (replicate.image.3d.<ClassName>). Verify Run these in order after adding the config entry: # 1. Generate the manifest (fetches from Replicate API; uses cache by default) export REPLICATE_API_TOKEN="r8_..." npm run generate:replicate # 2. Regenerate without cache if the model schema was recently updated REPLICATE_API_TOKEN="r8_..." npm run generate --workspace=packages/replicate-codegen -- --all --strict --no-cache # 3. Build — replicate-nodes loads from dist/, so rebuild is required npm run build:packages # 4. Type-check the codegen package npm run lint --workspace=packages/replicate-codegen # 5. Run codegen tests npm run test --workspace=packages/replicate-codegen # 6. Smoke-test the node (confirm it appears in the registry and properties look right) npm run dev:nodetool -- node run replicate.image.generate.MyFluxVariant \ --props '{"prompt": "a red fox"}' --no-secrets --json # 7. Full check npm run check If the model id is wrong or the model is private/unavailable, the generator exits with an error listing the failures. Fix the id or remove the entry. How past PRs did it The Replicate codegen and manifest have been in place since the initial commit. The three commits on record for this branch (ca742050, f900e7a6, d1491abf) are version bumps. To see how a specific model was added, look for commits that touch packages/replicate-codegen/src/configs/ and packages/replicate-nodes/src/replicate-manifest.json together: git log --oneline -- packages/replicate-codegen/src/configs packages/replicate-nodes/src/replicate-manifest.json The pattern is always: one config file edit + the regenerated manifest in the same commit. Contributing Open a PR against main at github.com/nodetool-ai/nodetool. Before pushing: npm run lint # must pass npm run typecheck # must pass Questions? Join the Discord.--- # Reve: Add Models & Nodes Source: https://docs.nodetool.ai/developer/providers/reve.html Navigation: docs/developer/index.md → Reve provider guide Audience: coding agents and contributors adding new Reve image nodes or model version strings to the @nodetool-ai/reve-nodes package. TL;DR Add a new class in packages/reve-nodes/src/nodes/ extending BaseNode. Export it from packages/reve-nodes/src/index.ts and push the class into REVE_NODES. Call reveGenerate from reve-base.ts with one of the three API endpoints: "create", "edit", or "remix". npm run build:packages — compiles from src/ to dist/ (the package loads from dist/). npm run typecheck && npm run lint — must pass before committing. Where things live What Path Shared API utilities packages/reve-nodes/src/reve-base.ts Create Image node packages/reve-nodes/src/nodes/create-image.ts Edit Image node packages/reve-nodes/src/nodes/edit-image.ts Remix Image node packages/reve-nodes/src/nodes/remix-image.ts Package entry point packages/reve-nodes/src/index.ts Runtime provider (generic image picker) packages/runtime/src/providers/reve-provider.ts Provider registration packages/runtime/src/providers/index.ts line 221 PROVIDER_IDS.REVE packages/protocol/src/api-types.ts line 887 Unit tests packages/reve-nodes/tests/nodes.test.ts Registration test packages/reve-nodes/tests/registration.test.ts How Reve models and nodes are defined Reve exposes three POST endpoints, each mapping to one TypeScript class: Endpoint Class Node type POST /v1/image/create CreateImageNode reve.CreateImage POST /v1/image/edit EditImageNode reve.EditImage POST /v1/image/remix RemixImageNode reve.RemixImage There is no manifest or code-generation script. Each node is a hand-written BaseNode subclass in its own file under packages/reve-nodes/src/nodes/. Fields are declared with the @prop decorator from @nodetool-ai/node-sdk. Shared utilities in reve-base.ts: REVE_ASPECT_RATIOS — the seven aspect ratios all endpoints accept. REVE_POSTPROCESSING — optional post-generation operations (upscale, remove_background, fit_image, effect). getReveApiKey(secrets) — reads REVE_API_KEY from this._secrets then process.env, throws if absent. reveGenerate(apiKey, endpoint, body) — POSTs to https://api.reve.com/v1/image/{endpoint} with Authorization: Bearer, returns the parsed ReveImageResponse. refToBase64(ref, context) — converts an ImageRef-like value to a base64 string; handles inline data, data URIs, asset://, file://, and remote URLs. reveImageToRef(base64) — wraps the response image field into a NodeTool ImageRef object; attaches dimensions via sharp when available. The runtime provider (reve-provider.ts) separately implements textToImage and imageToImage for the generic image-generation picker in the UI (the two hardcoded model ids reve-create and reve-edit). This is a thinner path — new Reve API endpoints should be added as full nodes in reve-nodes, not as branches in the provider. Add a new model version To add a new dated model version pin (e.g. reve-create@20260101), open the relevant node file and extend the version prop’s values array: // packages/reve-nodes/src/nodes/create-image.ts @prop({ type: "enum", default: "latest", title: "Version", description: "Model version to use.", values: ["latest", "reve-create@20250915", "reve-create@20260101"] // ← add here }) declare version: any; That is the entire change. Rebuild and verify (see Verify below). Add a new node Use this pattern when Reve releases a new endpoint (or a sufficiently different variant that warrants its own node type). Step 1. Create packages/reve-nodes/src/nodes/your-endpoint.ts: import { BaseNode, prop } from "@nodetool-ai/node-sdk"; import type { NodeClass } from "@nodetool-ai/node-sdk"; import { getReveApiKey, postprocessingArray, reveGenerate, reveImageToRef, REVE_ASPECT_RATIOS, REVE_POSTPROCESSING } from "../reve-base.js"; export class YourEndpointNode extends BaseNode { static readonly nodeType = "reve.YourEndpoint"; static readonly body = "content_card"; static readonly title = "Your Endpoint"; static readonly description = "One-line description of what this node does.\n" + "image, reve, keyword1, keyword2\n\n" + "Use cases:\n" + "- First use case\n" + "- Second use case"; static readonly metadataOutputTypes = { output: "image" }; static readonly inlineFields: string[] = ["prompt"]; static readonly requiredSettings = ["REVE_API_KEY"]; static readonly autoSaveAsset = true; @prop({ type: "str", default: "", title: "Prompt", description: "Text prompt (max 2560 chars)." }) declare prompt: any; @prop({ type: "enum", default: "3:2", title: "Aspect Ratio", description: "Proportions of the generated image.", values: [...REVE_ASPECT_RATIOS] }) declare aspect_ratio: any; @prop({ type: "enum", default: "latest", title: "Version", description: "Model version to use.", values: ["latest"] }) declare version: any; @prop({ type: "enum", default: "none", title: "Postprocessing", description: "Optional postprocessing operation applied to the result.", values: [...REVE_POSTPROCESSING] }) declare postprocessing: any; @prop({ type: "int", default: 1, title: "Test Time Scaling", description: "Effort multiplier for quality (1-15). Higher costs more.", min: 1, max: 15 }) declare test_time_scaling: any; async process(): Promise<Record<string, unknown>> { const apiKey = getReveApiKey(this._secrets); const prompt = String(this.prompt ?? ""); if (!prompt) throw new Error("Prompt is required"); const result = await reveGenerate(apiKey, "create", { // ← swap endpoint name prompt, aspect_ratio: String(this.aspect_ratio ?? "3:2"), version: String(this.version ?? "latest"), postprocessing: postprocessingArray(this.postprocessing), test_time_scaling: Number(this.test_time_scaling ?? 1) }); return { output: await reveImageToRef(result.image) }; } } export const YOUR_ENDPOINT_NODES: readonly NodeClass[] = [YourEndpointNode]; Nodes that accept a reference image (like EditImageNode) must take a context argument in process and pass it to refToBase64: async process( context?: Parameters<BaseNode["process"]>[0] ): Promise<Record<string, unknown>> { const referenceImage = await refToBase64(this.image, context); // ... } Step 2. Wire the new class into packages/reve-nodes/src/index.ts: import { YOUR_ENDPOINT_NODES } from "./nodes/your-endpoint.js"; export { YourEndpointNode } from "./nodes/your-endpoint.js"; export const REVE_NODES: readonly NodeClass[] = [ ...CREATE_IMAGE_NODES, ...EDIT_IMAGE_NODES, ...REMIX_IMAGE_NODES, ...YOUR_ENDPOINT_NODES // ← add here ]; No other files need to change. The node automatically appears in the registry because registerReveNodes iterates REVE_NODES. Step 3. Add tests in packages/reve-nodes/tests/nodes.test.ts. Follow the existing pattern: stub fetch, construct the node with props, call process(), assert the fetch URL and body fields. Verify Run these commands in order after any change: # 1. Compile src/ → dist/ (required — the package loads from dist/) npm run build:packages # 2. Type-check the whole repo npm run typecheck # 3. Lint npm run lint # 4. Run the reve-nodes unit tests npm run test --workspace=packages/reve-nodes # 5. Confirm the new node type resolves in the CLI registry # Expected: a REVE_API_KEY error, not "unknown node type" npm run dev:nodetool -- node run reve.YourEndpoint \ --props '{"prompt":"a red fox"}' \ --no-secrets # 6. Static validation (no API key required) npm run dev:nodetool -- validate workflow.json A REVE_API_KEY is not configured error from step 5 means the node registered correctly — it loaded from dist/ and resolved the type. An unknown node type error means the build or export wiring is broken. How past commits did it The Reve node package and runtime provider were introduced together in commit d1491abf (“add claude agent package”). That commit added: packages/runtime/src/providers/reve-provider.ts packages/reve-nodes/src/reve-base.ts packages/reve-nodes/src/nodes/create-image.ts packages/reve-nodes/src/nodes/edit-image.ts packages/reve-nodes/src/nodes/remix-image.ts packages/reve-nodes/src/index.ts packages/reve-nodes/tests/nodes.test.ts packages/reve-nodes/tests/registration.test.ts The three endpoints (create, edit, remix) map to three node classes with no factory or manifest layer. For reference, XAI image/video support was added in PR #3951 (commit 69dd6f88) by extending getAvailableImageModels and getAvailableVideoModels in the existing provider — that pattern applies when the provider’s generic image picker (not a standalone node pack) is the target. Contributing Source: https://github.com/nodetool-ai/nodetool Discord: https://discord.gg/WmQTWZRcYE Before opening a PR, run npm run check (typecheck + lint + tests). Add a test for every new node; the existing nodes.test.ts stubs fetch globally, so new test cases fit cleanly in the same file.--- # Together AI: Add Models & Nodes Source: https://docs.nodetool.ai/developer/providers/together.html Audience: Contributors and coding agents adding Together AI models or nodes to this repo. TL;DR Edit IMAGE_MODELS, VIDEO_MODELS, TTS_MODELS, or ASR_MODELS in scripts/generate-manifest.mjs. Run npm run gen:manifest — rewrites src/together-manifest.json. Run npm run build in the package, then npm run typecheck && npm run test. For TTS / ASR / embedding provider-side models only: also update the static arrays in packages/runtime/src/providers/together-provider.ts. Verify with nodetool node run together.<module>.<ClassName>. Where things live What Path Manifest generator (source of truth) packages/together-nodes/scripts/generate-manifest.mjs Generated manifest (do not hand-edit) packages/together-nodes/src/together-manifest.json Node factory (manifest → BaseNode subclasses) packages/together-nodes/src/together-factory.ts HTTP helpers / API executors packages/together-nodes/src/together-base.ts Node registration entry point packages/together-nodes/src/index.ts Provider class (chat, image, video, TTS, ASR, embeddings) packages/runtime/src/providers/together-provider.ts Manifest→model-list bridge packages/runtime/src/providers/manifest-models.ts Provider registration packages/runtime/src/providers/index.ts line 227 Pack registration (websocket server) packages/websocket/src/node-registry-setup.ts line 104 Pack catalog entry packages/protocol/src/builtin-packs.ts PROVIDER_IDS.TOGETHER constant packages/protocol/src/api-types.ts line 877 How Together models and nodes are defined Image and video nodes — generated from the manifest src/together-manifest.json is generated code. The source of truth is scripts/generate-manifest.mjs, which defines four model catalogs (IMAGE_MODELS, VIDEO_MODELS, TTS_MODELS, ASR_MODELS) and expands each (model × task) pair into a manifest entry. Each entry looks like this: { "className": "FLUX1SchnellTextToImage", "moduleName": "image", "modality": "text_to_image", "modelId": "black-forest-labs/FLUX.1-schnell", "outputType": "image", "title": "FLUX.1 Schnell — Text to Image", "description": "FLUX.1 Schnell — text to image via Together AI serverless.", "fields": [ ... ], "supportedTasks": ["text_to_image"] } The factory (together-factory.ts) turns every entry into a BaseNode subclass at runtime. The node type is together.<moduleName>.<className>. Fields declared in "fields" become node properties; the "modality" value routes execution to the matching HTTP executor in together-base.ts. The same manifest is read by manifest-models.ts (via loadImageModels / loadVideoModels) to populate TogetherProvider.getAvailableImageModels() and getAvailableVideoModels(). This means the node catalog and the provider’s model catalog stay in sync automatically — there is no second list to update for image and video models. Chat / language models — fetched dynamically TogetherProvider.getAvailableLanguageModels() calls https://api.together.xyz/v1/models at runtime and filters entries where type === "chat" || type === "language" || type === "". No static list exists for chat models; Together’s live catalog drives what appears in the model picker. No code change is required when Together adds a new chat model. TTS, ASR, and embedding models — static arrays in the provider These are hardcoded in packages/runtime/src/providers/together-provider.ts: TOGETHER_TTS_MODELS — 3 entries (Orpheus 3B, Kokoro 82M, Cartesia Sonic), each with a voices list. TOGETHER_ASR_MODELS — 3 entries (Whisper Large v3, Voxtral Mini 3B, Parakeet TDT 0.6B v3). TOGETHER_EMBEDDING_MODELS — 1 entry (Multilingual E5 Large, 1024 dims). TTS models are also in the manifest (so they get workflow nodes). ASR and embedding models currently appear only in the provider arrays (no dedicated workflow nodes yet). Add a new model or node The steps differ slightly by modality. Image or video model 1. Open the generator and add an entry to the right catalog. // packages/together-nodes/scripts/generate-manifest.mjs const IMAGE_MODELS = [ // existing entries ... { id: "black-forest-labs/FLUX.3-ultra", // Together API model id name: "FLUX.3 Ultra", // display name → drives className tasks: ["text_to_image", "image_to_image"] // which node(s) to generate } ]; For video, append to VIDEO_MODELS with tasks: ["text_to_video", "image_to_video"] (or just one if the model is task-specific). 2. Regenerate and build. cd packages/together-nodes npm run gen:manifest # rewrites src/together-manifest.json npm run build # tsc + copies manifest into dist/ npm test That’s all. The factory auto-generates the node class, and manifest-models.ts picks up the new model for the provider’s model lists. TTS model TTS models need entries in both the manifest generator and the provider array. 1. Add to the generator’s TTS_MODELS. // packages/together-nodes/scripts/generate-manifest.mjs const TTS_MODELS = [ // existing entries ... { id: "vendor/new-tts-model", name: "New TTS Model", voices: ["voice_a", "voice_b"] } ]; 2. Regenerate and build (same as above). 3. Add the provider-side entry. // packages/runtime/src/providers/together-provider.ts const TOGETHER_TTS_MODELS: TTSModel[] = [ // existing entries ... { id: "vendor/new-tts-model", name: "New TTS Model", provider: "together", voices: ["voice_a", "voice_b"] } ]; ASR model ASR nodes are not yet generated from the manifest, so update only the provider array: // packages/runtime/src/providers/together-provider.ts const TOGETHER_ASR_MODELS: ASRModel[] = [ // existing entries ... { id: "vendor/new-asr-model", name: "New ASR Model", provider: "together" } ]; Embedding model Same pattern as ASR: // packages/runtime/src/providers/together-provider.ts const TOGETHER_EMBEDDING_MODELS: EmbeddingModel[] = [ // existing entries ... { id: "vendor/new-embedding-model", name: "New Embedding Model", provider: "together", dimensions: 1024 } ]; Verify 1. Build and type-check. npm run build --workspace=packages/together-nodes npm run typecheck 2. Run package tests. npm run test --workspace=packages/together-nodes 3. Exercise the node in isolation (requires TOGETHER_API_KEY in secrets or env). For a text-to-image node the type is together.image.<ClassName>: npm run dev:nodetool -- node run together.image.FLUX1SchnellTextToImage \ --props '{"prompt":"a red panda on a skateboard","width":512,"height":512}' For a video node: npm run dev:nodetool -- node run together.video.Veo30TextToVideo \ --props '{"prompt":"timelapse of clouds","aspect_ratio":"16:9","resolution":"720p","duration":6}' For a TTS node: npm run dev:nodetool -- node run together.audio.Orpheus3BTextToSpeech \ --props '{"text":"Hello from Together AI","voice":"tara","format":"mp3"}' 4. Static workflow check (no API call). npm run dev:nodetool -- validate workflow.json 5. Combined lint + typecheck + test. npm run check How past PRs did it The entire packages/together-nodes package — generator script, manifest (56 entries), factory, base helpers, provider arrays, and tests — was introduced in commit d1491abf (“add claude agent package”). That single commit is the reference for how the package was structured from the start. It adds: packages/together-nodes/scripts/generate-manifest.mjs (214 lines, the generator) packages/together-nodes/src/together-manifest.json (4274 lines, 56 entries) packages/together-nodes/src/together-factory.ts (405 lines) packages/together-nodes/src/together-base.ts (553 lines) packages/runtime/src/providers/together-provider.ts (780 lines) Full test coverage: tests/together-base.test.ts and tests/providers/together-provider.test.ts To see the full diff: git show d1491abf. Contributing Repository: https://github.com/nodetool-ai/nodetool Discord: https://discord.gg/WmQTWZRcYE Before opening a PR, run npm run check (typecheck + lint + tests). PRs that break any of the three will not merge. Follow the writing style in docs/WRITING_STYLE.md for any Markdown you touch.--- # Topaz: Add Models & Nodes Source: https://docs.nodetool.ai/developer/providers/topaz.html Navigation: docs/developer/index.md → Topaz provider guide Audience: coding agents and contributors adding new Topaz image or video endpoints, or new model variants within existing endpoints. TL;DR Edit packages/topaz-nodes/src/topaz-manifest.json — one JSON object per endpoint (node). npm run build:packages — copies the manifest to dist/ and compiles the factory. npm run typecheck && npm run lint — must pass before committing. No new TypeScript files needed for standard endpoints; the factory generates the node class at runtime. Where things live What Path Manifest (all nodes) packages/topaz-nodes/src/topaz-manifest.json API + HTTP utilities packages/topaz-nodes/src/topaz-base.ts Runtime node factory packages/topaz-nodes/src/topaz-factory.ts Package entry point packages/topaz-nodes/src/index.ts Runtime provider (upscale only) packages/runtime/src/providers/topaz-provider.ts Manifest loader (for ImageModel list) packages/runtime/src/providers/manifest-models.ts Provider registration packages/runtime/src/providers/index.ts Unit tests packages/topaz-nodes/tests/topaz-base.test.ts How Topaz models and nodes are defined The manifest is the single source of truth. Every Topaz node is described by one JSON object in packages/topaz-nodes/src/topaz-manifest.json. There is no code-generation script — the manifest is hand-maintained. At package load time src/index.ts reads the JSON and calls loadTopazNodesFromManifest, which feeds each entry to createTopazNodeClass in topaz-factory.ts. The factory builds a full BaseNode subclass at runtime — no decorators, no separate .ts file per node. The build script (package.json build) compiles the TypeScript and then copies src/topaz-manifest.json to dist/topaz-manifest.json so the package.json exports entry "./topaz-manifest.json" resolves correctly. The runtime provider (topaz-provider.ts) loads the same file via require() to build its list of ImageModel objects, but only exposes the enhance and enhance-gen endpoints as upscale-task models. All other endpoints (sharpen, denoise, lighting, matting, restore) live solely as workflow nodes. Image nodes call a single multipart POST → poll status → download flow implemented in topazExecuteImageTask (topaz-base.ts). Video nodes drive a five-step pipeline: create job → accept (get presigned upload URLs) → PUT bytes → complete-upload → poll status → download. This requires ffprobe installed locally to probe the source video’s resolution, frame rate, and duration before the job is created. Add a new model or node Case A — new model variant in an existing endpoint Open packages/topaz-nodes/src/topaz-manifest.json and add the variant name to the values array of the relevant entry’s model field: { "name": "model", "type": "enum", "default": "Standard V2", "title": "Model", "required": false, "values": [ "Standard V2", "High Fidelity V2", "Upscale High Fidelity V3", "Low Resolution V2", "CGI", "Transparency Upscale", "Detail", "Faces", "Text Refine", "Standard V3" ] } That is the entire change. The factory picks up the new variant automatically. If the new variant is in the enhance or enhance-gen endpoint, the runtime provider will also surface it as a new ImageModel entry named topaz/image/enhance/Standard V3 (the modelId/variant pattern used in buildVariantMap). Case B — new image endpoint (new node) Add one object to the top-level array in topaz-manifest.json. Copy the shape of an existing image entry: { "className": "RemoveBackground", "moduleName": "image", "modelId": "topaz/image/remove-bg", "title": "Topaz Remove Background", "description": "Topaz Image API — background removal.\n\n topaz, image, background, removal\n\n Multipart POST /image/v1/remove-bg/async.", "outputType": "image", "submitEndpoint": "https://api.topazlabs.com/image/v1/remove-bg/async", "statusEndpoint": "https://api.topazlabs.com/image/v1/status/{process_id}", "downloadEndpoint": "https://api.topazlabs.com/image/v1/download/{process_id}", "pollInterval": 3000, "maxAttempts": 400, "fields": [ { "name": "image", "type": "image", "title": "Image", "description": "Source image.", "required": true, "uploadField": true }, { "name": "output_format", "type": "enum", "default": "png", "title": "Output Format", "required": false, "values": ["jpeg", "png", "tiff"] } ], "validation": [] } Key rules: className — PascalCase, unique across the manifest. Becomes the node class name and part of the node type: topaz.<moduleName>.<className>. modelId — slash-separated string, unique across the manifest. Used as the primary key in the provider’s variant map. Exactly one field must set "uploadField": true. This is the asset (image or video) the factory reads and sends to Topaz. submitEndpoint must be an absolute URL. statusEndpoint and downloadEndpoint use {process_id} as a placeholder. pollInterval is in milliseconds; maxAttempts × pollInterval sets the total timeout budget. Case C — new video endpoint Copy either EnhanceVideo or InterpolateVideo from the manifest. Video entries add: "outputType": "video" "videoKind": "upscale" or "videoKind": "interpolate" "requiredRuntimes": ["ffmpeg"] — tells the UI the node needs ffmpeg/ffprobe installed "acceptEndpoint" and "completeEndpoint" for the two extra pipeline steps A "video" typed field with "uploadField": true The factory detects outputType === "video" and routes through topazExecuteVideoTask, which calls ffprobe to probe the source before creating the job. Verify After editing the manifest, run these commands in order: # 1. Build the package (compiles TS and copies manifest to dist/) npm run build:packages # 2. Type-check the whole repo npm run typecheck # 3. Lint npm run lint # 4. Run the topaz-nodes unit tests npm run test --workspace=packages/topaz-nodes # 5. Run a node in isolation to confirm it loads from the manifest # (replace 'topaz.image.SharpenImage' with your new className) npm run dev:nodetool -- node run topaz.image.SharpenImage \ --props '{"image":{"type":"image","uri":"https://example.com/photo.jpg"}}' \ --no-secrets # 6. Static validation: confirm the manifest-derived node type is known npm run dev:nodetool -- validate workflow.json Step 5 will hit a missing-API-key error in --no-secrets mode, which is expected — the goal is to confirm the node type resolves and all fields load without a panic. A TOPAZ_API_KEY not configured error means the node registered correctly. How past PRs did it The Topaz package was introduced in commit b0bb9b0b (branch react-doctor safe sweep) which added all 11 manifest entries, the factory, base utilities, and the runtime provider together. The full file set is visible in git show --stat b0bb9b0b | grep topaz: packages/runtime/src/providers/topaz-provider.ts | 345 + packages/runtime/tests/providers/topaz-provider.test.ts | 225 + packages/topaz-nodes/package.json | 46 + packages/topaz-nodes/src/index.ts | 45 + packages/topaz-nodes/src/topaz-base.ts | 652 + packages/topaz-nodes/src/topaz-factory.ts | 270 + packages/topaz-nodes/src/topaz-manifest.json | 1084 + packages/topaz-nodes/tests/topaz-base.test.ts | 411 + The pattern for adding a single new endpoint is smaller: one manifest entry, no other files. Contributing Source: https://github.com/nodetool-ai/nodetool Discord: https://discord.gg/WmQTWZRcYE Before opening a PR, run npm run check (typecheck + lint + tests). The manifest is the right place to document API-level details (endpoint URL, poll interval, field constraints) — keep the description field useful for future readers.--- # xAI (Grok): Add Models Source: https://docs.nodetool.ai/developer/providers/xai.html XAIProvider sits at packages/runtime/src/providers/xai-provider.ts. It extends OpenAIProvider but overrides every model-discovery method and every generation method to call xAI’s REST API directly. Chat models are discovered at runtime by fetching /v1/models; image and video models come from the same listing, filtered by output_modalities. Audience: coding agents and contributors adding new xAI Grok models (chat, image generation, video generation). TL;DR Chat and image/video models are all discovered dynamically from xAI’s /v1/models endpoint. To support a new model: Store XAI_API_KEY via npm run dev:nodetool -- secrets store XAI_API_KEY. Verify the model appears in the /v1/models listing and that classifyModel() assigns it the right modality. If classifyModel() misclassifies the model (no output_modalities field and an ambiguous id), add an id.includes() branch in classifyModel(). Run npm run check. No static lists to edit. No registry changes. No rebuilds needed for the provider itself. Where things live Concern Path Provider class packages/runtime/src/providers/xai-provider.ts Provider registration packages/runtime/src/providers/index.ts line 235 Provider ID constant packages/protocol/src/api-types.ts (PROVIDER_IDS.XAI = "xai") Cloud profile allowlist packages/protocol/src/cloud-profile.ts (CLOUD_PROVIDER_IDS, CLOUD_NODE_NAMESPACES) Provider tests packages/runtime/tests/providers/xai-provider.test.ts Type definitions packages/runtime/src/providers/types.ts How xAI models are defined All three modalities — chat, image, video — are discovered at runtime from a single call to xAI’s /v1/models endpoint (fetchModelRows()). The private classifyModel() function sorts models into "language" | "image" | "video" by inspecting output_modalities first, then falling back to the model’s id string: // packages/runtime/src/providers/xai-provider.ts function classifyModel(row: XAIModelRow): ModelModality { const out = (row.output_modalities ?? []).map((m) => m.toLowerCase()); if (out.includes("video")) return "video"; if (out.includes("image")) return "image"; if (out.includes("text")) return "language"; const id = row.id.toLowerCase(); if (id.includes("video")) return "video"; if (id.includes("image")) return "image"; return "language"; // fallback } getAvailableLanguageModels(), getAvailableImageModels(), and getAvailableVideoModels() each call fetchModelRows() and filter the result through classifyModel(). There are no static model lists in this provider. Image models are tagged supportedTasks: ["text_to_image", "image_to_image"]. Video models are tagged supportedTasks: ["text_to_video", "image_to_video"]. Both sets are returned with provider: "xai". Generation calls go directly to xAI’s REST API: Task Endpoint textToImage POST /v1/images/generations imageToImage POST /v1/images/edits textToVideo / imageToVideo POST /v1/videos/generations (async, polls /v1/videos/{request_id}) Image inputs are converted to base64 data URIs before sending — xAI’s JSON API rejects multipart uploads. Add a new model Chat (language) model No code change required. If xAI adds a new chat model and exposes it via /v1/models with output_modalities: ["text"], NodeTool picks it up automatically on the next getAvailableLanguageModels() call. To verify the model appears: curl -s https://api.x.ai/v1/models \ -H "Authorization: Bearer $XAI_API_KEY" | jq '.data[] | {id, output_modalities}' If the response includes the new model with output_modalities containing "text", you are done. Run npm run check and open a PR only if you needed to change classifyModel(). Image model Same as chat: no static list to edit. xAI must return output_modalities: ["image"] for the model. If it does not (the field is missing and the model id does not contain "image"), add a guard in classifyModel(): // packages/runtime/src/providers/xai-provider.ts — inside classifyModel() const id = row.id.toLowerCase(); if (id.includes("video")) return "video"; if (id.includes("image")) return "image"; if (id.includes("grok-imagine")) return "image"; // ← add this if needed return "language"; getAvailableImageModels() will then include the model with supportedTasks: ["text_to_image", "image_to_image"] and route calls through textToImage / imageToImage already in the provider. No supportedTasks override is needed unless xAI adds a task the model does not support (e.g., image editing); in that case narrow the array: // hypothetical future: image-only model (no editing) supportedTasks: ["text_to_image"] That change goes in getAvailableImageModels() after the classifyModel() filter, keyed on row.id. Video model Same discovery path via classifyModel(). If the id contains "video" or output_modalities contains "video", it surfaces in getAvailableVideoModels() tagged supportedTasks: ["text_to_video", "image_to_video"]. The async polling loop in generateVideo() handles all video models uniformly. xAI video parameters: duration: 1–15 seconds (mapped from params.durationSeconds or derived from numFrames) aspect_ratio: passed through if set resolution: passed through if set No new code is needed unless the model requires a parameter xAI did not previously support. Add it in textToVideo or imageToVideo: // packages/runtime/src/providers/xai-provider.ts override async textToVideo(params: TextToVideoParams): Promise<Uint8Array> { const request: Record<string, unknown> = { model: params.model.id, prompt: params.prompt }; const duration = XAIProvider.resolveVideoDuration(params); if (duration !== undefined) request.duration = duration; if (params.aspectRatio) request.aspect_ratio = params.aspectRatio; if (params.resolution) request.resolution = params.resolution; // Add new xAI-specific param here, e.g.: // if (params.style) request.style = params.style; return this.generateVideo(request, params.timeoutSeconds); } Verify # 1. Confirm the model appears in the live listing curl -s https://api.x.ai/v1/models \ -H "Authorization: Bearer $XAI_API_KEY" | jq '.data[].id' # 2. Type check and lint npm run typecheck npm run lint # 3. Run the xAI provider tests npm run test --workspace=packages/runtime # 4. Smoke-test: list available models via the CLI (requires XAI_API_KEY set) npm run dev:nodetool -- info --json | jq '.providers[] | select(.id=="xai")' # 5. Full check npm run check If you changed classifyModel() or any generation method, also run the provider unit tests directly: npm run test --workspace=packages/runtime -- --reporter=verbose xai-provider How PR #3951 did it Commit 69dd6f88 (“Add image and video generation support to XAI provider”, PR #3951) is the canonical reference for this provider. Before the PR, XAIProvider had no classifyModel() logic. It called super.getAvailableLanguageModels(), which returned every model from /v1/models — including Grok Imagine image and video models — as language models. getAvailableImageModels() and getAvailableVideoModels() were not overridden, so they returned the parent OpenAIProvider’s lists (OpenAI models, not xAI ones). What the PR changed (two commits): Commit 1 — added classifyModel(), fetchModelRows(), and three overriding getAvailable*() methods. Chat models now come from rows whose output_modalities contains "text". Image and video models come from the same listing, classified and returned with provider: "xai" and the correct supportedTasks. Commit 2 — overrode all four generation methods (textToImage, imageToImage, textToVideo, imageToVideo) to call xAI’s REST API directly instead of using the OpenAI SDK’s multipart upload path (which xAI rejects). Added detectImageMime() and bytesToDataUri() to inline image bytes as base64 data URIs. Added the generateVideo() async polling loop for the /v1/videos/generations → /v1/videos/{id} flow. The PR also added 276 lines of unit tests in packages/runtime/tests/providers/xai-provider.test.ts, covering all four generation methods and the model classification logic with mocked fetch responses. The pattern to mirror: if xAI adds a new endpoint category (e.g., audio generation), follow the same two-step shape — first add classification + discovery in the appropriate getAvailable*() override, then add the generation method calling xAI’s REST API directly with this._xaiFetch. Contributing Open PRs at https://github.com/nodetool-ai/nodetool. Before pushing: npm run check # typecheck + lint + test across all workspaces Join the discussion on Discord.--- # Suspendable Nodes Source: https://docs.nodetool.ai/developer/suspendable-nodes.html Suspendable Nodes Suspendable nodes allow workflows to pause execution, save state, and resume later. This enables human-in-the-loop workflows, external callbacks, and checkpoint-based processing. Overview A suspendable node can: Suspend the workflow and save state to the event log Resume when triggered externally (via API or UI) Restore state from the saved suspension data This is useful for: Human approvals - Wait for a manager to approve before continuing External callbacks - Pause for webhook or API responses Checkpoints - Save progress in long-running computations Interactive workflows - Pause for user input The Built-in WaitNode (a delay, not a suspension) Important: WaitNode (nodetool.triggers.Wait, in packages/automation-nodes/src/nodes/triggers.ts) is not a suspendable node. It is a simple in-process delay: process() sleeps for timeout_seconds via setTimeout, then passes its input through. It does not use SuspendableState, does not suspend or resume the workflow, and does not wait for any external approval or callback. Use it for rate-limiting and fixed delays. For true pause/resume, see Creating Custom Suspendable Nodes below. import { WaitNode } from "@nodetool-ai/automation-nodes"; // A node that delays for a fixed number of seconds, then forwards `input`. const waitNode = new WaitNode(); waitNode.timeout_seconds = 5; // Seconds to wait (0 = no wait, pass through immediately) waitNode.input = { request_id: "REQ-123" }; WaitNode Properties Property Type Default Description timeout_seconds number 0 Seconds to wait before continuing. 0 = no wait (pass through immediately). input any "" Input data passed through to the output after the delay WaitNode Output After the delay, the WaitNode outputs: { data: { /* input data passed through */ }, resumed_at: "2026-03-16T12:00:00.000Z", // ISO timestamp after the delay waited_seconds: 5.0 // Actual seconds slept } The output field names (resumed_at, waited_seconds) read like suspension semantics, but they only reflect the setTimeout delay — nothing was suspended. Creating Custom Suspendable Nodes True suspension is provided by the SuspendableState helper class (in packages/kernel/src/suspendable.ts, exported from @nodetool-ai/kernel). SuspendableNode is the interface that describes the capability; SuspendableState is the concrete implementation you compose into a node. Calling suspendWorkflow(reason, state, metadata?) throws a WorkflowSuspendedError, which the runner catches to persist state and pause the run. import { SuspendableState } from "@nodetool-ai/kernel"; import { BaseNode, prop } from "@nodetool-ai/node-sdk"; class ApprovalNode extends BaseNode { static readonly nodeType = "custom.ApprovalNode"; @prop({ type: "str", default: "" }) declare document_id: string; // Compose a SuspendableState helper for this node private _suspend = new SuspendableState(ApprovalNode.nodeType); async process(): Promise<Record<string, unknown>> { // Check if resuming from suspension if (this._suspend.isResuming()) { const savedState = this._suspend.getSavedState(); if (savedState.approved) { return { status: "approved", approved_by: savedState.approved_by, approved_at: savedState.approved_at, }; } else { return { status: "rejected", reason: savedState.rejection_reason, }; } } // First execution — suspend and wait for approval this._suspend.suspendWorkflow( `Waiting for approval of document ${this.document_id}`, { document_id: this.document_id, submitted_at: new Date().toISOString(), }, { approver_email: "admin@example.com", timeout_hours: 24, }, ); // Execution never reaches here on first run. // suspendWorkflow() throws WorkflowSuspendedError. } } API Methods SuspendableNode Methods isSuspendable(): boolean Returns true to indicate this node supports suspension. isResuming(): boolean Check if the node is resuming from a previous suspension. if (this._suspend.isResuming()) { // Resumption path — get saved state const saved = this._suspend.getSavedState(); } else { // First execution path — suspend this._suspend.suspendWorkflow(reason, state); } getSavedState(): Record<string, unknown> Get the state that was saved when workflow suspended. const savedState = this._suspend.getSavedState(); const approvalStatus = savedState.approved ?? false; Throws Error if called when not resuming. suspendWorkflow(reason: string, state: Record<string, unknown>, metadata?: Record<string, unknown>): never Suspend workflow execution and save state. this._suspend.suspendWorkflow( "Waiting for user input", { partial_result: computedValue }, { timeout: 3600 }, ); This method: Logs NodeSuspended event with state Logs RunSuspended event Throws WorkflowSuspendedError to exit execution Never returns (workflow is suspended) Suspension Flow 1. Initial Execution WorkflowRunner.run() ├─> NodeActor executes node ├─> node.process() calls suspendWorkflow() ├─> WorkflowSuspendedError thrown ├─> Runner catches exception ├─> Logs NodeSuspended event (with state) ├─> Logs RunSuspended event ├─> Sends JobUpdate(status="suspended") to frontend └─> Exits cleanly 2. External Resume (via UI or API) User clicks Resume button OR API call to resume endpoint ├─> WorkflowRecoveryService.resume_workflow() ├─> Loads saved state from event log ├─> Logs NodeResumed event ├─> Sets node.setResumingState() └─> WorkflowRunner.run() continues 3. Node Resumption NodeActor executes node (resuming=True) ├─> node.isResuming() returns true ├─> node.getSavedState() returns saved state ├─> node.process() continues from saved state └─> Workflow completes normally Frontend Integration When a workflow suspends: Backend sends JobUpdate(status="suspended", message="...") Frontend state changes to "suspended" UI shows: Notification with suspension reason Purple Resume button in toolbar Stop button remains enabled When user clicks Resume: Frontend sends resume_job command via WebSocket Backend resumes workflow from saved state Frontend state changes back to "running" Best Practices Always check isResuming() - Handle both first execution and resumption paths Save minimal state - Only save what’s needed to resume Use descriptive reasons - Make suspension reason clear for users Add metadata - Include context like timeout, approver email, etc. Handle timeouts - Consider what happens if workflow isn’t resumed Test both paths - Test both suspension and resumption code Example: Webhook Callback import { SuspendableState } from "@nodetool-ai/kernel"; import { BaseNode, prop } from "@nodetool-ai/node-sdk"; class WebhookWaitNode extends BaseNode { static readonly nodeType = "custom.WebhookWaitNode"; @prop({ type: "str", default: "" }) declare callback_url: string; private _suspend = new SuspendableState(WebhookWaitNode.nodeType); async process(): Promise<Record<string, unknown>> { if (this._suspend.isResuming()) { const state = this._suspend.getSavedState(); return { webhook_id: state.webhook_id, callback_data: state.callback_data ?? {}, }; } // Register webhook and get ID const webhookId = await registerWebhook(this.callback_url); // Suspend until webhook is called this._suspend.suspendWorkflow( "Waiting for webhook callback", { webhook_id: webhookId, callback_url: this.callback_url }, { external_service: true }, ); } } See Also Workflow Editor - User guide for pause/resume controls Trigger Nodes - Nodes that fire on external events Workflow API - API endpoints for workflow control--- # TypeScript DSL Guide Source: https://docs.nodetool.ai/developer/ts-dsl-guide.html The TypeScript DSL (@nodetool-ai/dsl) provides type-safe factory functions for building NodeTool workflows in code. Define workflows programmatically with full IDE autocompletion, then serialize them to the same JSON format used by the visual editor. Table of Contents Installation Core Concepts Basic Workflow Connecting Nodes Multi-Output Nodes Building the Workflow Graph Namespaces Code Generation Best Practices Installation Install from npm: npm install @nodetool-ai/dsl Or inside the NodeTool monorepo, all workspace packages are available after npm install at the repo root. Import namespaces directly: import { math, constant, text, image } from "@nodetool-ai/dsl"; import { workflow } from "@nodetool-ai/dsl"; Or import from a specific namespace for tree-shaking: import { add, multiply } from "@nodetool-ai/dsl/generated/nodetool.math"; Core Concepts OutputHandle When you create a node, you get back a DslNode object. Its output() method returns an OutputHandle — a symbolic reference to one of the node’s output slots. You pass handles as inputs to other nodes to create connections. const a = constant.integer({ value: 5 }); a.output() // → OutputHandle<number> — reference, not the value itself Connectable Every input field accepts either a literal value or an OutputHandle: const sum = math.add({ a: 1, b: 2 }); // literal values const sum2 = math.add({ a: a.output(), b: 2 }); // connection + literal DslNode The frozen object returned by every factory function: const node = constant.integer({ value: 42 }); node.nodeId // unique UUID node.nodeType // "nodetool.constant.Integer" node.inputs // { value: 42 } node.output() // OutputHandle for the node's default output slot Basic Workflow A workflow follows three steps: create nodes, connect them, build the graph. import { constant, math } from "@nodetool-ai/dsl"; import { workflow } from "@nodetool-ai/dsl"; // 1. Create nodes const x = constant.float({ value: 3.14 }); const y = constant.float({ value: 2.0 }); // 2. Connect nodes by passing output handles const sum = math.add({ a: x.output(), b: y.output() }); // 3. Build the workflow graph const wf = workflow(sum); console.log(wf.nodes); // 3 nodes console.log(wf.edges); // 2 edges (x→sum, y→sum) The workflow() function traces all connections from the terminal nodes back to their sources, producing a serializable Workflow object with nodes and edges. Connecting Nodes Linear Chain const a = constant.integer({ value: 5 }); const b = math.add({ a: a.output(), b: 1 }); const c = math.multiply({ a: b.output(), b: 2 }); const wf = workflow(c); // 3 nodes, 2 edges: a→b→c Diamond (Shared Dependencies) A node’s output can be connected to multiple downstream nodes. The graph builder deduplicates automatically. const shared = constant.float({ value: 10 }); const left = math.add({ a: shared.output(), b: 1 }); const right = math.multiply({ a: shared.output(), b: 2 }); const final = math.add({ a: left.output(), b: right.output() }); const wf = workflow(final); // 4 nodes, 4 edges — `shared` appears only once Multiple Terminal Nodes Pass multiple nodes to workflow() to trace all branches: const branch1 = math.add({ a: x.output(), b: 1 }); const branch2 = math.multiply({ a: x.output(), b: 2 }); const wf = workflow(branch1, branch2); Multi-Output Nodes Some nodes produce multiple outputs (e.g., If has if_true and if_false). Use output("slotName") to select the slot you want: import { control } from "@nodetool-ai/dsl"; const branch = control.if_({ condition: true, value: "hello" }); // Access named outputs branch.output("if_true") // → OutputHandle branch.output("if_false") // → OutputHandle // Calling output() without a slot throws when there is no default output Each output slot is individually typed, so TypeScript catches type mismatches at compile time. Building the Workflow Graph workflow() function workflow(...terminals: DslNode<any>[]): Workflow; Traces from terminal nodes via BFS, discovers all connected nodes and edges, performs topological sort, and returns a frozen Workflow object. The result can be serialized to JSON: const wf = workflow(outputNode); const json = JSON.stringify(wf, null, 2); This JSON is compatible with the NodeTool workflow format used by the visual editor and the workflow runner. run() / runGraph() async function run(wf: Workflow, opts?: RunOptions): Promise<WorkflowResult>; async function runGraph(...terminals: DslNode<any>[]): Promise<WorkflowResult>; run() executes the graph locally via WorkflowRunner. By default it resolves executors from NodeRegistry.global, or you can pass an explicit registry via RunOptions.registry. Namespaces All 800+ nodes are organized by namespace. Import the namespace object and call factory functions: Import Description Example constant Fixed-value nodes constant.integer({ value: 5 }) math Math operations math.add({ a: 1, b: 2 }) text Text processing text.concat({ a: "hi", b: " there" }) image Image I/O image.loadImageFile({ path: "..." }) audio Audio processing audio.sliceAudio({ ... }) video Video processing video.trimVideo({ ... }) control Flow control control.if_({ condition: true, value: x }) agents AI agents agents.agent({ prompt: "..." }) kieImage KIE image services kieImage.removeBackground({ ... }) geminiText Google Gemini geminiText.gemini({ ... }) openaiText OpenAI text openaiText.chatGPT({ ... }) skills* Skill agents skillsBrowser.browserAgent({ ... }) See the full list in packages/dsl/src/generated/index.ts. Code Generation The factory functions are auto-generated from node metadata. To regenerate after adding or modifying nodes: npm run codegen --workspace=packages/dsl This reads all nodes registered in @nodetool-ai/base-nodes, introspects their metadata (inputs, outputs, types, defaults), and emits typed factory functions into packages/dsl/src/generated/. Generated files are committed to git. The codegen script is at packages/dsl/scripts/codegen.ts. Type Mapping Node Type TypeScript Type str string int, float number bool boolean image ImageRef audio AudioRef video VideoRef list[T] T[] dict[K,V] Record<K, V> enum string literal union any unknown Best Practices Use namespace imports — import { math } from "@nodetool-ai/dsl" gives you autocompletion for all math nodes. Let TypeScript catch errors — the DSL is fully typed. If you pass a string where a number is expected, the compiler tells you. Don’t reuse handles across builds — after calling workflow(), the internal registry is cleared. Handles from previous builds are stale and will throw if used in a new workflow() call. Build workflows linearly — create source nodes first, then processing nodes. The immutable API prevents cycles by construction. Serialize for interop — JSON.stringify(workflow(node)) produces a workflow that can be loaded in the visual editor or executed via the API.--- # Docker Resource Management Source: https://docs.nodetool.ai/docker-resource-management.html Use these guidelines to keep multi-tenant or long-running NodeTool deployments stable. The examples use Docker’s standard --memory / --cpus flags and volume mounts. Baseline Settings Set memory and CPU caps for every container to avoid noisy-neighbor issues. Start with --memory 8g and --cpus 4 for GPU hosts, then tune from docker stats. Keep Hugging Face caches mounted read-only; mount workspaces read/write only where necessary. Keep /tmp and workspace volumes on fast disks; avoid sharing /tmp across unrelated services. Local Docker Runs For standalone containers or quick tests: docker run --gpus all \ --memory 8g --cpus 4 \ -v /data/nodetool/workspace:/workspace \ -v /data/hf-cache:/hf-cache:ro \ -e HF_HOME=/hf-cache \ -p 7777:7777 \ nodetool:latest Use --memory-reservation to set soft limits when co-locating multiple servers. Keep per-run tmp data on the workspace volume (e.g., TMPDIR=/workspace/tmp) so it persists through restarts. Mount shared caches read-only (:ro) to prevent accidental mutation. Monitoring and Tuning Watch docker stats to catch containers hitting limits or restarting. If runs are OOM-killed, lower batch sizes in your workflows or increase --memory incrementally. For storage-heavy jobs, pair limits with the guidance in Storage to ensure caches and outputs have enough space.--- # Editor Panels Source: https://docs.nodetool.ai/editor-panels.html The NodeTool Workflow Editor is surrounded by four dockable panels that host the workflow explorer, inspector, runtime diagnostics, and quick actions. This page covers each panel in depth. Left Panel Opens from the icons down the left edge. It’s a tabbed drawer — click an icon to expand, click the same icon to collapse. The top-level views are: Nodes, Workflows, Sketches, Timelines, Settings, History, Favorites, and Assets. Nodes Tab The node browser. Search and browse all available nodes, organized into sub-tabs (All, I/O, Image, Image AI, Video, Video AI, Audio, Audio AI, 3D, Agents, Control). Drag a node onto the canvas to add it. Workflows Tab Your saved workflows. Search, filter, and double-click to open in a new tab. Sketches Tab Quick image sketches you can drop into the workflow, edited with the built-in layered sketch editor. See Sketch Editor. Timelines Tab Timeline-based media arrangements used by the workflow. Settings Tab Workflow-level settings. History Tab Recent edits and activity for the current workflow. Favorites Tab Your starred nodes for quick access. Assets Tab Folder tree plus file grid. Drag a file onto the canvas to instantly create the matching input node. Right Panel (Inspector) Press i or click the icon in the top right to toggle. The right panel hosts only the Inspector — its contents switch based on what’s selected on the canvas. (Logs, Queue, Trace, Version History, and Workspace are not here — they live in the Bottom Panel.) Inspector — Node Properties When a node is selected, the Inspector renders every property with the right input type (number, slider, model picker, asset selector, dropdown, color picker, and so on). Inspector — Workflow Properties When no node is selected, the Inspector shows workflow-level metadata: title, description, tags, thumbnail. Bottom Panel The bottom panel docks runtime diagnostics and secondary workflow tools. Drag its top edge to resize. Its views are grouped: Run — Logs, Queue, Sandboxes, Workers Workflow — Versions, Workspace Debug — Trace Logs Raw logs from the current run. Filter by level (debug, info, warn, error) and search. Queue Background jobs queued by your workflows — long-running fine-tunes, downloads, and batch runs. Sandboxes & Workers The code-runner sandboxes and worker processes backing the current run. Versions Every save is versioned. Review past versions and roll back. Workspace File hierarchy of the backing workspace (on local installs) or the assigned workspace (on server installs). Trace The full execution trace of the most recent run — per-node timing and the call tree. Floating Toolbar An overlay on the canvas with the most-used runtime controls. Button When shown Action ➕ Add node Graph view Open the node menu 💬 Conversation When a conversation exists Toggle the in-canvas conversation overlay ⏹ Stop While running/paused/suspended Cancel the run ▶ Run Always Run the workflow (shows elapsed time while running) ⇄ Auto Layout Graph view Auto-arrange the graph 💾 Save Always Save the workflow ⋮ More Always Overflow menu (see below) The ⋮ overflow menu contains: Chain View / Graph View (toggle), Instant Update (on/off), Resume (when paused or suspended), Stop (while running), Mini Map (show/hide), Download JSON, and Panels… (on mobile). There is no separate Pause or Fit button in the toolbar. Right Side Buttons A stack of toggles along the right canvas edge: Inspector — open / close the right panel. Run as App — jump to the Mini-App view for this workflow. Notifications — pending warnings and agent messages. System Stats — inline CPU/RAM preview. App Menu (logo dropdown) The logo at the top of the left rail opens the app menu: Dashboard, Examples, Costs, Model Manager, Collections, Workspaces (when enabled), Settings, Help, and Downloads. See User Interface → App Menu for details. Customizing the Layout Every panel is a dockview tab — drag tabs between panels, out of panels to float them, or onto other tabs to stack them. The editor remembers your layout per workspace. To reset: open the command menu (Ctrl+K / ⌘+K), type “reset layout”, and hit Enter. Next Steps Workflow Editor — building on the canvas Chat — how the in-editor chat works Configuration — settings that affect the editor--- # Desktop App Views Source: https://docs.nodetool.ai/electron-views.html The NodeTool desktop app is built on Electron. It reuses the web app for all workflow editing but adds a handful of native surfaces: a boot splash, package manager, log viewer, tray menu, pinned windows, and a native menu bar. This page catalogs every desktop-only view with screenshots. Everything inside the main window is covered by User Interface and Workflow Editor. Boot Message (Splash) A minimal splash appears while the embedded Python runtime, backend server, and web app all come online. The splash streams progress messages — “Starting Python…”, “Warming models…”, “Connecting to server…”. It closes the moment the backend reports ready. There is no separate install wizard — the embedded Python environment and packages are checked and set up in the background at startup, and node packs and runtimes are managed from the Package Manager window. Package Manager Opens from the app menu or ⌘K → Open Package Manager. A native window sized 1200×900 that manages: Installed node packs and their upgrade availability. Optional AI runtimes (torch CUDA, torch MPS, llama.cpp). Conda environment health (Python version, cached wheels). Integrity checks — re-download a broken runtime in one click. See Node Packs for where packs come from and how to publish your own. Log Viewer A persistent window that tails the backend log. Helpful for debugging without dropping to a terminal. Each row is color-coded by severity — info, warning, or error. The viewer is a straightforward table tail with no level/component filters, search, or jump-to-source. Update Notification When a new desktop release is available, a discreet toast appears with Install now and Later actions. The app restarts into the new build; there’s no mandatory update interrupting a run. Pinned Windows Electron adds a few borderless, always-on-top window types: Workflow Execution Window A frameless window that runs a single workflow with the inputs collapsed and the outputs centered. Great for a pinned dashboard on a second monitor. Mini-App Window A self-contained Mini-App launched from the tray. Maximum size 1200×900. Closing it doesn’t quit NodeTool. Chat Window Standalone chat opened from the tray. Same content as /standalone-chat but in its own window. System Tray The tray icon stays running while NodeTool is active and exposes quick actions: Action What it does Show NodeTool Bring up the main window Chat Launch the standalone chat window Mini Apps Submenu of mini-apps published from your workflows Package Manager Open the Package Manager window Log Viewer Tail the backend log Settings Native settings window Open Log File Open the raw log file in the OS file handler Quit NodeTool Stop the backend and exit Above these actions the tray shows non-clickable status lines (server and managed-service state) plus a Managed Model Services submenu — for example a Llama.cpp entry to Install / Start / Stop the local server and toggle “Start on App Startup” — along with Start Service / Stop Service to control the backend and an On Close Behavior submenu. Native Menu Bar On macOS and Windows a native menu bar is attached to the main window. On macOS the app menu (NodeTool) holds the standard About / Services / Hide / Quit items; Quit uses the platform default and has no explicit accelerator in the menu. File Item Shortcut Action Save Ctrl/⌘ + S Save the active workflow New Workflow Ctrl/⌘ + T Open a blank workflow in a new tab Close Tab Ctrl/⌘ + W Close the active editor tab Edit Undo, Redo, Cut, Copy, Paste, Duplicate, Duplicate Vertical, Group, Select All, Align, and Align with Spacing — mapped to the same shortcuts used on the canvas. View Item Shortcut Action Fit View Ctrl/⌘ + 0 Fit the graph to the viewport Reset Zoom   Reset the canvas zoom Zoom In   Zoom the canvas in Zoom Out   Zoom the canvas out Toggle Fullscreen   Enter/exit fullscreen Tools Item Action Chat Open the standalone chat window Package Manager Open the Package Manager window Log Viewer Open the Log Viewer window Performance Monitor Open the performance monitor window Settings Open the native settings window There is no in-menu Reload or DevTools item; DevTools toggles with Ctrl/⌘ + Shift + I and only in unpackaged (dev) builds. Window / Help Window offers the platform-standard Minimize. Help has Learn More (opens https://nodetool.ai) and System Information (a native dialog with app, OS, and runtime versions). Where the Electron Code Lives For contributors: electron/src/main.ts — entry point, window lifecycle. electron/src/window.ts — main window. electron/src/workflowWindow.ts — pinned workflow and mini-app windows. electron/src/tray.ts — tray icon and menu. electron/src/menu.ts — native menu bar. electron/src/components/ — React components for the native windows. See the Electron developer guide for build and debugging tips. Related Docs Installation — download and first-run Configuration — settings persisted per-user Troubleshooting — boot and install issues Deployment — self-hosted / cloud alternatives--- # Agent Configuration Examples Source: https://docs.nodetool.ai/examples/agents/ Agent Configuration Examples This directory contains example YAML configuration files for the nodetool agent command. Run an example with nodetool agent run <file> and supply the objective with --objective (or pipe it via stdin, or set an objective: field in the YAML): nodetool agent run <file>.yaml --objective "Your objective here" Available Examples research-agent.yaml Autonomous research agent for gathering and synthesizing information from the web. Use cases: Market research and competitive analysis Literature reviews and fact-finding Data collection from multiple sources Automated research workflows Tools: Web search (Google) Web browsing and content extraction File read/write operations Example: nodetool agent run research-agent.yaml \ --objective "Research the latest developments in quantum computing" code-assistant.yaml AI coding assistant for development tasks, debugging, and code review. Use cases: Writing clean, efficient code Debugging and fixing issues Code refactoring and optimization Explaining complex code Note: this example’s tools: list includes execute_code and terminal, which are not wired into the agent CLI’s tool registry — they are ignored at run time with a warning. For code execution use run_code; there is no shell/terminal tool. Update the file to run_code (and remove terminal) before relying on code execution. Example: nodetool agent run code-assistant.yaml \ --objective "Create a Python script to parse CSV files and generate visualizations" content-creator.yaml Creative writing agent for generating blog posts, marketing content, and documentation. Use cases: Blog posts and articles Marketing copy and social media content Technical documentation Creative storytelling Tools: Web research File operations Example: nodetool agent run content-creator.yaml \ --objective "Write a blog post about sustainable software development practices" minimal.yaml Minimal agent configuration showing the core fields. Use cases: Starting point for custom agents Simple task automation Testing agent functionality Example: nodetool agent run minimal.yaml \ --objective "Summarize the main features of NodeTool" Creating Custom Agents Basic Structure name: my-agent description: Agent description system_prompt: | Agent instructions and behavior model: provider: openai id: gpt-4o planning_agent: enabled: true model: provider: openai id: gpt-4o-mini tools: - write_file - read_file max_tokens: 128000 max_steps: 10 Best Practices Start with minimal.yaml and add complexity as needed Be specific in system_prompt about expected behavior and workflow Choose appropriate models for the task complexity and budget Enable only necessary tools to reduce complexity and improve focus Set a reasonable max_steps to bound task length Test incrementally with simple objectives before complex tasks Configuration Tips System Prompt: Clearly define the agent’s role and responsibilities Include specific workflow steps Provide guidelines for tool usage Set expectations for output format and quality Model Selection: Planning model: Use a fast, cost-effective model (gpt-4o-mini) Main model: Match the model to task complexity Simple tasks: gpt-4o-mini, gemini-3.5-flash Complex reasoning: gpt-4o, claude-sonnet-4-6 Code tasks: claude-sonnet-4-6 (large context) Tool Configuration: Start minimal (read_file, write_file) Add tools progressively based on needs (e.g. run_code, grep, google_search, browser) Document tool usage in the system_prompt Parameters: max_tokens: per-step context budget (default 128000) max_steps: bound the number of steps (e.g. 5 simple, 10 standard, 20 complex) Testing Agents Validate a Config nodetool agent test your-agent.yaml This reports a missing model.provider/model.id, lists the resolved tools, and warns about unknown tool names. Quick Run nodetool agent run your-agent.yaml --objective "Simple test task" Automated Testing # Test multiple objectives for objective in "Test 1" "Test 2" "Test 3"; do nodetool agent run your-agent.yaml --objective "$objective" done Documentation For complete documentation, see: Agent CLI Documentation — Complete reference Chat & Agents — Agent system overview NodeTool CLI — Full CLI reference Troubleshooting Agent doesn’t complete tasks Solution: Increase max_steps or improve system_prompt specificity. max_steps: 20 # Allow more steps Tool errors Solution: Verify the tool name against the available tools list. Unknown names are ignored with a warning; nodetool agent test reports them. Rate limiting Solution: Use a local model. model: provider: ollama id: llama3.2:3b Performance issues Solution: Use a faster model for planning. planning_agent: model: provider: openai id: gpt-4o-mini Contributing To add new example configurations: Create a descriptive YAML file (e.g., data-analyst.yaml) Include a detailed system_prompt with clear instructions Choose an appropriate model and tools Validate with nodetool agent test, then run with sample objectives Update this README with the new example Submit a pull request License These examples are provided under the same license as NodeTool (AGPL-3.0).--- # Execution Strategies Source: https://docs.nodetool.ai/execution-strategies.html This page covers two distinct mechanisms that are easy to conflate: Workflow execution — how the kernel runs a graph of nodes. This is an in-process actor model with message-passing; there are no per-job threads, subprocesses, or containers. Code execution — how individual code-runner nodes (Python, JavaScript, Bash, …) run arbitrary user code. This is the only place Docker / subprocess sandboxing applies, and it sandboxes the code inside one node, not the workflow. See also Architecture for the system overview and Automatic Message Correlation for the lineage model the scheduler relies on. Workflow Execution: the actor model A workflow is a DAG. The runtime lives in packages/kernel/: WorkflowRunner (packages/kernel/src/runner.ts) orchestrates one job. NodeActor (packages/kernel/src/actor.ts) runs a single node. NodeInbox (packages/kernel/src/inbox.ts) buffers each node’s inputs per handle and tracks how many upstream sources remain open. There is no JobExecutionManager and no pluggable “threaded / subprocess / docker” strategy for running a workflow. The whole graph runs concurrently in one event loop via async actors that pass messages to each other’s inboxes. What WorkflowRunner does WorkflowRunner.run(request, graphData) performs, in order: Bypass rewrite — rewriteBypassedNodes re-routes around nodes flagged ui_properties.bypassed. Invalid-edge filtering — drops edges whose source or target node is missing. Correlation analysis — analyzeCorrelation (mandatory) computes the static lineage scope of every node input/output. Issues abort the run with a GraphValidationError before any actor starts. Graph + node validation — structural validation plus an optional per-node validateNode callback (e.g. missing required fields). Inbox initialization — one NodeInbox per node, seeded with the count of incoming data edges per handle (and one __control__ upstream per unique controller). Node initialization — resolves and caches one executor instance per node. Input dispatch — _dispatchInputs runs each external input node’s process() once and delivers values to downstream inboxes. Non-streaming inputs then signal end-of-stream (EOS); streaming-output inputs (e.g. RealtimeAudioInput) stay open for later pushInputValue() calls. Actor spawning — _processGraph creates a NodeActor per node and runs them all with Promise.all. Actors block on their inboxes until upstream data arrives, so nodes whose inputs are ready run concurrently while the rest wait. The runner also routes outputs downstream (_sendMessages), tracks per-edge message counters for the UI (throttled edge_update events), propagates EOS (_sendEOS), and emits job_update / output_update messages through the ProcessingContext. Job outcomes run() returns a RunResult with status completed, failed, cancelled, or suspended. Precedence at finalization is cancel > suspend > failed > completed: a node throwing WorkflowSuspendedError yields a suspend payload (human-in-the-loop pause), a node error fails the whole job, and cancel() aborts the run-level AbortController and closes every inbox to unblock waiting actors. The four actor modes NodeActor._runImpl picks a mode from the node’s hydrated behavior flags. (The runner requires a hydrated graph: the flags below must be set, or streaming nodes would silently run as one-shot process() calls.) Mode Trigger Behavior Buffered default Gathers inputs via the correlated scheduler and calls process() once per ready input set. Streaming input is_streaming_input Node drains its inbox itself via NodeInputs (run(inputs, outputs, ctx)), emitting through NodeOutputs. Streaming output is_streaming_output Calls genProcess(), which yields partial frames; each yield is routed downstream as it is produced. Controlled is_controlled Waits for control events on the __control__ handle, caches data inputs, and re-runs process() per run event until controllers signal EOS. Buffered and streaming-output nodes share the same correlation-aware gather path (_runCorrelated); the difference is whether the executor exposes process() or genProcess(). Correlation-aware scheduling (_runCorrelated) Buffered/streaming-output nodes do not simply wait for “all inputs.” They schedule per correlation key so that fan-out/fan-in over iterations stays correctly paired. _runCorrelated (in actor.ts) classifies each connected data handle by its static scope: max-scope — scope length equals the node’s invocation scope; bucketed per projected lineage key. One handle may be the repeating driver (repeats_per_key), firing once per arriving envelope. strict-prefix sticky — a shorter non-empty scope; the latest value is kept per projected parent key (side inputs that change less often). empty — empty scope; a single sticky value (or the node’s declared property default). As envelopes arrive on iterAnyWithEnvelope(), the actor records each into its bucket and re-evaluates which keys are now ready. A key is ready when every handle has a value for it (lists wait for the handle to close so the full set is captured). Each ready key fires process()/genProcess() with its matched inputs, and the produced outputs inherit (or mint) the correct lineage so downstream joins line up. Source nodes (no connected data handles) fire once with empty inputs. Inbox safety limits (max_pending_keys, max_pending_messages_per_key) abort the run if pending keys grow without bound — typically a missing upstream close. See correlation-design.md for the full model: lineage as root-id → item-token maps, static scope analysis, and done / dropped-key / scope-close propagation. Sync modes (legacy framing) The older zip_all / on_any / sticky sync-mode vocabulary maps onto the correlation scheduler: zip_all-style “wait for all handles” is the max-scope readiness rule, sticky is the strict-prefix / empty-scope handle classes, and on_any is the per-arrival driver behavior. New nodes declare correlation via input scope and output_correlation rather than a single sync_mode flag. Control events Controller nodes (e.g. agent nodes) drive controlled nodes over control edges. The runner builds a _control_context describing each controlled node’s properties and “run” action schema, injects it as an input, and exposes sendControlEvent(targetNodeId, properties) (via ProcessingContext.setSendControlEvent) so a controller can dispatch a run and await that node’s next output. Responses are tracked FIFO per node so a burst of concurrent control calls doesn’t drop a waiter. Code Execution: sandboxed runners Separately from workflow scheduling, code-runner nodes execute arbitrary user code in a sandbox. These live in packages/code-runners/ and are consumed by the nodes in packages/code-nodes/ (Python, JavaScript, Bash, Ruby, Lua, shell command). This is the only place “docker / subprocess” legitimately applies, and it isolates the code in one node, not the workflow. StreamRunnerBase StreamRunnerBase (packages/code-runners/src/stream-runner-base.ts) is the base class. Its stream(userCode, envLocals, options) async generator yields [slot, value] tuples where slot is "stdout" or "stderr". It supports two mutually exclusive modes: "docker" (default) — run inside a Docker container (via dockerode). "subprocess" — run as a local child process on the host. stop() cooperatively aborts: it force-removes the active container or terminates the child (SIGTERM, escalating to SIGKILL after a grace period). Docker sandbox defaults The constructor applies security-first defaults for untrusted code: Option Default Notes image "bash:5.2" Subclasses override (e.g. python:3.11-slim, node:22-alpine). timeoutSeconds 10 Max container/process lifetime. memLimit "256m" Container memory cap. nanoCpus 1_000_000_000 CPU quota (1e9 = 1 CPU). networkDisabled true No network for code runners. ipcMode "private" Never shares host/other-container IPC or shared memory. capDrop ["ALL"] All Linux capabilities dropped. securityOpt ["no-new-privileges"] Blocks setuid privilege escalation. user "1000:1000" Never runs as root; null keeps the image’s user. readonlyRootfs false Opt-in read-only root FS (with a writable /tmp tmpfs). workspaceMountPath / dockerWorkdir "/workspace" Bind-mounted workspace; readonlyWorkspace opt-in. These are node/runner options, not environment variables. Runner subclasses Exported from packages/code-runners/src/index.ts: PythonDockerRunner — wraps env vars as key = repr(value) lines, runs python -c. JavaScriptDockerRunner, BashDockerRunner, RubyDockerRunner — per-language code runners. LuaRunner / LuaSubprocessRunner — Lua execution. CommandDockerRunner — shell-command execution. ServerDockerRunner — starts a long-running server in a container, publishes an ephemeral host port, and yields an ["endpoint", url] message once the port is TCP-reachable (networking enabled). ServerSubprocessRunner — same endpoint pattern as a local subprocess; can download and cache a remote binary on disk. The code-nodes (e.g. the Python/JavaScript code nodes in packages/code-nodes/src/nodes/code.ts) expose image and execution_mode ("docker" | "subprocess") as node properties and construct the matching runner, then collect its streamed output. Cancellation and shutdown Workflow: WorkflowRunner.cancel() aborts the run-level AbortController (observed by node code via inputs.signal) and closes every inbox so waiting actors unblock; the job finalizes as cancelled. Code runner: StreamRunnerBase.stop() force-removes the container or terminates the subprocess (SIGTERM → SIGKILL escalation). Related Architecture — system components, message types, job lifecycle. Automatic Message Correlation — lineage, scopes, and the scheduler design. packages/kernel/ — runner.ts, actor.ts, inbox.ts, correlation-analysis.ts. packages/code-runners/ — sandboxed code execution runners.--- # Quick Start Source: https://docs.nodetool.ai/getting-started.html Install, open a template, run it, edit it, ship it. About 10 minutes, no account required. Prefer to watch first? Here’s a full workflow running on the canvas — see the Tutorials page for all of them. Step 1 — Install Requirements Component Minimum Recommended RAM 8 GB 16 GB+ Disk 10 GB free 50 GB+ for models GPU None 8 GB+ VRAM for local inference OS macOS 13+, Windows 10+, Ubuntu 22+ Latest No GPU? Use cloud providers with your keys. See hardware notes. Install Download from nodetool.ai. Run the installer. Launch. No setup wizard. Providers: open Settings → Providers and paste a key from OpenAI, Anthropic, or Google. One key is enough to run the templates below. Step 2 — Run a workflow Open a template from the Dashboard. Movie Posters Dashboard → Templates → Movie Posters. The graph opens: inputs left, agent middle, image generator right, preview last. Fill the inputs: Title: Ocean Depths Genre: Sci-Fi Thriller Audience: Adults who love mystery Select a model for the Agent and List Generator (requires provider setup). Press Ctrl/⌘ + Enter to run. Nodes light up as they process. Outputs stream into the Preview node on the right — that’s your finished poster. Creative Story Ideas Dashboard → Templates → Creative Story Ideas. Set inputs: Genre: Cyberpunk Character: Rogue AI detective Setting: Neon-lit underwater city Select a model for the Agent and List Generator (requires provider setup). Press Ctrl/⌘ + Enter to run. Step 3 — Edit The graph is yours to change. Edit, re-run, repeat — every run reflects the current graph. Change an input or swap a model, then press Ctrl/⌘ + Enter to re-run. Save your version with Ctrl/⌘ + S. Click a node to inspect it. Hover an edge to see the data flowing through. Press Space, search “Preview”, drop one anywhere on the canvas to watch any value. Optional node packs NodeTool ships hundreds of nodes. To keep the node menu focused, advanced and niche namespaces — file system, databases, document conversion, web scraping, code execution, and more — are grouped into optional packs, and provider nodes follow your API keys. Press Space to open the node menu, then click Optional packs at the bottom of the namespace list. Categories — turn on a group (Documents, Image & Graphics, Web & Scraping, …) to reveal its nodes in the menu. Search always finds every node, even when its pack is off — so hiding a pack only declutters browsing, it never hides a node from search. Providers — a provider’s nodes appear once you add its API key. The key is the switch: set it in Settings → API Keys (or use Add API key right here) and the provider’s pack enables automatically, no restart. Locally-run packs like Transformers.js need no key, so they keep a manual toggle. Opening a workflow that uses a node from a disabled pack enables that pack for you, so shared workflows just work. Step 4 — Build an app A Mini-App hides the graph and exposes inputs and outputs only — a form anyone can use without touching the canvas. Open the workflow. Click Mini-App in the toolbar. Fill the inputs and run. Same workflow, no graph. Custom UI? See App Builder. The bigger picture You just ran the loop NodeTool is built around: collect assets, build a workflow, generate outputs, and share them as a Mini-App. Two more editing surfaces plug into the same loop — the Sketch Editor for layered still images and the Video Editor for timelines — and everything they produce is an asset the others can read. All four surfaces share one asset store and the same model/provider system. See Key Concepts → How everything fits together for the full loop and diagram. Next Goal Page How workflows work Key Concepts Full interface User Interface, Workflow Editor More examples Gallery, Cookbook Models and providers Models & Providers Deploy Deployment Stuck Troubleshooting, Debugging Discord · GitHub · Glossary--- # Chat & Agents Source: https://docs.nodetool.ai/global-chat-agents.html Chat runs workflows, streams results, and runs the unified agent loop on every turn — the assistant plans and calls tools on its own as a task needs it. Jump to the right reference below. Core guides Chat — Interface layout, thread management, and how chat invokes workflows or tools. NodeTool Agent System — How planning, tool usage, and execution loops work under the hood. Agent CLI — Run autonomous agents from the command line with YAML configuration files. Chat CLI & Chat Server — Automate conversations or integrate with custom frontends. Chat API — Programmatic access for running chats, streaming outputs, and issuing tool calls. Typical flows Run any saved workflow from chat Save the workflow in the editor, open Chat, and choose a workflow in the composer. Let the agent plan complex tasks The unified agent loop runs on every turn — the LLM decomposes goals into tasks and calls tools as needed. See the Agent guide. Run autonomous agents from command line Use the Agent CLI to run agents for fully automated task execution. Expose agents via APIs Use the Chat API from backend services so agents can operate in headless environments. Ready to go deeper? Pair this with the Cookbook for agent patterns or the Deployment Guide to move agents into production.--- # Chat Source: https://docs.nodetool.ai/global-chat.html Chat is the always-on chat surface inside NodeTool. It talks to any provider, runs tools, kicks off agents, generates images, video, and speech, and triggers your saved workflows — all from one composer. Overview 20+ providers (OpenAI, Anthropic, Gemini, Ollama, …) One composer for chat, image, video, and speech generation Tools: web search, files, code execution, HTTP, and more Every turn runs the agent loop — the assistant plans and calls tools on its own as a task needs it Permission modes (Plan / Default / Auto) set how much the agent may do without asking Run saved workflows from chat Multiple threads, persistent history Standalone tray window Persistent WebSocket connection — reconnects after reloads. Getting Started Opening Chat From the App: Click Chat in the navigation menu Standalone Window: Click the NodeTool system tray icon and select Chat for a dedicated, focused window Choosing a Model Click the model chip in the composer to open the model picker. Available models depend on your configured providers: Cloud models – OpenAI GPT, Anthropic Claude, Google Gemini (requires API keys) Local models – Ollama, LM Studio models (requires local installation) Configure providers in Settings > Providers. See Models & Providers. Composer Modes The same composer generates more than text. Click the mode chip to switch between: Mode What it does Chat Talk to the model, run tools, and drive agents Generate Images Text-to-image with your chosen image model Edit Images Image-to-image edits on a dropped image Generate Videos Text-to-video Animate Image Turn a still image into a clip Generate Speech Text-to-speech with a voice picker Pi Agent The workspace-aware agent that works over your files Each mode swaps in its own controls — resolution, aspect ratio, duration, voice — and attaches them to the message so the server routes to the right provider call. Mentions: @ for assets and entities Type @ in the composer to reference things by name instead of hunting for them: Entities — characters, locations, styles, and props from your entity library appear first. Picking one inlines its name into the message and attaches its reference image, so generations stay consistent with the entity’s canonical look. Assets — recent and saved library files. Picking one attaches it to the message like a drag from the asset library. The same picker works inside the workflow editor’s Prompt node, where a picked entity becomes a chip that expands to its descriptor and reference image at generation time. Conversation Threads Chat organizes conversations into threads: Create threads – Click the New Chat button to start a fresh conversation Switch threads – Use the sidebar to navigate between conversations Delete threads – Remove conversations you no longer need Message history – Scroll through past messages with cursor-based pagination Message caching – Recent messages are cached locally for fast loading Each thread keeps its own model, permission mode, and history. Agents The Agent Loop Every turn in Chat runs the same agent loop — there’s no separate mode to turn on. The assistant decides for itself, per request, whether to break a task into steps, select tools, and execute a multi-step plan, or just answer directly. Planning – The agent analyzes your request and, when the task warrants it, creates a plan with ordered steps Tool selection – For each step, the agent chooses from the available tools Execution – Steps run in sequence, with results feeding the next step Adaptation – The agent adjusts its plan based on intermediate results Reporting – Progress, tool calls, and reasoning stream in real time Permission Modes The permission chip sets how far the agent may act on its own. It’s per-thread, so a scratch thread can run wide open while a production one stays cautious: Mode Behavior Plan Read and propose only. No actions taken. Default Reads run automatically; actions ask first. Auto Everything runs, no prompts. Agent Capabilities When a request calls for it, the assistant can: Capability Examples Web research Search the web, browse pages, extract content File operations Read, write, and organize files in your workspace Code execution Run JavaScript in a sandboxed environment Data analysis Perform calculations, query vector databases Document processing Extract text from PDFs, process emails Asset management Create, organize, and index assets HTTP requests Call external APIs and process responses Workflow execution Run saved NodeTool workflows with custom inputs Watching an Agent Work As the agent runs, the thread shows the task plan, each step’s status as it starts, completes, or fails, the tools it calls and their results, and — on models that support it — its reasoning. Workflow Integration You can ask the agent to run a saved workflow as part of a turn: it calls the workflow with your inputs and streams the results back into the chat. You can also ask it to build or modify workflows — it uses workspace tools to write the workflow configuration for you. Save a workflow in the workflow editor first, then reference it from chat. Available Tools Chat agents have access to these tools: Tool Category What It Does Browser Navigate web pages, extract content, take screenshots Search Web search via multiple search providers Filesystem Read/write files, list directories, manage workspace Code Execute JavaScript in a sandboxed environment Calculator Perform mathematical calculations HTTP Make HTTP requests to external APIs PDF Extract text and data from PDF documents Email Read and process email messages Assets Upload, organize, and manage NodeTool assets Vectors Query and manage vector database collections Google Interact with Google APIs (search, drive, etc.) Workspace Manage NodeTool workspace settings and files MCP Tools (Model Context Protocol) NodeTool supports MCP for connecting to external tool servers, so you can integrate custom tools and services beyond the built-in set. See the MCP documentation for available MCP servers. Standalone Chat Window Access chat in a focused, dedicated window outside the main app: Click the NodeTool icon in your system tray Select Chat from the menu A new window opens with just the chat interface The standalone window is useful for: Quick questions without switching to the full app Running agents in the background while doing other work Using chat as a general-purpose AI assistant Next Steps Chat & Agents – Agent CLI and API integration Chat API – Programmatic access for running chats Chat CLI – Command-line chat interface Agent CLI – Run autonomous agents from the terminal Models & Providers – Configure AI providers Cookbook – Agent workflow patterns--- # Glossary Source: https://docs.nodetool.ai/glossary.html NodeTool terminology in plain language. Simple explanations followed by technical details where relevant. Core Concepts NodeTool The platform you’re using! NodeTool is a visual environment for building AI workflows. You connect nodes (building blocks) to create automations without writing code. Workflow A workflow is your project – a collection of connected nodes that accomplish a task together. Think of it like a recipe: each step (node) does something specific, and together they create the final result. Technical: A directed acyclic graph (DAG) of nodes describing an end-to-end task. Node A node is a single building block in your workflow. Each node does one specific job, like “generate text” or “resize image.” You connect nodes together to build workflows. Technical: A processing unit with typed inputs, outputs, and configurable properties. Edge / Connection The lines that connect nodes together. They show which node’s output feeds into another node’s input – like pipes carrying water between stations. Input Where data enters – either into a workflow (like uploading a file) or into a specific node (like connecting another node’s output). Output Where results come out. Output nodes show your final results; other nodes have outputs that connect to the next node in the chain. AI & Models Model A pre-trained AI program that has learned to do a specific task. For example, a language model generates text, an image model creates pictures. You don’t train models – you just use them. Example: GPT is a language model; Stable Diffusion is an image model. Provider A service that runs AI models for you. Providers can be: Cloud providers like OpenAI, Anthropic, or Google Local engines like Ollama or llama.cpp that run models on your computer Technical: Adapter that talks to an external AI service (OpenAI, Anthropic, Gemini, Ollama, ComfyUI, etc.). Agent A special type of AI that can plan and execute multi-step tasks. Unlike simple chat, an agent can break down complex requests, use tools, and work through problems step by step. Technical: Multi-step planner/executor that can call tools or workflows. LLM (Large Language Model) A type of AI model that understands and generates text. LLMs power chatbots, writing assistants, and text analysis tools. Examples: GPT-4, Claude, Llama. Inference The process of using a trained AI model to generate results. When you “run” a model, you’re performing inference. User Interface Canvas The main work area where you build workflows by placing and connecting nodes. Pan around by dragging, zoom with scroll. Preview Node A special node that shows you intermediate results while your workflow runs. Add previews anywhere to debug or monitor progress. Mini-App A simplified interface for running a workflow. Mini-Apps hide the complexity and show only the inputs and outputs, making them easy to share with non-technical users. Chat NodeTool’s AI assistant interface. Chat with AI models, run workflows conversationally, or let the agent plan and execute multi-step tasks on its own. Inspector / Properties Panel The panel (usually on the right) that shows settings for the selected node. This is where you configure how each node behaves. Dashboard Your home screen showing recent workflows, templates, and chat threads. Running & Execution Run Execute your workflow and see the results. Click the Run button or press Ctrl/⌘+Enter. Job A single workflow execution. When you click Run, NodeTool creates a job that tracks progress, handles errors, and delivers results. Technical: A single workflow execution orchestrated by the kernel’s WorkflowRunner, which runs each node as a NodeActor. Streaming Receiving results progressively as they’re generated, rather than waiting for everything to complete. Many AI nodes stream their output so you see progress in real-time. Execution Strategy How NodeTool runs your workflow – either in the same process, a separate subprocess, or inside a Docker container. Technical options: threaded, subprocess, Docker. Infrastructure Server The background process that actually runs your workflows. Servers connect to the server and execute jobs. Technical: Process that runs workflows and exposes HTTP/WebSocket endpoints (via nodetool serve, optionally with --host/--port). API Server The backend service that handles requests, manages workflows, and coordinates servers. Technical: Node.js HTTP server (via @nodetool-ai/websocket) handling REST endpoints such as /v1/chat/completions and /api/workflows. Proxy An optional service that sits in front of NodeTool to handle security, routing, and SSL certificates for production deployments. Technical: Reverse proxy that terminates TLS and forwards to the API server. Thread ID An identifier that tracks a conversation in Chat. Each chat thread has its own history and context. Technical: Conversation identifier for chat/agent sessions; used by WebSocket and SSE streams. Data & Storage Asset Any file you use in NodeTool – images, audio, documents, etc. Assets are stored and managed so you can reuse them across workflows. Collection A searchable database of documents used for RAG (Retrieval-Augmented Generation). Collections let AI answer questions using your own documents. Vector Database A special database that stores text as mathematical vectors, enabling semantic search (finding content by meaning, not just keywords). AI Techniques RAG (Retrieval-Augmented Generation) A technique where an AI model answers questions using your own documents as context. Instead of relying only on its training data, the model retrieves relevant snippets from your documents first, then generates an answer based on that context. This dramatically reduces hallucinations and keeps answers grounded in your data. In NodeTool: Use the HybridSearch node to retrieve documents, then pass them to an Agent node via FormatText. Embedding A mathematical representation of text (or images) as a list of numbers (a “vector”). Embeddings capture meaning, so similar concepts have similar numbers. This is what makes semantic search possible — finding content by meaning rather than exact keyword matches. Prompt The text instruction you give to an AI model. Good prompts are specific and clear. In NodeTool, prompts are usually set as text inputs or constructed using FormatText nodes that combine variables into a template. Temperature A setting that controls how creative or deterministic an AI model’s output is. Low temperature (0.0–0.3) produces consistent, factual responses. High temperature (0.8–1.2) produces more varied, creative results. Quantization A technique for making AI models smaller and faster by reducing the precision of their numbers. A Q4 model uses 4-bit precision (smaller, slightly less accurate), while Q8 uses 8-bit (larger, more accurate). Quantized models let you run larger models on less powerful hardware. Fine-tuning The process of further training an existing AI model on your own data to specialize it for a particular task. NodeTool can use fine-tuned models from providers like HuggingFace or OpenAI. Development Terms DAG (Directed Acyclic Graph) The technical name for how workflows are structured. “Directed” means data flows one way; “Acyclic” means no loops. NodeTool handles this automatically — you just connect nodes and it figures out the execution order. DSL (Domain Specific Language) NodeTool’s TypeScript API (@nodetool-ai/dsl) for building workflows in code rather than the visual editor. Useful for automation, testing, and custom integrations. Node Pack A collection of related nodes bundled together. Install node packs to add new capabilities to NodeTool (e.g., additional model providers or data processing tools). API Key A secret string that authenticates you with a cloud AI provider. You get API keys from provider dashboards (OpenAI, Anthropic, Google, etc.) and enter them in NodeTool’s Settings → Providers panel. WebSocket A communication protocol that allows real-time, bidirectional data flow between NodeTool’s frontend and backend. This is what enables live streaming of workflow results and chat responses. SSE (Server-Sent Events) A one-way streaming protocol used by NodeTool’s Server API to push workflow progress and results to clients in real-time. Similar to WebSocket but simpler and HTTP-based. See Also Key Concepts – Deeper explanation of core ideas Getting Started – Hands-on tutorial Models & Providers – Setting up AI models--- # HuggingFace Integration Source: https://docs.nodetool.ai/huggingface.html HuggingFace Integration Nodes for running HuggingFace Hub models in NodeTool workflows. Supports text, image, audio, and multimodal tasks with optional quantization, CPU offload, and LoRA adapters. All HuggingFace nodes live in the huggingface.* namespace. Node Categories 🎨 Image Generation Text-to-Image Nodes Stable Diffusion - Generate high-quality images from text prompts Custom width/height settings (256-1024px) Configurable inference steps and guidance scale Support for negative prompts Use cases: Art creation, concept visualization, content generation Stable Diffusion XL - Enhanced image generation with SDXL models Higher resolution outputs (up to 1024px) Improved image quality and detail Support for IP adapters and LoRA models Use cases: Marketing materials, game assets, interior design concepts Qwen-Image - High-quality general-purpose text-to-image generation Nunchaku quantization support for efficient memory usage True CFG scale control for precise guidance Supports MLX for Apple Silicon optimization Use cases: General-purpose image generation, quick prototyping, production workflows Flux - Image generation with memory-efficient quantization Supports schnell (fast) and dev (high-quality) variants Nunchaku quantization (FP16, FP4, INT4) for reduced VRAM usage CPU offload support for large models Configurable max_sequence_length for prompt complexity Use cases: High-fidelity image generation with limited hardware Flux Control - Controlled image generation with depth/canny guidance Depth-aware and edge-guided generation Control image input for structural guidance Quantization support (FP16, FP4, INT4) Use cases: Controlled composition, maintaining structure while changing style Chroma - Flux-based model with advanced attention masking Professional-quality color control Attention slicing for memory optimization Use cases: Professional photography effects, precise color grading Text2Image (AutoPipeline) - Automatic pipeline selection for any text-to-image model Auto-detects best pipeline for given model Flexible generation without pipeline-specific knowledge Use cases: Testing different models, rapid prototyping Image-to-Image Transformation Image to Image - Transform existing images using Stable Diffusion Strength parameter controls transformation amount Support for style transfer and image variations Use cases: Style transfer, image enhancement, creative remixing 🗣️ Speech & Audio Processing Audio Classification Audio Classifier - Classify audio into predefined categories Recommended models: MIT/ast-finetuned-audioset-10-10-0.4593 ehcalabres/wav2vec2-lg-xlsr-en-speech-emotion-recognition Use cases: Music genre classification, speech detection, environmental sounds, emotion recognition Zero-Shot Audio Classifier - Classify audio without predefined categories Flexible classification with custom labels Use cases: Dynamic audio categorization, sound identification Automatic Speech Recognition Whisper - Convert speech to text with multilingual support Supports 100+ languages Translation mode (translate any language to English) Timestamp options (word-level or sentence-level) Multiple model sizes (tiny to large-v3) Recommended models: openai/whisper-large-v3 - Best accuracy openai/whisper-large-v3-turbo - Fast inference openai/whisper-small - Lightweight option Use cases: Transcription, translation, subtitle generation, voice interfaces ChunksToSRT - Convert transcription chunks to SRT subtitle format Automatic timestamp formatting Time offset support Use cases: Video subtitling, accessibility features Audio Generation Text-to-Speech - Generate natural-sounding speech from text Multiple voice options Configurable speaking rate and pitch Use cases: Voiceovers, accessibility, content creation Text-to-Audio - Generate audio effects and sounds from text descriptions Creative sound generation Use cases: Sound effects, audio design, music production 📝 Text Processing Text Generation Text Generation - Generate text using large language models Streaming output support Extensive model support including: Qwen3 series (0.6B to 32B parameters) Meta Llama 3.1 series Ministral 3 series Gemma 3 series TinyLlama for lightweight deployment Quantized model support (BitsAndBytes 4-bit) Configurable parameters: Temperature (0.0-2.0) - Controls randomness Top-p (0.0-1.0) - Controls diversity Max tokens (up to 512 default) GGUF model support for efficient inference Use cases: Chatbots, content generation, code completion, creative writing Text Analysis Text Classification - Classify text into categories Sentiment analysis Topic categorization Use cases: Content moderation, sentiment analysis, document organization Token Classification - Identify and classify tokens in text Named entity recognition (NER) Part-of-speech tagging Use cases: Information extraction, text analysis Fill Mask - Predict masked tokens in text BERT-style masked language modeling Use cases: Text completion, grammar correction Question Answering Question Answering - Extract answers from context Recommended models: distilbert-base-cased-distilled-squad bert-large-uncased-whole-word-masking-finetuned-squad Returns answer with confidence score and position Use cases: Document Q&A, customer support, information retrieval Table Question Answering - Query tabular data with natural language Works with DataFrames Recommended models: google/tapas-base-finetuned-wtq microsoft/tapex-large-finetuned-tabfact Use cases: Database queries, spreadsheet analysis Text Transformation Translation - Translate text between languages Multiple language pairs Use cases: Localization, multilingual content Summarization - Generate concise summaries of long text Extractive and abstractive summarization Use cases: Document summarization, news digests 🖼️ Image Analysis Image Classification Image Classifier - Classify images into predefined categories Recommended models: google/vit-base-patch16-224 - Vision Transformer microsoft/resnet-50 - ResNet architecture Falconsai/nsfw_image_detection - Content moderation nateraw/vit-age-classifier - Age estimation Returns confidence scores for each category Use cases: Content moderation, photo organization, age detection Zero-Shot Image Classifier - Classify images without training data Uses CLIP models for flexible classification Custom candidate labels Recommended models: openai/clip-vit-base-patch32 laion/CLIP-ViT-H-14-laion2B-s32B-b79K Use cases: Dynamic categorization, custom tagging Image Understanding Image Segmentation - Segment images into different regions Instance and semantic segmentation Use cases: Object isolation, background removal Object Detection - Detect and locate objects in images Bounding box outputs Multi-object detection Use cases: Surveillance, counting, automation Depth Estimation - Estimate depth from 2D images Monocular depth prediction Use cases: 3D reconstruction, AR/VR, robotics 🎭 Multimodal Processing Video Generation Text-to-Video (CogVideoX) - Generate videos from text prompts Large diffusion transformer model High-quality, consistent video generation Longer video sequences Use cases: Video content creation, animated storytelling, marketing videos, cinematic content Image-to-Video - Convert static images into video sequences Animate still images Add motion to photographs Use cases: Photo animation, creating video from stills, dynamic presentations Image-Text Models Image to Text - Generate captions for images Automatic image captioning Use cases: Accessibility, content tagging, image search Image-Text-to-Text - Process images with text queries Visual question answering Image reasoning with text context Use cases: Document understanding, visual Q&A, scene description Multimodal - Process both image and text inputs Vision-language models Combined visual and textual understanding Use cases: Complex visual reasoning, document analysis, multimodal search 🎯 Model Customization LoRA (Low-Rank Adaptation) LoRA Selector - Apply LoRA models to Stable Diffusion Combine up to 5 LoRA models Adjustable strength per LoRA (0.0-2.0) 60+ pre-configured style LoRAs including: Art styles (anime, pixel art, 3D render) Character styles (Ghibli, Arcane, One Piece) Visual effects (fire, lightning, water) Use cases: Style customization, character consistency, artistic effects LoRA Selector XL - Apply LoRA models to Stable Diffusion XL SDXL-specific LoRA support Enhanced quality for high-resolution outputs Use cases: High-quality style transfer, professional artwork 🔧 Utility Nodes Feature Extraction - Extract embeddings from text or images Generate vector representations Use cases: Semantic search, similarity matching, clustering Sentence Similarity - Compute similarity between text pairs Use cases: Duplicate detection, semantic search Ranking - Rank documents by relevance Use cases: Search engines, recommendation systems Installation & Setup Requirements Python 3.10+ PyTorch 2.9.0+ CUDA support recommended for optimal performance HuggingFace nodes are included with NodeTool by default. No additional installation is required for basic usage. Model Downloads Models are automatically downloaded from HuggingFace Hub on first use. Downloaded models are cached in ~/.cache/huggingface/ by default. Authentication Some models require HuggingFace authentication: Create a HuggingFace account at https://huggingface.co Generate an access token at https://huggingface.co/settings/tokens Set your token in NodeTool: Desktop App: Settings → Providers → HuggingFace Token Environment Variable: HF_TOKEN=your_token_here Gated Models Some models (like FLUX) require accepting terms on HuggingFace: Visit the model page on HuggingFace Hub Click “Agree and access repository” Ensure your HF_TOKEN is set in NodeTool settings Usage Examples Example 1: Text Generation Workflow Create a text generation workflow: Add a Text Generation node from huggingface.text_generation Configure the node: Model: Qwen/Qwen2.5-7B-Instruct Prompt: “Write a short story about a robot learning to paint” Max tokens: 512 Temperature: 0.8 Connect to an output node Run the workflow Example 2: Image Generation with Stable Diffusion Generate images from text: Add a Stable Diffusion node from huggingface.text_to_image Configure the node: Prompt: “A serene landscape with mountains and a lake at sunset, highly detailed” Negative prompt: “blurry, low quality, distorted” Width: 512, Height: 512 Inference steps: 50 Guidance scale: 7.5 Connect to an image output node Run and view the generated image Example 3: Speech-to-Text Transcription Transcribe audio files: Add an Audio Input node with your audio file Add a Whisper node from huggingface.automatic_speech_recognition Configure Whisper: Model: openai/whisper-large-v3 Task: Transcribe Language: English Timestamps: Word level Connect audio input to Whisper node Connect Whisper output to text output Run to get transcription with timestamps Example 4: Image Classification Classify images into categories: Add an Image Input node with your image Add an Image Classifier node from huggingface.image_classification Configure classifier: Model: google/vit-base-patch16-224 Connect image input to classifier Connect classifier output to output node Run to see classification results with confidence scores Example 5: Complete Workflow - Audio to Summary with Image Build a multi-modal workflow: Audio Transcription: Audio Input → Whisper node Text Summarization: Whisper output → Text Generation node Prompt: “Summarize the following text in 2-3 sentences: {transcription}” Image Generation: Summary output → Stable Diffusion node Prompt: “Create an illustration for: {summary}” Outputs: Connect all outputs to appropriate output nodes This workflow transcribes audio, generates a summary, and creates a matching image. Performance Tips Memory Optimization Use quantized models (INT4, FP4) for reduced VRAM usage Enable CPU offload for large models in node properties Use smaller model variants when possible (e.g., whisper-small vs whisper-large) Enable attention slicing for memory-intensive operations Close other GPU applications when running large models Speed Optimization Use CUDA/GPU when available for best performance Select appropriate model sizes based on your needs: Development: Use tiny/small/turbo variants Production: Use large/optimized variants Use optimized models (e.g., whisper-large-v3-turbo vs whisper-large-v3) Enable PyTorch compilation for frequently-used models Pre-download models before production runs Quality vs Performance Trade-offs Fast + Low Memory: Quantized models with CPU offload Example: FLUX Schnell with INT4 quantization Balanced: FP16 models on GPU with moderate inference steps Example: Stable Diffusion with 30 steps Best Quality: Full precision models with high inference steps Example: FLUX Dev with 50 steps Available Workflow Examples The HuggingFace integration includes pre-built workflow examples: Image to Image - Transform images using Stable Diffusion Movie Posters - Generate movie poster-style images Transcribe Audio - Convert speech to text with Whisper Pokemon Maker - Generate Pokemon-style creatures Depth Estimation - Extract depth information from images Add Subtitles To Video - Automatically generate and add subtitles Object Detection - Detect and locate objects in images Summarize Audio - Transcribe and summarize audio content Segmentation - Segment images into regions Audio To Spectrogram - Visualize audio as spectrograms These examples demonstrate best practices and can be customized for your needs. Troubleshooting Common Issues CUDA Out of Memory RuntimeError: CUDA out of memory Solutions: Enable CPU offload in node advanced properties Use quantized models (INT4/FP4) Reduce image size or inference steps Close other GPU applications Use smaller model variants Model Not Found OSError: Model not found Solutions: Ensure model name is correct (check HuggingFace Hub) Verify internet connection for model download Check HF_TOKEN is set for gated models Clear cache and retry: rm -rf ~/.cache/huggingface/ Slow Inference Solutions: Verify CUDA is available: Check Settings → System Info Use smaller or quantized models Enable attention optimizations in node properties Consider using turbo/fast model variants Pre-compile models with torch.compile Authentication Errors HTTPError: 401 Client Error: Unauthorized Solutions: Generate a new token at https://huggingface.co/settings/tokens Set token in Settings → Providers → HuggingFace Token Accept model terms on HuggingFace Hub for gated models Ensure token has appropriate permissions Import Errors ModuleNotFoundError: No module named 'transformers' Solutions: Restart NodeTool to ensure dependencies are loaded Check Python environment has all required packages Reinstall NodeTool if issues persist Model Recommendations Text Generation Best Quality: Qwen/Qwen2.5-32B-Instruct Balanced: meta-llama/Llama-3.1-8B-Instruct Fast/Lightweight: TinyLlama/TinyLlama-1.1B-Chat-v1.0 Image Generation Best Quality: FLUX Dev with 50 steps Balanced: Stable Diffusion XL with 30 steps Fast: FLUX Schnell with 4 steps Speech Recognition Best Accuracy: openai/whisper-large-v3 Fast: openai/whisper-large-v3-turbo Lightweight: openai/whisper-small Image Classification General Purpose: google/vit-base-patch16-224 Content Moderation: Falconsai/nsfw_image_detection Zero-Shot: openai/clip-vit-base-patch32 Related Documentation Models Guide - Overview of all model types in NodeTool Providers Guide - Provider configuration and usage Nodes Reference - Complete node documentation Workflow Examples - Pre-built workflow templates Cookbook - Workflow patterns and recipes External Resources HuggingFace Hub - Browse available models HuggingFace Transformers - Library documentation Diffusers Documentation - Image generation library--- # image-editor.md Source: https://docs.nodetool.ai/image-editor.html This page has moved. Please go to Sketch Editor.--- # Indexing & Vector Stores Source: https://docs.nodetool.ai/indexing.html NodeTool ships a lightweight ingestion pipeline for semantic search and retrieval-augmented generation (RAG) tasks. The indexing logic is split across @nodetool-ai/vectorstore (store and embedding) and @nodetool-ai/deploy (collection routes). Overview Collection metadata (CollectionResponse in @nodetool-ai/protocol packages/protocol/src/api-types.ts) stores ingest configuration, including an optional workflow ID. Vector store – the default backend is SQLite-vec (@nodetool-ai/vectorstore packages/vectorstore/src/sqlite-vec-store.ts). Embeddings flow through the VectorProvider abstraction — see Vector Storage for swapping backends (Pinecone, Supabase/pgvector). Indexing route – handleCollectionIndex() (@nodetool-ai/deploy packages/deploy/src/collection-routes.ts) validates the upload and delegates the actual ingestion to a caller-supplied indexFn callback (typed IndexFileToCollectionFn). The route itself does not resolve collections or run workflows; that logic lives in the provided callback. Default Flow The HTTP layer receives an uploaded file and calls handleCollectionIndex() with the collection name, file path, MIME type, and an indexFn. The indexFn callback performs the ingestion: it resolves the target collection (e.g. via resolveCollection() in @nodetool-ai/vectorstore packages/vectorstore/src/index.ts), splits the document with splitDocument(), embeds it, and stores embeddings in SQLite-vec. handleCollectionIndex() returns an IndexResult ({ path, error }) per file, or throws a CollectionHttpError on failure. Messages & Progress While custom workflows run, the service streams JobUpdate, NodeUpdate, and progress messages (from @nodetool-ai/protocol packages/protocol/src/messages.ts). Tests under packages/deploy/tests/collection-routes.test.ts cover expected message sequences. Configuring the vector store The default backend is local SQLite-vec. Switch backends with NODETOOL_VECTOR_PROVIDER. Variable Description Default NODETOOL_VECTOR_PROVIDER sqlite-vec, pinecone, or supabase sqlite-vec VECTORSTORE_DB_PATH Local SQLite-vec database file ~/.local/share/nodetool/vectorstore.db PINECONE_API_KEY Required when provider is pinecone — SUPABASE_URL / SUPABASE_SERVICE_ROLE_KEY Required when provider is supabase — See Vector Storage for backend-specific setup. Custom Ingestion Workflows Collections can reference bespoke workflows to process files before embedding. The workflow should expect: A CollectionInput node receiving Collection(name=…). A FileInput node receiving FilePath(path=…). Return values can include summaries, metadata, or alternate embeddings. Review packages/deploy/tests/collection-routes.test.ts for a template. CLI & API Integration POST /api/collections/:name/index (see @nodetool-ai/websocket packages/websocket/src/collection-api.ts) triggers ingestion via HTTP (multipart/form-data file upload). The MCP server (@nodetool-ai/websocket packages/websocket/src/mcp-server.ts) exposes read-only collection tools — get_collection, query_collection, list_collections, get_asset, list_assets — for IDE plug-ins. It does not index assets. Admin routes under @nodetool-ai/deploy packages/deploy/src/admin-routes.ts provide remote ingestion endpoints for deployed servers. Troubleshooting Missing collection metadata – ensure the collection exists and includes the required workflow entry when using custom workflows. Remote backend errors – for pinecone or supabase, verify credentials and network reachability; fall back to local SQLite-vec by setting NODETOOL_VECTOR_PROVIDER=sqlite-vec. Large files – ensure VECTORSTORE_DB_PATH has disk headroom, or move to a remote backend; the default ingestion workflow streams chunks to reduce memory usage. Related Documentation Providers – selecting embedding models for ingestion nodes. Workflow API – details on RunJobRequest. Storage Guide – configuring persistent storage for uploaded documents.--- # Install NodeTool on Windows, macOS, or Linux Source: https://docs.nodetool.ai/installation.html NodeTool opens on first launch with no setup wizard. Python, Conda, and inference engines install on demand the first time you need them. Quick start Download from nodetool.ai Run the installer Launch NodeTool The download is a .dmg on macOS, .exe on Windows, and an AppImage on Linux — see macOS, Windows, and Linux below for the exact steps and platform notes. No local AI stack? Skip the download entirely: open Settings → Providers and add a key from OpenAI, Anthropic, or Google. See Providers. macOS Download the .dmg from nodetool.ai. Separate builds ship for Apple Silicon (arm64) and Intel (x64) — pick the one matching your Mac (Apple menu → About This Mac). Open the DMG and drag Nodetool into Applications. Launch it from Applications. Release builds are code-signed and notarized, so Gatekeeper shouldn’t block the first launch. Recording audio or video in a workflow prompts for microphone/camera permission the first time — approve it to use those nodes. Apple Silicon Macs run local models through Apple’s MLX framework automatically; no extra setup. Windows Download the installer (Nodetool-Setup-<version>.exe) from nodetool.ai. Run it. You choose the install directory; the installer adds a desktop shortcut and launches NodeTool when it finishes. Approve the firewall prompt — NodeTool runs a local server on port 7777 that the UI connects to. The installer and app executable are code-signed. For local model acceleration, keep your NVIDIA driver current. Linux Download the AppImage from nodetool.ai — it’s the only Linux package NodeTool ships today. Make it executable and run it: chmod +x Nodetool-*.AppImage ./Nodetool-*.AppImage There’s no install step. The AppImage runs in place — move the file wherever you want it to live. Prefer Flatpak? The project publishes unsigned CI builds from every push to main — see Flatpak CI Builds (not yet on Flathub). What installs on demand The app itself is small. Everything else downloads the first time a workflow needs it: Python and Conda — installs when you run a workflow that uses a Python node (HuggingFace, MLX, Apple integrations). The backend’s PythonStdioBridge spawns the Python worker only at that point; a workflow built entirely from TypeScript nodes never triggers it. One-time download, roughly 3-5 GB. Local inference engines — Ollama and llama.cpp download when you install or run a model that needs them, from the Models panel. Model weights — each model you install pulls its own weights, typically 4-20 GB depending on the model. No GPU, or don’t want the download? Use a cloud provider with your own API key instead (Settings → Providers). See Providers and Models & Providers. What Different Tasks Need Hardware maps to inference engine, not to a specific GPU model: Hardware Engine Good for NVIDIA GPU Nunchaku (4-bit diffusion), llama.cpp/GGUF Image generation, quantized LLMs Apple Silicon MLX LLMs, vision models, Flux ported to MLX CPU only llama.cpp, Transformers Works, slower No GPU Cloud providers (BYOK) Every modality, no download See Supported Models for the full engine comparison. Alternative installs Don’t need the desktop app: CLI only — install the standalone nodetool CLI without Electron: curl -fsSL https://raw.githubusercontent.com/nodetool-ai/nodetool/main/install.sh | bash or through npm: npm install -g @nodetool-ai/cli nodetool serve See the CLI Reference. Self-hosted server (Docker) — run the backend on your own machine or a remote host: cp .env.example .env docker compose up -d See Self-Hosted Deployment for auth modes, upgrades, and remote hosts, or the Deployment Guide for the full server/worker picture. From source — for contributors: nvm use npm install npm run build:packages npm run dev Requires Node.js 22.22.1 (.nvmrc) and, for Python nodes, Python 3.11+ with conda. Full setup in the repo README. Troubleshooting Installation Most install problems are one of these. For workflow and runtime issues once NodeTool is running, see the Troubleshooting Guide. On-demand Python/Conda setup fails — needs internet access and about 5 GB free disk space. Restart NodeTool; partial downloads resume automatically. GPU not detected — check your driver with nvidia-smi in a terminal (the same check NodeTool’s own Help → System Information dialog runs). No dedicated GPU? NodeTool falls back to CPU, or use a cloud provider instead. Model download fails or stalls — usually disk space or network. See Model Download Troubleshooting for the full list: disk space, resuming, and HuggingFace rate limits. Can’t connect to the local server — approve the firewall prompt for NodeTool’s local server (port 7777 by default). Running the self-hosted Docker image instead? See Deployment Troubleshooting. Still stuck — ask in Discord or file a GitHub Issue. Include your OS and NodeTool version (Help → About). Uninstalling Windows — Settings → Apps → Nodetool → Uninstall. macOS — drag Nodetool from Applications to the Trash. Linux — delete the AppImage file. Settings live in ~/.config/nodetool/settings.yaml (macOS/Linux) or %APPDATA%\nodetool\settings.yaml (Windows) — remove that folder too for a clean reset. Next Steps Ready to build your first workflow? See the Getting Started guide.--- # Key Concepts Source: https://docs.nodetool.ai/key-concepts.html The ideas you need to build on the canvas. What NodeTool is A node-based canvas for creative AI. You wire nodes instead of writing code. Each node is a single operation: a model call, a transform, a tool. Your machine. Local models run on your hardware. Outputs stay on disk. Your keys. Cloud calls go directly to OpenAI, Anthropic, Gemini, FAL, KIE, Replicate, and others. No credit markup. Mixed. Local and cloud models in the same graph. Open. AGPL-3.0. Self-host the same code we host. Building blocks Nodes A node does one thing. Node Does Example Image Generator Image from text “Sunset over mountains” → image Agent Plans and executes a multi-step task “Summarize this document” → structured summary TextToSpeech Text → audio Blog post → narration Filter Drop items that don’t match Conditional branching Each node has inputs (left), outputs (right), and settings (right panel when selected). Workflows A workflow is connected nodes. When you run it, inputs enter on the left, each node executes when its inputs are ready, results stream into preview and output nodes on the right. Examples: Prompt → image model → save PDF → chunk → index → search → answer Story → characters → portraits → video Workflows are the automation layer. Use them when you want a repeatable pipeline: generate media, transform files, call models, index documents, or prepare assets for another editor surface. Connections Lines between ports. Types are checked — image outputs only connect to image inputs. Drag from an output port to an input port. Hover to inspect data in flight. Assets An asset is a stored file: image, video, audio, PDF, text file, 3D model, or any other media a node can read or write. Assets live in the Asset Explorer and can be reused across workflows, sketches, timelines, and chats. Assets are the shared material between surfaces: Drop an image asset onto the workflow canvas to create an image input. Save a workflow output as an asset so you can use it later. Drag video, audio, or image assets onto a timeline as clips. Open an image asset in the Sketch Editor for painting, masking, or layered edits. Put document assets in collections for indexing and RAG. Sketches A sketch is a layered image document. It is for painting, masking, retouching, compositing, and generating image layers in place. Sketches sit between manual editing and workflow automation: Start with a blank canvas or open an existing image asset. Paint or build up layers by hand. Bind a layer to an image workflow so it can regenerate when inputs change. Render the sketch back into an image asset. Use the rendered image in a workflow or timeline. Timelines A timeline is a multi-track media sequence. It holds clips arranged over time: video, audio, still images, overlays, and generated clips. Timelines are where outputs become finished media: Drag existing assets into tracks as imported clips. Add a generated clip backed by a workflow. Trim, split, reorder, and layer clips. Regenerate stale generated clips when the bound workflow or parameters change. Export the sequence to a video asset. Agents An agent node takes a natural-language goal, breaks it into steps, and uses tools (web search, files, code) to finish. Use it when the path isn’t fixed. Mini-Apps A workflow with the graph hidden — just inputs and outputs. Share with people who shouldn’t have to read a node graph. How everything fits together NodeTool has one common loop: Collect or create assets. Upload files, generate outputs from workflows, paint sketches, or export timelines. Build workflows around them. Workflows read assets, call models, transform media, and write new assets. Edit the result in a surface built for the medium. Use sketches for layered still images and timelines for time-based media. Feed the edited result back into workflows. A rendered sketch or exported timeline is just another asset. Share the workflow or final output. Keep the graph for repeatable work, expose it as a Mini-App, or publish the exported asset. The surfaces are separate because they solve different editing problems, but they share the same asset store and model/provider system. graph LR A[Assets] --> B[Workflow] B --> C[Generated assets] C --> D[Sketch] C --> E[Timeline] D --> A E --> A B --> F[Mini-App] Example: make a short product video Upload product photos as assets. Use a workflow to generate copy, background images, and voiceover. Open a hero image in a sketch to mask and retouch it. Drag the image, voiceover, and generated clips into a timeline. Bind one clip to an image-to-video workflow and regenerate until it fits. Export the timeline as the final video asset. Models Type Output Use Image Pictures Posters, concept art, mockups Video Clips Animation, motion Audio Sound Narration, music, SFX Text Tokens Scripts, summaries, analysis Local vs. cloud   Local Cloud Cost Free after download Provider’s price, billed to your key Where Your machine Provider’s servers Speed Your hardware Provider’s hardware Internet Offline OK Required Setup Download (4–20 GB each) Paste API key Mix freely. Pick the best model for the job per node. Glossary Term Meaning Workflow Connected nodes Node One operation Edge / connection Data flow between nodes Input / output Entry and exit ports Preview Node that displays intermediate data Run Execute the graph Asset Stored file used by workflows, sketches, timelines, chats, or collections Sketch Layered image document for painting, masking, compositing, and image generation Timeline Multi-track sequence for arranging video, audio, image, and generated clips Clip Media item placed on a timeline track Generated clip / layer Timeline clip or sketch layer backed by a workflow or model call Stale Generated output whose inputs, prompt, or bound workflow changed since the last generation Model A trained network you call from a node Provider Where the model runs (local, OpenAI, FAL, …) How a run works On Ctrl/⌘ + Enter: NodeTool walks the graph for dependencies. Each node runs the moment its inputs are ready. Independent nodes run in parallel. Results stream live into preview and output nodes. graph LR A[Input: Prompt] --> B[Agent: Plan] B --> C[Image Generator] B --> D[Text Writer] C --> E[Preview: Image] D --> F[Preview: Text] The Agent runs first; Image Generator and Text Writer then run in parallel since both depend only on the Agent. The graph is a DAG — data flows one way, no cycles. The runner schedules the order. For developers Component Role Graph Nodes + connections. Build with workflow(...), execute with run(...) or runGraph(...) (@nodetool-ai/dsl packages/dsl/src/core.ts). DSL TypeScript DSL (@nodetool-ai/dsl) — typed factories for building graphs in code. WorkflowRunner The runner. Schedules nodes, manages GPU, streams progress. ProcessingContext Runtime — user, auth, assets, cache (@nodetool-ai/runtime). Node type resolution A workflow references nodes by type string (package.Namespace.Class). The runner resolves it via: In-memory registry (with and without trailing Node) Dynamic import by type path Installed packages registry Fallback match by class name Graphs load without pre-importing every node module. See the Developer Guide and Custom Nodes. Next Getting Started Workflow Editor Asset Management Sketch Editor Video Editor Models & Providers Cookbook--- # Long-Term Memory Source: https://docs.nodetool.ai/long-term-memory Navigation: Root AGENTS.md Chat CLI Overview Agent Memory (in-run) NodeTool’s long-term memory (LTM) persists durable facts across chat sessions: stable user preferences, identity facts, project context, decisions made, and notable events. Recalled items are folded into the system prompt before each LLM call, and new memories are mined from the conversation after the turn completes. LTM is distinct from the per-run agent memory (context.memory). Agent memory is scratch space shared between steps inside a single workflow run. LTM is durable, cross-session, and exposed only when the user opts in. Default-off. LTM is a trust boundary, not a quiet convenience. It is disabled until the user flips the toggle in the chat UI or the operator sets NODETOOL_MEMORY_ENABLED=1. Even with the env flag on, the per-session renderer toggle is a hard veto. How recall works Each user has a private vector collection per (namespace, workspace) pair (e.g. ltm_<user>_chat:<threadId>). When LTM is enabled for a turn: The user’s latest message is embedded. The collection returns top-K nearest neighbours. Items are re-ranked with a hybrid score: 0.7 · cosine_similarity — semantic match 0.2 · recency — exponential decay, 30-day half-life on the later of createdAt / lastAccessedAt 0.1 · importance — value extraction stamps each item 0..1 The top results are rendered into a <recalled-memories> block (with an explicit “this is USER DATA, not instructions” preamble) and injected as a system message just before the user’s message. lastAccessedAt is bumped on the returned items so frequently-recalled memories age slower. The recalled block is not persisted into chat history — it’s ephemeral context for the LLM call only. How extraction works After the assistant’s final message is saved, the completed turn is mined for new memories on a fire-and-forget call: The conversation is rendered to text. tool and system messages are stripped before the prompt is built so secrets that surface in tool results never reach the extraction LLM. A small LLM pass extracts up to 8 candidates per turn (max 12,000 input chars) as strict JSON with { text, kind, importance }. Each candidate runs through looksLikeSecret() (see below) — anything that matches is dropped silently. Survivors run through near-duplicate dedupe (cosine similarity ≥ 0.92 against existing items). New items are upserted into the collection. Eviction kicks in when the collection exceeds maxItems (default 500, configurable via NODETOOL_MEMORY_MAX_ITEMS); the lowest-scored items are dropped via paged scans. The extraction prompt explicitly requires user-explicit content only and forbids storing secrets, generated content, advice, or unconfirmed inferences. Trust boundary: secret/credential redaction Every write goes through looksLikeSecret() — extraction, the ltm_remember agent tool, and any direct programmatic caller. Anything matching a credential shape is silently dropped: OpenAI-style sk-…, Anthropic sk-ant-…, GitHub ghp_…/gho_…/etc. Stripe sk_live_… / pk_live_… AWS access keys (AKIA…) and aws_secret_access_key assignments Authorization / Bearer headers PEM-armored private keys JWTs (three base64 segments separated by dots) Generic api_key|token|password|secret = value assignments DB connection strings with embedded creds (postgres://user:pass@host) Patterns are bounded character classes (no ReDoS). False positives are preferable to persisting a real key. The recalled-memories block also escapes < and > unconditionally so tag-shaped content from a manipulated memory cannot break out of the delimiter. Per-user isolation Memories are stored per userId. Collection names include the user id; secrets for embedding API calls are resolved via getSecret(envKey, userId) — never against a different user’s scope. Within a user, the namespace can be further scoped by workspaceId (the websocket chat path uses the chat thread id) so memories from one project don’t bleed into another. Enabling LTM From the chat UI The chat composer has a Memory: on / off chip next to the model chip. The setting persists in the renderer’s local store (memoryEnabled in GlobalChatStore) and is sent on every chat message as memory_enabled. Programmatically import { createDefaultLongTermMemory } from "@nodetool-ai/agents"; const memory = await createDefaultLongTermMemory({ userId, // required — memory is per-user namespace: "chat", // logical bucket workspaceId: threadId, // optional, appended to namespace extractionProvider: provider, // BaseProvider used to mine memories extractionModel: model, // model id for the extraction call enabled: true // explicit opt-in (overrides env) }); if (memory && memory.isReady()) { const recalled = await memory.recall(userInput); // hybrid-scored // ... render recalled into the prompt ... await memory.rememberConversation(messages); // fire-and-forget } createDefaultLongTermMemory returns null when: enabled !== true and NODETOOL_MEMORY_ENABLED is not truthy in the env enabled === false (caller veto) no embedding model can be resolved (no OPENAI_API_KEY / GEMINI_API_KEY / OLLAMA_API_URL / NODETOOL_MEMORY_EMBEDDING_MODEL) Agent tools Agents can drive the store directly via two auto-attached tools when LTM is wired into their session: ltm_recall(query, k?) — return ranked memories for a query. ltm_remember(text, kind?, importance?) — persist a fact. The same secret filter applies. Agent runs do not auto-mine the objective + final result by default. To re-enable that for a specific agent, pass autoPersistMemory: true in AgentOptions. Configuration reference Variable Default Effect NODETOOL_MEMORY_ENABLED unset (off) 1 / true / yes / on makes LTM the default-on for sessions that don’t pass enabled explicitly. 0 / false is a hard global veto. NODETOOL_MEMORY_EMBEDDING_MODEL auto Force an embedding model regardless of which provider keys are configured. NODETOOL_MEMORY_EMBEDDING_PROVIDER auto Pair with the model override above. NODETOOL_MEMORY_MAX_ITEMS 500 Soft cap per (user, namespace) collection. 0 disables eviction. NODETOOL_VECTOR_PROVIDER sqlite-vec Reroutes LTM along with every other vector consumer (Pinecone, Chroma, etc.). Storage backend LTM uses the shared VectorProvider abstraction — there is no SQLite-vec-specific code path. Switching the global vector provider via NODETOOL_VECTOR_PROVIDER reroutes LTM along with every other vector consumer in the system. Privacy and reset Memories live in the configured vector store, encrypted only if the underlying backend encrypts at rest. A user’s collection is named ltm_<user>_<namespace>[:workspaceId]. Dropping the collection (e.g. via the vector store’s admin tool) clears that user’s memories for that scope. Programmatically: memory.clear() drops the entire collection; memory.forget(id) removes a single item. Related Agent Memory — in-run scratch space (context.memory), tool-driven access. Different system from LTM. Chat CLI Overview — chat surfaces and composer. Models & Providers — embedding provider configuration.--- # Mobile App Source: https://docs.nodetool.ai/mobile-app.html Run Mini-Apps and chat with models from your phone or tablet. Connects to any NodeTool server — your desktop, a self-hosted instance, or NodeTool Cloud. New here? Start on desktop with Getting Started. Overview Feature Notes Chat Streaming responses across providers Mini-Apps Run your workflows with a simple UI Platforms iOS, Android, browser Server Connect to any NodeTool server Getting the App From App Stores (Coming Soon) The app will be available on: iOS: Apple App Store Android: Google Play Store For Developers Build and run from source: # Clone the NodeTool repository git clone https://github.com/nodetool-ai/nodetool.git cd nodetool/mobile # Install dependencies npm install # Start the development server npm start Then: Press i for iOS Simulator (macOS only) Press a for Android Emulator Press w for web browser Scan the QR code with Expo Go on your phone Connecting to Your Server The mobile app requires a running NodeTool server. Configure Server URL Open the app Go to Settings (gear icon) Enter your server URL Tap Test Connection Save when successful Server URLs by Platform Platform Server URL iOS Simulator http://localhost:7777 Android Emulator http://10.0.2.2:7777 Physical Device http://<your-computer-ip>:7777 Physical devices must be on the same network as your NodeTool server. AI Chat Chat with AI models from your mobile device. Features Streaming responses – See text appear in real-time Model selection – Choose from available AI models Markdown rendering – Code blocks, formatting, syntax highlighting Stop generation – Tap stop to cancel long responses Multiple threads – Keep separate conversation topics How to Chat Tap the Chat button on the Workflows list screen Select a model (tap model name) Type your message Tap Send Watch the AI respond in real-time Tips Use the + button to start a new conversation Tap Stop to halt long responses Scroll up to review conversation history Switch models mid-conversation if needed Mini Apps Run your NodeTool workflows with a simplified mobile interface. There is no separate “Mini Apps” screen — the home screen is the Workflows list (WorkflowsListScreen), and tapping a workflow opens the graph editor (GraphEditorScreen), which hosts the Mini-App runner for that workflow. What Are Mini Apps? Mini Apps are workflows converted to simple interfaces. They hide the complexity of the workflow and show only: Input fields Run button Results Running a Mini App Open the Workflows list (the home screen) Tap a workflow to open it Fill in the inputs (text, number, boolean toggles, image, audio, …) Tap Run View results below Supported Input Types Type Description Text Single or multi-line text input Number Integer or float values Boolean On/off toggle switches Image Image input Audio Audio input File path File reference Mobile Graph Editor The mobile app includes a touch-friendly version of the workflow editor. Workflows render as a vertical chain of cards that you scroll through — there is no free-form pan-and-zoom canvas. Overview Empty State New workflows open with a single prompt to add your first node. Node Picker Tap the + button to open the full-screen node picker, which filters by input/output compatibility. Linear Chain Workflows render as a vertical chain of cards on mobile — easier to scroll and tap. Interactions: Action Result Tap a card Open its properties Up / Down buttons on a card Reorder it within the chain Duplicate / Remove buttons Duplicate or delete the node + button Add a node via the full-screen picker Mobile Settings Configure the mobile app from the gear icon: Section Purpose Appearance Light / Dark / System theme Server URL + Test Connection Which NodeTool server to talk to, with a one-tap connectivity check Manage Shortcuts to API Keys, Collections, and Jobs Account Signed-in email and Sign Out About App version, build info, links Mobile Language Model Selection Tapping the model name at the top of a chat opens a two-step picker: Select a provider — the providers your server reports as supporting message generation. Select a model — the models offered by that provider. A search box appears in either step once the list is long enough, and a back arrow returns from models to providers. There is no API-key gating, disabled styling, or docs links in this picker. Screens The app is made up of 12 screens: Login, Workflows list, Graph editor, Settings, Chat, Language Model Selection, Assets, Asset Viewer, Secrets, Collections, Jobs, and Threads. Server Requirements Your NodeTool server must be: Running – Start with nodetool serve --port 7777 Accessible – On same network as your device Configured – With the models you want to use Starting the Server # Activate environment conda activate nodetool # Start server nodetool serve --port 7777 The server runs at http://localhost:7777 by default. Firewall Settings If connecting from a physical device, ensure: Port 7777 is open on your computer’s firewall Your router allows local network connections Both devices are on the same WiFi network Troubleshooting Cannot Connect to Server Symptoms: “Connection failed” or timeout errors Solutions: Verify the server is running Check the server URL in Settings For physical devices: Use your computer’s IP address (not localhost) Ensure same WiFi network Check firewall settings Android Emulator Connection Issues Problem: Cannot reach localhost Solution: Use http://10.0.2.2:7777 instead of localhost:7777. This is Android’s special IP for the host machine. App Crashes on Startup Solutions: Clear app data and restart Check that all dependencies are installed: npm install Reset Metro bundler: npm start --reset-cache Chat Not Streaming Symptoms: Responses appear all at once Solutions: Check WebSocket connection Verify server is running current version Try a different AI model Platform Notes iOS Requires Xcode for development (macOS only) Use iOS Simulator for testing Production builds via EAS Build Android Requires Android Studio for development Use Android Emulator for testing Use 10.0.2.2 for localhost access Web Works in any modern browser Good for testing without mobile device Run with npm run web Building for Production Use Expo’s EAS Build for production apps: # Install EAS CLI npm install -g eas-cli # Log in to Expo eas login # Build for Android eas build --platform android --profile preview # Build for iOS eas build --platform ios --profile preview For store submissions: # Google Play Store (AAB format) eas build --platform android --profile production # Apple App Store eas build --platform ios --profile production See EAS Build Documentation for details. Related Topics Getting Started – Desktop setup and first workflow User Interface – Full UI guide Mini Apps – Creating Mini Apps Chat & Agents – Chat features in detail API Reference – Server API documentation--- # Models & Providers Source: https://docs.nodetool.ai/models-and-providers.html Overview / start here. For the full catalog of model families see Supported Models; for connecting and configuring each provider see Providers; for the desktop download panel see Models Manager. Run models locally, through cloud APIs with your own keys, or both in the same graph. Local vs. cloud Local Data stays on your disk Free after download Works offline 4–20 GB per model; needs a capable GPU or Apple Silicon Cloud (BYOK) No download Runs on the provider’s hardware Latest model releases Billed by the provider, at the provider’s price — no NodeTool markup Requires internet; data goes to the provider Mixed Pick the best provider per node: ASR (Whisper) — local for sensitive audio Image generation — Flux locally for control, FAL/KIE cloud for speed Document processing — local for confidential files Cloud models NodeTool reaches thousands of cloud models across 30+ providers, all through the same generic nodes. The catalog below is grouped by what you’re generating — text, images, video, speech and music, 3D, embeddings. Every model runs on your own API key (BYOK): you’re billed by the provider at the provider’s price, with no NodeTool markup. Add a key in Settings → Providers and the models show up in the node’s model dropdown. Chat models are fetched live from each provider’s API, so a provider’s list always reflects its newest releases; the families below are the ones you’ll find there. Image, video, audio, and 3D models are drawn from the manifests each provider node package ships (packages/*-nodes/), so they track what NodeTool actually exposes. Text & chat models (LLMs) The generic nodetool.agents.Agent and chat nodes route to whichever provider owns the model you pick. Swap claude-opus for gpt-5 or gemini-3.1-pro-preview without changing the rest of the graph. Provider Model families OpenAI GPT-5, GPT-5 mini, GPT-5 nano, GPT-4.1, GPT-4o, o-series reasoning models Anthropic Claude Opus 4.x, Claude Sonnet 4.x, Claude Haiku 4.x, Claude 3.5/3.7 Sonnet Google Gemini Gemini 3.5 Flash, Gemini 3.1 Pro, Gemini 3.1 Flash-Lite, Gemini 2.5 Pro xAI Grok 4, Grok 3, Grok Code DeepSeek DeepSeek-V3, DeepSeek-R1 (reasoning) Groq Llama, Qwen, GPT-OSS, Kimi K2 on LPU inference Mistral Mistral Large, Mistral Medium, Mistral Small, Mixtral, Codestral, Magistral Cerebras Llama, Qwen, GPT-OSS on wafer-scale inference GMI Cloud Llama, DeepSeek, Qwen (open-weight) Moonshot Kimi K2, Kimi latest MiniMax MiniMax-Text, MiniMax M2 OpenRouter 300+ models proxied through one key (Claude, GPT, Gemini, Llama, Qwen, DeepSeek, …) Together AI Llama, Qwen, DeepSeek, Mixtral, GLM, Kimi, and more open models Evolink GPT, Claude, Gemini, DeepSeek through one gateway key kie.ai GPT-5.5, Claude Opus/Sonnet/Haiku 4.x, Gemini 3.1 Pro (chat gateway) Codex (OpenAI OAuth) GPT chat models via your ChatGPT/Codex login Claude Agent SDK Claude via your local claude CLI subscription Ollama · vLLM · LM Studio · llama.cpp Any local open-weight model — Llama, Qwen, Gemma, DeepSeek, Mistral, Phi, GPT-OSS HuggingFace Chat inference over Hub models (500,000+) Reasoning, tool calling, and vision input are exposed where the provider supports them. For local text generation without an API key, see Supported Models (llama.cpp, MLX, Transformers). Image generation models Generate and edit images through nodetool.image.TextToImage and nodetool.image.ImageToImage. Flagship models Model Provider Capabilities Key features Black Forest Labs FLUX.2 BFL T2I with control Photoreal images, multi-reference consistency, accurate text, flexible control Google Nano Banana 2.0 / Pro Google High-res T2I/Edit 2K output, 4K upscaling, better text and character consistency GPT Image 2 OpenAI T2I/Edit Photorealistic generation and instruction-based editing Ideogram V3 Ideogram T2I/Edit Typography rendering and artistic style control Seedream 4.5 ByteDance T2I/Edit High-fidelity generation and instruction-based editing Z-Image Turbo Z-AI T2I Fast generation with strong prompt adherence Full catalog by provider Provider Image models OpenAI GPT Image 2, GPT Image 1.5, GPT Image 1, GPT Image 1 mini Google Gemini 3.1 Flash Image (Nano Banana 2), Gemini 3 Pro Image (Nano Banana Pro) Black Forest Labs FLUX.1 [schnell], FLUX.1 [dev], FLUX.1 Pro, FLUX 1.1 Pro, FLUX 1.1 Pro Ultra, FLUX.2 [dev/flex/pro/max], FLUX.2 Klein, FLUX Kontext [pro/max], FLUX Fill, FLUX Canny, FLUX Depth, FLUX Redux, FLUX PuLID Stability AI Stable Diffusion 1.5, SD 2.1, SDXL, SDXL Lightning, Stable Diffusion 3 Medium, SD 3.5 [medium/large/large turbo], Stable Cascade Qwen Qwen Image, Qwen Image Edit (2509/2511/Plus), Qwen Image Max, Qwen Image Layered ByteDance Seedream 3.0, Seedream 4.0, Seedream 4.5, Seedream 5 Lite Ideogram Ideogram V2, V2 Turbo, V2a, V3 [balanced/quality/turbo], Ideogram Character Recraft Recraft V3, Recraft 20B, Recraft V4, Recraft V4 Pro (raster + SVG) Others Kolors, HiDream, Hunyuan Image 3, OmniGen v1/v2, Sana, PixArt-Σ, Aura Flow, CogView4, Luma Photon, GLM Image, Reve, Bria, ERNIE Image, Emu 3.5, Lumina, F-Lite, Fibo, Playground v2.5, Juggernaut, DreamShaper, Proteus, Recraft, Chroma, Janus, MiniMax Image-01, xAI Grok Imagine Upscale & restore Topaz, Real-ESRGAN, ESRGAN, SUPIR, Clarity Upscaler, SeedVR, GFPGAN, CodeFormer, AuraSR, SwinIR, DDColor Access aggregators — FAL (450+ image endpoints), Replicate (170+), and kie.ai — carry most of these families plus hundreds of community fine-tunes, LoRAs, and control variants. Video generation models Generate video through nodetool.video.TextToVideo and nodetool.video.ImageToVideo. Flagship models Model Provider Capabilities Key features OpenAI Sora 2 Pro OpenAI T2V/I2V up to 15s Realistic motion, refined physics, synchronized native audio, 1080p Google Veo 3.1 Google T2V/I2V with references Multi-image references, extended length, native 1080p with synced audio ByteDance Seedance 2.0 ByteDance T2V/I2V Cinematic video with stable characters and smooth motion Runway Gen-4 / Aleph Runway T2V/I2V/Extend Precise motion control and high fidelity Luma Ray 2 Luma AI T2V/I2V/edit Fast generation and creative editing xAI Grok Imagine xAI T2V/I2V/T2I Coherent motion with synchronized audio Alibaba Wan 2.6 Alibaba Multi-shot T2V/I2V Affordable 1080p, stable characters, native audio MiniMax Hailuo 2.3 MiniMax High-fidelity T2V/I2V Expressive characters, complex motion and lighting Kling 3.0 Kling T2V/I2V with audio Synchronized speech, ambient sound, and effects Full catalog by provider Provider Video models OpenAI Sora 2, Sora 2 Pro Google Veo 3.1, Veo 3.1 Fast, Veo 3.1 Lite Kling Kling 2.1 [standard/pro/master], Kling 2.5 Turbo Pro, Kling 2.6, Kling 3.0, Kling Avatar, Kling Lip Sync ByteDance Seedance 1.0 [lite/pro], Seedance 1.5 Pro, Seedance 2.0 Alibaba Wan 2.1, Wan 2.2, Wan 2.5, Wan 2.6, Wan VACE MiniMax Hailuo 02, Hailuo 2.3, Video-01 [director/live] Runway Gen-3 Alpha, Gen-4, Gen-4 Turbo, Gen-4.5, Aleph Luma AI Dream Machine, Ray 2 [540p/720p], Ray Flash 2 PixVerse PixVerse v4, v4.5, v5, v5.6, v6 Vidu Vidu 2.0, Vidu Q1 Others LTX Video, LTX-2, Hunyuan Video (+ v1.5, Avatar, Foley), Mochi v1, CogVideoX-5B, Marey, Lucy, Magi, Kandinsky 5, Sana Video, SkyReels, Stable Video Diffusion, Zeroscope, Pika, xAI Grok Imagine Lip sync & avatars LatentSync, Lipsync 2 [pro], MultiTalk, OmniHuman, SadTalker, Live Portrait, Stable Avatar, DreamActor, InfiniteTalk Upscale Topaz Video, SeedVR, Crystal Video Upscaler Speech, audio & music models Generate speech through nodetool.audio.TextToSpeech, transcribe with nodetool.text.AutomaticSpeechRecognition, and create music through provider music nodes. Text-to-speech (TTS) Provider TTS models OpenAI TTS 1, TTS 1 HD, gpt-4o-mini-tts ElevenLabs Eleven v3, Turbo v2.5, Flash v2.5, Multilingual v2 Google Gemini TTS, Gemini 3.1 Flash TTS MiniMax Speech 02 [hd/turbo], Speech 2.6, Speech 2.8 Open models Kokoro, Orpheus, Chatterbox [HD/Pro/Turbo/Multilingual], Dia, F5-TTS, XTTS v2, VibeVoice, Zonos, CSM-1B, Index-TTS 2, Qwen 3 TTS, Maya, StyleTTS2, Parler-TTS, Tortoise, VoiceCraft, OpenVoice, Cartesia Sonic Speech recognition (ASR) Provider ASR models OpenAI Whisper Google Gemini audio transcription ElevenLabs Scribe (speech-to-text) Together AI Whisper Large v3, Voxtral, Parakeet HuggingFace Whisper and other ASR models, plus MLX Whisper locally Music & sound Provider Music & sound models Suno (via kie.ai) Suno — song generation, extend, cover, remix ElevenLabs V3 Dialogue, Sound Effects MiniMax Music 01, Music 1.5, Music 2.6 Google Lyria 3 Clip, Lyria 3 Pro Open models MusicGen, Stable Audio 2.5, ACE-Step, DiffRhythm, YuE, Riffusion, Flux Music, MMAudio, ThinkSound 3D generation models Generate 3D assets through nodetool.3d.TextTo3D and nodetool.3d.ImageTo3D. Model Provider Capabilities Key features Meshy AI Meshy (MESHY_API_KEY) T2M/I2M Textured mesh generation Rodin AI Rodin (RODIN_API_KEY) T2M/I2M High-fidelity 3D creation Open 3D families — Hunyuan3D (v2.1/v3), Trellis / Trellis 2, TripoSR / Tripo, Shap-E, Point-E, OmniPart, Era3D — run through HuggingFace / base-node 3D nodes (HFTextTo3D, HFImageTo3D) and FAL/Replicate rather than dedicated runtime providers. See Providers for details. Embedding models Power RAG and semantic search through embedding nodes. Provider Embedding models OpenAI text-embedding-3-small, text-embedding-3-large, ada-002 Google gemini-embedding-2 Mistral mistral-embed Cohere embed-v4.0, embed-english-v3.0, embed-multilingual-v3.0 Voyage AI voyage-3.5 and the Voyage line Jina AI jina-embeddings-v3 Together AI m2-bert, BGE, and other open embedding models Ollama · HuggingFace nomic-embed, mxbai-embed, sentence-transformers (local, no key) Using these models Access these models through NodeTool’s generic nodes: For Video: Use nodetool.video.TextToVideo or nodetool.video.ImageToVideo For Images: Use nodetool.image.TextToImage For 3D: Use nodetool.3d.TextTo3D or nodetool.3d.ImageTo3D For Music: Use kie.ai-backed Suno nodes (Suno Generate, Extend, Cover) Select Provider: Click the model dropdown in the node properties Configure API: Add provider API keys in Settings → Providers Access via kie.ai (recommended for broad model support): Many of these models are available through kie.ai, an AI provider aggregator that often offers competitive or lower pricing compared to upstream providers. Configure using KIE_API_KEY in Settings → Providers Access via fal.ai: Configure using FAL_API_KEY in Settings → Providers Cost Considerations: Cloud models typically charge per generation. Check each provider’s pricing before extensive use. Local models are free after download but require capable hardware. Getting Started Option 1: Start with Local Models (Recommended) Open Models → Model Manager in NodeTool Install these starter models: GPT-OSS (~4 GB) – Text generation and chat Flux (~12 GB) – High-quality image generation Wait for downloads to complete Run templates – they’ll work offline! Option 2: Start with Cloud Providers Get an API key from a provider: OpenAI – GPT, Whisper, Sora Anthropic – Claude models Google – Gemini models In NodeTool, go to Settings → Providers Paste your API key Select the provider when using AI nodes Detailed Guides General Models Manager – Download and manage AI models Getting Started – First workflow Local AI Supported Models – List of local models (llama.cpp, MLX, Whisper, Flux) Cloud AI Providers Guide – Set up OpenAI, Anthropic, Google HuggingFace Integration – Access 500,000+ models Advanced Self-Hosted Deployment – Secure deployments Deployment Guide – Cloud infrastructure--- # Models Manager Source: https://docs.nodetool.ai/models-manager.html Desktop app panel for downloading and managing local models. For the model catalog see Supported Models; for provider keys see Providers. The Models Manager helps you browse, download, and manage AI models available on your system. Opening the Manager The Models Manager is a full page at the /models route — open it from the app navigation. It shows all downloaded models, recommended models, and available models from configured providers. Browsing Models Filter by Type The sidebar on the left filters by HuggingFace pipeline tag rather than simplified labels. Tags include, for example: Pipeline tag Example Use text-generation Text generation, chat, reasoning text-to-image Text-to-image generation image-to-image Image transformation text-to-video Text-to-video generation automatic-speech-recognition Transcription, dictation text-to-speech Voice synthesis, narration feature-extraction Embeddings / vector search The available tags reflect what’s actually present in your model list; each shows a count of matching models. Search and Sort Search by name or repository to quickly find specific models Favorites – Star frequently used models for quick access Recent – See models you’ve used recently Downloading Models Find the model you want in the browser Click Download to start fetching it to your local cache Track progress in the Downloads bar at the bottom of the screen Download Details Downloads continue in the background while you navigate the app The bottom bar shows total progress, speed, and estimated time remaining Click the Downloads bar to expand and see individual file progress The download WebSocket reconnects automatically on connection loss (up to 5 attempts with exponential backoff); this reconnection is separate from any per-file download behavior Storage Location Downloaded models are stored in your local HuggingFace cache (~/.cache/huggingface/) or provider-specific locations (e.g., ~/.ollama/ for Ollama models). Managing Models Per-Model Actions Download – Fetch a model to your local cache Delete – Remove a model you no longer need to free disk space Show in Explorer – Open the model folder on your computer README – Read the model’s documentation on Hugging Face Recommended Models Many workflow nodes specify recommended or required models. The Models Manager highlights these under a Recommended section with direct install links, so you can quickly get the models your workflows need. Model Selection Dialogs Each property role has a type-aware picker: Language Model — LLM selector with provider grouping Image Model — image-generation models only Video Model — video-generation models only TTS / ASR Model — speech models only Embedding Model — vector embedding models only HuggingFace Model — search any HF repo Cloud Provider Models Models from cloud providers (OpenAI, Anthropic, Google, etc.) appear in the manager based on your configured API keys. These don’t require downloading – they run remotely when you use them in workflows. Configure API keys in Settings > Providers. See Models & Providers for setup details. Next Steps Models & Providers – Configure providers and API keys Installation – Hardware requirements for local models HuggingFace Integration – Browse and use HuggingFace models--- # Supported Models Source: https://docs.nodetool.ai/models.html The model catalog. For how to connect a provider, see Providers. For the desktop app’s download panel, see Models Manager. New here? Start with Models & Providers. NodeTool runs models from many providers — proprietary and open. Generic nodes (TextToImage, Agent, RealtimeAgent, …) work across providers, so swapping a model doesn’t change the graph. Local inference engines 1,655+ local models across the engines below. For provider-based local inference (Ollama, vLLM), please refer to the Providers documentation. llama.cpp & GGUF Format llama.cpp is a highly optimized C/C++ inference library that enables efficient LLM inference on CPU and GPU hardware using the GGUF format. It supports 1.5-bit through 8-bit integer quantization for significantly reduced memory usage. Models: Supports 300+ GGUF quantized models including Qwen, Llama, Gemma, DeepSeek, and GPT variants. MLX Framework (Apple Silicon) MLX is Apple’s open-source machine learning framework specifically optimized for Apple Silicon’s unified memory architecture. It enables efficient on-device AI for Mac users. Capabilities: LLMs: Native optimization for Llama, Qwen, Mistral, and others. Vision: Multimodal models and FastVLM support. Image Gen: FLUX models ported to MLX for faster generation. Nunchaku (NVIDIA GPU) Nunchaku is a high-performance inference engine specifically designed for 4-bit diffusion models on NVIDIA GPUs. It implements SVDQuant to maintain visual fidelity while reducing memory usage by 3.6x compared to BF16 models. It is ideal for running large diffusion models (like FLUX.1) on consumer NVIDIA GPUs. HuggingFace Transformers Transformers is the standard library for working with ML models across text, vision, audio, and multimodal tasks. It provides access to the HuggingFace Hub with over 500,000 pre-trained models and supports automatic device detection (GPU/Apple Silicon/CPU). Comparison Matrix Framework Throughput Memory Efficiency Ease of Use Best Hardware Use Case llama.cpp Medium Excellent Medium CPU, GPU Quantized models, edge devices MLX Good Excellent Good Apple Silicon Mac, iOS, privacy Nunchaku Excellent Excellent Medium NVIDIA GPU High-performance Diffusion Transformers Medium Good Excellent Any Research, flexibility Supported Model Types NodeTool supports a wide range of model types across different domains. Below is an overview of the supported types and their available execution variants. Variants Key Full Precision: Standard execution using HuggingFace Transformers/Diffusers (supports CUDA, MPS, CPU). MLX: Optimized execution for Apple Silicon (M-series chips). Nunchaku: High-performance 4-bit quantization for NVIDIA GPUs. Image Generation Model Type Description Variants Flux Text-to-image generation ✅ Full Precision✅ MLX✅ Nunchaku Flux Fill Inpainting/Outpainting for Flux ✅ Full Precision✅ MLX Flux Depth Depth-guided generation ✅ Full Precision✅ MLX Flux Redux Image variation and mixing ✅ Full Precision✅ MLX Flux Kontext Context-aware generation ✅ Full Precision✅ MLX Stable Diffusion XL SDXL base and refiner models ✅ Full Precision✅ Nunchaku Stable Diffusion 3 Latest Stable Diffusion architecture ✅ Full Precision Stable Diffusion SD 1.5, 2.1, and variants ✅ Full Precision Qwen Image Qwen-based text-to-image ✅ Full Precision✅ MLX✅ Nunchaku Qwen Image Edit Instruction-based image editing ✅ Full Precision✅ MLX ControlNet Structural guidance (Canny, Depth, etc.) ✅ Full Precision✅ MLX (Flux) Text to Image Generic text-to-image models ✅ Full Precision Image to Image Image transformation models ✅ Full Precision Inpainting Mask-based image editing ✅ Full Precision Vision & Video Model Type Description Variants Image Text to Text Vision-Language Models (VLM) ✅ Full Precision✅ MLX (Qwen2-VL) Visual QA Visual Question Answering ✅ Full Precision Document QA Document understanding and QA ✅ Full Precision OCR Optical Character Recognition (GOT-OCR, etc.) ✅ Full Precision Depth Estimation Monocular depth estimation ✅ Full Precision Image Classification Categorize images ✅ Full Precision Object Detection Detect objects in images ✅ Full Precision Image Segmentation Pixel-level segmentation ✅ Full Precision Zero-Shot Detection Open-vocabulary detection ✅ Full Precision Mask Generation Segment Anything (SAM) variants ✅ Full Precision Video Classification Categorize video content ✅ Full Precision Text to Video Generate video from text ✅ Full Precision Image to Video Animate images ✅ Full Precision Text to 3D Generate 3D assets from text ✅ Full Precision Image to 3D Generate 3D assets from images ✅ Full Precision Natural Language Processing Model Type Description Variants Text Generation LLMs (Llama, Qwen, Mistral, etc.) ✅ Full Precision✅ MLX Text to Text T5, BART, and seq2seq models ✅ Full Precision Summarization Text summarization ✅ Full Precision Translation Machine translation ✅ Full Precision Question Answering Extractive QA ✅ Full Precision Text Classification Sentiment analysis, etc. ✅ Full Precision Token Classification NER, POS tagging ✅ Full Precision Zero-Shot Class. Open-vocabulary classification ✅ Full Precision Sentence Similarity Semantic similarity / Embeddings ✅ Full Precision Reranker Search result reranking ✅ Full Precision Feature Extraction General embeddings ✅ Full Precision Fill Mask BERT-style masked modeling ✅ Full Precision Audio Model Type Description Variants Text to Speech Generate speech from text ✅ Full Precision✅ MLX Speech Recognition ASR (Whisper, etc.) ✅ Full Precision✅ MLX Audio Classification Categorize audio events ✅ Full Precision Voice Activity VAD (Silero, etc.) ✅ Full Precision Audio to Audio Voice conversion, enhancement ✅ Full Precision Components & Adapters Model Type Description Variants LoRA Low-Rank Adaptation weights ✅ Full Precision (SD, SDXL, Qwen) IP Adapter Image Prompt Adapters ✅ Full Precision VAE Variational Autoencoders ✅ Full Precision CLIP Text/Image Encoders ✅ Full Precision T5 Encoder Text Encoders for diffusion ✅ Full Precision RealESRGAN Image Upscaling ✅ Full Precision Cloud Models In addition to local models, NodeTool provides access to cloud-based models through provider integrations. These models offer the latest capabilities in video, image, and audio generation. Video Generation (Cloud) Model Provider Key Features Resolution Max Duration Sora 2 Pro OpenAI Realistic motion, refined physics, native audio 1080p 15s Veo 3.1 Google Realistic motion, multi-image refs, synced audio 1080p Extended Seedance 2.0 ByteDance High-quality cinematic video, stable characters 1080p Variable Runway Gen-3 Alpha Runway Precise motion control, professional fidelity 1080p Variable Runway Aleph Runway Next-gen Runway video generation 1080p Variable Luma Luma AI AI-powered video modification and editing 1080p Variable Grok Imagine xAI Multimodal T2V/I2V with coherent motion 1080p Short clips Wan 2.6 Alibaba Multi-shot, stable characters, affordable 1080p Variable Hailuo 2.3 MiniMax Expressive characters, complex lighting 1080p+ Variable Kling 3.0 Kling Synced speech & effects, audio-visual coherence 1080p Variable Access via: nodetool.video.TextToVideo, nodetool.video.ImageToVideo nodes Image Generation (Cloud) Model Provider Key Features Output Quality FLUX.2 Pro Black Forest Labs Photoreal, multi-reference consistency, accurate text High Nano Banana 2.0 Google 2K native, 4K scaling, enhanced text & characters Very High GPT Image 2 OpenAI Photorealistic generation and instruction-based editing High Ideogram V3 Ideogram Exceptional typography, artistic style control High Z-Image Turbo Z-AI Fast generation with strong prompt adherence High Seedream 4.5 ByteDance High-fidelity generation and instruction-based editing High Imagen 4 Google Ultra-detailed photorealistic images Very High Access via: nodetool.image.TextToImage node Music & Audio Generation (Cloud) Model Provider Key Features Suno Suno Full song creation from text, extend/cover/remix, instrumental support ElevenLabs V3 Dialogue ElevenLabs Multi-speaker dialogue with emotional control ElevenLabs TTS Turbo 2.5 ElevenLabs Ultra-fast, natural text-to-speech ElevenLabs Sound Effect ElevenLabs Generate sound effects and ambient audio from text Access via: nodetool.audio.TextToSpeech node; Suno and ElevenLabs advanced features via kie.ai Advantages of Cloud Models Latest Technology: Access to newest architectures and training data No Local Resources: Run on any hardware without GPU requirements Instant Availability: No download or installation needed Continuous Updates: Models improve without local updates Considerations API Costs: Per-generation pricing varies by provider Internet Required: Cannot run offline Data Privacy: Content is processed on provider servers Rate Limits: Subject to provider API quotas Cost-Effective Alternative: kie.ai All the cloud models listed above are available through kie.ai, an AI provider aggregator that: Offers unified access to multiple providers through a single API Often provides competitive or lower pricing than upstream providers Simplifies API key management (one key for all models) Enables easy cost comparison and optimization across providers Important: Many models (ByteDance Seedance, Runway, Luma, xAI Grok Imagine, Alibaba Wan 2.6, Kling 3.0, Ideogram V3, Z-Image Turbo, Suno) currently require kie.ai for access. Models with direct NodeTool API key support include OpenAI Sora 2 Pro, Google Veo 3.1, MiniMax Hailuo 2.3, and OpenAI GPT Image 2. This can be particularly beneficial for workflows using multiple SOTA models from different providers. For detailed provider configuration and usage, see the Providers Guide.--- # Real-Time Collaborative Workflow Editing (“Multiplayer”) Source: https://docs.nodetool.ai/multiplayer-design.html Real-Time Collaborative Workflow Editing (“Multiplayer”) Author: Matti Georgi Status: Draft — for review Last updated: 2026-07-10 Reviewers: TBD Implementation plan: multiplayer-implementation-plan.md Summary NodeTool’s workflow editor is single-writer. Two people who open the same workflow silently overwrite each other; the only concurrency mechanism we have is a coarse “this resource changed, refetch it” push that discards the loser’s work. This doc proposes making the workflow canvas multiplayer: shared live editing with presence (cursors, selections), per-user undo, and convergent conflict resolution, built on a CRDT (Yjs) synchronized over our existing WebSocket server and persisted in our existing database. The central architectural decision is that the CRDT is an implementation detail of the editor, not a new source of truth for the platform. The workflow’s materialized graph JSON remains the interface for the runner, the REST/tRPC API, the CLI, version history, export, and validation. Everything outside the editor keeps working unchanged. That constraint is what makes this project a bounded, reversible investment rather than a rewrite. Estimated cost: ~2 engineer-weeks for Phase 0 (presence + write-conflict safety), ~7 engineer-weeks for Phase 1 (CRDT co-editing). Phase 0 ships value on its own and is a prerequisite for nothing; if Phase 1 is cancelled, Phase 0 stands. 1. Context NodeTool is a visual AI workflow platform: a React/ReactFlow editor over an actor-model execution kernel, deployed both as a desktop app (Electron, local single-user server) and as a hosted service (Fly.io, api.nodetool.ai). Over the last several quarters the platform grew the full perimeter of a collaborative product: Hosted multi-user deployment with auth and public/private workflow access (packages/websocket/src/trpc/routers/workflows.ts). Version history with manual and autosave snapshots (packages/models/src/schema/workflow-versions.ts, web/src/components/version/). Sharing surfaces: public workflows, mini-apps, portable .nodetool bundles. A live push channel: the server observes model changes and pushes resource-change events over WebSocket; the client invalidates TanStack Query caches (web/src/stores/resourceChangeHandler.ts). What is missing is the center of that perimeter: two people cannot work on the same canvas. Saves are last-write-wins. There is no presence, no awareness, no merge. The resource-change channel makes this worse in one respect: user B’s screen refetches and discards their in-progress edits when user A saves. AI workflows are collaborative artifacts by nature — a prompt author, a domain expert, and an engineer iterating on the same graph is the normal working mode, not the exception. Every adjacent category has already internalized this (Figma for design canvases; n8n shipped collaborative editing for workflow graphs). For a product with a hosted offering, multiplayer is not a feature among features; it is the difference between a tool and a platform. 1.1 Current editing pipeline (what we’d be changing) Each open workflow gets its own Zustand store created by createNodeStore (web/src/stores/NodeStore.ts), holding ReactFlow nodes/edges. Undo/redo is client-local via the zundo temporal middleware. Saves go through tRPC mutations to the workflows router, which writes the whole graph JSON column. No optimistic-concurrency check: the write path accepts whatever graph the client sends. Version snapshots (save_type: manual | autosave) are rows in nodetool_workflow_versions keyed by workflow. All realtime traffic (runs, chat, resource changes) flows through one WebSocket (/ws, Fastify + @fastify/websocket, MsgPack frames) behind the client’s GlobalWebSocketManager singleton. These are good bones. The transport, auth, persistence, and versioning layers all exist; we are adding a synchronization layer, not infrastructure. 2. Goals G1 — Convergent co-editing. N users edit the same workflow concurrently; all replicas converge to the same graph without lost updates. Target p95 remote-edit visibility < 500 ms on the hosted deployment. G2 — Presence. Live cursors, selections, and a “who’s here” roster, with join/leave latency < 2 s. G3 — Per-user undo. Undo reverts my operations only, never a collaborator’s. G4 — Zero blast radius outside the editor. No changes to the execution kernel, REST/tRPC API shapes, CLI, DSL export, bundles, validation, or mini-apps. The materialized graph JSON remains authoritative for all of them. G5 — Runs on our actual infrastructure. SQLite and Postgres schema parity, single-node Fly deployment, self-hostable with no new external services. G6 — Reversible rollout. Feature-flagged; disabling the flag loses no data and returns to today’s behavior. Non-goals Collaborative text editing inside prompt fields (character-level merge via Y.Text). Phase 2 candidate; v1 treats each property value as an atomic register. Comments, annotations, follow-mode, emoji reactions. Presence primitives make these cheap later; they are product work, not sync work. Offline-first editing with deferred merge. Desktop users editing purely local workflows are out of scope; collab requires a live connection to the owning server. (The CRDT gives us most of the machinery for this later; we are deliberately not paying the UX cost now.) Horizontal scale-out of the sync server. We run one Fly machine. The design documents the scale-out path (§7.4) but does not build it. Shared execution sessions. Runs stay per-user (per-user generations, per-user results). Broadcasting run status to collaborators is cheap and included; sharing run outputs across users touches asset ownership and is deferred. A new permissions system. We reuse access: public | private plus one additive collaborators table (§5.6). Org/team modeling is its own project. 3. Design overview One page, top to bottom: Document model. Each collaborative workflow has a Yjs Y.Doc: nodes and edges as Y.Maps keyed by id, node properties as nested Y.Maps with last-writer-wins leaf values. This mirrors the existing graph JSON one-to-one. Transport. A new MsgPack message family (collab_*) on the existing /ws socket: join/leave, Yjs sync protocol (state vectors + updates), and awareness (presence) frames. No new port, no new server process, no new auth path. Server. A CollabRoomManager in packages/websocket: one in-memory room per open workflow, holding the loaded Y.Doc, broadcasting updates to members, appending updates to the database, and periodically (a) compacting the update log into snapshots and (b) flushing the materialized graph JSON through the existing save path — which keeps versions, resource-change events, and every downstream consumer working untouched. Client. A YjsGraphBinding that two-way binds the Y.Doc to the existing per-workflow NodeStore. ReactFlow and every component above it are unaware anything changed. Undo/redo switches from zundo to Y.UndoManager (scoped to local origin) when a workflow is in collab mode. Persistence. Two additive tables: an append-only update log and periodic snapshots. Recovery = load latest snapshot + replay tail. The materialized JSON is always a valid fallback: worst case, we discard the CRDT state and re-seed it from workflow.graph, losing nothing but sub-second granularity. Phasing. Phase 0 — Safety + presence (no CRDT). Optimistic-concurrency check on workflow saves (reject stale writes instead of clobbering), presence roster + live cursors over an awareness channel, and a “someone else is editing” banner. Small, independently shippable, kills the silent-data-loss bug on day one. Phase 1 — CRDT co-editing. Everything above. Phase 2 (sketch only). Y.Text for prompt properties, comments, follow-mode, offline merge. ┌────────────┐ Yjs updates + awareness ┌─────────────────────────┐ │ Client A │◄────────────────────────────►│ /ws (existing socket) │ │ NodeStore │ │ CollabRoomManager │ │ ▲ binding│ │ ┌───────────────────┐ │ │ Y.Doc │ │ │ Room: workflow_id │ │ └────────────┘ │ │ Y.Doc (memory) │ │ ┌────────────┐ │ └───┬─────────┬─────┘ │ │ Client B │◄────────────────────────────►│ │ │ │ └────────────┘ └──────┼─────────┼────────┘ append│ │debounced flush ▼ ▼ crdt_updates/ workflow.graph JSON crdt_snapshots (unchanged interface: (new tables) runner, API, versions, export, validate, CLI) 4. Why a CRDT, and why Yjs The decision that shapes everything else. Stating the reasoning explicitly because it’s the part most likely to be re-litigated: A workflow graph under concurrent editing has exactly the conflict profile CRDTs are good at: a keyed collection of objects (nodes, edges) where concurrent operations mostly touch different keys (you move your node, I edit my prompt), and where the rare same-key conflict has an acceptable arbitrary-but-deterministic resolution (last writer wins on a property value). We do not need server-ordered intent preservation (OT’s strength, at the cost of a server that must serialize and transform every operation); we need convergence, offline-tolerant buffering during reconnects, and per-key merge — CRDT territory. Yjs specifically, over Automerge: Yjs is the ecosystem default (mature, MIT, ~10 years of production use, the engine behind most collaborative ReactFlow implementations), has an order-of-magnitude edge in update size and apply throughput in public benchmarks, ships the awareness protocol we need for presence as a sibling package (y-protocols), and — decisive for us — its sync protocol is transport-agnostic bytes we can frame in MsgPack on our existing socket rather than adopting a new server. Automerge’s nicer git-like history model buys us nothing here because version history already exists at the platform level. The considered-and-rejected alternatives are in §8. 5. Detailed design 5.1 Document model One Y.Doc per workflow. Top-level shared types: doc.getMap('nodes') // Y.Map<node_id → Y.Map> doc.getMap('edges') // Y.Map<edge_id → plain object (atomic)> doc.getMap('meta') // workflow-level fields editable in the canvas (name, description) Per node (Y.Map): Key Type Conflict semantics type, parent_id plain value LWW (rarely concurrent) position plain {x,y} value LWW, whole-point (concurrent drags of the same node: one wins; visually fine) width/height, ui state plain values LWW properties nested Y.Map<prop → value> LWW per property — the case that matters. A edits prompt while B edits temperature on the same node: both survive. Edges are atomic values, not nested maps: an edge’s identity is its endpoints, so “merging” two concurrent edits of one edge is meaningless — replace wholesale. Structural conflicts follow Yjs map semantics: concurrent delete-node vs. edit-node resolves to delete-wins (the map key is gone). The binding then garbage-collects edges referencing missing nodes inside the same transaction, preserving the graph invariant the kernel’s validateGraph expects. This is the one invariant the CRDT cannot express by itself and must be enforced in the binding on every remote transaction — it gets a dedicated property-based test suite (§7.3). Property values are the same JSON-serializable values stored in graph today, so Y.Doc ⇄ graph JSON conversion is mechanical and lossless in both directions (docFromGraph, graphFromDoc — pure functions, heavily unit-tested, shared between client and server via a new packages/collab workspace package). Not in the doc: run results, generations, node execution state, selection. Execution state stays in ResultsStore/LiveRunStore and flows through existing run channels; selection is presence (ephemeral), not document state. 5.2 Wire protocol New MsgPack frames on the existing /ws connection, multiplexed by workflow id. We reuse Yjs’s binary sync protocol as an opaque payload — we frame it, we don’t reimplement it. Message Direction Payload collab_join C→S {workflow_id} — authz check, room attach collab_joined S→C {state_vector, snapshot, members[]} collab_sync both {workflow_id, data: bytes} — y-protocols sync1/sync2 (state-vector exchange → diff) collab_update both {workflow_id, update: bytes} — incremental Yjs update; server rebroadcasts to other members and appends to log collab_awareness both {workflow_id, data: bytes} — y-protocols awareness delta (cursor, selection, viewport, user meta). Ephemeral: never persisted, never ordered. collab_leave C→S room detach; server broadcasts awareness removal Reconnect is the sync protocol’s normal case: the client keeps its Y.Doc, reconnects, exchanges state vectors, and receives/sends exactly the missed diff. Edits made while disconnected are buffered in the local doc and merge on reconnect — short-outage tolerance falls out of the design for free (distinct from offline-first, which we are not doing; the buffer lives only as long as the tab). Rate shaping at the client: position updates during drags are throttled to ~30 Hz and batched per Yjs transaction; awareness to ~15 Hz. Yjs batches transactions into single updates natively. 5.3 Server: CollabRoomManager Lives in packages/websocket, wired into the unified socket runner next to the existing message handlers. Room lifecycle. First collab_join for a workflow: load latest snapshot from workflow_crdt_snapshots, apply tail from workflow_crdt_updates; if neither exists (first-ever collab session), seed the doc from workflow.graph via docFromGraph. Last member leaves: flush, then evict after a 60 s grace period (fast rejoin skips the reload). On each update: apply to the room doc, rebroadcast to other members, append the raw update to workflow_crdt_updates (append-only insert — cheap, safe on SQLite and Postgres). Compaction. When a room’s log exceeds 500 updates (or on eviction), encode the doc as a snapshot row and delete the covered updates, in one transaction. Back-of-envelope: a heavy session is ~10 updates/s sustained ⇒ compaction roughly every minute under load; snapshot for a 500-node workflow is O(100 KB); the log never grows unbounded. Materialized flush. Debounced (2 s idle / 15 s max), the room writes graphFromDoc(doc) through the existing workflow save path. Consequences, all intentional: The ModelObserver → resource-change event fires as today, so non-collab clients (CLI, another user’s read-only view, mini-apps) see updates at the granularity they already expect. Autosave version snapshots keep accruing in nodetool_workflow_versions with no changes to that system. A run started mid-session sees a graph at most ~2 s stale — identical to today’s autosave behavior, and the run submit path can force a flush first (§5.7). Authority. While a room is live, the room doc is authoritative and direct graph saves from collab-mode clients are suppressed (the binding replaces the save path). A non-collab writer (CLI workflows run-adjacent tooling, older client) that writes workflow.graph while a room is live is detected via the Phase 0 optimistic-concurrency token; the room responds by rebasing: diffing the incoming JSON against graphFromDoc(doc) and applying the delta as a Yjs transaction. This is the ugliest corner of the design; it is bounded, and the alternative (locking out the CLI) breaks G4. 5.4 Client: YjsGraphBinding A per-workflow object owned by WorkflowManagerStore, created when a workflow opens in collab mode. Responsibilities: Doc → store: on remote transactions, translate changed keys into the minimal NodeStore mutation (add/remove/patch node or edge). ReactFlow, properties panels, and everything above the store are untouched — they already re-render from store changes. Store → doc: the store’s mutation actions (add node, move node, change property, connect edge) additionally write into the Y.Doc inside a transaction tagged with a local origin. Origin tagging is what prevents echo loops (remote transactions don’t re-enter the doc) and is what scopes undo. Undo/redo: Y.UndoManager tracking only local-origin transactions replaces the zundo temporal store while in collab mode. This is a real behavior change (undo history no longer survives a reload; a collaborator deleting the node you just edited makes your undo a no-op) — both are the industry-standard semantics and match user expectations from Figma/Google Docs. Presence rendering: awareness states drive remote cursors (canvas overlay), selection highlights (colored node outlines), and the avatar roster in the editor header. Each user gets a stable color derived from user id. Single-user and local-desktop workflows do not create a binding and keep today’s path bit-for-bit, including zundo. Yes, this is a dual code path (risk §9); the seam is one object with a narrow interface, and Phase 1 exit criteria include e2e coverage of both paths. 5.5 Presence & awareness y-protocols/awareness, payload per user: {user_id, name, color, cursor: {x,y} | null, selection: node_ids[], viewport?}. Ephemeral by construction — 30 s timeout evicts crashed clients, nothing persisted, nothing in the doc. Presence ships in Phase 0, before the CRDT: the awareness channel has no dependency on shared document state, and “I can see you’re here, and where” removes most accidental-conflict pain even before merges are automatic. 5.6 Permissions & sharing Reuse the existing model, one additive table: workflow_collaborators (workflow_id, user_id, role: editor | viewer, invited_by, created_at) — SQLite and Postgres schemas, additive Drizzle migration. Edit (join room, send updates): owner + editor collaborators. Observe (join room read-only, receive updates + awareness, send only awareness): viewer collaborators; optionally public workflows (flag-gated — live-observing a public workflow is a great demo and a real load consideration; default off at launch). Authorization enforced at collab_join and per collab_update server-side (a viewer socket sending an update is dropped and logged). All room traffic rides the already-authenticated socket. Invite UX (email/link) is product work layered on this table; out of scope for this doc beyond the schema. 5.7 Interaction with execution & caching The rule: runs consume the materialized JSON; the CRDT never touches the kernel. Run submit from a collab client forces a room flush, then follows today’s path. The kernel WorkflowRunner, actors, and the debug/validate CLI see an ordinary graph. Per-user run state is unchanged. Run status (node running/completed) is additionally mirrored into the initiating user’s awareness state so collaborators see “Anna is running this branch” — cheap, ephemeral, no new channels. The editor’s cached partial-run machinery (runSubgraph.ts, computeRunSignatures.ts, generations) already keys reuse off input signatures of current graph state; remote edits dirty signatures exactly like local edits. No changes, but this interaction gets explicit e2e coverage because it’s subtle: a collaborator editing an upstream node must invalidate my “run from here” cache, and signature hashing makes that automatic. 5.8 Version history Unchanged and load-bearing. Autosaves continue via the materialized flush; manual “save version” works from any collaborator. Version restore while a room is live is implemented as a room-level operation: server rebases the restored graph onto the doc as one transaction (same mechanism as §5.3 external-writer rebase), so a restore doesn’t fork the room. Versions remain our coarse-grained, human-meaningful history; the CRDT update log is fine-grained machinery, not a user-facing timeline (per-user attribution of changes is recorded in update metadata, so a future “who changed this node” feature has the data it needs). 5.9 Failure modes Failure Behavior Server restart / deploy Rooms are memory; clients reconnect, rooms lazily reload from snapshot+log. In-flight updates not yet appended are re-sent by clients via sync (client docs hold full state). Worst case loss: none, by protocol. Client crash mid-edit Their unflushed local ops are lost (same as today); doc consistent. Awareness timeout clears their cursor. Divergence bug (binding or protocol defect) Detection: clients periodically send a state-vector hash; mismatch → client discards local doc and re-syncs from server (server wins), incident logged with both docs’ encoded state attached. Corrupt/unloadable CRDT state Drop snapshot+log for that workflow, re-seed from workflow.graph. Loses at most the last flush interval (≤15 s) of granularity, never the graph. This is the design’s ultimate safety property and the reason G4 exists. Log growth (compaction failing) Alert at 10× threshold; room degrades to read-only presence rather than unbounded writes. 6. Rollout Flag: collab (server setting + per-workspace override). Off = today’s behavior exactly; Phase 0’s stale-write rejection ships behind its own sub-flag first since it changes save semantics for racing writers (from “silently clobber” to “second writer gets a conflict prompt” — strictly better, but a visible change). Phase 0 (≈2 eng-weeks): optimistic-concurrency token on workflow saves; awareness channel; presence UI (roster, cursors, “N others editing” banner). Dogfood on the team’s own hosted workspace. Phase 1 (≈7 eng-weeks): packages/collab (doc model + conversions, property-based tests), server room manager + persistence + compaction, client binding + undo swap, permissions table, e2e suite. Dogfood ≥2 weeks; exit criteria: zero divergence incidents, p95 edit latency <500 ms, both editor code paths green in CI. GA: default-on for hosted; self-hosters get it via normal upgrade; desktop-vs-cloud unchanged (local workflows never enter collab mode). Kill switch: disabling the flag stops room creation; materialized JSON is current to within the flush interval, so rollback is a no-op for data. Observability: OTel spans for room lifecycle and sync (collab.room.load, collab.update.apply); metrics: concurrent rooms, members/room, update rate, broadcast fan-out latency, compaction duration, divergence count (this one pages), flush lag. All through the existing tracing stack (NODETOOL_TRACE_FILE et al.). Testing: property-based convergence tests on the doc model (random concurrent op interleavings ⇒ identical graphFromDoc output, graph invariants hold); fuzzed delete-vs-edit races for the edge-GC invariant; multi-context Playwright e2e (two browsers, one workflow: edit propagation, cursor rendering, conflict cases, undo isolation, reconnect); chaos tests reordering/dropping/duplicating update frames at the transport seam. 7. Scale & capacity (back of envelope) Hosted deployment today: one Fly machine, hundreds of users, tens concurrent. Numbers for 50 concurrent rooms × 4 members, heavy editing (10 updates/s/room, ~150 B/update): broadcast egress ≈ 50 × 10 × 3 × 150 B ≈ 225 KB/s — noise. Memory: 50 loaded docs × O(1 MB) worst case ≈ 50 MB — fine. DB: 500 inserts/s worst-burst append-only — fine on Postgres; on SQLite, batched in the room’s write loop (single writer per process anyway). CPU: Yjs applies are microseconds. The binding constraint is rooms are process-local, so multiplayer requires all members of a room on one machine. Single machine today ⇒ non-issue. The documented scale-out path, when needed: consistent-hash workflow_id → machine (Fly-Replay header routing), which preserves the room-per-process model without a Redis/pubsub layer. Explicitly not built now. 8. Alternatives considered A. Per-node locking (“check out” a node) + today’s refetch. Simple, and some enterprise tools live with it. Rejected: locks rot (crashed clients, forgotten sessions), pessimistic UX at exactly the granularity people collide least, doesn’t solve the whole-workflow save race, and builds no foundation for anything in Phase 2. We’d pay UX cost forever for a dead end. B. Rebroadcast Zustand actions (naive event sync). Tempting because the store already exists. Rejected: no convergence guarantee under reorder/loss/reconnect — this is the architecture that produces ghost edges and split-brain canvases, and then you rebuild a worse CRDT case by case. Listed because someone always proposes it. C. Operational Transformation. Proven (Docs), but requires the server to serialize and transform every op with per-type transform functions — high-complexity server code for a graph type nobody has a battle-tested OT library for. CRDTs give convergence with an off-the-shelf, transport-agnostic library. Rejected on build-cost and risk. D. Automerge instead of Yjs. Credible; better history ergonomics. Rejected on update-size/throughput (order-of-magnitude in public benchmarks), smaller production footprint for canvas apps, and because its git-like history duplicates what our version system already provides. E. Hosted sync service (Liveblocks, PartyKit, etc.). Fastest path on the hosted product. Rejected on a product invariant: NodeTool is self-hostable (desktop, docker-compose, packages/deploy), and a collaboration feature that only works on our cloud — or that ships customer workflow content to a third party — breaks that promise. Yjs costs us the room manager (§5.3), which at our scale is small. F. CRDT as the sole source of truth (drop the JSON graph). Architecturally purer; rejected emphatically. It couples the kernel, API, CLI, exports, and versioning to the CRDT library, makes rollback a migration instead of a flag flip, and turns a 9-week project into a rewrite. The materialized-view pattern (§5.3) is the load-bearing decision of this doc. 9. Risks & open questions Dual editor code path (collab vs. solo binding, Y.UndoManager vs. zundo) is the top maintenance risk. Mitigation: narrow seam, both paths in CI. Open question for post-GA: converge solo mode onto a local-only Y.Doc to delete the fork. External-writer rebase (§5.3) is the fiddliest logic. Mitigation: it degrades safely (worst case equals today’s last-write-wins for that one write) and is fuzz-tested. ReactFlow render cost under remote-update storms (many nodes patched at 30 Hz). Mitigation: transaction batching + rAF coalescing in the binding; perf budget test with a 500-node graph and a simulated 4-editor storm. SQLite write contention on busy self-hosted single-file DBs. Mitigation: batched appends, compaction thresholds; needs measurement in dogfood, not speculation. Open: do public workflows allow anonymous observers at GA, or authenticated-only? (Load and abuse surface vs. shareability. Default: authenticated-only, revisit with data.) Open: awareness on mobile (mobile/ app is read-mostly today) — roster yes, cursors probably meaningless. Decide during Phase 0 UI work. Appendix A — Schema sketch (Drizzle, additive) workflow_crdt_updates (id, workflow_id, seq, update BLOB, user_id, created_at) index (workflow_id, seq) workflow_crdt_snapshots (id, workflow_id, seq_through, snapshot BLOB, created_at) index (workflow_id, seq_through desc) workflow_collaborators (workflow_id, user_id, role, invited_by, created_at) pk (workflow_id, user_id) Mirrored in schema/ and schema-pg/; migration is additive only — no changes to workflows or nodetool_workflow_versions. Appendix B — Glossary CRDT — Conflict-free Replicated Data Type: data structures whose concurrent edits merge deterministically without coordination. Awareness — Yjs’s ephemeral presence side-channel: per-client state (cursor, selection) that is broadcast but never part of the document. State vector — compact summary of “which updates a replica has,” used to compute the minimal diff on (re)connect. LWW — last-writer-wins register semantics for a single value. Materialized flush — writing graphFromDoc(doc) into the existing workflow.graph column so non-collab consumers stay oblivious.--- # Implementation Plan: Real-Time Collaborative Workflow Editing (“Multiplayer”) Source: https://docs.nodetool.ai/multiplayer-implementation-plan.html Implementation Plan: Real-Time Collaborative Workflow Editing (“Multiplayer”) Author: Matti Georgi Status: Draft — for review Last updated: 2026-07-10 Design doc: multiplayer-design.md This plan turns the design into ordered, reviewable PRs with file-level targets, exit criteria, and a rollout checklist. Estimates assume one engineer full-time; workstreams marked ∥ can run in parallel with a second engineer. 0. Ground rules Every PR passes npm run check (typecheck + lint + test) and follows repo conventions: strict TS, no any, ui_primitives only (no raw MUI), design tokens, Zustand selectors, tests in __tests__/. New backend code lands in a new workspace package packages/collab plus surgical changes to packages/websocket, packages/models, and web/. No changes to packages/kernel, packages/runtime, or any node package — if a PR touches those, it violates design goal G4 and needs a design amendment first. All schema changes are additive, mirrored in packages/models/src/schema/ and schema-pg/, registered in packages/models/src/migrations/versions.ts, and covered in migrations.test.ts (the suite asserts tableExists). Feature flags via the existing settings registry (packages/websocket/src/settings-registry.ts → registerSetting): collab_occ (Phase 0 save safety), collab_presence (Phase 0 UI), collab (Phase 1 CRDT). Everything defaults off until the rollout checklist says otherwise. Phase 0 — Save safety + presence (no CRDT) · ~2 eng-weeks Ships standalone value: kills silent last-write-wins data loss and adds presence. Nothing in Phase 0 depends on Yjs. PR-1: Optimistic concurrency on workflow saves (~3 days) Server — packages/websocket/src/trpc/routers/workflows.ts: Extend updateInput (the update mutation, ~line 489) and autosaveInput (~line 706) with optional base_updated_at: string. When present and ≠ the row’s current updated_at, throw a tRPC CONFLICT (the error formatter already maps ALREADY_EXISTS/CONFLICT; add a distinct STALE_WRITE code in packages/websocket/src/error-codes.ts so clients can discriminate). Return the server’s current updated_at and graph in the error payload. Omitted token ⇒ today’s behavior (older clients, CLI keep working). Client — web/src/stores/WorkflowManagerStore.ts (saveWorkflow, ~line 314) and the autosave path: Track updated_at from the last successful load/save per workflow; send as base_updated_at; update it from every save response. On STALE_WRITE: conflict dialog (ui_primitives) with three actions — Reload theirs (refetch, discard local), Overwrite (resend without token), Keep both (save local graph as a manual version via the existing versions.create mutation, then reload). Autosave never overwrites: on conflict it silently snapshots local state as an autosave version and backs off until the user decides. Interplay fix (required, not optional): web/src/stores/resourceChangeHandler.ts currently invalidates ["workflows"] on every workflow change, which refetches and clobbers a dirty open editor. Gate that invalidation: if the changed workflow is open and dirty, surface the conflict banner instead of refetching. Tests: router unit tests (token match / mismatch / omitted); WorkflowManagerStore tests for token threading and each conflict action; regression test for the dirty-editor refetch clobber. Exit: two clients racing on one workflow can no longer silently lose work, with collab_occ on. PR-2: Awareness channel, server side (~2 days) New module packages/websocket/src/collab/presence-registry.ts: in-memory Map<workflow_id, Map<socket_id, AwarenessState>>, join/leave/update, 30 s stale eviction, broadcast to co-members. Wire three message types into the dispatch switch in packages/websocket/src/unified-websocket-runner.ts (the case block starting ~line 5849): collab_join, collab_leave, collab_awareness. MsgPack frames like every other message; no new socket, no new port. Authz on collab_join: reuse the access check pattern from the workflows router (access !== "public" && user_id !== ctx.userId ⇒ deny, plus collaborators table once PR-9 lands). Message schemas in packages/protocol/src/ (new collab.ts), since protocol is the shared-types package. Note: protocol changes require cd packages/protocol && npm run build before mobile typecheck — CI already handles this. OTel: collab.presence.join/leave events; gauge for members per workflow. Tests: registry unit tests (join/leave/evict/broadcast); an integration test in packages/websocket/tests/ driving two fake sockets through join → awareness → leave. PR-3: Presence UI (~4 days) ∥ after PR-2’s protocol types merge web/src/lib/collab/PresenceChannel.ts: thin client over globalWebSocketManager (web/src/lib/websocket/GlobalWebSocketManager.ts — singleton, per CLAUDE.md never a new socket): join on workflow open (hook into WorkflowManagerStore open/close lifecycle), throttle outgoing cursor/selection to ~15 Hz, resend state on reconnect (subscribe to the manager’s reconnect event). web/src/stores/CollabPresenceStore.ts: Zustand store keyed by workflow id → member list with cursor/selection; selectors per member to keep re-renders scoped. UI (all ui_primitives + design tokens): Roster: avatar row in the editor header; stable per-user color derived from user id hash. Remote cursors: absolutely-positioned overlay inside the ReactFlow viewport (transform flow-coords → screen via ReactFlow’s viewport state; rAF-batched). Selection highlights: colored outline on nodes present in a remote member’s selection. Banner: “N others viewing/editing” using the existing notification/banner primitives. Own selection/cursor published from existing ReactFlow callbacks (onSelectionChange, pane onMouseMove throttled). Tests: store tests; RTL component tests for roster/banner; one Playwright two-context smoke (second browser’s cursor appears) — this bootstraps the multi-client e2e harness Phase 1 needs. Phase 0 exit / dogfood gate: flags on for the team workspace ≥1 week; zero clobbered-save reports; presence latency < 2 s observed. Phase 1 — CRDT co-editing · ~7 eng-weeks PR-4: packages/collab — document model (~1 week) ∥ with PR-2/3 New workspace package (depends only on @nodetool-ai/protocol; register in root workspaces and build:packages order — it has no decorators, so no dist/-loading caveats): src/doc-model.ts: shared-type layout (nodes/edges/meta maps per design §5.1); typed accessors. src/convert.ts: docFromGraph(graph): Y.Doc, graphFromDoc(doc): Graph — pure, lossless, the contract everything else leans on. src/invariants.ts: enforceGraphInvariants(doc, transaction) — edge GC for dangling endpoints after delete-vs-edit races, duplicate-edge collapse; runs inside the same Yjs transaction. src/protocol.ts: sync/awareness frame encode/decode wrapping y-protocols (opaque bytes in MsgPack fields). Tests (the heart of this PR): round-trip property tests (fast-check) over generated graphs: graphFromDoc(docFromGraph(g)) ≡ g; convergence property tests — N docs, random interleaved concurrent ops, sync pairwise ⇒ identical graphFromDoc output and invariants hold; targeted delete-vs-edit fuzz for edge GC. Budget half the PR’s time for these tests; they are the cheapest place this project can fail. Deps: yjs, y-protocols, fast-check (dev). Pin exact versions; both are pure-JS ESM (no native modules, no sandbox install pain). PR-5: Persistence — schema + migrations (~2 days) packages/models/src/schema/workflow-crdt.ts (+ schema-pg/ mirror): workflow_crdt_updates (id, workflow_id, seq, update blob, user_id, created_at; index workflow_id+seq) and workflow_crdt_snapshots (id, workflow_id, seq_through, snapshot blob, created_at; index workflow_id+seq_through desc). Export from both schema/index.ts files. workflow_collaborators (workflow_id, user_id, role, invited_by, created_at; pk workflow_id+user_id). Migrations create_workflow_crdt_tables, create_workflow_collaborators appended in packages/models/src/migrations/versions.ts with down-migrations, covered in migrations.test.ts. DAO packages/models/src/workflow-crdt.ts: appendUpdate, loadSince(seq), latestSnapshot, compact(seqThrough, snapshot) (transactional), collaboratorsFor(workflowId). PR-6: Server — CollabRoomManager (~2 weeks) packages/websocket/src/collab/room-manager.ts + room.ts, replacing PR-2’s presence registry internals (presence becomes room-scoped; wire protocol unchanged so PR-3’s client needs no changes): Lifecycle: first join loads snapshot + update tail via the DAO, else seeds from workflow.graph with docFromGraph; last-leave flush + 60 s grace eviction. Update path: authz (editor role) → apply to room doc → rebroadcast to other members → batched append to workflow_crdt_updates (single writer loop per room; batch inserts for SQLite friendliness). Sync: collab_join reply carries state vector + snapshot; handle sync1/sync2 frames via packages/collab protocol helpers. Compaction: threshold 500 updates or on eviction; snapshot + delete-covered-updates in one transaction. Materialized flush: debounced 2 s idle / 15 s max; write graphFromDoc(doc) through the same internal save routine the update mutation uses (so updated_at advances, the ModelObserver resource-change event fires, and autosave versions accrue). Tag the write with a room origin so the OCC check (PR-1) knows it’s the authority. External-writer rebase: when a non-collab write lands on a workflow with a live room (detected via the OCC token path), diff incoming JSON vs graphFromDoc(doc) and apply the delta as a Yjs transaction with a rebase origin. Divergence guard: on a client-reported state-vector-hash mismatch, send authoritative re-sync; count it; log both encoded states at debug level. Run-submit hook: the run_job path forces a flush for workflows with a live room before job creation. OTel spans collab.room.load, collab.update.apply, collab.flush, collab.compact; metrics per design §6. Tests: room unit tests (lifecycle, authz, batching, compaction transactionality); integration tests with two fake sockets (join → concurrent updates → convergence → flush lands in workflows table → resource event observed); rebase fuzz tests; restart simulation (kill room, rejoin, sync from persistence). PR-7: Client — YjsGraphBinding (~2 weeks) ∥ with PR-6 against a stub room web/src/lib/collab/YjsGraphBinding.ts, owned per-workflow by WorkflowManagerStore: Store → doc: wrap the NodeStore mutation surface — onNodesChange, onEdgesChange, onConnect, addNode, updateNode, updateNodeData, deleteNode(s), setNodes, setEdges (all in web/src/stores/NodeStore.ts) — mirroring each mutation into a Y transaction tagged localOrigin. Position drags coalesced to ~30 Hz, one transaction per frame. Doc → store: deep-observe handler translating remote transactions into minimal store mutations, entering the store through a remote: true path so wrapped actions don’t echo back into the doc, and marked to not dirty the workflow (the store already has a shouldn’t-dirty pattern, NodeStore.ts ~line 808) — collab edits must not trigger the legacy autosave. Save suppression: in collab mode, saveWorkflow/autosave become no-ops for the graph (server room flush is the writer); manual “save version” calls the versions mutation directly. Undo/redo: Y.UndoManager over nodes/edges/meta, trackedOrigins: {localOrigin}; the editor’s undo/redo entry points dispatch to it in collab mode, zundo temporal path untouched otherwise. Keyboard shortcuts unchanged. Sync client: join/sync/update/reconnect state machine over globalWebSocketManager; buffered local edits merge on reconnect; periodic state-vector hash for the divergence guard; on server-forced re-sync, rebuild store from doc. Mode selection: binding created only when collab flag is on and the workflow is server-backed and shared/multi-member-capable; pure-local desktop workflows never enter collab mode. Tests: binding unit tests against an in-memory Y.Doc pair (each store action round-trips; remote ops produce exact store mutations; echo prevention; undo isolation); store integration tests for save suppression and no-dirty remote edits; jsdom perf smoke for the 500-node update-storm budget. PR-8: Collaborative signals in the editor (~3 days) Run-status mirroring: initiating client publishes {running_nodes} into its awareness state; collaborators render the existing node-running affordance in the runner’s color. Cache interplay e2e: collaborator edits an upstream node ⇒ my “run from here” signature dirties (should be automatic via computeRunSignatures; this PR proves it with a test, changes nothing). Version restore while a room is live: route the versions.restore mutation through the room rebase path (PR-6 mechanism) instead of a raw graph write. PR-9: Permissions + sharing surface (~3 days) Enforce workflow_collaborators roles at collab_join (editor/viewer/deny) and per collab_update (viewer updates dropped + logged). tRPC mutations: collaborators.list/add/remove on the workflows router; minimal share dialog in the editor header (ui_primitives) — add by email/user id, role toggle. Public-workflow live observation stays behind its own flag, default off. PR-10: Multi-client e2e + chaos suite (~1 week, overlaps 6–9) Extend the Playwright harness from PR-3: two browser contexts, one workflow — edit propagation, both-edit-same-node property LWW, delete-vs-edit edge GC, undo isolation, reconnect-and-merge, presence rendering, conflict dialog (Phase 0 path with flag off). Chaos at the transport seam: inject a frame-mangling proxy (drop/reorder/duplicate collab_update) and assert convergence via state-vector hash. Perf budget test: 500-node graph, 4 simulated editors at full rate, assert frame time and store-update latency budgets. Sequencing Phase 0: PR-1 ──► dogfood ─────────────────────────────┐ PR-2 ──► PR-3 ─────────────► dogfood │ ▼ Phase 1: PR-4 ─┬─► PR-6 ─┬─► PR-8 ─► PR-9 ─► PR-10 ─► dogfood ─► GA │ │ PR-5 ─┘ PR-7 ─┘ (PR-6 ∥ PR-7 via stubbed counterpart) Total: ~9 eng-weeks serial; ~6–7 calendar weeks with two engineers (PR-3∥PR-4, PR-6∥PR-7). Rollout checklist Phase 0 dogfood: collab_occ + collab_presence on for the team workspace ≥1 week. Gate: zero lost-work reports, zero false-positive conflict dialogs. Phase 0 GA: collab_occ default-on everywhere (it is pure safety); presence default-on for hosted. Phase 1 dogfood: collab on for team workspace ≥2 weeks. Gates: zero divergence-counter increments; p95 remote-edit latency < 500 ms; flush lag p99 < 20 s; compaction keeping logs < 2× threshold; both editor code paths (collab/solo) green in CI. Phase 1 GA (hosted): default-on; self-hosters receive via normal upgrade (migrations are additive; flag still available as kill switch). Kill switch drill (before GA): disable collab mid-session in staging; verify clients fall back to solo mode, materialized JSON is current, no data loss. Monitoring to have in place before step 3: divergence counter (pages), room count / members gauge, update rate, broadcast latency histogram, flush lag, compaction duration, workflow_crdt_updates row-count alert at 10× threshold. Risk register (implementation-level) Risk Mitigation Where NodeStore has mutation paths not on the wrapped action list (bulk ops, timeline/sketch side stores writing nodes) Audit every set() touching nodes/edges in NodeStore.ts during PR-7; add a dev-mode assertion that the doc and store hash-match after each transaction PR-7 Legacy autosave fires in collab mode and double-writes Save suppression tested explicitly; server rejects graph writes from collab-mode clients (origin check) PR-6/7 resourceChangeHandler refetch fights the binding Collab-mode workflows opt out of workflow query invalidation (binding is the sync channel) PR-7 Yjs update floods from position drags degrade ReactFlow 30 Hz coalescing + rAF apply batching; perf budget test PR-7/10 SQLite write stalls under append load on self-hosted boxes Batched appends in room writer loop; measured in dogfood before GA PR-6 Protocol drift between web and mobile Frames defined once in packages/protocol; mobile ignores unknown message types today (verify in PR-2) PR-2 Explicit non-goals of this plan (deferred with the design doc) Y.Text prompt merging, comments/annotations, follow-mode, offline-first merge, horizontal room scale-out, anonymous public observers, org/team permissions.--- # Node Packs Source: https://docs.nodetool.ai/node-packs.html Node Packs extend NodeTool with additional nodes, integrations, and workflow templates. Packs are distributed through the public registry and can be installed directly from the app or via the command line. What’s in a Pack A Node Pack can include any combination of: Component Description Node definitions New processing nodes that appear in the Node Menu npm dependencies Required libraries bundled with the pack Workflow templates Pre-built workflows that demonstrate the pack’s capabilities Chat tools Tools accessible from chat agents Installing Packs from the App Open the Package Manager Launch the desktop app Go to Tools > Packs to open the Package Manager Browse and Search Search by name – Use the search box to filter by package name, description, or repository Search by node – Use the “Search nodes” field to find specific nodes across all packs. You can install the required pack directly from node search results. Install, Update, or Remove Click Install to add a pack. It downloads as an npm package. Click Update when a new version is available. Click Uninstall to remove a pack you no longer need. Once installed, the pack’s nodes automatically appear in the Node Menu under new namespaces. Installing Packs via CLI For terminal users: # List all available packs nodetool package list -a # Install a pack npm install @nodetool-ai/<pack-name> # Update a pack npm update @nodetool-ai/<pack-name> # Remove a pack npm uninstall @nodetool-ai/<pack-name> Built-in Node Libraries NodeTool ships with extensive built-in nodes organized by provider: Library Categories Examples nodetool Agents, audio, code, constants, control flow, data, documents, generators, images, input/output, text, video Core processing nodes for any workflow openai Agents, audio, image, text GPT models, GPT-Image, Whisper, TTS gemini Audio, image, text, video Google Gemini multimodal models lib 30+ utility categories NumPy, Pillow, PDF, Excel, SQLite, HTTP, RSS, and more Browse the full node library in the Node Reference. Anthropic (Claude), HuggingFace, and MLX are reached through the provider system and generic nodes (e.g. nodetool.agents.Agent), not as standalone TypeScript node namespaces. See Providers. Publishing Your Own Pack Create and share your own Node Packs with the community: Follow the guidelines in the NodeTool Packs Registry on GitHub Structure your pack as an npm package with node definitions Publish to the registry so others can discover and install your work For details on building custom nodes, see the Developer Guide and Custom Nodes Guide. Next Steps Developer Guide – Build custom nodes and packs Custom Nodes Guide – Step-by-step node development Node Patterns – Common node implementation patterns CLI Reference – Package management commands--- # Package Registry Guide Source: https://docs.nodetool.ai/packages.html NodeTool packages bundle reusable nodes, assets, and example workflows. The package registry discovers and registers node classes so workflows can reference them at runtime. Package Anatomy A package is a standard npm workspace package that exports node classes and a registration function: package.json – declares the package name, dependencies, and build scripts. src/nodes/ – node implementations, one file per domain (e.g. list.ts, audio.ts). src/index.ts – exports all node classes and a register*Nodes() function. tsconfig.json – extends the workspace base config. examples/ – optional workflow examples. assets/ – optional static assets used by nodes. Example package.json { "name": "@nodetool-ai/base-nodes", "type": "module", "version": "0.1.0", "main": "dist/index.js", "types": "dist/index.d.ts", "scripts": { "build": "node -e \"require('node:fs').rmSync('dist', { recursive: true, force: true })\" && tsc", "test": "vitest run", "lint": "tsc --noEmit" }, "dependencies": { "@nodetool-ai/node-sdk": "latest" } } Every node package depends on @nodetool-ai/node-sdk, which provides BaseNode, the @prop decorator, and the NodeRegistry type. Example tsconfig.json { "extends": "../../tsconfig.base.json", "compilerOptions": { "outDir": "dist", "rootDir": "src" }, "include": ["src"] } Node Registration Each package exports a constant array of node classes and a registration function. The pattern used by @nodetool-ai/base-nodes: import type { NodeClass, NodeRegistry } from "@nodetool-ai/node-sdk"; import { LIST_NODES } from "./nodes/list.js"; import { TEXT_NODES } from "./nodes/text.js"; export const ALL_BASE_NODES: readonly NodeClass[] = [ ...LIST_NODES, ...TEXT_NODES, // ... additional node groups ]; export function registerBaseNodes(registry: NodeRegistry): void { for (const nodeClass of ALL_BASE_NODES) { registry.register(nodeClass); } } At startup, the runtime creates a NodeRegistry and calls each package’s registration function. Workflows referencing nodetool.list.Length or mypack.math.AddOffset resolve through the registry without manual imports. Managing Packages via CLI List Packages nodetool package list nodetool package list --available # fetch registry index Displays installed packages (local metadata) or remote entries hosted at the package index URL. Initialize a Package nodetool package init Scaffolds a new package in the current directory. Generate Documentation nodetool package docs # single index.md in ./docs nodetool package docs --output-dir docs # custom directory (default: docs) nodetool package docs --compact # shorter summaries for LLM prompts package docs writes a single index.md overview. For per-node Markdown pages, use node-docs: nodetool package node-docs # one page per node (default: docs/nodes) nodetool package workflow-docs # docs for workflow examples (default: docs/workflows) The full set of package subcommands is: list, init, docs, node-docs, and workflow-docs. (Note: the top-level nodetool install / nodetool uninstall commands configure the MCP server, not node packages.) Building Packages Compile TypeScript and prepare the package for use: npm run build For type checking without emitting output: npm run lint For running tests: npm run test Publishing Packages Implement nodes under src/nodes/ extending BaseNode with @prop decorators. Export all node classes and a registration function from src/index.ts. Run npm run build to compile. Add example workflows in examples/ and assets in assets/ if relevant. Publish to npm or provide a Git URL. To add the package to the public index, create an entry in the registry repository so package list --available surfaces it. Workflow Integration Installed packages automatically register nodes with the runtime: Node metadata is merged during startup so workflows referencing package.namespace.Node resolve without manual imports. Run npm run codegen --workspace=packages/dsl to regenerate typed factory functions from node metadata. Related Documentation CLI Reference – package subcommands. Configuration Guide – where package metadata is cached. Custom Nodes Guide – step-by-step node implementation. TypeScript DSL Guide – type-safe workflow definitions with @nodetool-ai/dsl.--- # packs.md Source: https://docs.nodetool.ai/packs This page has moved. Please go to Node Packs.--- # Examples Revamp Plan Source: https://docs.nodetool.ai/plans/examples-revamp.html Examples Revamp Plan Status: draft, 2026-07-17. Scope: the 37 shipped templates in packages/base-nodes/nodetool/examples/nodetool-base/ and their generated mini apps (scripts/generate-template-apps.mjs, web/public/app-preview/). Why The examples are the product’s first impression three times over: the template gallery in the editor, the mini apps on /apps/* marketing pages, and the first thing a new user runs. Today they undersell on all three surfaces. The workflows are mostly linear prompt pipes that don’t justify a node editor, and the generated apps are 37 copies of the same form. Meanwhile the platform just gained the features that would make them compelling — node-property bindings, reactive subgraph runs, run pacing, container-responsive app layout — and no example uses any of them. Analysis Measured across all 37 templates (graph JSON + generated app_doc): Workflow shape 15 of 37 graphs are purely linear (max fan-out 1); median size ~7 nodes. The dominant shape is Input → Prompt template → LLM → Output — a prompt wrapper that a chat box does better. Control flow is nearly absent: 14 nodetool.control nodes and 6 nodetool.boolean nodes across the whole set, almost all in one teaching example (Conditional Logic Engine). Node namespace census: nodetool.input 72, nodetool.text 60, nodetool.agents 24, nodetool.generators 19, image 34, video 11, lib.sqlite 3. No vectorstore/RAG usage, no workflow-as-tool composition, no streaming showcase. Models gpt-5-mini appears 41 times — effectively every text node. Images are all fal-ai/flux/schnell (16). Video is veo-3.1 (5, expensive). Providers: openai 43, fal 22, gemini 6, claude_agent_sdk 1. Zero local models (Ollama/MLX). A fresh install without an OpenAI key has ~30 broken examples and no private/offline example at all. Content hygiene Shipped prompt text includes typos and one-liners (“Create a 3 compelling Pokémons”). No system prompts, no output-format instructions. Prompts are the actual product of these workflows and read as first drafts. Hardcoded content that should be inputs: RSS feed URL (BBC Europe), search topics, folder paths. FolderPathInput appears in two templates and is meaningless in a hosted app. Node titles/descriptions are sparse; comments decorate rather than teach. Outputs 30 of 37 templates have exactly one Output; nearly everything funnels into a text/markdown blob. Flashcards, calendars, SEO fields, and generated data are prose instead of structured objects — which caps what the apps can render (widget census over all 37 apps: 28 Markdown + 37 Text vs 13 Image, 7 Video, 1 Audio, 1 Json). Apps All 37 app docs share one generated shape: heading, tagline, input form, Run button, results panel. Zero Slider/Select/Switch widgets anywhere (71 WorkflowInput widgets, 37 Buttons). 8 templates have no inputs at all — their app is a Run button (Agent Google Search, Color Boost Video, Data Generator, Fetch Papers, Hacker News Agent, Image To Audio Story, Summarize RSS, YouTube Research Agent). Apps open with empty fields (seeded demo values exist only in the marketing preview bundles), and none use pacing, node-property bindings, variables, or an iteration loop (regenerate/variants/gallery). Marketing surface The examples feed the marketing site directly: every template gets a /apps/<slug> landing page (marketing/scripts/generate-miniapp-entries.mjs → miniAppEntries.generated.ts, 37 pages live), a screenshot (web/scripts/screenshot-app-previews.mjs → marketing/public/apps/<slug>.png, 37 present), and a /templates/<slug> page via the generated template catalog. Consequences of the current state: All 37 screenshots show the same empty form layout — a grid of visually identical pages that read as one product with 37 names. Conversion surface, currently underselling. The 8 no-input apps are marketed as apps while being a Run button. Landing pages exist for near-duplicate templates, splitting search intent (three YouTube/research agents, two thumbnail pipelines, three audio summarizers) instead of concentrating it on one strong page each. Root cause The generator (generate-template-apps.mjs) is sound infrastructure with a timid curation table, and the workflows were authored to demo single nodes, not to be products. Both are fixable without new platform features — the platform is now ahead of its own examples. Goals Every shipped example runs on first try and passes nodetool validate. Every example demonstrates at least one thing a chat box can’t do (parallel comparison, media pipeline, control flow, RAG, live parameters, structured data). A keyless install has at least 3 fully working local examples. The featured apps feel like tools: seeded defaults, real controls, visible progress, an iteration loop. Fewer, better: target ~20 curated examples instead of 37 near-duplicates. Plan Phase 0 — Guardrails (do first, half a day) CI job: run nodetool validate over every example JSON; --warnings-as-errors for unknown types, dangling edges, unselected models. This is the rot detector for everything that follows. Add a cheap smoke tier: nodetool debug on 3–4 keyless/local examples. Skip paid-model examples in CI; run them in a manual pre-release pass. Phase 1 — Hygiene pass on all survivors (1–2 days) For each kept template: Rewrite prompts properly: system prompt, task structure, explicit output format, no typos. Prompts are the product; treat them like code review. Parametrize hardcoded values into inputs (feed URL, topic, style). Name and describe every node; make comments explain the why of the graph. Set sensible input defaults so Run works with zero typing. Model pass: keep a cheap fast default for text, but vary providers deliberately where it shows breadth; flag expensive nodes (veo) in the description with a cost note. Phase 2 — Restructure the catalog (2–3 days) Disposition of all 37 (keep = hygiene only; fix = keep + targeted change; merge/retire = fold into another or drop): Template Disposition Change Ad Creative Factory keep showcase; structured variant outputs Agent Google Search merge into Research Agent (topic input, streaming) Audio To Image keep starter; seed demo audio Brand Asset Generator keep hero app Cold Outreach Co-Pilot fix structured outputs (draft objects) Color Boost Video fix VideoInput + slider-bound intensity (live grading) Concept Art Iteration Board keep hero; variant gallery loop Conditional Logic Engine keep teaching example for control flow Creative Story Ideas merge into one “Prompt Template” starter Data Generator fix schema input → dataframe output → table app Fetch Papers merge into Research Paper Summarizer (topic input) Flashcard Generator fix emit card objects, card-flip app rendering Hacker News Agent fix topic input, streaming markdown Hook & Thumbnail Factory keep hero (absorbs YouTube Thumbnail Pipeline) Image Enhance fix FLAGSHIP: sliders bound to node properties, pace: release Image To Audio Story fix add ImageInput Image to Video Animation keep   Learning Path Generator merge into Prompt Template starter Meeting Transcript Summarizer fix action items as dataframe (absorbs Summarize Audio) Movie Posters keep hero Movie Trailer Generator keep showcase; cost note Music Video Visualizer keep showcase Photo Enhancement Suite fix replace FolderPathInput (multi-image) or mark local-only Podcast Repurposing Studio keep showcase Pokemon Maker fix hero; rewrite prompt, style Select, variant gallery Product Mockup Generator keep hero Product Video Generator keep   Research Paper Summarizer fix absorbs Fetch Papers SEO Content Engine fix typed field outputs Social Media Calendar Filler fix calendar as dataframe Story to Video Generator retire overlaps Movie Trailer; FolderPath-bound Summarize Audio merge into Meeting Transcript Summarizer Summarize RSS fix feed URL input; digest by topic Transcribe Audio fix instant tool: auto-run on upload (change event, paced) Wikipedia Agent merge into Research Agent YouTube Research Agent merge into Research Agent YouTube Thumbnail Pipeline merge into Hook & Thumbnail Factory Net: 37 → ~24, minus retire → ~23, plus new below → ~28; prune to ~20 at final curation if the merged families hold up. Marketing follow-through for every merge/retire: keep the old /apps/<slug> and /templates/<slug> URLs as redirects to the absorbing template’s page (the slugs are already indexed), and grep docs/tutorials for retired template names in the same pass. Phase 3 — New differentiator examples (2–3 days) Each maps to a marketing story the catalog currently can’t tell: Chat With Your Documents — RAG: index a folder into the vectorstore, ask questions with cited answers. (First vectorstore usage in examples.) Research Agent — the merged agent family: topic input, real tool use (search + browse), streaming markdown brief with citations. Model Arena — one brief fanned out to 3 providers side-by-side. Earns the graph visually and doubles as a great comparison app. Private Assistant (local) — Ollama/MLX end-to-end, no API keys. Along with 2 other local-capable starters, satisfies the keyless goal. Workflow as a Tool — a workflow exposed via tool_name, consumed by an agent in another workflow. Shows composition, which nothing demos. Phase 4 — App layer (2 days, extends the curation table) Extend generate-template-apps.mjs curation entries with per-template hints instead of hand-editing app docs: sliders: node-property bindings with ranges (autofill already derives min/max/step from metadata), pace: "release" for anything expensive. selects: finite choices (aspect ratio, style, tone, platform) instead of free-text inputs. seeds: shipped default values in the app itself, not just the marketing preview — first Run must work with zero typing. gallery: append-disposition outputs rendered as a grid with a “Make another” button for the creative heroes. featured: flag the ~8 hero apps for the gallery and /apps landing pages (Image Enhance, Pokemon Maker, Movie Posters, Brand Asset Generator, Hook & Thumbnail Factory, Product Mockup Generator, Concept Art Iteration Board, Music Video Visualizer). Progress widget on every template whose run exceeds a few seconds. Regenerate previews and /apps pages (screenshot-app-previews.mjs, marketing/scripts/generate-miniapp-entries.mjs) as the last step. Phase 4b — Marketing page impact What the revamp changes on the marketing site, and the work it implies: Screenshots become the sell. Today’s 37 near-identical empty-form screenshots become ~20 distinct ones showing seeded results: a photo mid-enhancement with sliders, a Pokémon variant gallery, a model-arena comparison, a calendar table. The screenshot pipeline needs seeded values rendered (the preview bundles already seed; verify the new gallery/slider states render before the screenshot settles). Fewer, stronger landing pages. ~20 pages that each answer one search intent instead of 37 splitting it. Merged/retired slugs redirect (see Phase 2); the /apps index orders by the featured flag with heroes first. New pages with real search demand. The Phase 3 examples add landing pages for terms the site can’t currently target: “chat with your documents / RAG”, “compare AI models side by side”, “local private AI assistant”, “AI research agent”. These are stronger queries than most existing template names. Copy pass per page. Landing copy currently describes the workflow; rewrite to describe the outcome and name the differentiator (“drag the slider, watch the photo update — runs the graph live”), reusing each template’s new description field so app, gallery, and landing page say the same thing. Mechanics. All of this flows through the existing generators — the work is the redirects map, the featured ordering, and the copy fields in the curation table; no new marketing infrastructure. Entries without a fresh screenshot stay noindexed by the existing rule, which safely gates half-finished templates out of search. Phase 5 — Verification and release nodetool validate green across the catalog (CI from Phase 0). nodetool app debug per hero app: bindings resolve, run trigger exists, display widgets receive values. Manual pass on paid-model examples before release; screenshot refresh. Sequencing and effort Phases 0–1 are independent of 2–4 and land first (~2 days, pure win, no decisions needed). Phase 2 dispositions need a maintainer sign-off on the merge/retire list, then 2–4 parallelize by template family. Total ~7–9 working days; the flagship (Image Enhance live editor) is worth pulling forward as the proof piece for the whole direction. Risks Cost: veo/fal examples are expensive to smoke-test. Mitigation: CI validates but doesn’t execute paid models; manual pre-release pass only. Model drift: pinned model ids rot. Mitigation: Phase 0 CI catches unknown models at validate time; prefer provider-default aliases where the registry supports them. User familiarity: retired/merged templates may be referenced in docs and tutorials. Mitigation: grep docs for template names during Phase 2; keep slugs redirecting on the marketing site. Scope creep: structured outputs may reveal missing table/gallery rendering in app widgets. The Json/Output widgets cover the minimum; a dedicated Table widget is a fast follow, not a blocker. Success criteria Fresh install, no keys: 3+ examples run end-to-end locally. Every example passes validate in CI; hero apps pass app debug. Every kept example demonstrates a named differentiator (recorded in its description). Featured apps: seeded first run, at least one real control (slider/select), progress on long runs, and an iteration loop on the creative ones. Marketing: no dead /apps or /templates URLs (redirects in place), every indexed page has a distinct seeded-result screenshot, heroes lead the /apps index.--- # Motion Design — Implementation Plan Source: https://docs.nodetool.ai/plans/motion-design-tasks.html Motion Design — Implementation Plan Companion to motion-design.md (the technical design). Every task below is written to be executed by an agent with no prior conversation context. Each task’s first step is: read docs/plans/motion-design.md in full — it is the contract; this file adds sequencing, file-level pointers, and acceptance criteria. Ground rules for every task: Repo root is a TypeScript monorepo; run nvm use && npm install if node_modules is missing (sandboxed environments: npm install --ignore-scripts, see AGENTS.md § Install in sandboxed environments). After changes: npm run typecheck && npm run lint, plus the package tests named in the task. npm run dev:nodetool -- affected maps changed files to the minimal workspaces to rebuild/test. packages/timeline is pure TypeScript (no DOM, no GPU, no store imports) — keep it that way. Web UI: primitives from web/src/components/ui_primitives/ only, design tokens per docs/DESIGN.md. Never import raw MUI. Commit per task with a descriptive message; do not reformat unrelated code. Model key (suggested per task): Opus 4.8 (claude-opus-4-8) — design-sensitive core, rendering internals. Sonnet 5 (claude-sonnet-5) — well-specified implementation and integration. Haiku 4.5 (claude-haiku-4-5-20251001) — mechanical plumbing with exact specs. Dependency graph: M1: T1 ──► T2 (schema) T1 ──► T3 (render) T1 ──► T4 (edit ops) T3 + T4 ──► T5 (agent tools) M2: T3 + T4 ──► T6 (inspector) T3 ──► T7 (chips) T3 + T5 ──► T8 (verify) M3: T3 ──► T9 (text clips) ──► T10 (text agent + UI, also needs T5, T6) Milestone 1 — Animation engine and integration T1 — Animation core in packages/timeline · Opus 4.8 Goal: the pure animation engine: types, easings, preset catalog, compiler, sampler, and edit-op semantics hooks. No web code. Read first: docs/plans/motion-design.md §§ Data model, Preset catalog, Editing semantics. Existing style references: packages/timeline/src/types.ts (doc-comment conventions), packages/timeline/src/splitClip.ts (pure-op shape). Create packages/timeline/src/animation/ with: types.ts — AnimationRole, EasingId, ClipAnimation, AnimationPresetId exactly as specified in the design doc. easing.ts — ease(id: EasingId, t: number): number, each easing pure, t ∈ [0,1] → [0,~1] (back/elastic may overshoot; clamp opacity at the composition site, not here). presets.ts — the v1 catalog table from the design doc (fade, slide, pop, spin, pulse, shake, bounce, kenBurns, float, breathe, rotate), each entry { id, roles, defaultDurationMs, defaultEasing, params: {name, default, min?, max?, options?}[], describe: string, curves(params, canvas): PropertyCurve[] }. Export ANIMATION_PRESETS and getAnimationPreset(id). compile.ts — compileClipAnimations(animations, clipDurationMs, canvas) → CompiledAnimation[] (window math per role incl. delayMs, clamping to clip duration, dropping degenerate windows, resolving normalized preset distances to px). kenBurns compiles as a full-clip one-shot with holdAfter: true, not a repeating loop. sample.ts — IDENTITY_SAMPLE, sampleAnimations(compiled, localMs) implementing the fold rules (add offsets/rotation, multiply scale/opacity; hold-before for in, hold-after for out, modulo for loop). Also export hasActiveAnimationWindow(compiled, localMs): boolean. index.ts barrel; re-export everything from packages/timeline/src/index.ts. Modify: packages/timeline/src/types.ts — add animations?: ClipAnimation[] to TimelineClip (after transitionIn), doc comment included. packages/timeline/src/splitClip.ts — left half keeps in, right keeps out, emphasis/loop copied to both, right-half ids regenerated. Follow how the file already handles caption/paragraphId for split-field precedent. packages/timeline/src/defaults.ts — no new factory needed; verify makeClip output stays valid with the field absent. Tests (packages/timeline/src/animation/__tests__/, Vitest — mirror the package’s existing test setup; if the package has no __tests__ yet, add the standard Vitest config used by sibling packages): easings: endpoints 0→0, 1→1; easeOut monotonic; back overshoots. window math per role, incl. out measured from clip end, delays, clamping when duration > clip length. sampler: identity outside windows, hold semantics, fold of in+loop concurrently, loop wraparound continuity (start==end value for every compiled animation with loop: true), delayed fade-in holds opacity 0 during delay, folded opacity clamped to [0,1] under easeOutBack. split semantics as above. determinism: same inputs → deeply equal compiled output. Acceptance: npm run test --workspace=packages/timeline green; npm run typecheck && npm run lint green; no imports from web/ or DOM types. T2 — Protocol schema + persistence round-trip · Haiku 4.5 Goal: animations survives save/load; older documents load unchanged. Depends on: T1 merged (types exist). Context: TimelineDocument persists as a JSON blob (no DB migration), but the wire schema strips unknown fields on PATCH — see the comment on storyboardBoardId in packages/timeline/src/types.ts. Modify: packages/protocol/src/api-schemas/timeline.ts — add to the clip schema: animations: z.array(clipAnimationSchema).optional() where clipAnimationSchema = z.object({ id: z.string(), role: z.enum(["in","out","emphasis","loop"]), preset: z.string(), durationMs: z.number(), delayMs: z.number().optional(), easing: z.string().optional(), enabled: z.boolean().optional(), params: z.record(z.union([z.number(), z.string(), z.boolean()])).optional() }). preset/easing are plain strings on the wire by design (forward compat); the compiler skips unknown presets. Check web/src/hooks/timeline/timelineDocumentPayload.ts and web/src/hooks/timeline/useLoadTimelineIntoStore.ts: if they pass clips through wholesale, no change; if they enumerate fields, add animations. Tests: protocol package round-trip test — a clip with two animations parses through the zod schema unchanged; a clip without the field parses; an animation with an unknown preset string parses (validation is not the schema’s job). Acceptance: npm run test --workspace=packages/protocol green; save → reload in the web store preserves animations (covered by the payload check above); typecheck/lint green. T3 — Preview + export rendering integration · Opus 4.8 Goal: animated transform/opacity visible in the live preview and identical in the MP4 export. Depends on: T1. Read first: design doc § Rendering integration; web/src/components/timeline/preview/sceneModel.ts (whole file — note the change-horizon contract on computeActiveLayersWithHorizon, which must NOT change); web/src/components/timeline/preview/PreviewCompositor.tsx; web/src/components/timeline/render/TimelineRenderer.ts; web/src/components/timeline/preview/gpu/transform.ts (position = canvas px from center, scale multiplies contain-fit, rotation radians). Create in sceneModel.ts (keep it pure): resolveAnimatedLayerProps(layer, currentTimeMs, canvas, cache) → { transform?: ClipTransform; opacity: number } per the design doc’s composition table. Identity fast-path: no enabled animations → return the layer’s existing values without allocating. hasActiveAnimation(layers, currentTimeMs, cache): boolean. An AnimationCompileCache (Map keyed by clip id, invalidated when the animations array reference changes) so compilation never runs in the rAF loop. Wire up: PreviewCompositor.tsx: where each ActiveLayer’s transform/opacity feed the GPU layer list, route through resolveAnimatedLayerProps. Then fix the redraw condition: the compositor skips redraws while the cached layer set is valid (nextChangeMs horizon) and nothing else changed. With animations, any tick where the playhead time changed and hasActiveAnimation(...) is true must redraw even though the layer set is cached — otherwise motion freezes mid-clip. A paused playhead needs only the one draw after the last seek: samples are pure functions of time, so a static frame stays correct. render/TimelineRenderer.ts: apply the same resolver at each stepped frame. The cache can live for the whole render. Caption layers get the same resolution (they already carry transform). Tests (Jest, web/src/components/timeline/preview/__tests__/): composition: static transform {position:{x:100,y:0}} + slide in from left at window midpoint → hand-computed expected transform/opacity. parity: for a seeded clip set, resolveAnimatedLayerProps output at t = {window start, mid, end, past-end} is identical when called the way the compositor calls it and the way the renderer calls it. horizon: nextChangeMs from computeActiveLayersWithHorizon is identical with and without animations on a clip — animations must never shrink the horizon; callers keep reusing cached layers and re-resolve properties per frame instead. Acceptance: cd web && npm test -- --testPathPattern=preview green; manual check via the debug harness or cd web && npm start optional; typecheck/lint green. Do not modify computeActiveLayersWithHorizon’s signature or horizon semantics. T4 — Store patch action + trim/duplicate semantics · Sonnet 5 Goal: clip animation edits flow through the Zustand store with undo/redo, and duplicate/trim behave per the design doc. Depends on: T1. Read first: design doc § Editing semantics; web/src/stores/timeline/TimelineStore.ts (find patchClip(clipId, patch) around the store actions; note the temporal undo middleware and pure-reducer style); packages/timeline/src/trimClip.ts. Do: Add a store action setClipAnimations(clipId, animations: ClipAnimation[]) (thin wrapper over patchClip is fine if patchClip already handles arbitrary fields — verify it does not shallow-merge arrays incorrectly). duplicateClip path (find the existing duplicate logic — store or useTimelineAgentBridge.ts): copy animations with fresh ids (crypto.randomUUID()). Trim: confirm no data rewrite is needed (compile-time clamping from T1 covers it) and add a comment where trimClip is called noting that. Verify split flows through the T1 splitClip change (store calls the pure op — confirm, don’t reimplement). Tests: store-level Jest test: set animations → undo → redo; split a clip with in+out+loop and assert the halves’ animation roles; duplicate regenerates ids. Acceptance: cd web && npm test -- --testPathPattern=timeline green; typecheck/lint green. T5 — Agent tools · Sonnet 5 Goal: an agent can discover the motion vocabulary, apply animations, and inspect them — the primary authoring surface. Depends on: T1, T3 (so results are visible), T4 (store action). Read first: design doc § Agent surface; web/src/lib/tools/builtin/timeline.ts (register pattern, targetParam, error convention); web/src/components/timeline/timelineAgentBridge.ts (TimelineAgentHandler); web/src/hooks/timeline/useTimelineAgentBridge.ts (handler implementation — see setClipParams at ~line 377 for the patch pattern). Do: Extend TimelineAgentHandler with setClipAnimations(target, animations: ClipAnimationInput[], mode: "add" | "replace") and clearClipAnimations(target, role?); implement in useTimelineAgentBridge.ts (resolve target by id/name/”selected” like the existing methods; fill defaults from the preset catalog; validate preset/role and throw descriptive errors that include the valid options). Register in web/src/lib/tools/builtin/timeline.ts: ui_timeline_animate_clip — zod params { target, mode: z.enum(["add","replace"]).optional(), animations: z.array(z.object({ role, preset, durationMs?, delayMs?, easing?, params? })) }. Description must include the compact vocabulary summary (preset names by role, that durations are ms, and the recommended loop: get_state → animate → get_clip_frames at window boundaries → adjust). ui_timeline_clear_animations — { target, role? }. ui_timeline_list_animation_presets — no params; returns ANIMATION_PRESETS mapped to { id, roles, params, defaults, describe }. Include animations in getSnapshot() clip entries (find the snapshot builder in useTimelineAgentBridge.ts). Tests: Jest tests beside existing frontend-tool tests (find the pattern under web/src/lib/tools/): registering is side-effectful on import, so test through FrontendToolRegistry.call with a stubbed handler — animate applies defaults, an unknown preset produces an error whose message lists the valid preset ids, clear with role filter keeps other roles. Acceptance: cd web && npm test -- --testPathPattern=tools green; typecheck/lint green; tool count in the manifest grows by 3. Milestone 2 — Manual UI + verification T6 — Inspector “Animate” section · Sonnet 5 Depends on: T3, T4. Read first: design doc § Manual UI; web/src/components/timeline/Inspector/ClipAdjustments.tsx (section pattern, fold behavior via usePersistedFold.ts), InspectorPrimitives.tsx (labeled rows, sliders, selects), web/src/components/ui_primitives/STRATEGY.md, docs/DESIGN.md. Do: an “Animate” section in the clip inspector: list current animations grouped by role; add-animation flow (role → preset filtered by roles → defaults applied); per-animation controls: duration (ms), delay, easing select, preset params (number params as sliders with catalog min/max, option params as selects); enable toggle; delete. All edits go through the T4 store action (single patchClip per change so undo granularity is per-edit). Constraints: ui_primitives only; SPACING/TYPOGRAPHY/MOTION tokens; no hardcoded px/transition strings; selection comes from the existing inspector wiring (TimelineInspector.tsx). Tests: RTL: renders animations from a stub clip; adding a pop in-animation dispatches the expected store patch; slider change patches params. Acceptance: cd web && npm test -- --testPathPattern=Inspector green; typecheck/lint green. T7 — Clip animation chips in track lanes · Haiku 4.5 Depends on: T3. Read first: web/src/components/timeline/Tracks/Clip.tsx (how fade handles / status badges are drawn at clip edges, zoom-dependent widths). Do: read-only affordances on clips that have animations: a left wedge spanning the in window’s width at current zoom, a right wedge for out, a small loop glyph (⟳ icon from the project’s icon set) when a loop/emphasis animation exists. Clicking them selects the clip (existing click path). Keep it cheap: pure derivation from clip.animations, no new store state; hide wedges below a minimum px width (~6 px) to avoid clutter at low zoom. Acceptance: visual check in cd web && npm start; no new lint/type errors; existing Tracks tests still green. T8 — End-to-end verification pass · Sonnet 5 Depends on: T3, T5. Goal: prove the loop the feature exists for: an agent animates a clip and the export contains the motion. Do: Add a seeded visual test to the existing web visual/debug-harness suite (web/tests/, see yts806379-everything-claude-code-e2e-testing patterns in repo tests): build a two-clip sequence in code (image clip + slide in + kenBurns loop; second clip pop in with delay), render frames at fixed timestamps through the real compositor, snapshot-compare. Export parity: run renderTimeline for the same sequence at a low fps (e.g. 5) for 3 s and assert frame N’s decoded pixels match the preview compositor’s output within tolerance (reuse whatever the existing export tests do — locate them under web/src/components/timeline/render/). Update docs/video-editor.md (and docs/timeline-editor-prd.md if it lists capabilities) with a short “Animations” section; follow docs/WRITING_STYLE.md. Acceptance: new tests green in cd web && npm test; docs updated; full npm run check green at repo root. Milestone 3 — Text clips (phase 2, separately mergeable) T9 — text clip type + rasterized rendering · Opus 4.8 Depends on: T3. Read first: design doc § Phase 2; web/src/components/timeline/preview/captionRender.ts (existing text rasterization — reuse its font/scale handling); web/src/components/timeline/preview/gpu/source.ts (how bitmaps become GPU sources); packages/timeline/src/types.ts. Do: packages/timeline: extend TimelineClip.mediaType union with "text"; add textStyle?: ClipTextStyle ({ text: string; fontFamily?: string; fontSizePx: number; fontWeight?: number; color: string; align?: "left"|"center"|"right"; maxWidthFrac?: number }); protocol schema field (same pattern as T2); makeClip support. Scene model: a text clip contributes a layer with kind: "image" semantics but sourced from a rasterized bitmap — introduce kind: "text" on ActiveLayer and teach both compositors (gpu/compositor.ts, gpu/canvas2dCompositor.ts) and the renderer to draw it: rasterize textStyle to an offscreen canvas at sequence resolution × devicePixelRatio, cache by hash of textStyle, upload like any image source. Text clips are sourceType: "imported" with status: "generated"-equivalent drawability — pick the minimal status handling that makes effectiveAssetId-gated paths treat text as always drawable (they have no asset id; adjust computeActiveLayers’ caption-only guard accordingly). Animations apply unchanged via resolveAnimatedLayerProps. Tests: scene-model unit tests (text layer present, no asset id required); rasterizer cache hit test; visual snapshot of a styled text frame. Acceptance: a text clip with pop in-animation renders in preview and export; npm run check green. T10 — Text clip authoring: agent tool + inspector · Sonnet 5 Depends on: T9, T5, T6. Do: Handler + tool ui_timeline_add_text_clip — { text, trackId?, startMs?, durationMs?, style? }, defaulting onto an overlay track (create one if none, same fallback logic as generateClip’s track selection in useTimelineAgentBridge.ts), default duration 3000 ms. Extend ui_timeline_set_clip_params (or the snapshot + a textStyle patch on setClipParams) so the agent can restyle existing text clips. Inspector: text section (content textarea, font size, weight, color, align) for mediaType === "text" clips, same primitives/tokens rules as T6. AddClipMenu.tsx: manual “Text” entry. Tests: tool test (adds clip on overlay track with defaults); RTL inspector test (edit patches textStyle). Acceptance: agent flow works end-to-end: ui_timeline_add_text_clip → ui_timeline_animate_clip (pop in, float loop) → ui_timeline_get_clip_frames shows animated text; npm run check green. Sequencing summary Order Task Model Parallelizable with 1 T1 animation core Opus 4.8 — 2 T2 protocol schema Haiku 4.5 T3, T4 2 T3 render integration Opus 4.8 T2, T4 2 T4 store + edit ops Sonnet 5 T2, T3 3 T5 agent tools Sonnet 5 — 4 T6 inspector Sonnet 5 T7 4 T7 clip chips Haiku 4.5 T6 5 T8 verification Sonnet 5 — 6 T9 text clips Opus 4.8 — 7 T10 text authoring Sonnet 5 — Ship gate for Milestone 1+2: T8 green. Text (M3) is separately mergeable.--- # Motion Design for the Timeline — Technical Design Source: https://docs.nodetool.ai/plans/motion-design.html Motion Design for the Timeline — Technical Design Status: implemented (July 2026). Companion doc: motion-design-tasks.md (implementation plan, agent-consumable tasks). Goal Jitter-style motion design inside the existing timeline editor: clips animate — slide in, pop, drift, loop — without the user placing keyframes. The authoring unit is a named animation preset attached to a clip (fadeIn, slideIn, kenBurns, …) with duration, delay, easing, and a few parameters. The primary authoring surface is the agent: a user says “make the title pop in and float”, the agent applies animations via ui_timeline_* tools, previews the result frame-by-frame, and adjusts. Manual UI (inspector panel, clip badges) is a thin layer over the same data. Presets compile to an internal keyframe/curve form evaluated by one sampler, so a curve editor or per-keyframe tools can be added later without a schema break. Implementation baseline The timeline supports preset motion on visual clips, including authored text and shapes. The table below records the baseline that this design extended. Concern Where it lives today Data model packages/timeline/src/types.ts — TimelineSequence, TimelineTrack, TimelineClip. Clips carry a static transform?: ClipTransform (position px from canvas center, scale multiplier on contain-fit, rotation radians, anchor normalized), opacity, effects[], transitionIn (crossfade), fadeInMs/fadeOutMs (audio-only, applied in web/src/components/timeline/preview/AudioGraph.ts). Scene resolution web/src/components/timeline/preview/sceneModel.ts — pure computeActiveLayersWithHorizon(tracks, clips, currentTimeMs) returns ActiveLayer[] plus nextChangeMs, the change horizon that lets the rAF loop skip recomputation between boundaries. Live preview web/src/components/timeline/preview/PreviewCompositor.tsx + preview/gpu/ (WebGPU with Canvas2D fallback; affine math in preview/gpu/transform.ts). Export web/src/components/timeline/render/TimelineRenderer.ts — steps in exact 1/fps increments, composites via the same scene model and GPU compositor, encodes MP4 via WebCodecs (mediabunny). Server ffmpeg path (packages/video-nodes/src/nodes/timeline.ts) is an explicit rough cut and stays one. Edit ops packages/timeline/src/splitClip.ts, trimClip.ts (pure); Zustand web/src/stores/timeline/TimelineStore.ts (patchClip(clipId, patch)), temporal undo/redo. Persistence TimelineDocument JSON blob in timeline-sequences (packages/models/src/timeline-sequence.ts); zod wire schema packages/protocol/src/api-schemas/timeline.ts. Fields absent from the zod schema are stripped on PATCH. Agent surface web/src/lib/tools/builtin/timeline.ts (13 ui_timeline_* tools) → TimelineAgentHandler interface in web/src/components/timeline/timelineAgentBridge.ts, implemented by web/src/hooks/timeline/useTimelineAgentBridge.ts. ui_timeline_get_clip_frames already lets the agent see rendered frames. Data model New module: packages/timeline/src/animation/ (pure, no DOM/GPU/store access), re-exported from packages/timeline/src/index.ts. // packages/timeline/src/animation/types.ts export type AnimationRole = "in" | "out" | "emphasis" | "loop"; export type EasingId = | "linear" | "easeIn" | "easeOut" | "easeInOut" // cubic | "easeOutBack" // overshoot (pop) | "easeOutElastic" | "easeOutBounce"; export interface ClipAnimation { id: string; // crypto.randomUUID() at creation role: AnimationRole; /** An AnimationPresetId. Typed `string` on purpose: documents saved by a * newer client may carry ids this build doesn't know — they parse fine and * are skipped at compile time (forward compat). Tool inputs and the UI * constrain to the catalog union. */ preset: string; /** Animation length in ms. For "loop", the period of one cycle. */ durationMs: number; /** * "in" | "emphasis" | "loop": offset from clip start to window start. * "out": offset from clip END back to window end (0 = ends exactly at clip end). * Default 0. */ delayMs?: number; easing?: EasingId; // default per preset /** Default true. Disabled animations are kept but not evaluated. */ enabled?: boolean; /** Preset-specific knobs; unknown keys ignored. See catalog. */ params?: Record<string, number | string | boolean>; } TimelineClip gains one optional field (after transitionIn): /** Motion-design animations evaluated at render time. Evaluation is * order-independent (the fold is commutative — see animation/sample.ts); * array order is presentation order in the UI only. */ animations?: ClipAnimation[]; Compiled form (internal substrate) Presets never render directly; they compile to curves. This is the extension point for a future curve editor and for baked custom motion. // packages/timeline/src/animation/compile.ts export type AnimatedProperty = | "offsetX" | "offsetY" // canvas px, resolved from normalized preset units | "scale" // uniform multiplier on ClipTransform.scale | "rotation" // radians, added to ClipTransform.rotation | "opacity" // multiplier on the layer's resolved opacity | "wipeProgress" // 0 hidden → 1 revealed (drives CompiledAnimation.mask) | "blur" // source px, added to the layer's blur radius | "brightness" // -1..1, added to the color grade's brightness term | "saturation"; // 0..4, multiplies the color grade's saturation export interface Keyframe { t: number; value: number; easing?: EasingId; } // t normalized 0..1 within the animation window; keyframes sorted by t; // first t === 0, last t === 1. export interface PropertyCurve { property: AnimatedProperty; keyframes: Keyframe[]; } export interface CompiledAnimation { role: AnimationRole; windowStartMs: number; // clip-local, resolved from role + delay + duration windowEndMs: number; loop: boolean; // repeat the window until clip end. Set by the // preset's compile, not derived from role alone: // most "loop"-role presets compile loop:true, but // kenBurns compiles loop:false (one-shot full clip) /** Hold curve endpoint values outside the window (in: hold t=0 before window; * out: hold t=1 after window). false → identity outside the window. */ holdBefore: boolean; // true for "in" holdAfter: boolean; // true for "out" curves: PropertyCurve[]; } /** Pure. canvas = sequence {width, height}; clipDurationMs for window math. */ export function compileClipAnimations( animations: ClipAnimation[], clipDurationMs: number, canvas: { width: number; height: number } ): CompiledAnimation[]; Preset position units are normalized (fraction of canvas width/height) inside the catalog; compileClipAnimations resolves them to px so the sampler and GPU stay unit-free. Sampler // packages/timeline/src/animation/sample.ts export interface AnimationSample { offsetX: number; // px, add to transform.position.x offsetY: number; scale: number; // multiply transform.scale.x and .y rotation: number; // radians, add to transform.rotation opacity: number; // 0..1, multiply layer opacity } export const IDENTITY_SAMPLE: AnimationSample; // {0, 0, 1, 0, 1} /** Pure. localMs = currentTimeMs - clip.startMs. */ export function sampleAnimations( compiled: CompiledAnimation[], localMs: number ): AnimationSample; Composition rules (deterministic, order-independent across roles): Evaluate every compiled animation independently, then fold: offsets and rotation add, scale and opacity multiply. After the fold, clamp the resulting opacity to [0,1] — overshoot easings (easeOutBack, easeOutElastic) may push individual samples past 1. Scale is clamped to ≥ 0. Within a window: map localMs to normalized t, find the bracketing keyframes, apply the segment’s easing (the easing stored on the right keyframe of the segment; ClipAnimation.easing overrides every segment), lerp. "in" before its window: hold the t=0 value — a delayed fadeIn keeps the clip invisible during the delay. After the window: identity. "out" after its window: hold the t=1 value (clip stays out). Before: identity. "emphasis" outside its window: identity. "loop": identity before windowStartMs; from there, t = ((localMs - start) % durationMs) / durationMs until clip end. Loop curves must start and end at the same value to avoid pops (catalog invariant, asserted in tests). Speed (speedMultiplier) does not rescale animation time: animations run on the timeline clock, not the source clock. (Matches how fades and crossfades behave today.) Preset catalog v1 packages/timeline/src/animation/presets.ts. Each entry: id, allowed roles, default duration/easing, params with defaults, and a curve generator (params, canvas) => PropertyCurve[]. AnimationPresetId is the union of ids. Preset Roles Params (defaults) Motion fade in, out — opacity 0↔1 slide in, out direction: "left"\|"right"\|"up"\|"down" (“left”), distance (0.3 = fraction of canvas) offset from/to direction, opacity 0↔1 pop in, out overshoot (1.08) scale 0.6↔1 with easeOutBack, opacity 0↔1 spin in, out turns (0.25) rotation ±turns·2π, opacity 0↔1 wipe in, out direction: "left"\|"right"\|"up"\|"down" (“left”; the edge the reveal starts from — “up” = top, “down” = bottom), softness (0.05 = feather width as fraction of the wipe axis, 0 = hard edge) directional mask reveal. Added after v1 with per-layer compositor masking: the preset compiles a wipeProgress curve (0 = hidden, 1 = revealed), direction/softness ride on CompiledAnimation.mask, and AnimationSample.mask carries the resolved mask to both compositors. Overlapping wipes fold by keeping the smaller progress (more hidden wins). The mask lives in the layer’s own quad space, so the wipe edge rotates with the clip blur in, out amount (12 = blur radius in source px, 0..40) blur amount→0 with an opacity fade (rack focus); reversed for out colorFade in, out — saturation 0→1 (grayscale → full color); reversed for out pulse emphasis intensity (0.06) scale 1→1+i→1 flash emphasis intensity (0.6, 0..1) brightness 0→+i→0 (brief exposure spike) shake emphasis intensity (0.02), cycles (4) offsetX zig-zag, fraction of canvas width bounce emphasis height (0.05) offsetY dip with easeOutBounce kenBurns loop zoom (0.12), direction (“in”), driftX/driftY (0.02) slow scale 1→1+z (or reverse) + drift. Compiles loop:false, window = full clip, holdAfter:true — it runs once over the whole clip, so durationMs and delayMs are ignored and the monotonic curve is exempt from the loop start==end invariant float loop amplitude (0.015), periodMs via durationMs offsetY sine bob (piecewise-cubic approximation over ≥8 keyframes) breathe loop intensity (0.03) scale sine rotate loop rpm via durationMs rotation 0→2π per cycle Defaults: in/out 500 ms, emphasis 600 ms, loop 3000 ms. in defaults to easeOut, out to easeIn, others per table. blur, flash, and colorFade animate GPU effect parameters. They drive blur/brightness/saturation curves that the sampler folds like the effect pre-pass aggregates (blur and brightness add, saturation multiplies). At the render site resolveAnimatedLayerProps composes them with the clip’s static effects: animated blur adds to the static blur radius, animated brightness adds to the grade’s brightness term, animated saturation multiplies the grade’s saturation — emitted as synthesized ClipEffects so both compositors reuse the existing color-grade / Gaussian-blur pipeline, keeping preview/export parity. Rendering integration The layer set stays cacheable; animated properties are resolved per frame. computeActiveLayersWithHorizon and its change-horizon contract are untouched — animation windows never add or remove layers, so nextChangeMs stays correct for set membership. New pure helper (in sceneModel.ts, since it is shared preview/export vocabulary): export interface AnimatedLayerProps { transform?: ClipTransform; opacity: number; } /** Compose a layer's static transform/opacity with its animation sample at t. * Returns the inputs unchanged when the clip has no enabled animations. */ export function resolveAnimatedLayerProps( layer: ActiveLayer, currentTimeMs: number, canvas: { width: number; height: number }, cache?: AnimationCompileCache // keyed by clip id + animations ref ): AnimatedLayerProps; Call sites (the only two, keeping 1:1 preview/export parity): Live preview — PreviewCompositor.tsx, where each ActiveLayer’s transform/opacity is handed to the GPU layer list. While any active layer has an enabled animation whose window (or loop tail) covers the current time, the compositor must redraw every rAF tick even when paused- idle short-circuits would otherwise skip drawing — expose hasActiveAnimation(layers, timeMs) next to the resolver for that check. Export — render/TimelineRenderer.ts, same resolver per stepped frame. No other change: WebCodecs encode path is agnostic. Composition into ClipTransform (identity when clip has none: position {0,0}, scale {1,1}, rotation 0, anchor {0.5,0.5}): position.x += sample.offsetX scale.x *= sample.scale position.y += sample.offsetY scale.y *= sample.scale rotation += sample.rotation opacity *= sample.opacity sample.opacity multiplies the already-resolved layer opacity (base × crossfade), so animations compose with transitionIn rather than fighting it. Caption layers (kind: "caption") get the same treatment — they carry the clip’s transform already. Compilation is memoized per clip (WeakMap on the animations array ref, or a Map<clipId, {ref, compiled}>) — the per-frame cost is sampling only: a handful of lerps per animated layer, no allocation in the steady state (sampler writes into a scratch AnimationSample). Persistence TimelineDocument is a JSON blob — no DB migration. Old documents lack animations; every consumer treats undefined as “no animations”. Required: add animations to the clip zod schema in packages/protocol/src/api-schemas/timeline.ts (see the existing storyboardBoardId comment in types.ts: fields missing from the schema are stripped by PATCH). Use a permissive z.array(clipAnimation) with preset: z.string() (not the enum) so older servers don’t reject documents from newer clients; unknown presets are skipped at compile time with a console warning. Autosave/undo need no changes: patchClip flows through the temporal middleware like any clip field. Editing semantics Implemented where the pure ops live (packages/timeline/src/): splitClip: left half keeps role: "in" animations, right half keeps "out"; "emphasis" and "loop" are copied to both halves (loop windows re-derive naturally from each half’s duration). Ids on the right half are regenerated. trimClip / duration changes: windows are clamped at sample time (compileClipAnimations clamps windowEndMs to clipDurationMs, and drops a window whose start ≥ clip duration), so trims need no data rewrite. When in+out overlap because the clip got too short, both still evaluate; fold rules keep the result continuous. duplicateClip: copies animations with fresh ids. Linked clips / speed / bake: no interaction — animations live on the timeline clock. Agent surface Extend TimelineAgentHandler (web/src/components/timeline/timelineAgentBridge.ts, implemented in web/src/hooks/timeline/useTimelineAgentBridge.ts) and register three tools in web/src/lib/tools/builtin/timeline.ts: ui_timeline_animate_clip — { target, animations: ClipAnimationInput[], mode?: "add" | "replace" } (replace default; add appends). Each input: { role, preset, durationMs?, delayMs?, easing?, params? } — ids and defaults filled in by the handler. Returns the updated clip including resolved animations. On validation errors (unknown preset, role not allowed for the preset) the handler throws with a message listing the valid options — the tool layer surfaces handler throws back to the agent, same as every existing ui_timeline_* tool. ui_timeline_clear_animations — { target, role? }. ui_timeline_list_animation_presets — no args; returns the catalog (id, roles, params with defaults and ranges, one-line description, default duration/easing). This is the runtime vocabulary discovery; tool descriptions carry a compact summary so simple requests skip the extra call. Also: include animations in getSnapshot() clip entries so ui_timeline_get_state shows current motion. The critique loop needs no new tools: ui_timeline_get_clip_frames + ui_timeline_seek already exist. Tool descriptions should state the intended loop: get state → animate → get frames at window boundaries → adjust. Manual UI (thin, after the agent surface) Inspector: an “Animate” section in web/src/components/timeline/Inspector/ (pattern: ClipAdjustments.tsx, primitives from InspectorPrimitives.tsx). Per role: preset select, duration, delay, easing, preset params. ui_primitives only; SPACING/TYPOGRAPHY/MOTION tokens (see docs/DESIGN.md). Clip chips: in web/src/components/timeline/Tracks/Clip.tsx, small in/out zone markers (left/right wedge spanning the window width at current zoom) and a loop glyph — read-only affordance, click selects the clip and opens the inspector section. Phase 2 — text and shape clips Jitter is mostly motion typography; this design animates whatever a clip can draw, so text arrives as a new clip kind, not an animation change: mediaType: "text" on TimelineClip + textStyle?: ClipTextStyle (content, font family/size/weight, fill, alignment, max width fraction). Rendered as a rasterized layer: reuse the caption rasterization approach (preview/captionRender.ts) — draw to an offscreen canvas at sequence resolution × device scale, hand the bitmap to the compositor as an image source keyed by a style hash. Animation then applies for free via the same resolveAnimatedLayerProps. Agent tool ui_timeline_add_text_clip. Shapes (rect/ellipse/line) follow the same rasterize-to-layer pattern later. Non-goals Lottie/GIF export — MP4 via the existing WebCodecs path only. Curve editor UI / dope sheet — the compiled-curve substrate exists for it, but no UI in this effort. Server-side render parity — the ffmpeg rough cut ignores animations, exactly as it ignores effects and transitions today. Per-property keyframe authoring by agents — presets + params only until real demand appears. Testing packages/timeline/src/animation/__tests__/ (Vitest): easings (endpoint + monotonicity), window math for every role incl. delays and clamping, sampler fold rules, loop wraparound and start==end invariant, split/trim semantics, compile determinism. web/src/components/timeline/preview/__tests__/ (Jest): resolveAnimatedLayerProps composition against hand-computed frames; hasActiveAnimation horizon behavior; export/preview parity by sampling both paths at the same t. Visual: one debug-harness / Playwright pass rendering a seeded sequence with each preset at t = window midpoints, snapshot-compared (existing web/tests/ visual patterns). Risks Skipped redraws during playback: the preview skips redraws while the cached layer set is valid (nextChangeMs horizon). With animations, any tick where the playhead moved must redraw even though the set is cached, or motion freezes mid-clip. A paused playhead needs only one draw — samples are pure functions of time. Caught by the T8 visual test and manual check, not by unit tests. Schema strip: forgetting the protocol zod field silently loses animations on save. Caught by a round-trip test in the protocol package. Perf: per-frame sampling is O(active animated layers × curves); with memoized compilation this is negligible next to video decode. Do not compile in the rAF loop.--- # Motion Typography — Per-Word Staggered Text Animation Source: https://docs.nodetool.ai/plans/motion-typography.html Motion Typography — Per-Word Staggered Text Animation Status: implemented (July 2026). Companion doc: motion-design.md (the preset/curve/sampler substrate this builds on). Goal Jitter is mostly motion typography, and the signature move is the stagger: words of a title animating in sequence, each offset in time from the previous — a headline whose words pop in one after another, a kicker that slides away word by word. The motion-design system animates a text clip as one block; this design splits the block into words without breaking the one-clip-one-layer model. Authoring stays preset-shaped: a stagger is not a new animation type but a modifier on an existing ClipAnimation — “run this pop-in once per word, 120 ms apart”. The agent (ui_timeline_animate_clip) is the primary authoring surface; the inspector gets a thin stagger row on text clips. v1 unit is words. Characters and lines can follow behind the same stagger.unit field without a schema break. Architecture: stagger inside the text rasterizer Two candidates were on the table: A (chosen). Time-dependent text raster. Keep one layer per text clip. The TextRasterizer already measures every word to wrap lines, so per-word geometry is one refactor away. At draw time each word evaluates its own animation sample (same compiled curves, per-word time offset) and is drawn into the block’s canvas with its own translate/rotate/scale/alpha — 2D canvas transforms per word are cheap. The compositor, layer model, and computeActiveLayersWithHorizon’s change-horizon contract are untouched. Preview/export parity is free because both paths already rasterize through the same class. B (rejected). One sub-layer per word. Explode the clip into N word layers in the scene model, each animated via resolveAnimatedLayerProps with a time offset. GPU-composited words stay crisper under extreme scale/rotate, but it breaks the layer-set caching and horizon assumptions, multiplies layer count by word count, and touches every layer consumer (compositors, gizmo, placeholder DOM, video-slot bookkeeping). That price buys quality headroom v1 does not need — a word rotating inside a sequence-resolution raster is more than good enough for titles. The cost of A is cacheability: while a stagger window is active the raster is a function of time, not just style. That is exactly the caption situation (karaoke highlight), and the same remedy applies (see Caching). Data model ClipAnimation gains one optional field (engine animation/types.ts, wire schema packages/protocol/src/api-schemas/timeline.ts — the zod field is mandatory or PATCH strips it): export interface AnimationStagger { unit: string; // "word" — unknown units compile un-staggered (forward compat) offsetMs: number; // delay between successive words; must be > 0 from?: "start" | "end" | "center"; // which word leads; default "start" } export interface ClipAnimation { // …existing fields… stagger?: AnimationStagger; } from ordering: "start" = first word first, "end" = last word first, "center" = middle words first, rippling outward (delay = |i − (N−1)/2| × offsetMs). Stagger is only meaningful on text clips. The agent bridge rejects it on any other clip; a hand-edited document that carries it elsewhere (or an unknown unit) compiles as a plain block animation — same degrade-gracefully posture as unknown presets. Window and duration semantics durationMs is the per-word duration; the total span stretches. Word i runs the full preset curve over durationMs, delayed delay(i) from the span start, so the span is durationMs + maxDelay where maxDelay = maxFactor(from, N) × offsetMs. This is the reading an agent setting it expects — “each word pops for 400 ms, 120 ms apart” — and it keeps durationMs meaning the same thing with and without a stagger. The alternative (squeeze N word-windows inside durationMs) makes the per-word motion speed depend on word count, which reads as a bug when the text changes. Anchoring per role: in / emphasis: span grows forward from delayMs; the first word starts where the un-staggered window would have. out: span grows backward, so the last word to leave still finishes at clipEnd − delayMs — the clip ends empty, like an un-staggered out. loop: no span change; delay(i) is a phase shift on the cycle. Short clips shrink the offset, never the word duration. When durationMs + maxDelay does not fit the clip, the effective offsetMs is scaled down so the last word still completes inside the clip. If no positive offset fits (the clip is shorter than one word’s duration), the stagger drops and the animation compiles as a plain block window, clamped as before. Compiled form: compileClipAnimations takes options.staggerCount (the word count — countStaggerUnits(text), whitespace-split, matching the rasterizer’s wrap tokenization). A compiled stagger rides on the animation as CompiledAnimation.stagger (count, effective offsetMs, from, unitDurationMs, maxDelayMs), and windowStartMs..windowEndMs covers the full span. That one choice keeps the rest of the system ignorant of staggering: hasActiveAnimationWindow (and therefore the preview’s redraw gate and the compile cache) see one window that happens to be longer. Sampling: block/word split A staggered animation’s curves are split across two samplers: sampleAnimations (block level, unchanged signature) folds only its effect/mask curves (blur, brightness, saturation, wipeProgress), over the full span. Transform/opacity curves are skipped — they belong to the words. sampleStaggeredAnimations(compiled, localMs, wordIndex, out?) (new, pure) folds only transform/opacity curves of staggered animations, each word evaluated at its own window [spanStart + delay(i), + unitDurationMs] with the role’s usual holds (in: hold t=0 before, identity after; out: identity before, hold t=1 after; loop: phase shift). Un-staggered animations are skipped — they already applied at the layer. So on a text clip, per-word offset/scale/rotation/opacity come from the raster; block-level animations (a Ken Burns drift, a wipe, an added fade) compose on the layer exactly as before, and a staggered blur preset still blurs the whole block while its opacity fade runs per word. Per-word masks and effect parameters are explicitly out of scope for v1 (see Non-goals). Rendering integration TextRasterizer.rasterize gains an optional stagger argument ({ compiled, localMs }). Without it the block draw path is byte-identical to before. With it, the rasterizer lays out word boxes (line breaks use the same measured-candidate rule as the block path, so a staggered title wraps exactly like its un-staggered self) and draws each word about its own center: translate(wordCenter + sample.offset) → rotate → scale → alpha → fillText The per-frame context is resolved by one shared helper, resolveTextStaggerContext(clip, timeMs, canvas, cache) in sceneModel.ts, called from the only three text draw sites — PreviewCompositor, TimelineRenderer, and rasterClipFrames (the agent’s ui_timeline_get_clip_frames path) — so preview, export, and agent frames render per-word motion from one code path. The preview’s redraw gate needs no change: hasActiveAnimation reads the compiled full-span window, so the compositor keeps redrawing every rAF tick until the last word lands. Caching The style-hash bitmap cache cannot serve a mid-stagger frame. Policy, per rasterize call: Active (clip-local time inside any staggered animation’s span): draw fresh, uncached — a new bitmap per frame, like captions during a karaoke highlight. The rasterizer keeps the latest active bitmap per style key and closes the previous one on the next call (the caller has composited it by then), so playback does not accumulate full-frame bitmaps. Per-frame cost is one offscreen canvas fill of a few dozen words plus a texture upload — the price captions already pay, negligible next to video decode. Time- bucketed caching was rejected: 64 cached 1080p frames is ~0.5 GB. Held (before/after the span): the frame is static (hold or identity) but differs from the plain raster, so it caches under the normal key plus a suffix of the compiled-array identity and a per-animation phase signature (before/after). Steady state after a stagger completes is one cached bitmap again — the GPU re-uses the uploaded texture. Agent surface ui_timeline_animate_clip accepts stagger: { unit: "word", offsetMs, from? } per animation, documented in the tool description with the motion-typography use case. Validation lives where preset/role validation already lives: buildClipAnimation checks the unit and offset, the bridge handler rejects stagger on non-text clips with a corrective message. Ids, defaults, and the returned clip node are unchanged. ui_timeline_list_animation_presets is untouched — stagger is orthogonal to the catalog and composes with any non-full-clip preset. Manual UI Text clips get two rows inside each animation’s editor in the Animate section: a “Stagger words” toggle and, when on, a word-offset pill (ms). from stays agent/JSON-only until demand appears. Hidden for non-text clips and full-clip presets (kenBurns ignores stagger). Editing semantics Nothing new to store: splitClip/duplicateClip copy animations wholesale (stagger included), and trims re-clamp at compile time exactly like plain windows — the compile cache keys on clip duration and word count, so editing the text re-derives the span. Non-goals Character and line units — the schema (unit) and the sampler (unit index/count) are ready; the rasterizer would need per-glyph layout. Per-word effect/mask properties — blur/brightness/saturation and wipes stay block-level; per-word canvas filters would cost a save/filter/restore per word per frame for little visible gain at title sizes. Caption stagger — captions have real word timings; their karaoke path stays separate. Per-word easing/curve overrides — one curve, offset in time, is the Jitter look; anything richer belongs to a future curve editor. Risks Stale raster mid-stagger: a cached bitmap served during the window would freeze the motion. Covered by the active-window cache bypass and a regression test; the phase-suffixed keys cover the held frames. Word-count drift: the engine counts words by whitespace split, the rasterizer by the same split during wrap. A divergence would mis-time later words. Both sides share the tokenization rule (countStaggerUnits is documented against the rasterizer) and the compile cache re-keys on count. Raster churn during playback: while a stagger is active over a playing video, every frame re-rasterizes and re-uploads the text. Accepted (caption precedent); if profiling ever flags it, quantizing localMs to the frame rate inside resolveTextStaggerContext is a local fix.--- # ui_primitives overhaul — verified brief, tech design, implementation plan Source: https://docs.nodetool.ai/plans/ui-primitives-overhaul.html ui_primitives overhaul — verified brief, tech design, implementation plan Status: planned. Verified against main @ 2cf277b (2026-07-18). All file:line references below were re-checked against that commit; when the code drifts, re-verify before implementing. 1. Problem statement web/src/components/ui_primitives/ is a naming layer over MUI, not a design system. The import policy is enforced (never import raw MUI outside ui_primitives/ and editor_ui/) and holds in practice — a sweep found zero raw TextField/Select imports in app code. But the visual decisions were never made, so a screen built 100% from primitives still looks amateur by default: fields with mismatched variants, invisible panels, four label treatments, and per-component invented heights. The measure of success: a bare <Panel><FlexColumn gap={3}><TextInput label="Name" /><SelectField label="Kind" options={...} /></FlexColumn></Panel> with no styling props renders as a professional form — visible panel, two identically-styled fields, labels above controls, one control height. Reference implementation of the target look: the storyboard header in web/src/components/storyboard/StoryboardBoard.tsx (as of 2cf277b). It hand-builds everything this plan systematizes: a local Field wrapper (StoryboardBoard.tsx:69) putting labels above controls, uniform 36px control heights via CSS override (StoryboardBoard.tsx:131-133), an uppercase .group-label (:112-116), a responsive .header-grid (:102-111), and an overflow: visible workaround for Panel clipping the floating label (:96-98). Note it does not solve everything — its Panel is still the invisible default background. Once the primitives are fixed, migrate this file to the shared versions and delete the local workarounds (Fix 4). 2. Defect inventory (verified) Every claim below was verified against the codebase; corrections from the original audit draft are marked ⚠. 2.1 No control-level tokens tokens.ts exports FONT_WEIGHT, FONT_SIZE_SANS/FONT_SIZE_MONO, TYPOGRAPHY, MOTION, reducedMotion, Z_INDEX, BORDER_RADIUS, and scrollbar helpers — but no control height, control padding, or field radius. Every primitive invents its own: Component Value Source EditorButton 24 / 28 raw px editor_ui/EditorButton.tsx:55 Editor node controls 28px theme.editor.heightNode (ThemeNodetool.tsx:70), used at themes/components/editorControls.ts:108 Inspector controls 32px theme.editor.heightInspector (ThemeNodetool.tsx:71), used at editorControls.ts:129 TagInput 48px ui_primitives/TagInput.tsx:51 SearchInput radius 8px ui_primitives/SearchInput.tsx:48 Editor control radius 6px theme.editor.controlRadius (ThemeNodetool.tsx:76) MuiButton radius theme.rounded.buttonLarge = 6px ThemeNodetool.tsx:241 Consequences: a compact EditorButton (24px) next to a node-canvas select (28px) misaligns by 4px; next to an inspector select by 8px. Three independent radius sources for controls. A second, parallel control scale lives on theme.editor that EditorButton does not read. EditorButton.tsx:65 does height: size ? undefined : height — passing size silently discards the density height. Off-grid values inside the primitives whose own docs ban them (spacing.ts:23 forbids 0.25/0.75/1.25/2.5/5 steps; canonical scale is [0, 0.5, 1, 1.5, 2, 3, 4, 6, 8] units of 4px): Slider.tsx:45 — 13px vertical padding (normal density) NodeSlider.tsx:56 — 3px margin-top (compact) Panel.tsx:39 — comfortable: 2.5 units; Panel.tsx:151 computes paddingValue / 2 ⚠ (a computed halving, not a literal 1.25 — yields 1.25 units = 5px for comfortable) ⚠ Additional offenders the original draft missed (same class of defect, fold into the Fix 1 sweep): File Defect TabGroup.tsx:86,105 minHeight: isSmall ? "36px" : "42px" — 42 is off-grid Chip.tsx:64 height: "22px" — off-grid UndoRedoButtons.tsx:51-52,68,73 padding: 6px, border-radius: 6px, 24px box MenuItemPrimitive.tsx:46,114,131,135 border-radius: 4px literals, min-height: 32px, min-width: 28px MobileBottomSheet.tsx:45-46,62-63 16px corner radii, 40px/4px grab handle WarningBanner.tsx:73 border-radius: 8px Breadcrumbs.tsx:56 border-radius: 4px ZoomControls.tsx:34 minWidth: "45px" — off-grid InfoTooltip.tsx:77 font-size: 14px — off the type scale NotificationBadge.tsx:56 font-size: 10px — off the type scale 2.2 Unrelated defaults across primitives TextInput.tsx:52-53 → variant="outlined", size="medium" SelectField.tsx:98-99 → size="medium", variant="standard" A no-props TextInput above a no-props SelectField renders a boxed field above an underline-only field — the default composition is broken. Of 29 real SelectField call sites, 17 rely on the standard default and 12 pass variant="standard" redundantly (7 of those in SettingsMenu.tsx alone). Surfaces: Panel.tsx:89-90 → bordered=false, background="default". Dark background.default is #08090A (paletteDark.ts:268) — same as the page. A default Panel is invisible. (Corrected during PR 3: the “42 call sites” here was a <Panel prefix-grep artifact matching PanelLeft, PanelHeadline, etc. — the primitive is actually used at 13 sites across 10 files. Also found: Panel’s numeric borderRadius: 6 went through the sx multiplier and rendered 36px, not 6px.) Surface.tsx:64-68 → background="paper", bordered=false, elevation=0 with explicit boxShadow: "none". Different background AND a different radius source (theme.rounded[...], Surface.tsx:92-95) than Panel (theme.shape.borderRadius, Panel.tsx:135). size semantics diverge: TextInput maps compact → MUI small (TextInput.tsx:73); SelectField’s standard variant barely responds to size; EditorButton discards its height when size is passed (see 2.1). Flex containers: FlexRow.tsx:55, FlexColumn.tsx:53, Stack.tsx:61 all default gap/spacing to 0. ~374 of ~1010 usages (37%) rely on that default, so it cannot be changed — form spacing must come from the Fix 4 composition primitives instead. 2.3 Four label treatments, none shared MUI floating label — TextInput, hand-correcting MUI’s transform (TextInput.tsx:95-96 translate(14px, 17px)), sized via theme.fontSizeNormal || "15px" ⚠ (token with fallback, not a hardcoded 15px), opacity: 0.6 (TextInput.tsx:89-92). MUI InputLabel — SelectField.tsx:123-129, same 15px, no opacity, no transform fix. Renders differently from TextInput’s at rest. Label.tsx — Typography component="label" (:81), 13px, weight 500, text.secondary, baked marginBottom: theme.spacing(0.5) = 2px (:88). FormField.tsx:85-96 — reimplements the same thing with its own Typography (does NOT use Label), duplicating the required-asterisk logic (FormField.tsx:98-108 vs Label.tsx:95-104). The theme sets a fifth treatment on MuiFormLabel (ThemeNodetool.tsx:249-262): capitalize, 13px, padding: 0 0 8px 0 — which TextInput/SelectField then override back to 15px, defeating the theme’s label token exactly where labels matter. The floating-label choice is also a bug class: the shrunk label renders above the input box, and Panel.tsx:136 is overflow: hidden, so a TextInput at the top of a Panel gets its label clipped. This shipped (the storyboard title) and is worked around in StoryboardBoard.tsx:96-98 with overflow: visible. Escape hatches are asymmetric: SelectField has hideLabel (SelectField.tsx:48); TextInput has none — the storyboard Field wrapper exists partly because you cannot stop TextInput from floating its label. 2.4 No usable form composition primitive FormField exists but has exactly 1 consumer (components/collections/CollectionForm.tsx); STRATEGY.md:47 counts 14+ places doing label+input+helper by hand. FormField double-labels: it renders its own label while TextInput/SelectField render theirs. No FormRow, FormSection, FieldGroup, or grid primitive. Only flex helpers — which default to gap=0 (2.2). Spacing ownership is incoherent: SelectField.tsx:117 comments “spacing is the parent’s job” while Label.tsx:88, FormField.tsx:120, AutocompleteTagInput.tsx, EmptyState, ProgressBar, SectionHeader, and Panel bake their own margins. Stacking three fields yields four different gaps. TextInput additionally inherits MUI’s default helper-text margin (3px 14px 0) which nothing overrides — off-grid and different from SelectField’s mt: 0.5 (SelectField.tsx:157). 2.5 The type scale is fake Text.tsx:14 advertises 8 sizes; ThemeNodetool.tsx:39-46 collapses them to 4 real values: giant = bigger = big = 18px; normal = 15px; small = 13px; smaller = tiny = tinyer = 11px. <Text size="tiny"> and <Text size="smaller"> are indistinguishable. Usage counts across web/src (for migration sizing): small 277, normal 71, big 42, smaller 39, tiny 36, bigger 17, tinyer 4, giant 2 — 488 total. The aliased names (bigger, tiny, tinyer, giant) cover 59 call sites. Nothing consumes the legacy CSS vars (--fontSizeTiny etc.) directly. 2.6 Zero-opinion passthroughs masquerading as primitives Box.tsx — 2-line re-export. (Acceptable as an import-policy shim, but should say so.) muiReexports.ts:107-130 re-exports Accordion/AccordionSummary/ AccordionDetails, Tabs/Tab, ToggleButton/ToggleButtonGroup, Modal, Fade, Toolbar, LinearProgress under a header comment (:8-10) claiming “the MUI theme overrides in ThemeNodetool already style them”. ⚠ The theme’s actual override set is larger than the original draft claimed — MuiCssBaseline, MuiTypography, MuiButton, MuiFormLabel, MuiFormControl, MuiPopover, MuiModal, MuiToolbar, MuiDialogTitle, MuiTooltip, MuiDivider in ThemeNodetool.tsx, plus MuiOutlinedInput, MuiSelect, MuiPaper, MuiMenu, MuiMenuItem, MuiSwitch from editorControls.ts — but the core claim holds: there is no MuiTabs, MuiToggleButton, MuiAccordion, or MuiLinearProgress override anywhere in themes/. Those four render stock MUI. Contrast: MuiTooltip (ThemeNodetool.tsx:301-319) gets backdrop blur, a divider border, and a layered shadow — more surface craft than any panel, card, or input receives. Inputs have no default field background: only editor-scoped controls get Paper.overlay, gated on the nt-editor-control marker class (editorControls.ts:19-23, class defined in constants/editorUiClasses.ts:10). SearchInput.tsx:49 gives itself action.hover — so search boxes read as fields and plain text inputs don’t. Paper.overlay exists in both schemes (#17181B dark, #F0EDE6 light), so a default field background is safe in both. 3. Tech design 3.1 Control tokens (CONTROL) Add to ui_primitives/tokens.ts: /** * Control-chrome tokens: the five control heights, the two horizontal * paddings, and the single field radius. Every interactive control in * ui_primitives/ and editor_ui/ derives its box from these. */ export const CONTROL = { /** px, all on the 4px grid */ height: { xs: 24, // toolbar-density buttons (EditorButton compact) sm: 28, // node-canvas controls, EditorButton normal md: 32, // inspector controls lg: 36, // default form controls (TextInput, SelectField, SearchInput) xl: 44, // touch targets (MobileBottomSheet actions, TagInput) }, paddingX: { compact: 8, normal: 12 }, // px, on-grid radius: BORDER_RADIUS.md, // 6px — matches editor controlRadius + buttonLarge } as const; Rationale for five steps rather than the three the draft proposed (28/36/44): the editor’s two densities (28 node, 32 inspector) are real, shipped, and correct for a dense canvas — collapsing them into a 3-step scale would either restyle the whole editor or leave theme.editor as a second unreconciled scale, which is the defect we’re fixing. Five named steps, one source. Radius: BORDER_RADIUS.md (6px) — the value the editor controls and MuiButton already use, so the flip is churn-free for the editor and only moves SearchInput (8→6). (Alternative considered: sm/4px, rejected — it would restyle every editor control and button for no gain.) Wiring: theme.editor.heightNode / heightInspector / controlRadius (ThemeNodetool.tsx:69-79) become derived: `${CONTROL.height.sm}px`, `${CONTROL.height.md}px`, CONTROL.radius. ThemeNodetool.tsx may import from ui_primitives/tokens.ts (tokens has no theme dependency beyond types — no cycle). theme.d.ts needs no change; the shape of theme.editor is unchanged. EditorButton: density === "compact" ? CONTROL.height.xs : CONTROL.height.sm; the size prop maps small → sm, medium → md instead of discarding the height (EditorButton.tsx:65 bug). TextInput / SelectField: compact/small → CONTROL.height.sm, default → CONTROL.height.lg, applied as minHeight on the input root (leave multiline free to grow, as the storyboard CSS does at StoryboardBoard.tsx:135). SearchInput: radius → CONTROL.radius; height → lg (or sm when its compact variant is set). TagInput: 48px → CONTROL.height.xl (44) as minHeight (chips can wrap taller). Off-grid fixes in the same pass: Slider 13px→12px, NodeSlider 3px→4px, Panel comfortable: 2.5→3 (= SPACING.lg) and replace the paddingValue / 2 computed bottom padding with snapSpacing, plus the 2.1 ⚠-table sweep (TabGroup 42→40, Chip 22→24 or pill height token, radius literals → BORDER_RADIUS.*, font literals → FONT_SIZE_SANS.*). 3.2 One label treatment Standardize on label above the control: fontSizeSmall (13px), weight 500, text.secondary, no text-transform, 4px gap to the control. This is what Label.tsx already implements and what the storyboard Field wrapper hand-rolls. Label becomes the single renderer. Add htmlFor support (already has the asterisk); change its baked marginBottom from 2px to 4px (theme.spacing(1)) — the one component allowed to own that gap. TextInput: stop forwarding label to MUI. Render <Label htmlFor={id}> above the field; generate id via React.useId() when the caller passes none, so label/input association survives. Delete the floating-label correction sx (TextInput.tsx:88-101). Add hideLabel for parity with SelectField (renders the label visually hidden but keeps aria-label). SelectField: drop the InputLabel path (SelectField.tsx:123-129); render the shared Label above. hideLabel keeps its contract. FormField: use Label; delete its own Typography block and asterisk duplicate (FormField.tsx:85-108). Theme: remove textTransform: "capitalize" and the 8px padding from MuiFormLabel (ThemeNodetool.tsx:249-262) — labels are no longer MUI-managed; keep the 13px/500 rules for any residual MUI-internal labels. Outlined inputs no longer need the label notch; MUI renders no legend when no label is passed, so nothing to patch. This kills the clipped-label bug class. The overflow: visible workaround in StoryboardBoard.tsx:96-98 and its .field-label CSS go in Fix 4’s migration. Call-site impact: ~56 TextInput label= sites (17 files) and 29 SelectField label= sites keep their code unchanged — the prop’s meaning changes from “floating label” to “label above”, which is the point. Visual diff at every one of those sites is expected and desired. 3.3 Coherent defaults SelectField: default variant="outlined". Remove the 12 redundant explicit variant="standard" props in the same PR (they only restated the default; keeping them would fork the look). The standard variant stays supported but undocumented-discouraged; if post-flip audit finds no legitimate use, delete it in a follow-up. Inputs get a default field background: Paper.overlay on the input root (both TextInput and SelectField, outlined variant), matching the editor controls’ treatment. Change SearchInput from action.hover to Paper.overlay so all fields read identically. Panel: default background="paper" and bordered=true; radius source moves to theme.rounded (aligning with Surface — one radius source for surfaces). Risk: 42 call sites, none pass background. Docked workspace panels (PanelLeft, PanelBottom, WorkspaceShell, PanelToolbar) plausibly want the flush look — audit all 42 during the flip and pin background="default" bordered={false} (or introduce variant="docked") where flush is intentional. Everything else inherits the visible default. FlexRow/FlexColumn/Stack: keep gap=0 (37% of ~1010 usages rely on it). Form spacing is owned by the Fix 4 primitives. EditorButton size fix lands with Fix 1 (same file). 3.4 Form composition primitives FormField rework: owns the label (via Label), the 4px label gap, and a helper/error line with a fixed on-grid margin (theme.spacing(1)), overriding MUI’s 3px 14px 0 helper margin. Provides a FormFieldContext { controlId, labelless: true }; TextInput and SelectField consume it and suppress their own labels + adopt the id — no double labels by construction, no convention to remember. FormSection: optional group label (uppercase, smaller/11px, letterSpacing: "0.08em", text.disabled — the storyboard .group-label treatment, StoryboardBoard.tsx:112-116) above a FlexColumn gap={3} body. FormGrid: n-column CSS grid (gridTemplateColumns from a columns prop, default minmax(0, 1fr) 260px two-column like .header-grid), gap from SPACING, stacking to one column below a stackBelow breakpoint prop. Proof migrations: StoryboardBoard.tsx (delete local Field, the .header-grid/.group-label/.field-label/36px CSS, and the overflow: visible workaround), CollectionForm.tsx (the one existing FormField consumer), and WorkflowForm.tsx. 3.5 Honest type scale Five real sizes, five names: name px change giant 22 gets a real value (was 18; 2 call sites, both want display size) big 18 unchanged normal 15 unchanged small 13 unchanged smaller 11 unchanged Codemod the aliases with ast-grep (see §4 tooling note; pattern <Text size="tiny" $$$REST>$$$C</Text> verified to match): bigger→big (17 sites), tiny→smaller (36), tinyer→smaller (4) — mechanical, zero visual change since the values were already identical. Then narrow TextProps["size"] to the five names and delete fontSizeGiant/fontSizeBigger/fontSizeTiny/fontSizeTinyer from the theme after adding fontSizeGiant: "22px" — grep confirmed nothing consumes the legacy CSS vars directly. Text.tsx’s size map shrinks to five entries. Review the 2 giant call sites for the 18→22 bump. 3.6 Passthrough honesty For each re-export in muiReexports.ts:107-130 that the theme does not style, add the missing override with tokens (preferred over relabeling — the primitives-first policy funnels all usage through these re-exports, so a theme override fixes every consumer at once): MuiTabs/MuiTab: minHeight: CONTROL.height.lg, label 13px/500, 2px indicator (match TabGroup’s look so the two tab systems agree). MuiToggleButton: CONTROL height/radius/padding. MuiAccordion: Paper.paper background, divider border, radius theme.rounded.md, no default elevation shadow. MuiLinearProgress: 4px track, BORDER_RADIUS.pill, primary bar (match ProgressBar). Rewrite the muiReexports.ts header comment to state which exports are themed and which are deliberate raw escape hatches (MuiAutocomplete, MuiDialog at :132-141 already are). Document Box.tsx as an import-policy shim. 4. Implementation plan Codemod tooling: the mechanical rewrites in PRs 3 and 5 (and future large migrations — this repo refactors heavily) use ast-grep, driven by the official agent skill vendored at .claude/skills/ast-grep/ (plus ast-grep-outline for structural surveys). Run it via npx --yes --package @ast-grep/cli ast-grep <args> — it is not a repo dependency. For multi-step, test-gated migrations beyond single-pattern rewrites (e.g. the raw-MUI → primitives sweep STRATEGY.md tracks), evaluate the Codemod CLI + MCP stack (npx codemod ai), which layers AST inspection and codemod tests on top of ast-grep. Six PRs, in order. 1–3 change no call-site code (PR 3 removes 12 redundant props); every existing screen improves without migration. Each PR: cd web && npm run typecheck && npm run lint && npm test, plus npm run lint:design, plus the visual-suite step below. PR 1 — control tokens (mechanical) Add CONTROL to tokens.ts; export from the barrel; unit test alongside tokens.test.ts (all heights on 4px grid, radius is a --rounded-* var). Wire: EditorButton (incl. the size-discards-height fix), theme.editor derivation in ThemeNodetool.tsx, TextInput, SelectField, SearchInput, TagInput. Off-grid sweep: Slider, NodeSlider, Panel padding, plus the ⚠ table in 2.1 (TabGroup, Chip, UndoRedoButtons, MenuItemPrimitive, MobileBottomSheet, WarningBanner, Breadcrumbs, ZoomControls, InfoTooltip, NotificationBadge). Add a DesignTokens.Control Storybook story (heights/paddings/radius swatches) next to the existing DesignTokens.* stories. Acceptance: an EditorButton (normal), a compact TextInput, a compact SelectField, and a node-canvas select in one FlexRow share one height (28px), one radius, one horizontal padding; grep gates (§5) pass. PR 2 — one label treatment Label: add htmlFor; margin 2px→4px. TextInput: label-above via Label + useId; delete floating-label sx; add hideLabel. SelectField: drop InputLabel; label-above via Label. FormField: render via Label; delete duplicate asterisk logic. Theme: trim MuiFormLabel override. Tests: update TextInput.test.tsx (label association via getByLabelText must still pass — it proves the htmlFor wiring); add the missing SelectField.test.tsx (render, a11y, ref, label association, hideLabel). Acceptance: TextInput, SelectField, and a Field-wrapped model select render pixel-identical labels; no label renders outside the control’s box (Panel overflow: hidden no longer clips anything). PR 3 — coherent defaults SelectField default variant="outlined"; delete the 12 redundant variant="standard" props (SettingsMenu.tsx ×7, LayerRow, GetVariableBody, PropertiesPanel, WorkersPanel, SearchProviderSetupDialog). Default field background Paper.overlay for TextInput/SelectField; switch SearchInput from action.hover; verify both schemes (dark #17181B, light #F0EDE6). Panel defaults background="paper", bordered=true; radius from theme.rounded; audit all 42 call sites and pin the docked panels (PanelLeft, PanelBottom, PanelToolbar, WorkspaceShell, timeline/ sketch inspectors as judged) to the flush look explicitly. Update Panel.test.tsx defaults; visual-QA the 17 SelectField sites that relied on standard. Acceptance: the bare no-props composition from §1 renders as a visible bordered panel containing two identically-styled fields. PR 4 — form composition primitives Rework FormField (context-based label suppression); add FormSection, FormGrid; barrel exports; tests per the STRATEGY.md new-primitive checklist (render + axe + ref for each); Storybook stories. Migrate StoryboardBoard.tsx header — delete local Field, the .header-grid/.group-label/.field-label/minHeight: 36px CSS, and the overflow: visible workaround. Migrate CollectionForm.tsx and WorkflowForm.tsx as proofs. Acceptance: storyboard header is visually unchanged (within visual-suite tolerance) but contains zero local form CSS. PR 5 — honest type scale Theme: fontSizeGiant: "22px"; delete fontSizeBigger/fontSizeTiny/ fontSizeTinyer keys (and their CSS-var emission). Codemod size="bigger|tiny|tinyer" → canonical (59 sites incl. giant review); narrow TextProps["size"]; update Text.test.tsx. Review the 2 giant sites at 22px. PR 6 — passthrough honesty Theme overrides for MuiTabs/MuiTab, MuiToggleButton, MuiAccordion, MuiLinearProgress per §3.6. Rewrite the muiReexports.ts header comment; document Box.tsx as a shim. 5. Verification Per PR: cd web && npm run typecheck && npm run lint && npm test and npm run lint:design. Visual regression: npm run test:visual (Playwright, web/tests/visual/, committed baselines, 1% diff tolerance). PRs 1–3 and 5–6 intentionally change rendering — regenerate baselines with --update-snapshots, review the image diffs in the PR, and say so in the PR body. design-system.spec.ts covers the primitives gallery and is the primary gate; settings.spec.ts catches the SelectField flip. Manual before/after screenshots per PR: storyboard header (/workspace storyboard tab), node inspector, SettingsMenu, CollectionForm — in both color schemes. Grep gates after PR 1 (run in web/src/components/ui_primitives/ and editor_ui/, excluding __tests__/ and *.md): no minHeight: "2[248]px" / height: "2[24]px" control literals no border-?[rR]adius: "?[468](px)?" literals (use BORDER_RADIUS/CONTROL.radius) no 13px/3px/5px/42px/45px box values (exception: the 7px/3px vertical paddings inside TextInput/SelectField/SearchInput — they are derived from the CONTROL heights to make MUI’s content box land exactly on 36/28px, not layout spacing) no font-size: 1[04]px literals After PR 2: grep -rn "MuiInputLabel" web/src/components/ui_primitives/ returns nothing. After PR 4: grep -n "header-grid\|group-label\|field-label" web/src/components/storyboard/StoryboardBoard.tsx returns nothing. 6. Risks and open decisions Panel default flip — resolved in PR 3. The audit found 13 real sites (not 42); 10 docked inspector/transcript sites were pinned to the flush look explicitly, content panels (cost estimate, storyboard header) inherit the new visible default. Rendered radius corrected from the accidental 36px to a true 6px (theme.rounded.md). Label-above changes every labelled field’s geometry (~85 sites gain ~17px of height). Dialogs and dense inspectors may need spot re-layout; the visual suite will surface them. standard-variant removal is deferred until post-flip audit; the variant remains functional in PR 3. TagInput at 44px vs 36px: chosen 44 (xl) to keep the touch-friendly chip row; revisit if it reads oversized next to 36px fields in forms. giant at 22px restyles 2 call sites; trivially reviewable. Line numbers in §2 are pinned to 2cf277b; PRs land sequentially and each shifts the next PR’s targets — re-grep, don’t trust stale lines.--- # Connect OpenAI, Anthropic, Gemini & Ollama to NodeTool Source: https://docs.nodetool.ai/providers.html A provider is the adapter between a NodeTool node and an AI service — OpenAI, Anthropic, Gemini, a local Ollama daemon, or one of the 30+ others below. Every node that calls an LLM or a media model exposes a model property backed by a provider id. Pick a different provider from that same dropdown and the rest of the graph — edges, other nodes — doesn’t change. This is bring-your-own-key (BYOK): NodeTool never marks up a provider’s price, and cloud usage is billed directly by the provider. Add a key in Settings → Providers, or skip keys entirely and run everything through local models (Ollama, vLLM, LM Studio, llama.cpp). New to models and providers generally? Start with Models & Providers for the local-vs-cloud overview, or Supported Models for the full model catalog. This page is the provider reference: what each one does, which key it needs, and where to read more. Capability matrix Checked against each provider’s implementation in packages/runtime/src/providers/ — a blank cell means that provider doesn’t expose the modality through NodeTool’s generic nodes, even where the underlying service might. Provider Text Image Video TTS ASR Embeddings 3D OpenAI ✅ ✅ ✅ ✅ ✅ ✅   Anthropic ✅             Google Gemini ✅ ✅ ✅ ✅ ✅ ✅   xAI (Grok) ✅ ✅ ✅         DeepSeek ✅             Groq ✅             Mistral ✅         ✅   Cerebras ✅             GMI Cloud ✅             OpenRouter ✅ ✅           Together AI ✅ ✅ ✅ ✅ ✅ ✅   Moonshot (Kimi) ✅             MiniMax ✅ ✅ ✅ ✅       Codex (OpenAI OAuth) ✅ ✅           Claude Agent SDK ✅             Evolink ✅ ✅ ✅         kie.ai ✅¹ ✅ ✅ ✅       AKI ✅ ✅           Replicate ✅ ✅ ✅ ✅ ✅ ✅   FAL   ✅ ✅ ✅       HuggingFace ✅ ✅ ✅² ✅ ✅ ✅   Ollama ✅         ✅   vLLM ✅             LM Studio ✅             llama.cpp ✅             ElevenLabs       ✅ ✅³     Topaz   ✅⁴           Reve   ✅           AtlasCloud   ✅ ✅         Cohere           ✅   Voyage AI           ✅   Jina AI           ✅   Meshy AI             ✅ Rodin AI             ✅ ¹ Chat only for a short gateway list (GPT-5.5, Claude Opus/Sonnet/Haiku 4.x, Gemini 3.1 Pro) — most kie.ai models are image, video, or audio. ² Text-to-video only; no image-to-video. ³ Via a dedicated Speech-to-Text node, not the generic ASR picker. ⁴ Upscale and enhancement, not text-to-image generation. Provider capabilities NodeTool derives this matrix by checking which optional methods a provider overrides — getAvailableImageModels for text-to-image/image-to-image, getAvailableTTSModels for text-to-speech, getAvailableASRModels for speech recognition, getAvailableEmbeddingModels for embeddings, getAvailable3DModels for 3D — rather than a hand-maintained flag per provider. To make a model usable outside the node graph (an agent, the generate CLI command, the generation API), implement the matching method on the provider; the capability then shows up automatically everywhere NodeTool lists what a provider can do. OpenAI OpenAI covers chat (GPT), image generation (GPT-Image), Sora 2 Pro video, TTS, Whisper transcription, and embeddings — six of the seven modalities in the matrix above. Cloud only, keyed by OPENAI_API_KEY. Chat models are fetched live from the OpenAI API; image models are a maintained static list. See the OpenAI provider guide. Anthropic Anthropic runs Claude chat models with tool calling and image input for vision tasks. It doesn’t generate images, video, or audio. Cloud only, keyed by ANTHROPIC_API_KEY; models are fetched live from the Anthropic API. See the Anthropic provider guide. Google Gemini Gemini handles chat with native multimodal input (images, audio, video as Blobs), Nano Banana / Imagen image generation, Veo video, audio transcription, and text embeddings. Cloud only, keyed by GEMINI_API_KEY. Text models auto-fetch; Imagen and Veo are static lists. See the Gemini provider guide. xAI (Grok) xAI runs Grok chat models plus Grok Imagine for text-to-video, image-to-video, and text-to-image, all classified from the same /v1/models response. Cloud only, keyed by XAI_API_KEY. See the xAI provider guide. DeepSeek DeepSeek is chat only — DeepSeek-V3 and the R1 reasoning line — reached through an OpenAI-compatible endpoint. Cloud only, keyed by DEEPSEEK_API_KEY. See the OpenAI-compatible providers guide for how its models are fetched. Groq Groq runs chat models on its LPU inference hardware for low-latency responses. Text only — no image, video, or audio generation. Cloud only, keyed by GROQ_API_KEY. See the OpenAI-compatible providers guide. Mistral Mistral runs chat models (Mistral, Mixtral) plus one embedding model, mistral-embed. Cloud only, keyed by MISTRAL_API_KEY. See the OpenAI-compatible providers guide. Cerebras Cerebras runs chat models on its high-throughput inference hardware. Text only. Cloud only, keyed by CEREBRAS_API_KEY. See the OpenAI-compatible providers guide. GMI Cloud GMI Cloud is an OpenAI-compatible chat endpoint for open-weight models — Llama, DeepSeek, and Qwen variants. Text only. Cloud only, keyed by GMI_API_KEY. OpenRouter OpenRouter proxies 300+ chat models plus image generation through a single key. Cloud only, keyed by OPENROUTER_API_KEY. See the OpenAI-compatible providers guide. Together AI Together AI is one of the broadest providers in NodeTool: chat, manifest-driven image and video generation, TTS (Orpheus, Kokoro, Cartesia Sonic), ASR (Whisper Large v3, Voxtral, Parakeet), and one embedding model. Cloud only, keyed by TOGETHER_API_KEY. See the Together provider guide. Moonshot (Kimi) Moonshot runs Kimi chat models over an Anthropic-compatible endpoint — it subclasses the Anthropic provider, not OpenAI. Text only. Cloud only, keyed by KIMI_API_KEY. See the OpenAI-compatible providers guide. MiniMax MiniMax covers chat, image (Image-01), video (Hailuo 2.3), TTS, and music generation in one provider. ASR and embeddings exist in MiniMax’s API but are disabled in NodeTool’s provider (embeddings need a GroupId NodeTool doesn’t manage) — use another provider for those. Cloud only, keyed by MINIMAX_API_KEY. See the MiniMax provider guide. Codex (OpenAI OAuth) Codex reaches GPT chat models and GPT-Image 2 generation through your logged-in ChatGPT/Codex OAuth session instead of an API key — usage bills against that subscription. Cloud only, keyed by the stored CODEX_ACCESS_TOKEN; no OPENAI_API_KEY needed. Claude Agent SDK Claude Agent SDK reaches Claude by spawning your local, logged-in claude CLI instead of calling the Anthropic API directly, billing against your Claude subscription rather than per-token API spend. It supports tool calls through an in-process MCP bridge; images in the prompt are not forwarded to the CLI, so vision input doesn’t work through this path. No API key — requires the claude CLI installed and logged in. See the Anthropic provider guide. Evolink Evolink is an OpenAI/Anthropic-compatible gateway: one key for GPT, Claude, Gemini, and DeepSeek chat, plus image (GPT Image 2, Nano Banana 2, Seedream) and video (Seedance, Wan, Veo, Sora, Grok) generation. Cloud only, keyed by EVOLINK_API_KEY. kie.ai kie.ai is a multi-model aggregator: manifest-driven image, video, TTS, and music models (Seedance, Runway, Wan, Kling, FLUX.2, Suno, and more), plus chat for a short list of gateway models (GPT-5.5, Claude Opus/Sonnet/Haiku 4.x, Gemini 3.1 Pro). One key covers all of it, often at a lower price than the upstream provider directly. Cloud only, keyed by KIE_API_KEY. See the KIE provider guide. AKI AKI is an OpenAI-compatible gateway for chat plus text-to-image and image-to-image generation. Cloud only, keyed by AKI_API_KEY. Replicate Replicate runs chat, image, video, and music, plus curated TTS, ASR, and embedding models. Chat calls replicate.run()/.stream() on whatever model id you pass, so it isn’t limited to a fixed model list the way most other providers are; image, video, and music nodes are generated from Replicate’s schemas. Cloud only, keyed by REPLICATE_API_TOKEN. See the Replicate provider guide. FAL FAL generates image, video, TTS, and music from models whose nodes are generated straight from FAL’s OpenAPI schemas. No chat. Cloud only, keyed by FAL_API_KEY. See the FAL provider guide. HuggingFace HuggingFace exposes chat, image, text-to-video, TTS, ASR, and embeddings backed by Hub model discovery, plus a hand-written node pack for models the generic providers don’t cover, including 3D nodes (HFTextTo3D, HFImageTo3D). Optional HF_TOKEN — needed for gated or private models and higher rate limits. See HuggingFace Integration. Ollama Ollama runs chat and embedding models locally — no API key, no per-token cost. Pull a model with ollama pull <model> and it appears in NodeTool automatically. Configured via OLLAMA_API_URL (default http://127.0.0.1:11434). See the Ollama provider guide. vLLM vLLM points NodeTool at a self-hosted, OpenAI-compatible vLLM server for chat — models appear automatically from its /v1/models endpoint once the URL is set. Set VLLM_BASE_URL (and VLLM_API_KEY if your deployment requires one; no default). See Local Inference. LM Studio LM Studio connects to the local server LM Studio’s desktop app exposes — enable it in LM Studio → Local Server and NodeTool picks up loaded models automatically. Default URL http://127.0.0.1:1234, overridden with LMSTUDIO_API_URL. See Local Inference. llama.cpp llama.cpp points NodeTool at a local llama-server instance for chat. The OpenAI tool-call wire format isn’t reliably supported, so NodeTool falls back to parsing emulated function-call syntax out of the model’s text output. Set LLAMA_CPP_URL (required, no default). See Local Inference. ElevenLabs ElevenLabs covers text-to-speech, with a large static voice and model catalog, and speech-to-text, plus realtime WebSocket variants of both. Cloud only, keyed by ELEVENLABS_API_KEY. See the ElevenLabs provider guide. Topaz Topaz enhances existing images and video — upscale, sharpen, denoise, restore — rather than generating from a prompt. Only the enhance and enhance-gen endpoints appear in the generic image-model picker; the rest (sharpen, denoise, lighting, matting, restore) are dedicated nodes. Cloud only, keyed by TOPAZ_API_KEY. See the Topaz provider guide. Reve Reve creates, edits, and remixes images through three dedicated nodes (CreateImage, EditImage, RemixImage); the runtime provider also exposes create and edit to the generic image picker. Cloud only, keyed by REVE_API_KEY. See the Reve provider guide. AtlasCloud AtlasCloud generates image and video from a hand-maintained manifest (Seedance, GPT Image 2, Nano Banana, and more). No chat. Cloud only, keyed by ATLASCLOUD_API_KEY. See the AtlasCloud provider guide. Cohere Cohere provides text embeddings — embed-v4.0 and the English/multilingual v3 line. No chat and no reranking in NodeTool’s current provider. Cloud only, keyed by COHERE_API_KEY. Voyage AI Voyage AI provides text embeddings only. Cloud only, keyed by VOYAGE_API_KEY. Jina AI Jina AI provides text embeddings only. Cloud only, keyed by JINA_API_KEY. Meshy AI Meshy generates textured 3D meshes from text or a reference image, through the generic nodetool.3d.TextTo3D / ImageTo3D nodes. Cloud only, keyed by MESHY_API_KEY. Rodin AI Rodin generates 3D assets from text or a reference image, through the generic nodetool.3d.TextTo3D / ImageTo3D nodes. Cloud only, keyed by RODIN_API_KEY. Generic nodes: provider-agnostic workflows Nodes in the nodetool.* namespace take a model property and route to whichever provider owns that model — this is what makes switching providers a dropdown change, not a rewiring job. Node Switches between nodetool.agents.Agent OpenAI, Anthropic, Gemini, xAI, DeepSeek, Ollama, any chat provider nodetool.image.TextToImage FLUX.2, Nano Banana 2.0, GPT Image 2, Ideogram V3, Z-Image, HuggingFace, ComfyUI, MLX nodetool.image.ImageToImage HuggingFace, local servers, cloud services nodetool.video.TextToVideo Sora 2 Pro, Veo 3.1, Seedance 2.0, Runway, Grok Imagine, Wan 2.6, Hailuo 2.3, Kling 3.0, HuggingFace nodetool.video.ImageToVideo Sora 2 Pro, Veo 3.1, Seedance 2.0, Runway, Luma, Grok Imagine, Wan 2.6, Hailuo 2.3, Kling 3.0 nodetool.3d.TextTo3D / ImageTo3D Meshy AI, Rodin AI, plus HuggingFace 3D nodes (Hunyuan3D, Trellis, TripoSR, Shap-E, Point-E) nodetool.audio.TextToSpeech OpenAI TTS, ElevenLabs, HuggingFace, local TTS nodetool.text.AutomaticSpeechRecognition OpenAI Whisper, HuggingFace, local ASR A generic node drops parameters the selected provider doesn’t support instead of erroring — negative prompt, guidance scale, and seed apply mostly to HuggingFace/diffusion-style backends; GPT-Image and similar API-only models ignore them. Reach for a provider-specific node only when you need something the generic interface doesn’t carry: Claude’s thinking mode, OpenAI’s vision detail parameter, MiniMax’s emotion and pitch controls. Getting keys in The in-app Settings dialog is the easiest way to add a key: open Settings → Providers and paste it in. NodeTool stores it encrypted (AES-256-GCM) in a local SQLite database, not in a plaintext config file — see Configuration for how the encryption key itself is managed. From the CLI: nodetool secrets store OPENAI_API_KEY # prompts for the value, stores it encrypted nodetool secrets list # list stored keys (values are never shown) nodetool secrets get OPENAI_API_KEY # print a stored value Or set the variable directly in .env.development.local (or your shell environment) — environment variables always take precedence over stored secrets. See Configuration for the full load order. Local providers (Ollama, vLLM, LM Studio, llama.cpp) don’t need a key — point NodeTool at the server’s URL instead, either in Settings → Providers or via the matching *_URL environment variable. Adding a new provider Every provider is a class in packages/runtime/src/providers/ extending BaseProvider (or OpenAIProvider / AnthropicProvider for OpenAI- or Anthropic-compatible APIs), registered in provider-registry.ts with the environment variables it needs. The right approach differs by provider — live model discovery, a hand-maintained manifest, or codegen from upstream schemas — so follow the Provider Guides runbook for your case rather than reverse-engineering it from an existing provider. Run npm run check (typecheck, lint, test) before opening a PR. See also Models & Providers — local vs. cloud, mixed workflows, getting started Supported Models — the full model catalog and local inference engines Installation — install NodeTool on Windows, macOS, Linux Comparisons — NodeTool vs. ComfyUI, n8n, Dify, Flowise, Langflow, Weavy Chat API — WebSocket API for chat interactions and provider routing Global Chat and Agent Mode — using providers interactively and through the planning agent Workflow API — building workflows with providers Provider Guides — runbooks for adding models and nodes to each provider--- # Python Bridge Protocol Source: https://docs.nodetool.ai/python-bridge-protocol.html NodeTool runs Python nodes and local-compute providers in a separate Python worker process. The desktop app and server spawn that worker and communicate with it over a local stdio RPC protocol. Why this protocol exists NodeTool uses Python for: Python node execution local ML providers such as MLX and HuggingFace media processing and model-specific dependencies The TypeScript runtime uses stdio instead of a localhost socket for the worker because it is simpler to supervise, avoids port-management issues, and keeps the worker strictly parent-scoped. Transport Parent process spawns: python -m nodetool.worker --stdio stdin / stdout: binary protocol traffic stderr: worker logs and startup diagnostics Framing Each message is encoded as: [4-byte big-endian payload length][MessagePack payload] The payload is a MessagePack object with this envelope shape: { "type": "discover", "request_id": "uuid-or-null", "data": {} } Lifecycle TypeScript spawns the worker Python loads node packages and providers Python prints NODETOOL_STDIO_READY on stderr TypeScript sends discover Python responds with node metadata, protocol version, and any load errors TypeScript optionally requests worker.status Workflow execution uses execute / execute.stream, cancel, provider.*, and (on v2+ workers) models.* messages Message types discover Returns the worker’s executable node inventory. Response data: protocol_version nodes load_errors Example: { "type": "discover", "request_id": "d1", "data": { "protocol_version": 1, "nodes": [...], "load_errors": [ { "module": "nodetool.nodes.mlx.text_generation", "phase": "module_import", "error": "No module named 'mlx_lm'", "error_type": "ModuleNotFoundError" } ] } } worker.status Returns a structured worker health snapshot. Response data: protocol_version node_count provider_count namespaces load_errors transport max_frame_size execute Executes a Python node. Request data: node_type fields secrets blobs Possible response types: progress result error execute.stream Executes a Python node that streams results. Same request data as execute (node_type, fields, secrets, blobs), but the worker emits zero or more chunk messages (each carrying partial outputs/blobs) followed by a terminal result or error. cancel Requests cooperative cancellation for an in-flight execute or streaming provider request. provider.* Used for Python-only providers. The message families the TS bridge implements: provider.list provider.models provider.generate provider.stream provider.text_to_image provider.image_to_image provider.tts provider.asr provider.embedding Streaming providers (provider.stream, provider.tts) emit zero or more chunk messages followed by a terminal result or error. models.* Worker model management (HuggingFace cache). These were introduced in bridge protocol v2 and are gated by supportsModelManagement() — a v1 worker simply does not expose them (see Versioning). models.list_cached — list models cached on the worker’s HF_HOME (cache-only, no network) models.download — download a model onto the worker cache, streaming ordered progress frames then a terminal result models.delete — delete a cached model; returns whether it existed comfy.* ComfyUI proxy. Introduced in bridge protocol v3, gated by supportsComfy() — a worker offers these only when it fronts a co-located, loopback-only ComfyUI server AND reports worker.status.comfy.enabled: true. Route comfy.* requests only to such workers. The full field-level reference lives in docs/comfy-proxy.md in nodetool-core. comfy.execute — submit an API-format workflow; streams its lifecycle as dedicated comfy.event frames (see below), then a terminal result/error. Cancel with the standard cancel frame. comfy.queue — {queue_running, queue_pending} comfy.interrupt — global stop of the running job (admin-only) comfy.cancel — best-effort per-prompt cancel ({prompt_id}); the safe user-facing cancel comfy.upload / comfy.view — stage a file into / fetch a file from ComfyUI’s input dir comfy.object_info — full ComfyUI node catalog comfy.system_stats / comfy.status — health/capacity; comfy.status adds worker-level {enabled, url, reachable} comfy.free — unload models from VRAM without a cold restart comfy.models.list / comfy.models.download / comfy.models.delete — manage model files on the worker’s persistent volume. comfy.models.download streams generic progress frames (NOT comfy.event) then a terminal result. comfy.event comfy.execute does not stream progress frames — ComfyUI’s events don’t fit the {progress, total, message} shape. Instead it emits a dedicated frame: { "type": "comfy.event", "request_id": "<the execute request's id>", "data": { "event": "executing", "prompt_id": "p1", "node": "3" } } data.event is the discriminator, in emission order: queued → queue (repeatable) → started / cached → executing (per node) → progress → node_output → preview (only if previews: true) → completed or cancelled. result is always the last frame. Result, error, chunk, and progress result { "type": "result", "request_id": "e1", "data": { "outputs": { "text": "hello" }, "blobs": {} } } error { "type": "error", "request_id": "e1", "data": { "error": "Unknown node type: foo.Bar", "traceback": "..." } } chunk Used by streaming providers and streaming provider-like operations. progress The worker forwards NodeProgress messages from Python execution as protocol-level progress events: { "type": "progress", "request_id": "e1", "data": { "progress": 32, "total": 100, "message": "Downloading model" } } Binary data MessagePack allows binary payloads directly. NodeTool uses that for: input blobs in execute.data.blobs output blobs in result.data.blobs audio/image chunks for streaming provider APIs Diagnostics and failure handling The bridge now surfaces worker problems in-band: discover.load_errors lists node import and metadata extraction failures worker.status.load_errors provides the same information on demand error.traceback is returned for failed requests recent stderr lines are still retained on the TS side for debugging startup failures This matters because metadata may exist for a Python node even if its module failed to import in the worker. In that case the worker is connected, but the node is unavailable. load_errors is the authoritative signal for that situation. Limits and timeouts The worker and TS bridge enforce a maximum frame size via NODETOOL_BRIDGE_MAX_FRAME_SIZE The TS stdio bridge enforces a startup timeout (startupTimeoutMs, default 20s) Cancellation is cooperative Versioning The JS runtime and Python worker each report a BRIDGE_PROTOCOL_VERSION. Two distinct numbers govern compatibility (see packages/protocol/src/bridge-protocol.ts): BRIDGE_PROTOCOL_VERSION — the protocol the JS runtime currently speaks (presently 3). MIN_BRIDGE_PROTOCOL_VERSION — the hard floor (presently 1). The JS runtime rejects a worker only if it reports a protocol below this floor. Compatibility rules: Older worker, at or above the floor — connects normally. It does not fail startup. Additive features the worker predates are gated per-capability: a v1 worker connects fine and simply doesn’t expose the models.* family (gated by supportsModelManagement(), which requires v2+) or comfy.* (gated by supportsComfy(), which requires v3+ and comfy.enabled). Workers that predate the protocol_version field are treated as v1. Worker below MIN_BRIDGE_PROTOCOL_VERSION — rejected at discover with an actionable error (reinstall the Python environment). This is the only startup-failing case, reserved for genuine wire breaks. Newer worker protocol than the JS runtime — the runtime warns and assumes backward compatibility. Notes for contributors If you change the protocol: update the JS and Python protocol version constants together document the schema change here add or update end-to-end tests in nodetool-core/tests/ keep discover and worker.status authoritative for diagnostics Related Architecture Developer Guide nodetool-core/README.md--- # RFC: Generation Events — generation_complete, re-scoped output_update, autosave cutover Source: https://docs.nodetool.ai/rfc-generation-events.html RFC: Generation Events — generation_complete, re-scoped output_update, autosave cutover Status: Draft for sign-off · Owner: runtime/web Every line/file reference below was re-verified against the working tree before this revision. Findings that were wrong-as-written (the double-save rationale, the actor job_id field, “_messages authoritative for autosave”) are corrected in-body and catalogued in the Addressed review notes appendix. The scope was widened from “kernel→websocket→web-editor” to all message consumers (browser runner, mobile, CLI, mini-apps, timeline/sketch editors, chat) after a consumer census. 1. Summary We split the one overloaded “a node produced something” signal into three orthogonal channels: node_update — node lifecycle only (running / completed / error, timing, cost). One per node-run. Stops being the autosave/generation source. generation_complete (NEW) — a generator committed one complete artifact: { node_id, node_name, node_type, index, outputs } (plus job_id/workflow_id stamped downstream). Emitted once per process() result (once at stream-end for genProcess). Authoritative; drives liveGenerations + autosave; never suppressed. output_update — demoted to ephemeral display-only live feed (text tokens, progressive preview, realtime audio). Carries an explicit disposition: "append" | "replace". Never persisted, never creates a generation. Edge-suppression becomes harmless. The kernel change is small and TS-only. The breadth is in the consumers: every surface that today reads node_update.result or output_update.value as an artifact must move to generation_complete.outputs or accept a documented, graceful degradation to one artifact. 2. Problem & evidence The kernel collapses multi-execution runs into one node_update. _emitNodeStatus("completed", …) fires exactly once per node-run at actor.ts:440-443, carrying this._latestResult ?? {}. There are only 4 _emitNodeStatus call sites (actor.ts:278 running, :421 suspended, :426 error, :440 completed). _latestResult (declared actor.ts:157) is overwritten on every execution — never accumulated: actor.ts line mode write object identity 357 streaming-input run() = nodeOutputs.collected() fresh from collector 369 streaming-input legacy process() = outputs same ref as executor return 987 genProcess per-yield = {...collected} fresh spread each yield 994 genProcess stream-end = {...collected} fresh spread 1000 correlated process() = outputs same ref as executor return 1290 controlled-loop process() = outputs same ref as executor return (The object-identity column is load-bearing for §7 — see the corrected double-save analysis.) So a ListGenerator.item → TextToImage.prompt run that fires TextToImage 6× (the correlated while (isReady(key)) drain at actor.ts:716-720, each call overwriting _latestResult at :1000) emits one node_update{completed} carrying run #6 only. Autosave inherits the collapse. autoSaveAssets (unified-websocket-runner.ts:428-571) is called from the single site at unified-websocket-runner.ts:1932-1953, gated on outbound.type === "node_update" && status === "completed" && result != null && meta?.auto_save_asset. It walks outbound.result (= _latestResult, run #6) → persists 1 asset/run, 5 lost. (Idempotency guard at :468 if (assetValue.asset_id) continue skips already-asset_id‘d values by mutating that same object in place at :518-520; node_id/job_id stamped from opts at :506-507. Crucially, Asset records carry node_id+job_id but no index — there is no persisted-side dedupe key beyond the in-place asset_id mutation. This matters for replay; see §15.) output_update is the accidental sole carrier of the lost variants — and it’s overloaded three ways. Emitted at runner.ts:1286-1294, once per emit (per yield / per key / per control-run — NOT collapsed). But suppressed at runner.ts:1280-1284 for any handle with an outgoing data edge (findEdges(sourceNodeId, handle).some(isDataEdge) unless always_emit_output_updates). So an intermediate generator feeding a Preview emits nothing; only the terminal Preview re-emits and buffers all N. output_update simultaneously serves: (1) live incremental display (text tokens, progressive preview → setOutputResult(append=true)), (2) realtime audio transport (~50/s, coalesced to a worklet bus — the reason suppression exists, runner.ts:1257-1264), and (3) the accidental multi-execution variant carrier. It also can’t distinguish a chunk (append) from a whole value (replace) — workflowUpdates.ts:515 hardcodes append=true (latent progressive-preview bug). Net: a 6-image run produces 6 artifacts; the generation timeline and persistence see 1. 3. Goals / Non-goals Goals Every committed artifact of a multi-execution run is persisted (N assets/run) and appears as a navigable variant — on both the server and in-browser execution paths. Lifecycle, artifacts, and live display travel on three independent channels with clear contracts. output_update becomes correctness-irrelevant: edge-suppression can drop it freely. Zero nodetool-core / Python-worker change for the targeted multi-execution collapse (see §10). No double-save and no no-save during cutover. Every TS message consumer (web editor, browser runner, mobile, CLI, mini-apps, timeline/sketch, chat) is explicitly migrated or assigned a documented graceful-degradation decision. Non-goals Changing the realtime-audio fast-path (runner.ts:45-52 / bus). It stays byte-for-byte. Changing the TS↔Python bridge protocol. Reworking the salvaged UI navigator (groupByRun/NodeHistoryViewer). It is the correct consumer; it only needs a correct feed. A separate generation_started / progress channel (out of scope; node_update{running} + node_progress already cover it). Persisting N artifacts for multi-artifact-per-output-slot streaming generators (no such node exists today; see §10 — explicitly out of scope, with a guard test). 4. The three channels Channel Carries Rate Persisted? Creates Generation? Suppressible? node_update lifecycle: running/completed/error, timing, cost, property echo 1/run no no (changed) no generation_complete one complete artifact + stable variant index N/run (1 per process() result) yes (drives autosave) yes never output_update live display feed: text tokens, progressive preview snapshot, audio chunks high (≤~50/s audio) no no yes (harmless) 4.1 node_update (unchanged shape, narrowed role) messages.ts:166-178 stays as-is. After this change it no longer drives autosave or generations. Everything else (status/timing/cost/property-merge/error-notification/trace) is unchanged. The result field becomes advisory/back-compat only (still the last result); consumers must not read it for multi-execution artifacts. It remains the graceful-degradation path for consumers that cannot be fully migrated this cycle (mobile shows 1 artifact via result; see §8.6) and for skip-result nodes’ values (see Decision 7). 4.2 generation_complete (NEW) export interface GenerationComplete { type: "generation_complete"; node_id: string; node_name: string; node_type: string; /** k-th generation of this node in this run (backend-assigned, monotonic, stable). */ index: number; /** The complete result dict for this artifact (same shape as a process() return). */ outputs: Record<string, unknown>; /** Stamped downstream by the runner relay, NOT by the actor. */ job_id?: string | null; workflow_id?: string | null; } index is a per-actor monotonic counter (_generationIndex++), NOT the correlation lineage index (those reset per (root, parentKey) and are not a stable global variant id — see §5). A fresh actor is created per _processGraph and a fresh runner per job, so index does not collide across runs. job_id / workflow_id are optional in the actor emit and stamped uniformly downstream — see §5 and the corrected note below. outputs is the whole result dict, passed through unflattened. List-valued handles (num_images=N) stay as arrays; consumers flatten via runVariantValues / recursive autosave collect (Decision 1). Emitted from inside the serial actor paths, so per-node index ordering is deterministic. 4.3 output_update (re-scoped, + disposition) export interface OutputUpdate { type: "output_update"; node_id: string; node_name: string; output_name: string; value: unknown; output_type: string; metadata: Record<string, unknown>; /** NEW. "append" = value is a chunk to concatenate; "replace" = value is a whole snapshot. */ disposition?: "append" | "replace"; /** NEW (optional). Marks the final chunk of an append stream. */ done?: boolean; workflow_id?: string | null; } disposition is optional for back-compat: absent ⇒ treat as "append" (today’s behavior). Streaming chunks (text tokens, audio, genProcess yields) → "append"; whole non-generation values (a string into an Output node, a progressive-preview full-frame) → "replace". 5. generation_complete emission points Add to the actor, alongside _emitNodeStatus. The actor does NOT carry job_id — verified: packages/kernel/src/actor.ts has zero jobId/job_id/_jobId references, and _emitNodeStatus (actor.ts:1361-1380) emits node_update without a job_id field. job_id/workflow_id are backfilled for every outbound message at unified-websocket-runner.ts:1855-1861 (msg.job_id ?? active.jobId, msg.workflow_id ?? active.workflowId). A new type inherits this for free. (The browser path must apply the same backfill; see §8.5.) node_name mirrors node_update: this.node.name ?? this.node.type — not this.node.data?.title. private _generationIndex = 0; // new field near _latestResult (actor.ts:157) private _emitGenerationComplete(outputs: Record<string, unknown>): void { this._emitMessage({ type: "generation_complete", node_id: this.node.id, node_name: this.node.name ?? this.node.type, // mirror _emitNodeStatus node_type: this.node.type, index: this._generationIndex++, outputs // NO job_id here — runner relay stamps it (unified-websocket-runner.ts:1855-1861) }); } Wire it at each completed process()-result boundary — the same sites where _latestResult is assigned: # actor.ts site mode rule 1 after :1000 (correlated process()) one per ready key → N/run for a generator the primary 6×-collapse fix 2 after :1290 (controlled-loop process()) one per "run" control event   3 after :369 (streaming-input legacy process()) once also the path Python is_streaming_input nodes take (see §10) 4 after :994 (genProcess stream-end) ONCE per stream, using final _streamingCollectedOutputs — NOT per yield at :987 (yields are chunks → stay output_update) if also correlation-driven, reached once per key (correct). Load-bearing invariant — see below. 5 after :357 (streaming-input run()) once, nodeOutputs.collected() TS-only; Python nodes never reach run() (see §10). Parity-complete spot for filters/transforms; low priority Load-bearing invariant for site #4 (genProcess stream-end). _streamingCollectedOutputs is an overwrite-merge, not a concatenation: actor.ts:984 does Object.assign(this._streamingCollectedOutputs, routed), and :994 takes {...this._streamingCollectedOutputs}. Therefore the stream-end dict holds the last value written to each output slot, not all N values streamed through that slot. The “once at stream-end” rule is correct iff the generator’s final accumulated dict holds the complete artifact — true for nodes that emit a final whole-value yield (Summarizer-style: a terminal {text, output} consolidating yield; audio-with-done). It is wrong for a hypothetical generator that streams N distinct savable artifacts on the same output slot with no consolidating final yield — that would commit only artifact N (the exact collapse bug this RFC kills, reappearing on the streaming path). No such node exists today (e.g. ListGenerator yields per-item {item, index}, but its items propagate downstream as iteration-correlation envelopes that fire consumers N times → the N generation_complete land on the consumer at site #1, not on the generator, whose own timeline is not a savable sink). We scope multi-artifact-per-slot streaming generators OUT (§3 non-goal, Decision 1 corollary) and add a guard test (§13) asserting the documented limitation so a future node author hits a failing test, not a silent data loss. Constraints: Never routed through the runner.ts:1280 edge-suppression — generation_complete goes straight out via _emitMessage → _emit (runner.ts:999-1001, 1688-1708). Not audio-dropped — do NOT route it through the isAudioChunkOutputUpdate drop (runner.ts:1696-1703). It is pushed into _messages (which is truncated past MAX_RETAINED_MESSAGES = 10_000 via slice at runner.ts:1697-1700 — see §15 on what that does and doesn’t guarantee). No new backpressure: same fire-and-forget path as node_update, rate bounded by result count (low), does not touch inbox flow-control. Skip-result parity: for nodetool.constant.* / nodetool.input.* (the skipResult set at actor.ts:437-439), do not emit generation_complete — those aren’t generators. How their values still reach display sinks is specified in Decision 7. 6. output_update re-scope disposition drives append vs replace. runner.ts:1286-1294 sets disposition: "append" for streaming chunks (genProcess yields, audio), "replace" for whole-value emits. Every frontend consumer branches on it instead of hardcoding append (see §8.7 — there are 5 setOutputResult call sites, not 1), fixing the progressive-preview latent bug. Ephemeral clearing lifecycle. The outputResults buffer (ResultsStore.ts:609-694, keyed ${wf}:${job}:${node}) is cleared (a) on run start (existing clearResults/clearOutputResults at :424-443) and (b) on generation_complete for that (wf, job, node) artifact — once the artifact is committed as a generation, its live scratch buffer is stale. This keeps the display buffer from leaking partial chunks into the settled view. Edge-suppression is now harmless. No correctness depends on output_update reaching anyone — artifacts travel on generation_complete. The runner.ts:1280-1284 data-edge skip and the always_emit_output_updates opt-out remain as-is; they now only affect cosmetic live display, never persistence or variant counts. Audio path unchanged. isAudioChunkOutputUpdate (runner.ts:45-52), the live-stream-without-retention fast-path (runner.ts:1704-1706), queueAudioAppend / publishRealtimeAudioChunk (workflowUpdates.ts:513-523), realtimeAudioChunkBus.ts, AudioOutBody.tsx, useRealtimeAudioPlayback.ts — all untouched. Audio chunks are disposition: "append" and never become generations. Do not narrow value to chunk-only. Display sinks legitimately render whole non-generation values (a string into an Output node, chat string-streaming). Keep value: unknown. 7. Autosave cutover (server path) Move autosave from node_update{completed} to generation_complete — a hard switch, not dual-write: Add generation_complete to the processing gate at unified-websocket-runner.ts:1889-1892 so it gets node-type resolution, constant/input skip, and normalization. New branch (parallel to the deleted one), autosave before normalize (normalize strips inline bytes): if (outbound.type === "generation_complete") { const meta = this.getNodeMetadata?.(nodeType); if (meta?.auto_save_asset && outbound.outputs != null) { await autoSaveAssets(outbound.outputs as Record<string, unknown>, { userId: this.userId ?? "1", workflowId: active.workflowId, jobId: active.jobId, nodeId: String(outbound.node_id ?? ""), textOutputName: primaryTextOutputName(meta) }); } // then: normalizeOutputValue(outbound.outputs) — mirror :1958 on .outputs } Delete the node_update autosave at unified-websocket-runner.ts:1932-1953 in the same commit. Why hard-switch and not dual-write (corrected rationale). The original draft justified the hard switch by claiming the last-result object is a different instance on the two channels and would therefore double-save. That is inverted. Verified at actor.ts:1000/:1290: this._latestResult = outputs assigns the same object reference that _emitGenerationComplete(outputs) would carry — so on the process()/correlated path, node_update.result and the run-#N generation_complete.outputs are the same instance, and the reference-based idempotency guard (autoSaveAssets:468 keys on the in-place asset_id mutation at :518-520) would make a dual-write a no-op for that shared object, not a double-save. The genuine fresh-object case is the genProcess path (:994 does {...spread}), where the guard cannot see the prior save across channels. The hard switch is still required, for two real reasons: (a) Under-save, not double-save, is the dominant risk. Runs 1..N-1 of a multi-execution node exist only on generation_complete — node_update collapses to the last. A dual-write therefore under-saves the early runs regardless of object identity; the new channel must own autosave outright. (b) Cross-channel idempotency is reference-based and partial. It protects only the shared last-result object on the process() path; it does not protect the genProcess fresh-spread path, where dual-write would double-save the final stream-end artifact. Atomic swap is the only clean option. (If a dark-launch is mandated, gate both with one flag, never both ON.) Asset-volume / parity. A 6-image run now persists 6 assets (was 1), each tagged node_id, job_id. The consumer de-dupes by jobId: mergeGenerations (nodeGenerations.ts:74-82) drops live gens whose jobId already persisted, and all N variants share one jobId, so live N + persisted N collapse to N (not 2N). useNodeResultHistory.lastJobAssets (useNodeResultHistory.ts:75-81) already returns N assets for the last job → N tiles. The job_update{completed} asset reload (workflowUpdates.ts:712-725) now surfaces all N correctly. Note: persisted assets carry no index (§2), so mergeGenerations groups purely by jobId; this is sufficient for the live/persisted collapse but means replay idempotency must be addressed at the autosave layer (§15 / Decision 8). Storage volume note: flagged in §15 — runs that previously discarded intermediates now persist them; confirm desired for high-N generators (e.g. a 100-frame batch). 8. Frontend rewiring (all consumers) A census of every node_update / output_update consumer (grep across packages/, web/, mobile/) yields the surfaces below. The original draft covered only §8.1–§8.4; §8.5–§8.10 are the surfaces the consumer census surfaced. 8.1 workflowUpdates.ts reducer (web editor — primary) node_update branch (:866-1044): keep all lifecycle (error notification/state/ErrorStore/trace at :881-920 except the generation write; status/timing/cost at :931-974; property-merge at :1016-1043). DELETE the completed → upsertLiveGeneration write at :986-998. Keep a lightweight running → upsertLiveGeneration({status:"running"}) placeholder (:982-985) so a node shows a spinner before its first artifact (generation_complete only fires at commit). The node_update{error} → generation write (:894-901) stays on node_update (lifecycle owns errors; generation_complete represents committed artifacts only — see Decision 1). Both the running placeholder and the error settle are index-less patches — see §8.2 for how they route in the index-keyed store, and §9 for silent-scrub gating that must cover the running write too. output_update branch (:499-547): becomes display-only. Branch on disposition instead of hardcoded append=true at :515. Audio fast-path + logging + trace unchanged. NEW generation_complete branch: the sole generation driver. Appends/replaces by index, absorbs the silent-scrub gate (§9), clears the ephemeral outputResults buffer for the artifact. 8.2 ResultsStore.upsertLiveGeneration (:496-558) — DELETE the variant-inference, define index-less routing DELETE the Tier-2 “second completed appends a variant” hack: startsNewVariant (:522-525) + the synthesized-variant-id append branch (:527-548, the ${jobId}#${variantCount} scheme at :535). DELETE the isSilentJob-in-upsert special case (:519 + its comment :514-518). Silence moves to the message handler (§9). REPLACE the matcher with append/replace-by-index keyed ${workflowId}:${nodeId}. generation_complete{index:k} → set/replace slot k, id k===0 ? jobId : ${jobId}#${k} (preserve the back-compat id scheme — selected_generation values and tests depend on it). getLiveGenerations (:563-564) read API unchanged. Index-less-patch contract (NEW — addresses the under-specified seam). The surviving node_update{running} placeholder and node_update{error} settle carry no index. Specify: a patch without index targets the newest slot for that jobId — i.e. retain the existing findLastIndex(g => g.jobId === jobId) fallback (:507) only for index-less patches, while any generation_complete patch supplies an explicit index and uses index-keyed set/replace. This guarantees a node that errors before committing any artifact (no generation_complete, correct per Decision 1) still has its running placeholder settled by the index-less error patch — otherwise the index-keyed store has no slot for it and leaves a stuck running generation. 8.3 Preview / Output / content-card (web editor) PreviewNode.tsx (:235-298): DELETE the “accumulate output_updates as a variant array” reliance (the contract documented at :235-242). Preview reads its source node’s generations via the value edge — useNodeResultHistory(incomingValueEdge.source) (already wired at :277-284) / useNodeGenerations of the source — for discrete artifacts; streamBuffer (output_update) only for live text/audio. The sourceFallbackValue path becomes primary. OutputNode.tsx (:239-249): structurally unchanged — already layers streamBuffer ?? settledValue. Stream for live, generation (useNodeGenerations().current) for settled. ContentCardBody.tsx (:600-617): already generation-driven via useNodeResultValue/useNodeResultHistory. Its resolvePreviewValue array-flatten (:281-301) is now redundant for settled artifacts (runVariantValues covers N tiles); it still services the live in-flight buffer, so it stays. 8.4 SALVAGED UI — fed correctly, zero code change utils/nodeGenerations.ts (groupByRun/getCurrentRun/RunGroup/runVariantValues/mergeGenerations/assetToGeneration), useNodeGenerations ({runs, currentRun}), NodeHistoryViewer (the runs/variants/single navigator, defaultView flips to grid at variants.length > 1, :269-270), useNodeIO, nodeGenerationAccessor, useNodeExecState. These already expect index-keyed variants under a stable jobId; they just receive a clean feed instead of the node_update collapse. 8.5 In-browser execution path — normalize + autosave branches REQUIRED (was omitted) The dual-run-path architecture runs the same packages/kernel in a Web Worker, so generation_complete is emitted in-browser too. Two gaps: Normalize. browserRunner.worker.ts:187-203 (attachPreviewBitmaps / resolveImageRefForTransport) and browserWorkflowRunner.ts:435-441 (materializeBrowserOutputs / resolve) currently materialize only node_update.result and output_update.value. Add a generation_complete branch in both that applies the same resolve + materialize to .outputs (GPU read-back, raw-RGBA→PNG, inline-bytes→uri), or in-browser generation_complete carrying GPU refs / inline bytes renders broken and bitmaps are not transferred. The :432 “render identically” guarantee depends on it. Autosave. autoSaveAssets exists only in unified-websocket-runner.ts — the entire web/src/lib/workflow/ directory calls it nowhere (verified: zero matches). So in-browser generative runs do not persist via this code. Decision required (Decision 9): either (a) add a browser-side autosave hook keyed on generation_complete mirroring the server branch, or (b) confirm+document that browser-path persistence already round-trips to the server (the job_update{completed} → invalidateQueries(['assets']) + loadWorkflowAssets reload at workflowUpdates.ts:712-725 assumes the server already saved). The “every committed artifact is persisted” goal is unproven for the browser path until this is resolved; the rollout’s “each step shippable and green” claim is false for browser-eligible workflows until §8.5 lands. The silent-scrub case runs in-browser (§9), so this is on the critical path. Backfill. Confirm the browser relay applies the same job_id/workflow_id stamping the server does at unified-websocket-runner.ts:1855-1861; the bare actor emit (§5) relies on it. 8.6 Mobile (mobile/src/stores/WorkflowRunner.ts) — add a case, decide degradation Independent reducer: reads node_update.result (:354-359) and output_update.value (:384-396) into one nodeResults map; reads node_update only for errors otherwise. After this change, mobile keeps working at 1-artifact-per-node via node_update.result (the advisory last-result), which is acceptable graceful degradation. Required: (1) add a generation_complete case to the WorkflowRunner.ts switch — minimally store .outputs into nodeResults so multi-execution shows at least the latest, ideally a variant list if mobile surfaces variants; (2) regenerate mobile/src/api.ts message-type table (it currently has a generated output_update/node_update union with no generation_complete, :5005+); (3) state explicitly (Decision 10) that mobile’s continued read of node_update.result is an intended degradation, not an accident. Mobile is intentionally NOT a root workspace; build protocol first before its typecheck. 8.7 setOutputResult append/replace plumbing — 5 call sites, not 1 disposition must thread through ResultsStore.setOutputResult’s boolean append param. Verified call sites (excluding tests): workflowUpdates.ts:515, core/chat/chatProtocol.ts:514, hooks/timeline/useGenerateClip.ts:186, hooks/sketch/useGenerateLayer.ts:227. The original draft named only the first two. The timeline/sketch forwardWorkflowMessage paths call setOutputResult(..., true) unconditionally, so a disposition:"replace" value there would still append — re-introducing the exact progressive-preview bug elsewhere. Fix: change setOutputResult to take disposition (or derive append from it) and thread it through all four runtime call sites. (utils/imageRef.ts:51 is only a doc comment; update it too.) 8.8 Timeline & Sketch generation (useGenerateClip.ts, useGenerateLayer.ts) — migrate asset extraction Both share an identical pattern that makes output_update load-bearing for production asset extraction: on each message they jobOutputs.set(jobId, normalizeOutputUpdateValue(...)) gated on node_id === context.selectedOutputNodeId (useGenerateClip.ts:204-209, useGenerateLayer.ts:248-253), then at job_update{completed} call extractAssetId(jobOutputs.get(jobId)) (useGenerateClip.ts:228-235) and mark the whole job FAILED (“finished without producing an output asset”) if absent. Output nodes are terminal (no outgoing data edge) so output_update survives suppression today, but the new contract makes this a freely-droppable display artifact. Migrate the selectedOutputNodeId extraction from output_update to generation_complete.outputs (authoritative, never suppressed); keep output_update only for live preview inside the editors. This is the production path for both the video timeline and the sketch editor — it must move, not just honor disposition. 8.9 Mini-apps (web/src/components/miniapps/hooks/useMiniAppRunner.ts) — migrate result tiles Mini-app result tiles are built entirely from output_update: :84-103 creates one MiniAppResult per output_update (id node_id:output_name:timestamp) → upsertResult; node_update is read only for errors (:105-119). Under the new contract output_update is no longer the artifact carrier, so mini-app lists become incomplete/empty for edge-connected generators. Switch MiniAppResult construction to generation_complete.outputs (flatten per output handle for committed artifacts); keep output_update only for live-streaming display within a result. Mini-apps are a first-class surface. 8.10 Chat / agent path (web/src/core/chat/chatProtocol.ts) chatProtocol.ts independently consumes node_update (:1263) and output_update (:1294 → applyOutputUpdate at :492-521); it has no upsertLiveGeneration / generation path (only content_metadata.media_generation chunk handling at :430). Two requirements: (1) applyOutputUpdate must honor disposition (same demotion as §8.1; chat string-streaming at :523+ stays as legitimate whole-value display); (2) explicitly decide (Decision 11) whether the chat surface creates variants for a multi-execution generator invoked from a workflow tool, or intentionally ignores generation_complete (display-only). Recommend the latter for this cycle — document it — rather than leaving it implicit in a one-line note. If chat later needs variants it gets its own generation_complete branch. 8.11 CLI (packages/cli/src/websocket-client.ts) Protocol consumer used by nodetool workflows run --json and piped chat: handles node_update (:398-403, surfacing result) and output_update (:405-411, surfacing value); no generation_complete case. After the change a 6-image CLI run reports 1 collapsed result via node_update and the variants exist only on a type the CLI ignores. Decide (Decision 12): surface generation_complete events in the JSON stream (one event per artifact) or aggregate them; add the message-type entry to the CLI union (:36-38). Lowest-priority consumer but must not silently drop variants from --json output. 9. Silent-scrub handling useLiveSliderWriter reuses one jobId for an entire scrub, marks it silent (markJobSilent), and emits a running→completed arc per frame (runBrowserGraphJob({ jobId: previewJobIdRef.current })). Under the new model each frame emits generation_complete with incrementing index → without a gate, a 60-frame scrub = 60 variants. This runs in-browser (§8.5), so the browser-path normalize/autosave branches must exist for it. The gate relocates from ResultsStore.upsertLiveGeneration:519 into the new generation_complete handler in workflowUpdates.ts (frontend gate, Decision 2). Both writers that fire per scrub frame must honor it: // in the generation_complete handler: const slot = isSilentJob(jobId) ? 0 : index; // pin scrub frames to slot 0 upsertLiveGeneration(wf, node, jobId, { index: slot, outputs, status: "completed" }); // in the node_update{running} placeholder write (§8.1) — also fires per scrub frame: const slot = isSilentJob(jobId) ? 0 : undefined; // index-less for non-silent (newest-slot) upsertLiveGeneration(wf, node, jobId, slot === 0 ? { index: 0, status: "running" } : { status: "running" }); isSilentJob is already imported in workflowUpdates.ts:37. previewJobs.ts and useLiveSliderWriter are unchanged. Both halves matter: §8.2 deletes the isSilentJob special-case from upsertLiveGeneration, so if the per-frame running placeholder does not also pin slot 0, it would append/churn variants via the index-less newest-slot fallback. The covering test ResultsStore.variants.test.ts:154-183 (running→completed PER frame, 8 frames → 1 generation) relocates to a generation_complete-handler test that drives running + generation_complete per frame for 8 frames and asserts 1 live gen. Keep the kernel dumb (always increment index); silence is a pure slider-preview UI concern, so the gate lives frontend-side. The browser runner already owns the jobId; we do not push a silent flag into the kernel. 10. Python parity — VERDICT: TS-only for the targeted collapse. Zero nodetool-core change. Python nodes execute through the same NodeExecutor interface as TS nodes (node-executor.ts:127-171). PythonNodeExecutor implements only process() (python-node-executor.ts:223-237) and genProcess() (:239-258) — never run(). process() does the bridge RPC, calls materializeOutputs(), and returns a fully-materialized plain JS dict to the actor’s await; genProcess() yields materialized partials. For Python the actor assigns these to _latestResult at sites :369 (legacy single process()), :994 (genProcess stream-end), :1000 (correlated), :1290 (controlled-loop) — identically to TS. Two corrections to the “identical sites” claim (it was over-stated): Site #5 (run() / nodeOutputs.collected(), :357) is TS-only. PythonNodeExecutor has no run(); the actor’s run() path requires this._executor.run (actor.ts:286). A Python node flagged is_streaming_input therefore falls into the legacy single process() branch (actor.ts:358-371, site #3), never site #5. So for Python, site #5 is dead and site #357 is never reached. The zero-Python-change conclusion is unaffected; the coverage claim is corrected. Streaming-output multi-artifact nodes are out of scope (§5 site #4 invariant). Python streaming-output nodes deliver every artifact as a per-yield chunk frame (worker protocol.py:126-132), and the terminal result is deliberately empty (protocol.py:141-145, result_data={'outputs':{},'blobs':{}} for execute.stream). So for a Python generator that yields N distinct artifacts on the same slot, the only place those artifacts exist distinctly is the per-yield boundary — which we route to display-only output_update. The genProcess stream-end generation_complete would carry only artifact N. We scope this out (§3, Decision 1 corollary) rather than fix it, because the faithful fix would require the wire to distinguish a “complete artifact” yield from a progressive chunk — a final/complete marker on chunk frames — which would require a nodetool-core (worker/protocol.py) change, contradicting the zero-Python-change goal. No such node exists today; the §13 guard test documents the limitation. The bridge is pure request/response RPC keyed to request_id (python-bridge-base.ts:172-267) — it emits no node_update/output_update/generation/autosave concept. Therefore generation_complete is emitted entirely in the TS NodeActor. Python nodes get it for free for the targeted multi-execution collapse (the 6× ListGenerator→TextToImage case is a pure TS-actor correlated-drain phenomenon). No packages/protocol Python mirror, no bridge-protocol.ts change, no worker change. Python-side scope: none. 11. Protocol / serialization changes & full touch-point list msgpack is schemaless on both ends — server encodes with msgpackr pack (unified-websocket-runner.ts:7,1229), client decodes with @msgpack/msgpack decode (WebSocketManager.ts:242/249). No addExtension / Packr registry / type table anywhere. Any new type string round-trips for free; zero msgpack change. There is no zod schema for the streaming ProcessingMessage union (the api-schemas zod is REST chat-thread only) — zero zod change. Touch points (expanded to all consumers): packages/protocol/src/messages.ts: add interface GenerationComplete (after :178); add to the ProcessingMessage union (:633-653) — auto-flows into MessageType/MessageOfType, re-exported via index.ts:5. Add disposition? + done? to OutputUpdate (:201-210). packages/kernel/src/actor.ts: _emitGenerationComplete (bare, no job_id; node_name = node.name ?? node.type) + the 4–5 emit sites (§5); _generationIndex field. packages/kernel/src/runner.ts: set output_update.disposition at :1286-1294; ensure generation_complete is retained (not audio-dropped) in _emit (:1688-1708). packages/websocket/src/unified-websocket-runner.ts: add generation_complete to the processing gate :1889-1892; new autosave + normalize branch; delete node_update autosave :1932-1953. (Backfill at :1855-1861 already stamps job_id/workflow_id — no change.) web/src/lib/workflow/browserRunner.worker.ts + browserWorkflowRunner.ts: add generation_complete normalize/materialize branches (§8.5); confirm job_id backfill; add/confirm browser-path autosave (Decision 9). web/src/stores/workflowUpdates.ts: new generation_complete branch + silent-scrub gate (both writers); demote output_update to disposition-aware; delete completed→upsertLiveGeneration from node_update. web/src/stores/ResultsStore.ts: rewrite upsertLiveGeneration to append/replace-by-index with the index-less-patch contract; change setOutputResult to disposition-aware. web/src/core/chat/chatProtocol.ts: applyOutputUpdate honors disposition; documented generation_complete decision (Decision 11). web/src/hooks/timeline/useGenerateClip.ts + web/src/hooks/sketch/useGenerateLayer.ts: migrate asset extraction to generation_complete.outputs; thread disposition through their setOutputResult calls. web/src/components/miniapps/hooks/useMiniAppRunner.ts: build result tiles from generation_complete.outputs. mobile/src/stores/WorkflowRunner.ts + mobile/src/api.ts: add generation_complete case; regenerate message-type union; document node_update.result degradation (Decision 10). packages/cli/src/websocket-client.ts: add generation_complete to the union and the JSON event stream (Decision 12). 12. Rollout plan Ordered, each step shippable and green on both run paths (the original draft was green only for the server path): Protocol additive (no behavior change). Add GenerationComplete to the union + OutputUpdate.disposition?/done?. Build protocol. Nothing emits or consumes it yet. Back-compat: optional fields, additive union member — old clients ignore the new type. Kernel emits bare generation_complete at the 4–5 sites — no job_id and no index (per D8, both are stamped downstream: the server persist/relay seam derives index from DB ordering; the browser relay assigns an arrival-order index). Set output_update.disposition. Relay it through both the websocket (gate :1889-1892, backfill job_id+index, normalize .outputs) and the browser worker (assign arrival-order index, normalize .outputs, §8.5) without autosave yet. No consumer reacts → no behavior change. Add kernel emission tests (incl. the genProcess stream-end guard test). Frontend consumes generation_complete (new reducer branch + upsertLiveGeneration rewrite incl. index-less-patch contract + silent-scrub gate on both writers). Delete the node_update{completed}→upsertLiveGeneration write and the ResultsStore Tier-2 hack in the same commit. Migrate PreviewNode, mini-apps, timeline/sketch asset extraction, and mobile in the same wave (each is independently testable but must not be left reading the demoted channel for artifacts). Now the timeline shows N variants live. Autosave hard-switch. Add the generation_complete autosave branch and delete the node_update autosave atomically on the server (§7); land the browser-path persistence decision (Decision 9) in the same step. Now N assets persist on whichever paths are in scope. Demote output_update display (disposition-aware append/replace across all 5 setOutputResult sites + chatProtocol.ts; ephemeral-clear on generation_complete). Cosmetic. Cleanup: remove the now-dead node_update.result-as-artifact reads where a consumer fully migrated; tighten comments. (node_update.result survives as the documented mobile/skip-result degradation path.) Safe cutover sequence: steps 1–2 are invisible (additive + emit-only on both paths). Step 3 swaps the generation driver in one commit. Step 4 swaps the autosave driver in one commit. Never run both drivers of either {generations, autosave} simultaneously. Client/server version skew (NEW). Steps 1–2 (old-client/new-server) degrade safely — an old client ignores the new type and shows the old 1-variant behavior. The genuine risk is new-client/old-server: step 3 deletes the client’s node_update{completed}→upsertLiveGeneration write, but an old server never emits generation_complete → generations disappear entirely. Electron bundles client+server in lockstep (verified: web and websocket both 0.7.0-rc.23), so in-process is safe. For any remote nodetool serve deployment where web and the websocket server version independently, gate the step-3 deletion behind a server-capability probe, or keep the node_update generation write until the server is known to emit generation_complete. The optional NODETOOL_GENERATION_EVENTS flag stages a same-process cutover but does not cover client/server version skew. (Open — see §15.) 13. Test plan Kernel emission (packages/kernel): Correlated 6× generator (ListGenerator→TextToImage) emits exactly 6 generation_complete with index 0–5, each distinct outputs, one node_update{completed}. Controlled-loop node emits one per "run" event. genProcess node with a final whole-value yield: N output_update{disposition:"append"} yields + exactly one generation_complete at stream-end carrying the consolidated value (assert NOT per-yield, assert value is complete). Guard test (documented limitation): a genProcess node whose last yield does not carry the full value (overwrite-merge on one slot) loses prior content in the single generation_complete — assert this so a future multi-artifact-per-slot streaming node author hits a failing test, not silent loss. genProcess under correlation (multiple keys): one generation_complete per key. nodetool.constant.* / nodetool.input.* emit no generation_complete. generation_complete is never edge-suppressed (intermediate generator feeding a Preview still emits it) and never audio-dropped. Actor emit carries no job_id; assert the runner relay stamps it (server and browser paths). Autosave (packages/websocket): 6-gen run → 6 Asset rows, each with correct node_id/job_id. No double-save: assert node_update{completed} no longer triggers autoSaveAssets. Replay idempotency: re-emitting a fresh generation_complete object (asset_id unset, as a real reconnect would) is a no-op — requires the (job_id, node_id, index) guard from Decision 8, NOT just per-object asset_id. (Today’s per-object guard would duplicate.) Text-output autosave still fires off generation_complete.outputs. Browser path (web): In-browser ListGenerator→TextToImage→Preview end-to-end: 6 normalized variants render (assert generation_complete.outputs materialized — no broken images), and persistence matches Decision 9. Reducer / store (web): generation_complete{index:0,1,2} → 3 variants (rewrite ResultsStore.variants.test.ts:34-66). Index-less patch routing: node_update{running} then node_update{error} with no intervening generation_complete settles to exactly one errored generation (no stuck running). disposition:"append" concatenates; "replace" overwrites the display buffer — assert across workflowUpdates, chatProtocol, useGenerateClip, useGenerateLayer. node_update{completed} no longer mutates liveGenerations. mergeGenerations collapses live N + persisted N (same jobId) → N. Silent-scrub: 8 scrub frames, each driving node_update{running} and generation_complete under one silent jobId → 1 live generation (relocated from ResultsStore.variants.test.ts:154-183; covers both writers). Consumer surfaces: mini-app result list shows N tiles for a multi-execution generator; timeline/sketch extract the asset from generation_complete.outputs (not output_update); mobile shows ≥1 artifact via the new case; CLI --json emits N generation_complete events. 14. DECISIONS TO LOCK Status — locked 2026-06-20 D1 ✅ A — per-process() boundary; multi-artifact-per-output-slot streaming generators out of scope. D8 ✅ B (refined) — index from DB ordering, assigned at the persist/relay seam (not the actor). D9 ✅ Provider-generations only, always server-side; no browser autosave; browser normalize mandatory. D10 🔶 Provisional — mobile gets the generation_complete case + data now; full variant UI fast-follow (awaiting confirm). D11 ✅ B — chat shows variants. D2, D3, D4, D5, D6, D7, D12 ✅ accepted as recommended. Decision 1 — Generation boundary & list handling. Options: A) one generation_complete per process() result; list-valued handles carried as arrays, consumers flatten (runVariantValues / recursive autosave collect). B) fan a list-valued result into one generation_complete per element in the actor. Recommend: A — the actor already hands whole result dicts to _sendOutputs; passing the same dict avoids teaching the actor per-handle list semantics, and runVariantValues + autoSaveAssets’s recursive collect (:448-464) already flatten. Corollary (errors): keep node-level errors on node_update{error}; generation_complete represents committed artifacts only (no error variant in the shape). A run that errors before committing emits no generation_complete — correct. Corollary (streaming): the genProcess stream-end generation_complete carries the overwrite-merged _streamingCollectedOutputs, so multi-artifact-per-output-slot streaming generators are out of scope (§5/§10); guard-tested in §13. DECIDED: A — confirmed, including the streaming scope-out (multi-artifact-per-output-slot streaming generators are not covered this cycle; documented + guard-tested). Decision 2 — Silent-scrub gate location & mechanism. Options: A) backend pins index=0 for silent jobs (kernel must learn “silent”). B) frontend gate in the generation_complete handler and the node_update{running} placeholder: isSilentJob(jobId) → pin slot 0, else append by index (or newest-slot for index-less running/error). Recommend: B — silence is a pure slider-preview UI concern; the kernel stays dumb. The gate must cover both per-frame writers (generation_complete and the running placeholder), since §8.2 removes the isSilentJob special-case from upsertLiveGeneration. isSilentJob already imported in workflowUpdates.ts:37. Decision 3 — Autosave cutover. Options: A) dual-write (both node_update and generation_complete autosave during migration). B) hard switch (delete old, add new in one commit). Recommend: B — corrected rationale: runs 1..N-1 exist only on generation_complete (node_update collapses to the last), so dual-write under-saves the early runs regardless; and the cross-channel asset_id idempotency guard is reference-based — it protects the shared last-result object on the process() path (same instance at actor.ts:1000/1290, so dual-write would be a no-op there, NOT a double-save as the draft claimed) but does not protect the genProcess fresh-spread path (:994), where dual-write would double-save the final artifact. Atomic swap is the only clean option. (If a dark-launch is mandated, gate both with one flag, never both ON.) Decision 4 — output_update.disposition + ephemeral lifecycle. Options: A) add disposition: "append" | "replace" (+ optional done); clear the buffer on run-start and on generation_complete for that artifact; thread disposition through all 5 setOutputResult call sites. B) keep hardcoded append; clear only on run-start. Recommend: A — fixes the progressive-preview latent bug (workflowUpdates.ts:515 always-append) and makes whole-value snapshots correct. Absent disposition defaults to "append" for back-compat. Note the call-site count: workflowUpdates.ts:515, chatProtocol.ts:514, useGenerateClip.ts:186, useGenerateLayer.ts:227 — missing any re-introduces the bug in that consumer. Decision 5 — Keep the name output_update vs rename. Options: A) keep output_update. B) rename to display_update / output_chunk. Recommend: A — renaming churns the protocol union, both web reducers, chatProtocol.ts, mobile, CLI, the relay, and every display-sink component for a cosmetic gain; the disposition field already encodes the honest semantics. Revisit only if a future cleanup pass touches all sites anyway. Decision 6 — Do connected content-card/text nodes still emit output_update for live streaming? Options: A) yes — keep emitting for live token/preview display (now purely cosmetic). B) stop emitting for edge-connected handles (rely only on generation_complete). Recommend: A — live token streaming and progressive previews are real UX that generation_complete (fires only at commit) can’t provide. Edge-suppression already trims the firehose where it matters; the surviving emits are harmless display. Keep them. Decision 7 — How do constant/input/skipResult node values reach display sinks? Context: §5 skips generation_complete for nodetool.constant.* / nodetool.input.*; node_update.result is demoted to advisory; output_update is suppressible on edge-connected handles. That risks a value on none of the three authoritative channels. Options: A) these nodes’ downstream consumers are what emit generation_complete (the constant/input value flows as an input, and the consuming node commits its own artifact) — the constant/input node itself has no savable artifact and needs none; client-authored constants are already in the graph state, and server-resolved inputs surface via their consumer. Keep node_update.result as the explicit display fallback for the rare standalone constant/input preview. B) emit generation_complete for these too. Recommend: A — but state it explicitly rather than leaving the “client already holds the values” hand-wave: a constant/input feeding a connected node has its value carried into the consumer’s generation_complete; a standalone constant/input previewed directly reads node_update.result (the one sanctioned result read, scoped to skip-result node types). This reconciles “don’t read result for artifacts” (multi-execution generators) with “skip-result nodes use result” (they have no generation). Decision 8 — Replay/duplicate-asset idempotency. Options: A) per-object asset_id guard only (today). B) add a persisted-layer idempotency key on (job_id, node_id, index) — skip autosave for an already-persisted tuple. Recommend: B — a real reconnect/replay re-emits a fresh generation_complete (asset_id unset on a new object), so the reference-based guard does not fire → duplicate Asset rows sharing job_id but new ids. Since persisted assets carry no index today, add index to the autosave dedupe (either a stored column or an in-run (node_id, index) set per job) so replay is a true no-op. Without B, the §13 “replay is a no-op” test fails for genuine reconnects. DECIDED: B, refined — index is assigned from DB ordering at the server persist/relay seam, not by the actor. The unified-websocket-runner interception that backfills job_id (:1855-1861) persists the asset, derives index from the DB ordering of (job_id, node_id)’s assets, stamps it onto generation_complete before relay, and dedupes autosave on (job_id, node_id, index) → replay is a true no-op. Live ordering uses the stamped DB index (available because stamping happens in the same pre-relay pass). The browser path (no persist) falls back to an arrival-order index. Decision 9 — Browser-path persistence. Options: A) add a browser-side autosave hook keyed on generation_complete mirroring unified-websocket-runner’s branch. B) confirm + document that browser-path generative runs already round-trip persistence to the server (relying on the job_update{completed} asset reload), and that generation_complete in-browser is display-only. Recommend: decide before step 4 ships. The browser path calls autoSaveAssets nowhere (verified). If browser jobs do not reach the server autosave gate, option A is required or the “every committed artifact is persisted” goal fails for browser-eligible (and silent-scrub) workflows. Regardless of A/B, the browser normalize branch for generation_complete.outputs is mandatory (§8.5) for correct rendering. DECIDED: B — persistence is provider-generations only, always server-side; no browser autosave hook. Browser/non-provider generations are display-only and not persisted. Provider calls (fal/replicate/LLM/etc.) already execute server-side, so every persistable artifact hits the server autosave gate by construction. Option A (browser autosave) is rejected. The browser generation_complete.outputs normalize branch (§8.5) remains mandatory for rendering; only the persistence half is dropped from the browser path. Decision 10 — Mobile degradation. Options: A) mobile adds a generation_complete case but continues to show 1-artifact-per-node via node_update.result (graceful degradation, no variant UI). B) mobile renders full variant lists from generation_complete. Recommend: A for this cycle — state it as an intended degradation, regenerate mobile/src/api.ts types, and add the case so multi-execution at least shows the latest. Variant UI on mobile is a follow-up. DECIDED (provisional): add the generation_complete case + regenerate types now; full variant UI as a fast-follow. Mobile shows the latest immediately and does not block the cutover. Awaiting confirmation on whether mobile variant UI is wanted in this cycle. Decision 11 — Chat/agent path & generation_complete. Options: A) chat ignores generation_complete (display-only via output_update/media_generation); document why. B) chat gets its own generation_complete → variant handling. Recommend: A for this cycle — chat surfaces live media via existing chunk handling; workflow-tool multi-execution variants are not a chat UX today. Document the decision; don’t leave it implicit. applyOutputUpdate still honors disposition regardless. DECIDED: B — chat shows variants. Chat gets full generation_complete → variant handling (overrides the degrade recommendation). applyOutputUpdate still honors disposition. Decision 12 — CLI semantics. Options: A) surface each generation_complete as its own event in the --json stream. B) aggregate into the final result. Recommend: A — preserves the N-artifact information the kernel now exposes; add the type to the CLI union. Lowest priority but must not silently drop variants. 15. Risks & open questions Storage volume. N-per-run autosave means previously-discarded intermediates now persist (a 100-frame batch generator writes 100 assets). Confirm desired; consider a per-node auto_save_asset cap or an explicit “save intermediates” flag for very-high-N generators. (Open — needs product call.) Browser-path persistence (Decision 9). The single biggest open item: browser jobs call autoSaveAssets nowhere. Until Decision 9 lands, N-asset persistence is unproven for browser-eligible and silent-scrub runs. The browser normalize branch is mandatory independent of the persistence decision. Replay duplicate assets (Decision 8). Persisted assets carry no index; the only dedupe is the per-object asset_id mutation. A reconnect re-emits fresh generation_complete objects → duplicate rows. Needs (job_id, node_id, index) idempotency at the autosave layer. _messages truncation vs reconnect replay. generation_complete is pushed into _messages, but _messages is truncated to MAX_RETAINED_MESSAGES/2 past 10,000 (runner.ts:1697-1700) and is not the autosave source (autosave consumes the live relay stream in unified-websocket-runner, not _messages). Drop the draft’s “_messages authoritative for autosave” claim. For a very-high-N reconnect, early generation_complete entries can be evicted from replay; persisted assets recover them via the job_update{completed} reload — verify the reconnect replay path preserves index ordering for what survives, and document that very-high-N reconnects may drop early live variants from replay (persisted side recovers them). running placeholder + index-less patches. generation_complete fires only at commit, so the pre-first-artifact spinner relies on the node_update{running}→upsertLiveGeneration placeholder; a node that errors before any artifact must be settled by the index-less node_update{error} patch via the newest-slot fallback (§8.2). Tested in §13. job_id/node_name in the actor. Resolved (was asserted-resolved-but-wrong): the actor has no job_id; the runner relay stamps it at unified-websocket-runner.ts:1855-1861. node_name = node.name ?? node.type, not node.data?.title. The browser relay must apply the same stamping. Client/server version skew (§12). New-client/old-server erases generations (client deletes the node_update write; old server never emits generation_complete). Lockstep in Electron (verified 0.7.0-rc.23); for remote serve, gate the step-3 deletion behind a capability probe or keep the node_update write until the server is known to emit the new event. mergeGenerations jobId coupling. The live/persisted collapse depends on all N variants and N assets sharing one jobId (nodeGenerations.ts:74-82, assetToGeneration at :58-60). index must not leak into the jobId match. Verified the merge groups purely by jobId; confirm assetToGeneration keeps grouping intact when N assets land. All consumers migrated. The consumer census (§8.5–§8.11) must be complete; a missed reader of node_update.result/output_update.value as an artifact silently loses variants. Re-run the grep before sign-off. Appendix — Addressed review notes Findings verified against the working tree; each is either incorporated above or marked corrected-as-wrong with why. Double-save rationale inverted (high) — INCORPORATED + CORRECTED. Verified actor.ts:1000/:1290 assign _latestResult = outputs (same ref as the would-be generation_complete.outputs), and the autoSaveAssets guard (:468/:518-520) is reference-based. The draft’s “different instances → double-save” was inverted for the process() path (it’d be a no-op there); the real double-save risk is the genProcess fresh-spread path (:994). Rationale rewritten in §7 and Decision 3 around (a) under-save of runs 1..N-1 and (b) the genProcess-only double-save. genProcess stream-end overwrite-merge (high/critical) — INCORPORATED. Verified _streamingCollectedOutputs is Object.assign overwrite-merge (:984) then {...} (:994). Stated the load-bearing invariant in §5, scoped multi-artifact-per-slot streaming out (§3, Decision 1 corollary, §10), added a guard test (§13). Index-less running/error patches into an index-keyed store (medium) — INCORPORATED. Specified the index-less-patch contract in §8.2 (newest-slot-by-jobId fallback retained for running/error; explicit index only from generation_complete) + a §13 test for running→error-with-no-generation settling to one errored gen. 4./21. Actor job_id/node_name wrong-as-written (high/medium) — INCORPORATED + CORRECTED. Verified zero job_id in actor.ts; backfill at :1855-1861; _emitNodeStatus uses node.name ?? node.type. §5 emit snippet now omits job_id and uses node.name; §15 open question resolved. 5./15./16. Browser path autosave + normalize gaps (critical/high) — INCORPORATED. Verified autoSaveAssets only in unified-websocket-runner; browser normalize only handles .result/.value (browserWorkflowRunner.ts:435-441, browserRunner.worker.ts:187-203). Added §8.5, Decision 9, touch-point #5, browser test, and rollout/§3 corrections. 6./7. (Duplicate of 4 from a second reviewer.) — Same correction; consolidated. Silent-scrub running placeholder (high) — INCORPORATED. §9 now gates both the generation_complete write and the per-frame running placeholder on isSilentJob; §13 test drives both halves per frame. _messages retention vs “authoritative for autosave” (medium) — INCORPORATED + CORRECTED. Verified MAX_RETAINED_MESSAGES=10_000 slice (:1697-1700) and that autosave reads the live relay, not _messages. Dropped the “authoritative for autosave” claim; §15 documents truncation + reconnect implications. Replay duplicate assets (medium) — INCORPORATED. Verified Asset has node_id/job_id but no index; guard is per-object. Added Decision 8 ((job_id, node_id, index) idempotency) and a fresh-object replay test. New-client/old-server skew (medium) — INCORPORATED. Verified web and websocket are lockstep 0.7.0-rc.23. §12 documents Electron lockstep vs remote-serve risk and the capability-probe mitigation. 12./13. Python streaming-output + streaming-input site coverage (high/medium) — INCORPORATED. Verified PythonNodeExecutor implements only process()/genProcess() (no run()), so site #5 is TS-only and Python streaming-input uses site #3. Verified the empty terminal result for execute.stream. §10 corrected; streaming-output multi-artifact scoped out. Critical (mobile) — INCORPORATED (§8.6, Decision 10, touch-point #11). Critical (timeline/sketch) — INCORPORATED (§8.8, touch-point #9). Verified the selectedOutputNodeId/jobOutputs/extractAssetId/fail-job pattern. High (browser worker) — INCORPORATED (folded into §8.5). High (mini-apps) — INCORPORATED (§8.9, touch-point #10). Verified result tiles built entirely from output_update. High (browser autosave) — INCORPORATED (Decision 9, §15). Medium (CLI) — INCORPORATED (§8.11, Decision 12, touch-point #12). Medium (chat generation) — INCORPORATED (§8.10, Decision 11). Medium (constant/input leak) — INCORPORATED (Decision 7). Verified the skipResult set (actor.ts:437-439) and reconciled the result-read rules. Medium (setOutputResult call sites) — INCORPORATED (§8.7, Decision 4). Verified 4 runtime call sites + 1 doc comment. No findings were dropped as wrong; the two that were partially wrong (the double-save rationale and the _messages-authoritative claim) are corrected in-body rather than silently removed.--- # runpod-deployment-noext.md Source: https://docs.nodetool.ai/runpod-deployment This page has moved. Please go to Deployment Guide.--- # runpod-deployment-html.md Source: https://docs.nodetool.ai/runpod-deployment.html This page has moved. Please go to Deployment Guide.--- # Script Editor — Concept Source: https://docs.nodetool.ai/script-editor-concept.html Script Editor — Concept Status: Concept / brainstorm · Owner: matti A script as a first-class resource: text plus the audio takes voiced from it, authored in an ElevenLabs-Studio-style editor, importable into a timeline sequence for video. This document scopes the idea against what already exists in the codebase. The pitch Today the only place to write and voice narration is inside a timeline: the Studio transcript layer (TimelineTranscriptStore) projects a document view over voiceover clips, and the clips are the source of truth. That is the right model for post-production — word-accurate ripple cuts on media that already exists — but wrong for pre-production. A writer wants to draft, restructure, assign speakers, and audition voices before any timeline exists, and wants the same script reusable across several videos (a 60s cut and a 15s teaser). The Script editor inverts the transcript’s ownership: text is the source of truth, audio is derived. A script lives on its own, line by line, each line voiced into one or more takes (audio assets with word timings). “Send to timeline” assembles the current takes into a sequence — the same move the storyboard already makes for images and video. What already exists (the substrate) The concept is mostly assembly of parts that are in main: Piece Where What it gives the script editor First-class-resource pattern packages/models/src/schema/storyboards.ts, StoryboardStore, StoryboardSurface Table shape (id, user_id, project_id, name, document, timeline_id), tRPC CRUD, workspace tab, autosave Assemble → timeline web/src/components/storyboard/assembleTimeline.ts Resource → sequence creation with linkage keys on clips Back-sync after regen web/src/stores/storyboard/timelineSync.ts Re-voiced line updates the assembled timeline clip (CAS update, never throws) Voicing pipeline TimelineTranscriptStore.generateBeat text → generate_media (TTS) → probe duration → transcribe_audio (word timings) → reflow Word-timing model captionWord / clipCaption in packages/protocol/src/api-schemas/timeline.ts Clip-local word timings that survive re-flow Take/version model clipVersion (same file), ClipVersionHistory, ShotTakesGallery Versioned generations with param snapshots, favorites, cost Document editor infra web/src/components/timeline/transcript/ (Lexical: WordNode, SceneBreakNode, slash commands) The text-editing surface, word-level selection, per-word playback highlight Providers packages/runtime/src/providers/elevenlabs-provider.ts + every other TTS-capable provider Voice generation; ElevenLabs also returns native timestamps What does not exist: the script document schema, the scripts table, the standalone editor surface, a cast/speaker→voice binding, and the script↔timeline linkage keys. Document model Zod schema in packages/protocol/src/api-schemas/ next to the timeline and storyboard schemas. Sketch: Script ├─ id, name ├─ cast: Speaker[] { id, name, color, │ voice: { provider, model, voice, settings? } } └─ sections: Section[] { id, title? } // scene / chapter breaks └─ lines: Line[] ├─ id, speakerId?, text ├─ direction? // freeform performance note ("whispering, │ // tired"), passed through to providers │ // that accept it — no structured tag schema ├─ pauseAfterMs? // authored silence between lines ├─ takes: Take[] │ { id, assetId, durationMs, words: CaptionWord[], │ textSnapshot, voiceSnapshot, createdAt, favorite?, costCredits? } └─ currentTakeId? Derived, not stored: Line status: draft (no takes), voiced (current take’s textSnapshot and voiceSnapshot match the line), stale (text or voice changed since the take — show a re-voice affordance, exactly like the timeline’s dependencyHash staleness). Script duration: sum of current-take durations plus pauses. Placeholder duration for unvoiced lines (the transcript layer’s PLACEHOLDER_BEAT_MS move). CaptionWord is reused as-is; timings stay take-local so re-ordering lines never rewrites word timings — the same invariant the timeline already relies on. Takes are the clipVersion idea relocated onto the script: textSnapshot + voiceSnapshot play the role of paramOverridesSnapshot, and the take gallery is the shot-takes gallery with an audio player instead of a thumbnail. Storage scripts table mirroring storyboards byte for byte in shape: document JSON column, timeline_id back-pointer once assembled, indexes on user/project/ updated. tRPC router with the same get/list/create/update(CAS)/delete surface, autosave hook copied from the timeline’s. Audio takes are ordinary assets — the script references them by assetId, so asset lifecycle, storage backends, and .nodetool bundle export (graph + referenced asset bytes) all work unchanged. The script itself is a DB resource, not an asset, matching workflows/storyboards/sequences. Editor surface A new workspace tab type (WorkspaceTabsStore already routes storyboard / timeline / sketch surfaces), laid out like ElevenLabs Studio: Document pane — the script as continuous prose, one paragraph per line, speaker chip in the gutter. Reuse the Lexical transcript editor: it already has word nodes, scene breaks, and slash commands. Enter splits a line, backspace at start merges, drag reorders. Cast panel — speakers with voice pickers (provider/model/voice) and a “preview voice” button. Assigning a speaker to a line inherits the voice; per-line overrides allowed. Line affordances — voice / re-voice button with per-line spinner (clipStatus pattern), take gallery popover, stale badge, per-line playback. Play-through — sequential playback of current takes with the word highlight the transcript editor already renders; unvoiced lines are skipped or beep-placeholded. This is a plain Audio-element chain, no compositor needed. Voice all — batch-generate every draft/stale line, bounded concurrency, per-speaker voices respected. Voicing pipeline Identical to generateBeat, minus the timeline coupling: line text (+ direction) → generate_media RPC (mode audio, speaker’s provider/model/voice) → asset id probe duration word timings onto the take: providers that declare a ttsTimestamps capability return them natively with the synthesis (ElevenLabs with-timestamps endpoints — more accurate, one call fewer); everything else falls back to the transcribe_audio RPC (best-effort, as today — a failed transcription still leaves a playable take) append take, set currentTakeId Timeline import (“Send to timeline”) The storyboard’s assemble move, for audio: Create (or update) a sequence: one voiceover clip per line on an audio track — optionally one track per speaker — laid end to end with pauseAfterMs gaps. Each clip carries the take’s asset, its CaptionWord[] as clip.caption, speaker, and two linkage keys: scriptId + scriptLineId (the storyboard’s storyboardBoardId/storyboardShotId pattern). Set script.timeline_id. The assembled sequence opens in the timeline editor, where the existing transcript layer takes over for word-level post-production — the imported clips are indistinguishable from beats voiced in-timeline. Re-voice after assembly: mirror syncShotClipToTimeline — when a linked line gets a new current take, patch the matching clip’s currentAssetId, duration, and caption via a CAS document update. Log-and-continue on failure, never block the take. Structural drift: adding/removing/reordering lines after assembly is the hard case. V1 answer: sync only per-line asset swaps; structural changes prompt “re-assemble” (which re-lays the voiceover track but preserves other tracks). Anything cleverer (three-way merge against the timeline’s own edits) is explicitly out of scope until the simple thing proves limiting. Reverse import: “Extract as script” from a timeline — project the transcript document (buildTranscriptDoc) into a new script resource. Cheap to build because the projection already exists, and it closes the loop for recorded/imported media: transcribe a recording in the timeline, extract the script, re-voice it with a cast. Relation to the in-timeline transcript Keep both, with distinct jobs: the script is pre-production (authoring, casting, auditioning — text owns audio), the transcript is post-production (ripple cuts on real media — clips own words). The linkage keys are what stop this from becoming two competing sources of truth: a clip either belongs to a script line (script wins for text/voice, timeline wins for placement/trims) or it doesn’t (transcript behaves exactly as today). Unifying them into one model was considered and rejected for now — the transcript’s clips-as-truth invariant is load-bearing for ripple editing and undo, and inverting it underneath the existing editor is a rewrite, not a feature. Automation Agent tools: ui_script_* mirroring the timeline set — get_state, add_line, set_line_text, set_speaker, set_speaker_voice, voice_line, voice_all, send_to_timeline. Same bridge pattern (timelineAgentBridge), same assistant panel. This is what makes “paste a topic, get a voiced script” a chat interaction. Graph nodes: ScriptRef in the protocol type system, plus a small node family — LoadScript, VoiceScript (batch TTS over a cast), ScriptToTimeline. Enables headless pipelines: LLM writes script → voice → assemble → render. Exports: SRT/VTT straight from take word timings; audio-only mixdown (concatenate takes + pauses) as an asset; scripts included in .nodetool bundles with their take assets. Open questions Dialogue-mode models. ElevenLabs v3 renders multi-speaker dialogue in one call with better prosody than line-by-line synthesis, but produces one asset spanning many lines — it breaks the line↔take 1:1. Likely modeled as a section-level “dialogue take” whose word timings are split back onto lines. Phase 4 at the earliest. Take retention. Takes are cheap to keep (assets already exist); surface costCredits per take like clip versions do, revisit pruning only if it hurts. Collaboration. Same last-write-wins CAS as timeline and storyboard; nothing script-specific. Phasing Author + voice (implemented) — protocol schema (api-schemas/scripts.ts), scripts table + tRPC (scripts router), store autosave (ScriptStore, useScriptServerSync), workspace surface (ScriptSurface, script tab type), cast panel, per-line voicing (scriptVoicing, reusing generate_media / transcribe_audio), take gallery, play-through. The document pane is a plain per-line editor rather than the Lexical transcript editor — that reuse is deferred to a later pass. To timeline (implemented) — assemble the current takes into a voiceover sequence with scriptId/scriptLineId linkage keys (assembleScriptTimeline, useAssembleScriptTimeline, “Send to timeline”), per-line back-sync on re-voice / take switch (stores/script/timelineSync), re-assemble in place on structural drift (preserving foreign tracks), and extract-as-script from a timeline transcript (extractScript, useExtractScript, the transcript panel’s “Extract as script”). Automation (implemented) — ui_script_* agent tools (scriptAgentBridge, useScriptAgentBridge, builtin/script: get_state, add_speaker, set_speaker_voice, add_line, set_line_text, set_speaker, voice_line, voice_all, send_to_timeline) + an in-surface assistant panel (ScriptAgentPanel, a Cast/Assistant dock toggle), ScriptRef in the protocol type system with a script data type and property editor, and a graph node family (ConstantScript, LoadScript, VoiceScript — batch TTS over a cast, ScriptToTimeline) reading/writing scripts through new ProcessingContext script methods. Depth — SRT/VTT export (implemented): a shared, framework-agnostic subtitle codec (@nodetool-ai/timeline assembleSubtitleCues / cuesToSrt / cuesToVtt) that lays each line’s current take end to end with the authored pauses and renders line- or word-granularity cues from the take word timings, surfaced as a graph node (ScriptToSubtitles), an agent tool (ui_script_export_subtitles), and an “Export SRT” button on the script surface (exportScriptSubtitles). Still open: dialogue-mode rendering, provider-native timestamps, audio-only mixdown, and .nodetool bundle support. Phase 1 is deliberately shippable alone: a script you can write, cast, voice, and listen to is already the ElevenLabs-Studio use case, before any timeline integration.--- # Security Hardening Source: https://docs.nodetool.ai/security-hardening.html Use this page as a checklist before exposing NodeTool beyond your laptop. It covers security practices for each deployment stage and links to canonical guides for configuration, authentication, and proxy settings. Universal Checklist These apply to every deployment, regardless of environment: Network Security Require TLS at the proxy or ingress layer. Use real certificates (e.g., Let’s Encrypt) and redirect HTTP to HTTPS. Restrict Docker access: Run the proxy with a dedicated network (docker_network) and avoid exposing the Docker socket beyond the host. Firewall rules: Only expose the ports you need (typically 443 for HTTPS). Block direct access to internal service ports. Authentication ⚠️ NODETOOL_TRUST_LOCAL_NETWORKS bypasses login entirely. Every source IP in that list is trusted as admin user "1" with no password — full access to data, secrets, and API keys. The Docker Compose file trusts the Docker bridge (172.16.0.0/12) so a single-user local install works out of the box. Never set it to 0.0.0.0/0 on a public IP, and never widen it as a substitute for real auth — enable Supabase mode instead. See Authentication → Local mode in Docker. Enforce auth on any network-accessible deployment. The server enforces authentication only when it is in Supabase mode — that is, when both SUPABASE_URL and SUPABASE_KEY are set. Without them it runs in Local mode and trusts loopback connections (and any NODETOOL_TRUST_LOCAL_NETWORKS you configure). See Authentication. Lock down localhost trust. Behind a reverse proxy or in a container, set NODETOOL_TRUST_LOCALHOST=false (it already defaults off when auth is enforced) and list only your real proxies in NODETOOL_TRUSTED_PROXIES, so a proxy connecting from loopback cannot silently bypass auth. Keep NODETOOL_TRUST_LOCAL_NETWORKS tight. Trust the smallest range that works (the Docker bridge, not 0.0.0.0/0), and firewall the published port. It is ignored in Supabase mode. Rotate keys regularly – set calendar reminders to rotate Supabase service-role keys. Secrets Management Keep secrets out of Git: Load provider API keys and tokens from environment variables or a secrets manager. Never commit .env files with secrets. Add .env to .gitignore. Provide the encryption master key (SECRETS_MASTER_KEY) on every server so they share one key, and store all other deployment secrets in your platform’s secrets manager. Resource Limits Set container resource limits: Use mem_limit and cpus to prevent runaway workloads. Use read-only mounts where possible to limit filesystem write access. See Docker Resource Management for configuration details. Development (Local) Local development has the lowest risk but still deserves basic hygiene: Action Details Bind to localhost only Use 127.0.0.1 for all services; avoid publishing container ports to the LAN Use temporary tokens Generate throwaway tokens for demos; clear ~/.config/nodetool/deployment.yaml when finished Don’t expose to the internet Never use ngrok or similar tunneling without authentication in place Staging Staging environments should mirror production security with some allowances for debugging: Action Details Gate access Use VPN or IP allowlists; do not rely on obscurity Separate credentials Use distinct Supabase projects and tokens from production Rotate on personnel changes Rotate service-role keys when team members leave Enable end-to-end TLS Including internal hops if traversing untrusted networks Regular backups Back up workspace volumes and databases on a schedule; restrict who can read backups Audit logging Enable request logging at the proxy layer to track access patterns Production Production deployments require the strictest security posture: Authentication & Authorization Enable Supabase mode (set both SUPABASE_URL and SUPABASE_KEY) for any multi-user or network-accessible deployment so every request is authenticated Keep NODETOOL_TRUST_LOCALHOST off and restrict NODETOOL_TRUSTED_PROXIES to your real proxy addresses Unset NODETOOL_TRUST_LOCAL_NETWORKS (or leave it ignored under Supabase mode) — never carry a 0.0.0.0/0 trust rule into production Use dedicated service accounts for each deployment; avoid shared credentials Keep proxy.yaml free of embedded secrets – distribute bearer tokens via your secrets manager Infrastructure Set idle_timeout and per-service resource caps to prevent runaway workloads on multi-tenant hosts Use separate networks for the proxy, API server, and worker containers Pin container image versions (avoid latest tags in production) Monitoring & Alerting Centralize logging – Forward container logs to your logging platform (ELK, CloudWatch, etc.) Monitor for anomalies: Auth failures, 429 rate limits, container restarts, unusual traffic spikes Set up alerts for failed authentication attempts and service health degradation Track resource usage to detect potential abuse or misconfiguration Image Management Rebuild regularly to pick up base image security patches Track CVEs in your base images using tools like trivy or grype Prune unused images from registries and hosts to reduce attack surface Sign images if your workflow supports it, to ensure integrity Data Protection Encrypt data at rest – Use encrypted volumes for workspace and database storage Encrypt data in transit – TLS everywhere, including internal service communication Minimize data retention – Clear temporary assets (assets-temp bucket) on a schedule Backup strategy – Regular automated backups with tested restore procedures Quick Security Audit Run through this checklist before any deployment goes live: TLS enabled with valid certificates Supabase mode enabled (SUPABASE_URL + SUPABASE_KEY set) for any network-accessible deployment NODETOOL_TRUST_LOCAL_NETWORKS unset or tightly scoped — no 0.0.0.0/0 on a network-accessible deployment NODETOOL_TRUST_LOCALHOST off and NODETOOL_TRUSTED_PROXIES scoped to real proxies All API keys loaded from env vars or secrets manager (not hardcoded) Container resource limits configured Docker socket not exposed to containers Firewall rules restrict access to necessary ports only Logging enabled and forwarded to monitoring platform Backup schedule configured and tested Service tokens rotated within the last 90 days Base images updated within the last 30 days Related Authentication – Auth provider configuration Configuration Guide – Environment variables and settings Deployment Guide – Deployment overview and workflows Self-Hosted Deployment – Self-hosted setup details Docker Resource Management – Container resource limits Storage – Storage backend configuration--- # Self-Hosted Deployment Guide Source: https://docs.nodetool.ai/self-hosted-deployment.html This guide covers deploying NodeTool on your own infrastructure. Overview Self-hosted deployment runs NodeTool in a Docker container — on localhost or on a remote host reached over SSH. Docker is the only supported deployment type (SUPPORTED_TYPES = ["docker"]). Two paths: Docker Compose — a single docker-compose.yml you run yourself. The fastest way to stand up one server. See below. nodetool deploy CLI — a managed flow driven by deployment.yaml that also handles remote hosts over SSH, image transfer, and workflow sync. See Deployment Configuration. Docker Compose (reference) The repository ships a reference docker-compose.yml for running one server on a host you control. cp .env.example .env # fill in the provider keys you use docker compose up -d # open http://localhost:17777 The server binds to 0.0.0.0:7777 inside the container and is published on the host as ${NODETOOL_PORT:-17777}. All persistent state — SQLite database, assets, vector store, model cache, and the generated secret key — lives under /workspace, backed by the named nodetool-data volume, so it survives restarts and image upgrades. Common overrides (set in .env or the shell): Variable Default Purpose NODETOOL_VERSION latest Image tag to pull (pin a release in production) NODETOOL_PORT 17777 Host port mapped to the container’s 7777 NODETOOL_TRUST_LOCAL_NETWORKS 172.16.0.0/12 ⚠️ Source CIDRs trusted as user 1 without a login (Local mode). Docker bridge by default — never 0.0.0.0/0 on a public IP. Ignored in Supabase mode SECRETS_MASTER_KEY auto-generated 32-byte base64 key encrypting stored secrets — set explicitly in production (openssl rand -base64 32) OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY, FAL_API_KEY, HF_TOKEN unset Model provider keys Upgrade in place: docker compose pull docker compose up -d To store data in a host directory instead of the named volume, replace the nodetool-data:/workspace mount with a bind mount (e.g. ./nodetool-data:/workspace) and make sure the host directory is writable by the container’s node user. Authentication / login screen Auth is configured entirely on the backend. The web UI fetches its auth mode and public Supabase credentials at runtime from GET /api/config (a public, non-secret endpoint), so the same frontend build works with or without login — no rebuild, and it works even when the frontend is served from a different origin. Login off (default). With SUPABASE_URL/SUPABASE_KEY unset the server runs in Local mode: it trusts requests by source IP (loopback, plus any NODETOOL_TRUST_LOCAL_NETWORKS you set) and runs them as a single user; other requests are rejected, and the UI shows no login screen. In Docker the bundled Compose file trusts the Docker bridge (172.16.0.0/12) so a local install works out of the box — see the warning below. Login on. Set these three on the server to switch to Supabase mode — the server requires a valid Supabase JWT on every request and the UI shows the login screen: SUPABASE_URL=https://your-project.supabase.co SUPABASE_KEY=your-service-role-key # server-only, never sent to the browser SUPABASE_ANON_KEY=your-anon-key # public key the login screen uses Optionally set AUTH_REDIRECT_URL when serving behind a domain/proxy (it must be in the Supabase project’s redirect allow list). GET /api/config returns authMode, supabaseUrl, supabaseAnonKey, authRedirectUrl, and version — never the service-role key. See Authentication and Supabase Deployment. 🔒 Do not expose Local mode to the internet Local mode has no login. NODETOOL_TRUST_LOCAL_NETWORKS trusts a set of source IPs as admin user "1" with no password — full access to your data, secrets, and API keys. Anyone who can reach the published port from a trusted range gets in. Safe on a laptop or a private LAN/VPN behind a firewall. The default 172.16.0.0/12 trusts only Docker’s bridge, not the wider internet. Never change it to 0.0.0.0/0 on a public IP. Putting NodeTool on a public address or sharing it with untrusted users? Enable Supabase mode (above) so every request needs a real login, and terminate TLS in front of the server. Serving the web UI from a different origin (e.g. a CDN)? Point the frontend at the backend with the build-time VITE_API_URL, and add that origin to the server’s NODETOOL_ALLOWED_ORIGINS so the cross-origin GET /api/config and API calls are permitted. Everything else still comes from /api/config. Deployment Configuration Deployments are configured via deployment.yaml. Docker Deployment deployments: my-server: type: docker enabled: true host: 192.168.1.10 ssh: user: ubuntu key_path: ~/.ssh/id_rsa container: name: nodetool-server port: 8000 gpu: "0" paths: workspace: /data/nodetool hf_cache: /data/hf-cache image: name: ghcr.io/nodetool-ai/nodetool tag: latest For a local host, set host: localhost and omit the ssh block. Apply Flow Directory Creation: Ensures workspace and hf_cache directories exist. Image Check: Verifies the configured image exists locally/remote. deploy apply does not auto-pull. Image Transfer: For remote hosts, copies image to remote runtime if needed. Container Management: Restarts container with new configuration. Health Check: Verifies HTTP endpoint. End-to-End: Local Docker Deployment This walkthrough matches a common local setup flow: Pull the Docker image. Add a docker deployment interactively. Review the generated deployment. Apply deployment and validate health. Sync workflows. Run a synced workflow on the deployed instance. 0. Pull the Image First docker pull ghcr.io/nodetool-ai/nodetool:latest 1. Add Local Docker Deployment nodetool deploy add local --type docker --type docker is required. The command then prompts for the rest: Host address: localhost Docker image name: ghcr.io/nodetool-ai/nodetool Docker image tag: latest Container name: nodetool Port: 8000 GPU/workflows assignment: optional Workspace folder: $HOME/.nodetool-workspace HF cache folder: canonical local HF cache (auto-detected, usually $HOME/.cache/huggingface/hub) Path meanings: Workspace: stores assets and temporary runtime data. HF cache: stores downloaded Hugging Face artifacts/models. 2. Review Deployment Config nodetool deploy show local This dumps the deployment entry as YAML. You should see something like: local: type: docker host: localhost image: name: ghcr.io/nodetool-ai/nodetool tag: latest container: name: nodetool port: 8000 paths: workspace: <your workspace path> hf_cache: <your HF cache path> The server endpoint is at http://localhost:8000. The container EXPOSEs 7777; when container.port is 7777 the deployer maps it to host port 8000 (any other container.port is used as-is). You can also inspect the raw config: cat ~/.config/nodetool/deployment.yaml 3. Apply Deployment nodetool deploy apply local Expected successful output includes: directories created image check passed app container started health checks passing for http://127.0.0.1:8000/health Deployment successful secrets synced If the first apply fails (for example due to container/port conflicts), run apply again: nodetool deploy apply local Then confirm runtime state: {% raw %} nodetool deploy status local docker ps --filter name=nodetool --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}" curl http://127.0.0.1:8000/health {% endraw %} 4. Sync Workflows to the Deployment List local workflows first: nodetool workflows list Sync one workflow by ID to the deployed instance: nodetool deploy workflows sync local <workflow_id> This sync command also: uploads referenced assets downloads referenced models (HuggingFace/Ollama) on the target Verify remote workflows: nodetool deploy workflows list local 5. Test Workflow Execution on Deployed Instance Run a synced workflow remotely: nodetool deploy workflows run local <workflow_id> Optional params example: nodetool deploy workflows run local <workflow_id> -p prompt="hello" If a run fails, inspect logs: nodetool deploy logs local --tail 200 Manual Troubleshooting Container Logs nodetool deploy logs local --tail 200 For a remote host you can also read the container logs directly: ssh user@host "docker logs nodetool-server"--- # Programmatic SEO Playbook Source: https://docs.nodetool.ai/seo-programmatic Companion to SEO Strategy. That document is the audit and roadmap; this one is the build plan for programmatic pages. Source: a teardown of photoai.com (9,466 indexed URLs, ~78% generated from “user activity”, the rest templated per keyword). Written 2026-07-02, revised the same day to add the seeded-showcase engine, model pages, and the ranked list of further dimensions. File paths and counts below drift — re-verify before building. The thesis, adapted Photo AI treats pages as a product output, not a content cost: adding a keyword or generating an image both create a fully-optimized, schema-rich, interlinked page automatically. The thing to copy is the machine, not the pages. Three facts shape NodeTool’s version: Photo AI’s “UGC” pages did not wait for users. The founder seeded /photos/* with his own generations; user volume came later. NodeTool can do the same: batch-generate real outputs from its own workflows and publish each run as a page. The outputs are genuine (real model, real prompt, real workflow) — what’s seeded is who pressed the button. The line not to cross is fabricated attribution: no invented user names, no synthetic Review/aggregateRating markup. Google’s spam policies name fake review markup specifically; the generations themselves are fine. Search Console already shows what thin programmatic pages earn here. The 375 node-reference URLs on docs.nodetool.ai took 7.8k impressions for 12 clicks (SEO Strategy §0.4) because the queries were “official docs” lookups for third-party libraries. Page count without intent match is a proven zero on this domain. Every engine below has an index gate for this reason. NodeTool’s unique assets are workflows and multi-provider model access. A workflow has structure an image doesn’t: a rendered graph, a node list, typed inputs/outputs, a category. And the node packages expose the models people actually search for — Seedance, Veo, Kling, Sora, Hailuo, Wan, Seedream, FLUX, GPT-Image — each through several providers (Seedance alone via fal, kie, replicate, and atlascloud nodes). Same prompt, two models, side by side, with real outputs: content no single-model vendor can publish and no competitor bothers to. What survives the adaptation intact: fixed template per page type, output-first page layout, the dense internal-link mesh, per-page-type JSON-LD, one derived sitemap, and the principle that adding one row of data ships one complete page. Engine 1 — Template pages (/templates/*) 37 example workflows ship in packages/base-nodes/nodetool/examples/nodetool-base/*.json, each with name, description, tags, and the full graph. Three have marketing pages (/use-cases/*); 34 have none. Every one becomes a page at /templates/<slug>. Page anatomy, in render order: Sample output first. The generated video/image/audio leads the page (see Engine 2 for supply); the graph explains it below. Photo AI’s photo pages lead with the photo for the same reason: the output is what the searcher wants to judge. Slug and H1 from the workflow name (Movie Posters → /templates/movie-posters, H1 “Movie Poster Generator — free AI workflow template”). Meta description from the workflow’s description, extended with node count and category. Rendered graph. The static flow components in marketing/src/components/flow/ (NodeCanvas, FlowNode, FlowEdge, proven at marketing/src/app/flow-preview/page.tsx) render the actual graph server-side from the JSON’s ui_properties. No sibling page and no competitor has this DOM. “Nodes in this workflow”: one line per distinct node type — what it does, which provider runs it — with links into the model pages (Engine 3). How to run it: open in NodeTool Cloud CTA plus the download path. Related templates: 8–9 links by shared tags/category — Photo AI’s mechanism for circulating equity and crawl depth through the cluster. JSON-LD: HowTo + ItemList via the existing JsonLd.tsx. Build mechanics: a build-time script reads the example JSONs into a generated templateEntries.ts (same shape-discipline as use-cases/useCaseEntries.ts), one dynamic route app/templates/[slug]/page.tsx with generateStaticParams, and a /templates hub page in the footer. New example workflow in the repo → new page on deploy. That is the machine. Index gate: non-boilerplate description, graph renders, and a sample output. Below the bar, the page builds with noindex until the gap is filled. Engine 2 — Seeded showcase (/showcase/*): the UGC engine, without waiting for users The direct clone of /photos/* (7,428 pages, the biggest lever in the teardown), seeded the way Photo AI seeded it: generate the content ourselves. Every batch run of a workflow becomes a page. The pipeline. The CLI already does the hard part: nodetool workflows run <file> --params and nodetool generate <provider> <model> <prompt> produce outputs headlessly, and the debug harness captures per-run metadata. A seeding script iterates templates × models × a curated prompt list, writes each output plus a manifest (prompt, model, provider, workflow slug, duration, params) to the asset store, and emits one showcaseEntries row per run. Marginal cost per page is inference, not writing. Page anatomy (mirrors Photo AI’s /photos/* mechanics precisely): URL slug is the prompt, truncated and hyphenated, with a short id appended: /showcase/neon-jellyfish-drifting-through-a-flooded-subway-station-a7f3k2. The output is the hero — full-width video or image, playing on load. Title and meta description are the prompt text; H1 is the prompt plus the model name. Below the output: model chip (→ its Engine 3 page), workflow chip (→ its Engine 1 page), provider, generation params. This block is what makes the page more than a bare asset. ~9 related showcase links (shared model, shared workflow, shared tag) — the equity mesh. “Remix this” CTA: opens the workflow in NodeTool Cloud with the prompt pre-filled. JSON-LD: ImageObject/VideoObject + ItemList, matching Photo AI’s photo-page stack. Volume math: 37 templates × 3–4 models × 5–10 prompts is 500–1,500 pages from one weekend of compute, before any real user publishes anything. Start smaller: ~200 pages across the video and image templates, watch indexation for 4–6 weeks, then scale. Provenance rules (the adaptation of “fake UGC” that doesn’t blow up): outputs must be real generations from the named model and workflow — never stock, never misattributed to another model. Pages carry no user identity and no review markup; they are a showcase, not testimonials. Dedupe near-identical prompts before publishing (same model + cosine-similar prompt → one page, others canonical to it). Engine 2b — real publish loop, later. Once Cloud has a Publish action, user-published workflows and runs enter the same /showcase/* and /w/* templates behind the same index gate (description ≥ 140 chars, ≥ 5 nodes or a real output, no near-duplicate already indexed, noindex from birth until the gate passes). The seeded corpus is what makes the section already rank by the time real UGC arrives. Engine 3 — Model pages (/models/*) People search the model, not the tool: “seedance”, “veo 3”, “kling ai”, “sora 2”, “gpt image”, “seedream”, “wan 2.5”, “flux”. Every one of these is runnable in NodeTool today through multiple providers (verified in packages/*-nodes/src: Seedance via fal/kie/replicate/atlascloud, Veo via fal/kie/replicate/together, and so on). The honest angle writes itself: one canvas runs them all, through whichever provider is cheapest, on your own API keys. /models/<slug> (“Seedance”, “Veo 3”, “Kling 2.6”, …): Hero: 2–3 real outputs from Engine 2 (the showcase supplies every model page’s proof). What the model does, in NodeTool’s words — capability, resolution/duration limits, what it’s best at. This slice is hand-written per model; it’s the intent-match that node-docs pages lacked. Provider table: which node packages expose it, BYOK pricing note per provider. This is the differentiator no single-provider page has, and it’s generated from the package manifests. A template that uses the model (→ Engine 1), embedded graph and all. Prompt examples: 5–10 showcase items for this model (→ Engine 2), each linked. FAQ: “is Seedance free”, “Seedance vs Kling, which is better” (→ Engine 3b), “can I run it locally”. FAQPage markup. Title: “Seedance in NodeTool — run it in a visual AI workflow (2026)”. Engine 3b — model-vs-model comparisons (/models/veo-3-vs-kling-2-6 or similar). The highest-upside new dimension found in this revision. “veo vs kling”, “sora vs veo 3”, “seedance vs hailuo” are real queries with commercial intent, the SERPs are Reddit threads and listicles, and NodeTool can do the one thing they can’t: run the identical prompt through both models and embed both outputs side by side, generated by the same seeding pipeline as Engine 2. Feature table (resolution, duration, audio, price per clip via each provider), 3–4 same-prompt pairs, an honest “pick X when…” verdict, FAQ, and a CTA into the workflow that produced the comparison — which the reader can run with their own prompt. That last link is a page mechanic no static listicle can copy. Pairs are data rows too: ~10 video models → ship the ~15 pairs people actually search, not all 45. Freshness dimension: model releases are page events. When fal/kie add a new model (they ship within days of release), a NodeTool model page + a same-day showcase batch can rank on “how to try " while the query spikes and before the listicles consolidate. Keep a standing runbook: new model in a node package → entry in `modelEntries.ts` → seed 10 showcase runs → page live. Refresh version numbers in titles on each release (the Photo AI year-in-title trick, applied to model versions). Engine 4 — Keyword landing pages One master template, a swappable hero + proof slice, a shared body. /use-cases/* and useCaseEntries.ts are the half-built version at 3 pages. Keyword axes, each a data array feeding the same template: Task keywords from shipped templates: “AI product video generator”, “AI movie poster maker”, “chat with PDF open source”. 34 unshipped templates ≈ 34 pages. Task hubs (“image to video”, “lip sync AI”, “AI video upscaler”, “text to music”): one hub per capability listing the models that do it (→ Engine 3), the templates that wire it (→ Engine 1), and showcase output (→ Engine 2). These sit one level above model pages and catch the searcher who hasn’t picked a model yet. Persona keywords: four exist (/creatives, /agents, /developers, /marketing); add local-first/privacy and researcher/knowledge-worker (SEO Strategy §3 names both as uncovered). Trend/aesthetic keywords (Photo AI’s /old-money): weakest fit; skip until the axes above are exhausted. Do not copy the 15,000-word / 59-question body: Photo AI carries that on domain authority NodeTool doesn’t have. Target ~2,500 words with the keyword-specific slice ≥ 30%. Title formula: front-load the keyword, add the concrete differentiator, short suffix — “AI Product Video Generator — Free, Open Source, Your API Keys | NodeTool”. Numbers where honest; skip the emoji. Engine 5 — Product comparison and alternatives mesh Six /vs/* pages shipped 2026-07-01/02 with the honest-concession pattern (SEO Strategy §4.1). Remaining work, per the teardown: /alternatives/<competitor> twins — “comfyui alternative” and “nodetool vs comfyui” are different queries with different SERPs. One template: the competitor’s real limitation, a 4–6 tool list (NodeTool first, honestly framed), feature table, FAQ, CTA. Year in the title, computed at build: “NodeTool vs Langflow: Which Is Better? (2026)”. In-content cross-link block (“Compare NodeTool with other tools”) on every page, beyond the existing footer column — this is what makes the cluster rank as a unit. Wide net (~20 competitors × 2 page types): the shipped six plus Flora, Krea, Freepik Spaces, Griptape Nodes, Reflet.ai (creative canvases); Lindy, Gumloop, Activepieces, Zapier AI, Make AI (automation); LM Studio, Jan, Open WebUI (local-first). Factory-ize the hand-built ~250-line pages before scaling past ten. JSON-LD: BreadcrumbList + FAQPage (Photo AI deliberately uses a different stack here than on landing pages). Engine 6 — FAQ, ideas, and glossary hubs /faq/*: seed from questions with evidence — the GSC export’s conversational queries (“which is free”, “are any of these open source?”), docs/troubleshooting.md, provider-setup friction, and the FAQ blocks already written for /vs/*. Each question is a standalone page plus a row in the shared FAQ module that Engines 3–5 render, so every answer exists twice: standalone for question SERPs, embedded with FAQPage markup for snippets and AI-answer surfaces (§0.8 shows AI assistants are a live acquisition channel). /ideas/*: category roundups generated from Engine 1 metadata (“12 AI video workflow ideas”), each linking every template in its category. New template pages get their first internal links here on day one. Glossary: docs/glossary.md already exists; per-term pages (“what is image-to-video”, “what is a node-based editor”) are near-free definitional surface that AI assistants quote. Other dimensions considered, ranked Evaluated for this revision; the first two graduated into engines above, the rest are queued: Model-vs-model comparisons → Engine 3b. Highest upside: real query volume, weak SERPs, and a same-prompt side-by-side only NodeTool can generate. Task hubs → Engine 4 axis 2. “Ollama frontend” positioning page. “ollama gui”, “ollama frontend”, “ollama web ui” carry real volume; NodeTool genuinely is one (Ollama provider in packages/runtime). One hand-built page (/ollama), same honest-table pattern as /vs/*, linking the local-first persona page. Quick win, do alongside Engine 5. Prompt-library pages (“seedance prompts”, “veo 3 prompt examples”): curated collections of Engine 2 showcase items per model. Ship as /models/<slug>/prompts once a model has 20+ showcase items — a view over existing data, not a new engine. New-model release notes (“Wan 2.6 is out — how to try it”): blog posts tied to the Engine 3 freshness runbook. Needs the blog to exist (SEO Strategy §4.4). Integration pages (/integrations/supabase, Google Sheets, the Chrome extension): integration-nodes exists, but query volume for “ + AI workflow" is unproven — collect GSC evidence first. Free interactive mini-tools (workflow JSON viewer, prompt enhancer): strong link magnets, real engineering cost. Revisit after the engines ship. The technical layer (all engines) Page type JSON-LD Related-links block Title pattern /templates/* HowTo + ItemList 8–9 related templates <Name> — free AI workflow template /showcase/*, /w/* ImageObject/VideoObject + ItemList ~9 related showcase items the prompt, verbatim /models/* SoftwareApplication + FAQPage sibling models + its prompt/showcase items <Model> in NodeTool — run it in a visual AI workflow (<year>) /models/*-vs-* BreadcrumbList + FAQPage all sibling model comparisons <A> vs <B>: same prompt, side by side (<year>) Landing (tasks, personas) Product + FAQPage templates grid + sibling landings <Keyword> — Free, Open Source \| NodeTool /vs/*, /alternatives/* BreadcrumbList + FAQPage all sibling comparisons NodeTool vs <X>: Which Is Better? (<year>) /faq/* QAPage related FAQs + one money page the question, verbatim /ideas/* ItemList every template in category <N> <category> workflow ideas (<year>) Sitemap becomes derived, not maintained. marketing/src/app/sitemap.ts is a hand-ordered array of 21 entries; at 1,000+ pages that breaks. Each engine’s data module exports its entries; sitemap.ts concatenates them. Standing rules stay: priority tiers, and no route ships without a sitemap entry, an opengraph-image.tsx, and a row in marketing/tests/e2e/smoke.spec.ts (which should walk the same data modules). Internal-link floor. Photo AI runs ~400 internal links per page; NodeTool needs the invariant, not the number: every page reachable from a hub, every page linking onward to 8+ siblings, no orphans (the /vs/* pages spent their first day orphaned — SEO Strategy §1). Canonicals self-referencing everywhere; near-duplicate showcase pages canonical to the first-published copy. llms.txt regenerates from the same data modules, hubs first. Build sequence Revised: the showcase pipeline moves early because Engines 1, 3, and 4 all lead with its outputs. The factory + Engine 1 (template pages, ~35). Data modules, graph-render route, /templates hub, derived-sitemap refactor. Everything else reuses this plumbing. Engine 2 seeding pipeline + first ~200 showcase pages. CLI batch runner, asset store, showcaseEntries generation, the /showcase/* route. Gate scaling on indexation data. Engine 3 model pages for the searched models already in the node packages (Seedance, Veo, Kling, Sora, Hailuo, Wan, Seedream, FLUX, GPT-Image, Imagen, ~15 pages) + the top ~15 model-vs-model pairs (Engine 3b), heroes supplied by step 2. Engine 5 completion (~30 pages: /alternatives/*, year helper, mesh block, wide net) plus the /ollama page. Engine 4 task keywords and task hubs (~40–60 pages), then the two missing personas. Engine 6 FAQ/ideas/glossary (~40 pages) + the schema table everywhere. Engine 2b real publish loop once Cloud has the product surface. Steps 1–6 take nodetool.ai from 21 indexed pages to ~600–800, every one output-first and from a fixed template, before any real user publishes anything. Execution: NodeTool workflows run the machine The work splits cleanly into code that ships once and content that regenerates forever. The second half is exactly what NodeTool is for, so the SEO machine’s production layer is NodeTool workflows run headlessly — which is also a marketing story in itself (the site’s content is made by the product it sells, and the seeder workflows ship as /templates/* pages like any other). Layer 1 — the factory (TypeScript, one-time, ~3 PRs). Not workflows; this is Next.js build plumbing in marketing/: Data modules + /templates/[slug] route with the JSON→flow-components graph renderer, the /templates hub, derived sitemap, and the smoke-test walker. Everything later reuses this. /models/[slug] and /models/[a]-vs-[b] routes reading modelEntries.ts. /showcase/[slug] route reading the generated showcase manifest. Layer 2 — content production (NodeTool workflows in marketing/seo/workflows/, DSL .ts files, run via nodetool run <file> --params). Four workflows: Showcase Seeder — inputs: template slug, model list, prompt count. An agent or ListGenerator writes prompt variants in the template’s domain, each fans out through TextToImage/ImageToVideo per model, outputs land in marketing/seo/out/<batch>/ with one manifest row per run (prompt, model, provider, params, file). The same pattern as the shipped Image to Video Animation and Movie Posters examples, parameterized. Model Duel — one prompt, two model parameters, identical generation nodes; emits the paired outputs plus an agent-drafted observation block for the /models/a-vs-b page. Copy Writer — the shipped SEO Content Engine template (strategist agent → briefs → full Markdown per brief), retargeted: given a model id and its provider manifest, draft the hand-written slice of its model page (capability blurb, FAQ answers, title/meta) as Markdown for review. Drafts only — a human edits before merge. Model Watcher — a plain CI script, not a workflow (reliability beats dogfooding here): weekly generate <provider> --list-models --json diff against modelEntries.ts; a new model opens a PR with the stub entry and kicks a Seeder batch for it. Layer 3 — glue (GitHub Actions). One seo-seed.yml with cron + manual dispatch: runs the seeder/duel workflows with provider keys from repo secrets and an explicit per-batch spend cap in --params, then an ingest script that converts manifests into showcaseEntries.generated.ts, moves assets to the CDN path, and opens a PR. The PR is the quality gate: a human eyeballs every batch’s outputs before they go live — this is where the playbook’s index gate is enforced, not in code review of the pipeline. Merge deploys the site. What deliberately stays out of workflows: page rendering, sitemap, canonicals — all derived at Next.js build time from the checked-in data. The workflows produce data and assets; the build consumes them. That boundary keeps a bad generation batch revertable with git revert. Guardrails The node-docs lesson is the null hypothesis. Any engine can reproduce §0.4 (thousands of impressions, no clicks) if pages chase queries whose intent NodeTool can’t satisfy. The gate for every new page type: name the query, say why that searcher wants a workflow tool, and check the unique slice answers it. Model pages pass because “run X cheaper via any provider” is what a model searcher wants next; node-reference pages failed because “pdfplumber docs” searchers didn’t. Seeded content stays honest. Real generations, correctly attributed to model and workflow; no invented users, no review markup, no passing off one model’s output as another’s. The showcase label is load-bearing — it’s what distinguishes this from the spam Google actually targets. Watch site-level dilution, not just per-page failure. If a new engine’s pages sit indexed-but-not-ranking after a quarter, prune or noindex the tail rather than adding more. Photo AI’s thin tail is carried by unique images; NodeTool’s showcase pages have the same anchor, but template/model pages need their text to match intent too. Cap seeding spend. The showcase pipeline’s cost is inference; set a per-batch budget and prefer models NodeTool gets via cheap providers for the long tail, reserving expensive models (Veo, Sora) for pages that target their queries. Measure per engine. Segment GSC by URL prefix (/templates/, /showcase/, /models/, /vs/, /alternatives/, /faq/, /ideas/) plus conversions to /studio and /pricing. An engine with zero non-branded impressions after 8 weeks gets its keyword axis re-examined before its page count grows. Numbers in this file rot. 37 templates, 24 node packages, 21 sitemap entries were true on 2026-07-02. The factory design makes the counts irrelevant; the audit claims age worse.--- # SEO Strategy Source: https://docs.nodetool.ai/seo-strategy Research basis: an audit of marketing/ (the nodetool.ai Next.js site), docs/, the 61 example workflows in packages/base-nodes/nodetool/examples/, and the node packages in packages/, plus external research on competitor positioning and search behavior (sources linked inline). Written 2026-07-01, updated 2026-07-02 — re-audit the live site before acting on the numbers below if this file is more than a quarter old. Status: the movie-trailer sitemap fix shipped 2026-07-01. /vs/langflow and /vs/n8n (§4.1 items 1 and 3) and the footer “Compare” column (§1, orphaned-pages finding) shipped 2026-07-02. /marketing (§4.0), /vs/flowise, and /vs/dify (§4.1 items 2 and 5) shipped 2026-07-02. §0 (the Search Console audit) added 2026-07-02 — it re-prioritizes §7 with real query data. The programmatic build plan (template pages, the seeded showcase engine, model and model-vs-model pages, keyword landing matrix, comparison/alternatives mesh at scale, and FAQ/ideas hubs) lives in SEO_PROGRAMMATIC.md — it extends §4 from hand-built pages to data-driven page factories, adapted from a photoai.com teardown. Shipped from §0 on 2026-07-02: 79 redirect stubs under docs/redirects/ (§0.5, plus a fix to the three pre-existing stubs, whose trailing-slash redirect_to 404s on GitHub Pages); providers.md and installation.md rewrites (§0.6); comparisons.md cross-links to /vs/* (§0.6); /agents and /cloud retitles (§0.7); llms.txt on nodetool.ai and JSON-LD on /agents, /creatives, /marketing (§0.8). The Weavy → Figma Weave capture plan (§0.9) shipped 2026-07-16. 0.9 Weavy → Figma Weave capture plan (2026-07-16) Figma acquired Weavy in October 2025 and renamed it Figma Weave (weave.figma.com, separate billing and its own AI credits). The integration is deepening — Weave Workflows on Figma Community (April 2026), Weave tools inside Figma Design (Config 2026) — so search demand is splitting into two query sets, and the second one is growing: “weavy alternative” — the established term. Every ranking page now leads with the rebrand (Wireflow’s “Top 8 Weavy Alternatives (Weavy Is Now Figma Weave)”, Chase Jarvis’s “Best Weavy Alternatives for Creative Pros”, designtools.fyi). A /vs/weavy page that never mentions Figma reads stale to searchers and to the AI assistants in §0.8. “figma weave alternative” — the new term. Wireflow, Astorie, and Krea already run dedicated pages for it; NodeTool had zero surface. What shipped 2026-07-16: /vs/weavy and /alternatives/weavy refreshed — title, hero, bullets, and a new FAQ row (“What happened to Weavy?”) acknowledge the acquisition while keeping the BYOK/open-source argument. The slug and query target stay “weavy”. /vs/figma-weave and /alternatives/figma-weave added (competitorEntries.ts, isNew) — distinct copy angle from the weavy pages: ecosystem lock-in and post-acquisition ownership (“build on a canvas nobody can acquire”) rather than a re-tread of the credits argument, with an honest “when is Figma Weave the better pick?” FAQ and a free-tier concession row. Both pages are one product, so watch GSC for cannibalization between the weavy and figma-weave pages; if one set stops earning impressions, fold it into the other with a redirect. Keeping both mirrors how Wireflow serves the two query sets today. FAQ page /faq/open-source-figma-weave-alternative — targets the question form AI assistants ask (§0.8); JSON-LD comes with the FAQ template. Metadata: “Figma Weave alternative” keyword on / and /creatives; homepage JSON-LD and the homepage comparison card renamed; docs/comparisons.md updated; llms.txt regenerated. Distribution follow-ups (the §6 play, with named targets — these pages rank today and take listings): request inclusion in Wireflow’s “Top 8 Weavy Alternatives”, Chase Jarvis’s roundup, and designtools.fyi’s Figma Weave page. The migration audience — Weavy users re-evaluating after the acquisition — is the highest-intent segment this site can address, and “your workflows are files you own” is the argument acquisitions make for us. Watch queries (GSC, weekly, alongside the §0.2 set): “weavy alternative”, “figma weave alternative”, “figma weave pricing”, “figma weave vs”, “open source figma weave”. 0. What Search Console actually says (audit 2026-07-02) Everything below §0 was written from a content audit. This section is the first pass with real Google Search Console data (export pulled 2026-07-02, trailing window): 939 clicks / 48.2k impressions across 474 pages; 504 clicks / 11.1k impressions across the visible query set; 933 clicks / 40.1k impressions by country. Eight findings, each with the action it forces: Traffic is almost entirely branded. “nodetool”-variant queries deliver 389 of 504 visible query clicks (77%) at position ~1.7 and 23.6% CTR. Non-branded queries: 115 clicks on 9.4k impressions — 1.23% CTR. The doc’s premise (the site lacks non-branded surface) is confirmed, not just plausible. Exactly one non-branded cluster ranks, and it’s stuck at position 7–9. “ai node” (3,171 impressions, pos 7.4), “node ai” (1,093, pos 8.7), “node based ai” (797, pos 8.7), “ai node editor” (197, pos 5.8) and ~20 variants — all landing on the homepage. At position 7–9 CTR is ~1%; the same impressions at top-3 would be 10–25×. This cluster is also intent-ambiguous (Node.js AI? ComfyUI nodes?), which is why the better-intent comparison and use-case queries in §3–§4 matter more: the export shows zero impressions for “comfyui alternative”, “weavy alternative”, “langflow”, or “n8n alternative” — the /vs/* pages shipped 07-01/07-02 and have no data yet. Watch them weekly. The US is the weak market, and it’s the biggest. 17.3k of 40.1k impressions are US, but average US position is 12.5 vs 6.4–7.3 in every other top-10 country (US CTR 0.59% vs 3–5% elsewhere). That gap is authority/competition, not on-page tags — it upgrades §6 (distribution: directories, awesome-lists, Show HN, roundup outreach) from “ongoing” to a priority. Docs earn 39% of impressions and 0.4% CTR. docs.nodetool.ai: 75 clicks / 18.9k impressions. The 375 node-reference URLs in the export took 7.8k impressions for 12 clicks — the queries are “official docs” lookups for third-party libraries (pdfplumber, stable diffusion img2img, mlx_whisper) that will never click a NodeTool page. Do not invest in ranking node pages further; fix their meta (§4.3) and move on. The node-docs restructure orphaned 234 of 460 docs URLs in the export (~6.9k impressions) with no redirects. nodes/comfy/*, nodes/mlx/*, nodes/huggingface/image_to_image/*, nodes/lib/pdfplumber/*, and runpod-deployment all 404 now. The mechanism to fix it already exists (_layouts/redirect.html + redirect_to front matter, used by desktop-app.md, api.md, packs.md). Batch-generate redirect stubs where a successor page exists (lib/pdfplumber/* → lib/pdf/*, huggingface/image_to_image/* → huggingface/imagetoimage, runpod-deployment → deployment); let genuinely removed packs 404. Three docs pages carry real demand and rank badly. providers.html is the second-highest impression URL on either domain (3,181 impressions, position 20, 0.09% CTR); installation.html (1,904, pos 15.1); comparisons.html (716, pos 21 — it overlaps the new /vs/* pages and should link to them prominently). These three are the only docs pages worth active optimization: restructure providers.html with per-provider H2s and a capability table; expand installation.html with per-OS sections and troubleshooting. Marketing pages at position 5–9 get near-zero clicks — /cloud (716 impressions, pos 4.9, 0 clicks), /agents (664, pos 5.1, 0), /developers (1,570, pos 6.1, 0.51%), /creatives (1,034, pos 9.1, 0.48%), /studio (942, pos 6.8, 0.32%). Most of these impressions are likely sitelinks under branded queries, where the homepage absorbs the click — so title rewrites alone won’t fix CTR. The durable fix is giving each page a distinct non-branded query target so it earns its own impressions: /agents → “AI agent workflow builder” (its current title, “Agents for creative work”, targets nothing anyone searches), /cloud → “browser AI workflow builder / no-install”, /developers already targets “AI workflow SDK” reasonably. AI assistants are a real, visible acquisition channel. Dozens of queries in the export are LLM-generated: persona-length prompts (“shortlist node-based ai editors that let design teams…”), site:/after: operators, and conversational follow-ups (“which is free”, “are any of these open source?”, “you sure?”). NodeTool is being evaluated inside AI-assisted research sessions. Actions: add llms.txt to nodetool.ai (docs already has one); keep license/pricing/free-tier facts as plain crawlable text on /pricing and every /vs/* page; extend the existing JSON-LD (JsonLd.tsx, currently on a handful of pages) to all indexed routes. One stray demand signal: “real time agent api” (132 impressions, pos 36) plus the now-deleted nodes/openai/agents/realtimeagent.html (192 impressions) — there is search demand for a realtime agent guide; a proper docs page would inherit it. 1. Where nodetool.ai stands today The site (marketing/src/app/) has 13 indexed pages: /, /studio, /cloud, /pricing, /agents, /creatives, /developers, two comparison pages (/vs/comfyui, /vs/weavy), two use-case pages (/use-cases/product-video, /use-cases/movie-poster), and three legal pages. Five gaps stand out: /use-cases/movie-trailer exists but is missing from sitemap.ts (marketing/src/app/sitemap.ts). It has a page folder and an entry in useCaseEntries.ts but never reaches search engines. Fixed 2026-07-01. The /vs/* comparison pages are orphaned. Nothing on the site links to them — they’re reachable only through sitemap.xml. Pages with zero internal links get crawled less, rank worse, and pass no authority. Fixed 2026-07-02 with a “Compare” column in the shared footer (marketing/src/components/SiteFooter.tsx), which gives every comparison page a sitewide internal link. Keep this rule for every future page type: nothing ships without at least one internal link from an existing page. No blog. Every competitor identified in research below (Dify, n8n, Flowise, Langflow, Lindy, Gumloop, Vellum, StackAI, Wireflow) runs a blog and ranks on “X vs Y” and “best tools for Z” queries, which is exactly the traffic NodeTool needs and currently cedes to them. 3 use-case pages for 61 shipped example workflows. The useCaseEntries.ts pattern (a single data array plus a matching page folder) is already built to scale — it’s just underused by a factor of 20. No /marketing segment page. Fixed 2026-07-02. /marketing ships, featuring the Product Video Generator, in the nav, footer, and sitemap alongside Creatives/Agents/Developers. See §4.0. The existing meta setup is solid: layout.tsx already ships title/description/OG/Twitter tags and a keyword list (creative AI workspace, BYOK AI canvas, ComfyUI alternative, Weavy alternative, node-based AI canvas). robots.txt and sitemap.ts both exist. The problem is page count and content depth, not technical hygiene. 2. Competitive landscape Three groups of tools currently absorb the search intent NodeTool could capture: LLM/agent workflow builders — Dify, Flowise, Langflow, n8n. StackAI’s comparison names Dify “the best starting point for most teams in 2026” on debugging and knowledge-base features; Langflow is positioned for teams that need custom Python nodes and LangGraph; Flowise for the fastest path to a RAG chatbot; n8n where orchestration (retries, branches, schedules) is the hard part (Runchat comparison). None of these natively render image, video, or audio — they’re text/LLM-first, which is NodeTool’s opening. Node-based creative canvases — ComfyUI (already covered at /vs/comfyui), Weavy (covered at /vs/weavy and, post-rebrand, /vs/figma-weave — see §0.9; acquisition first flagged in Chase Jarvis’s review), plus Flora, Freepik Spaces, Krea, Layer AI, Griptape Nodes, and Reflet.ai — all covered in a single “2026: The Year of the Node-Based Editor” roundup and multiple “Weavy alternatives” posts (Wireflow). Most are browser-only, credit-metered SaaS with no local execution and no BYOK — the same gap NodeTool already argues against Weavy, reusable against this whole category. Local-first AI tooling — Ollama, LM Studio, Jan, Open WebUI cover local chat; none combine local inference with cloud generation or a visual workflow canvas (overchat.ai roundup, needaitool.com roundup). Net position: NodeTool is the only tool showing up across all three searches — LLM/agent builders don’t do media, creative canvases don’t do local models or agents, local-first tools don’t do workflows. That’s the argument every comparison page and use-case page should make concrete, not assert abstractly. 3. Audience segments and what they search Segment What they search Where they currently land instead Creatives (filmmakers, designers) “text to video pipeline”, “ComfyUI alternative no code”, “AI movie poster generator” Runway, Krea, Melies, Weavy alternatives posts Marketing teams (campaigns, growth, social) “product video AI tool”, “AI content calendar generator”, “brand asset generator AI”, “AI ad video generator” Lindy, Gumloop, Runway, Weavy alternatives posts Developers / ML engineers “open source AI agent framework”, “RAG pipeline no code”, “TypeScript AI workflow SDK”, “custom LLM node builder” Langflow, Flowise, LlamaIndex RAGArch Automation / ops teams “n8n alternative for AI agents”, “no-code AI workflow local”, “email triage AI automation” n8n, Activepieces, Gumloop Privacy-conscious / local-first users “run AI models locally desktop app”, “local LLM privacy no cloud”, “self-hosted AI agent” Jan, LM Studio, Ollama-based roundups Researchers / knowledge workers “chat with PDF tool”, “summarize research papers AI”, “document RAG open source” Kapa.ai, Prisme.ai The site already has landing pages for creatives, developers, and agents (automation). It has none targeting marketing teams specifically, even though the homepage copy names them by title (see §4.0) — nor the local-first/privacy segment (this is argued piecemeal on /studio and in comparisons, never as its own page), nor the researcher/knowledge-worker segment (despite shipping 5 document-processing example workflows: Chat with Docs, Index PDFs, Summarize Paper, Fetch Papers, Meeting Transcript Summarizer). 4. Page-type playbooks 4.0 Segment landing page — /marketing Shipped 2026-07-02. /creatives, /agents, /developers, and now /marketing are each a persona-framed rewrite of the same product, built to rank on that persona’s job-title searches and to give paid/social traffic a landing page that speaks their language instead of the generic homepage. /marketing (marketing/src/app/marketing/page.tsx) follows the /creatives pattern — headline, “why marketing teams choose NodeTool” section, a lead use-case feature, community/CTA — and is in sitemap.ts at priority 0.8/monthly, the main nav, and the footer “Solutions” column: Lead use case: Product Video Generator (use-cases/product-video, already category: "Marketing" in useCaseEntries.ts), featured with its own section and real assets. Supporting use cases still to build (§4.2 backlog): Social Media Calendar Filler, Brand Asset Generator, Cold Outreach Co-Pilot, YouTube Thumbnail Pipeline. /marketing lists these as “coming soon” teaser cards (no dead links — they get real hrefs once useCaseEntries.ts gains matching entries and page folders with real generated assets). Target keywords: “AI product video generator”, “AI content calendar tool”, “brand asset generator AI”, “AI ad video tool open source” (§3) — in the page’s metadata/keywords. Differentiation angle: same BYOK/no-markup argument as /creatives, reframed around output volume and cost-per-asset at campaign scale rather than single-artifact craft. 4.1 Comparison pages (/vs/*) Pattern already proven at marketing/src/app/vs/comfyui/page.tsx and vs/weavy/page.tsx (feature-table format, ~250 lines each, hand-built per page). Expand from 2 to a target list based on what people are already comparing: /vs/langflow — pull in the “no media generation” gap Shipped 2026-07-02. Angle used: both cover agents/RAG; only NodeTool generates media natively (Langflow is MIT, has a macOS/Windows desktop app — the honest table concedes both). /vs/flowise — same gap, plus BYOK vs. hosted credits Shipped 2026-07-02. Angle used: Flowise is the fastest path to a LangChain RAG chatbot; NodeTool covers the same agent/RAG ground and adds native image, video, and music generation plus editing tools on the same canvas. /vs/n8n — orchestration-only vs. orchestration + native generation Shipped 2026-07-02. Angle used: “workflows that create, not just connect”, plus AGPL-3.0 (open source) vs. Sustainable Use License (fair-code, commercially restricted) — a real differentiator n8n’s own docs confirm. The FAQ answers “when should I pick n8n instead?” honestly (400+ connectors, schedules/retries), which is what makes the rest of the page credible. /vs/flora (or a combined “AI canvas alternatives” page covering Flora, Freepik Spaces, Krea) /vs/dify — RAG/agent platform, no creative media Shipped 2026-07-02. Angle used: Dify is a strong text-first LLM app platform (prompt orchestration, knowledge bases, agent debugging); NodeTool starts from the same agent/RAG ground and adds native media generation and editing tools. License claim about Dify’s modified Apache 2.0 terms is hedged (“check Dify’s own license file for current terms”) rather than asserted precisely, since Dify’s commercial-use conditions change between releases. /vs/lmstudio or /vs/jan — local chat only vs. local + cloud workflow canvas Each page keeps the existing format: one comparison table, one differentiation section, one CTA. Do not invent a new template per competitor — the value here is coverage, not novelty. 4.2 Use-case pages (/use-cases/*) The useCaseEntries.ts array is the scaling mechanism: add an entry, build the matching page folder. 61 example workflows already exist in packages/base-nodes/nodetool/examples/nodetool-base/ with titles, descriptions, and (per the internal audit) natural category groupings. Prioritize by search-intent match, not by what’s easiest to build: Ship first (clear existing search terms, per §3): Research Paper Summarizer, Chat with Docs (→ “chat with PDF”), Meeting Transcript Summarizer, Music Video Visualizer Ship second: Hacker News Agent / YouTube Research Agent (agent-mode examples double as /agents landing-page proof) Marketing segment (feature on /marketing, see §4.0): Cold Outreach Co-Pilot, Social Media Calendar Filler, Brand Asset Generator, YouTube Thumbnail Pipeline Fix immediately: add /use-cases/movie-trailer to sitemap.ts (see §1) At 61 candidates, treat this as a backlog, not a one-time push — 3-5 new use-case pages per month sustains a steady stream of long-tail landing pages without a content team. 4.3 Node / provider pages docs/nodes/<provider>/ and docs/developer/providers/<provider>.md already exist for FAL, KIE, Gemini, ElevenLabs, Together, Replicate, Ollama, HuggingFace, Topaz, AtlasCloud, and more (20+ providers). These are developer docs, not landing pages, but each one is a near-zero-cost target for provider-name search traffic (“FAL nodes for NodeTool”, “use Kling in NodeTool”) once they carry proper titles/descriptions and get linked from /developers. No new pages needed here — just meta tags on what already exists (see §5). 4.4 Blog (net new) No blog exists. This is the biggest structural gap relative to every competitor in §2. Recommended scope, not a general-purpose blog: Comparison/roundup posts that NodeTool can legitimately win a mention in: “best ComfyUI alternatives 2026”, “open source Weavy alternatives”, “local-first AI tools 2026” — these are the exact post titles already ranking for competitors (§2 sources). Writing NodeTool’s own version, plus getting listed in the external ones (see §6), covers both sides. Cookbook walkthroughs built directly from docs/cookbook/core-concepts.md and the 15+ recipes in docs/cookbook.md — these already exist as docs; a blog post is a rewrite with a narrative frame and screenshots, not new research. Technical posts for the developer segment: actor-model workflow execution, the Python bridge (PythonStdioBridge), the planning agent system (packages/agents/) — these map to real architecture (docs/architecture.md, docs/AGENTS.md) and target “how do AI agent frameworks work” / “TypeScript LLM orchestration” searches from engineers evaluating tools. 5. Technical SEO checklist Add /use-cases/movie-trailer to marketing/src/app/sitemap.ts (shipped 2026-07-01) Internal-link the /vs/* pages — footer “Compare” column in SiteFooter.tsx links all four comparison pages sitewide (shipped 2026-07-02); every future page needs at least one internal link before it ships Build /marketing (§4.0) and add it to sitemap.ts at priority 0.8 / monthly — same tier as /agents, /creatives, /developers — plus main nav (shipped 2026-07-02) Confirm every new page under /vs/, /use-cases/, /marketing, and any future /blog/ gets an opengraph-image.tsx (all six vs/* pages and /marketing do — keep the pattern) Add priority/changeFrequency entries to sitemap.ts for every new page as it ships — don’t let the list drift out of sync with app/ folders again (/marketing, /vs/flowise, /vs/dify added 2026-07-02) Add every new indexed route to marketing/tests/e2e/smoke.spec.ts — it guards title, single-h1, and render status per route (/marketing, /vs/flowise, /vs/dify added 2026-07-02) Verify docs/nodes/<provider>/index.md pages carry unique titles and descriptions (currently generated docs; check for duplicate/boilerplate meta across providers) Internal linking: /agents, /creatives, /developers should each link to the use-case pages relevant to their segment (per §3’s mapping) — /marketing now does this for its lead use case, but the use-case showcase otherwise only appears on the homepage and /creatives 6. Distribution (off-site) Content only ranks if something points at it: “Alternatives to X” roundups (Wireflow, Lindy, Vellum, Gumloop, StackAI, and similar sites identified in §2 already publish and rank these lists) — get NodeTool listed via outreach when a new comparison or use-case page ships. This is lower effort than link-building from scratch and targets people already close to a decision. Directories already listing NodeTool: futuretools.io, theaidb.com, SourceForge (confirmed via search in this audit). Keep listings current with each release; check for others in the same category (There’s An AI For That, AI Tools Directory sites). Adjacent projects to note for competitive tracking, not for outreach: alvinreal/awesome-opensource-ai and light-and-ray/awesome-alternative-uis-for-comfyui on GitHub — both are community-curated lists where a PR adding NodeTool is a legitimate, low-effort listing (not unsolicited outreach). GitHub topics and README SEO: confirm the repo carries relevant GitHub topics (ai-workflow, comfyui-alternative, llm-agents, node-based, local-first) — these surface in GitHub’s own search and get scraped into “awesome-*” lists. Reddit / Hacker News: no existing discussion threads found in this audit (searched directly). This is upside, not a gap to fix — a Show HN or r/LocalLLaMA / r/comfyui post timed to a specific release (e.g., a new local-model integration) reaches an audience already primed for exactly this positioning. 7. Prioritized roadmap Re-prioritized 2026-07-02 against the Search Console audit in §0. Immediate (hours): fix the movie-trailer sitemap omission (§1, §5) done 2026-07-01; internal-link the orphaned /vs/* pages (§1, §5) done 2026-07-02; batch-generate redirect stubs for the moved node-docs URLs (§0.5) done 2026-07-02 (79 stubs); link comparisons.html to the /vs/* pages (§0.6) done 2026-07-02; add llms.txt to nodetool.ai (§0.8) done 2026-07-02. Phase 1 (this month): rework providers.html and installation.html — the two highest-demand docs pages, both ranking on page 2 (§0.6) done 2026-07-02; give /agents and /cloud distinct non-branded query targets (§0.7) done 2026-07-02; build /marketing (§4.0) done 2026-07-02 — its four supporting use-case pages (Cold Outreach Co-Pilot, Social Media Calendar Filler, Brand Asset Generator, YouTube Thumbnail Pipeline) still need real generated assets before they can ship; ship the remaining “ship first” use-case pages (§4.2); add /vs/langflow and /vs/n8n done 2026-07-02, add /vs/flowise and /vs/dify done 2026-07-02 — next comparison target is /vs/flora (or a combined AI-canvas-alternatives page, §4.1 item 4). Phase 2 (next month): distribution (§6) — upgraded from “ongoing” because the US position gap (§0.3) is an authority problem: submit to the two GitHub awesome-lists, refresh directory listings, and time a Show HN / r/LocalLLaMA post to a release; stand up the blog with 3 posts — one comparison/roundup post, one cookbook walkthrough, one technical post (§4.4). Ongoing: 3-5 use-case pages/month from the remaining 61-workflow backlog; one comparison page per new credible competitor; internal-link every new page from its matching segment landing page (§5); check the /vs/* pages’ query impressions weekly until they register (§0.2). 8. Measurement Baseline (GSC export, trailing window ending 2026-07-02): 939 clicks / 48.2k impressions total; non-branded 115 clicks / 9.4k impressions / 1.23% CTR; US position 12.5; docs 75 clicks / 18.9k impressions. The numbers to move: non-branded clicks (proves §4’s content plan), US average position (proves §6’s distribution plan), and impressions on /vs/* and /use-cases/* (currently zero — proves the pages are entering the index at all). Track per page: organic impressions/clicks (Search Console), and which comparison/use-case pages convert to /studio or /pricing visits (the two highest-priority pages in the existing sitemap). Re-run the competitive scan in §2 quarterly — this is a fast-moving category and new entrants (Flora, Reflet.ai, both surfaced only in this audit’s research) can appear within months.--- # Sketch Editor Source: https://docs.nodetool.ai/sketch-editor.html Draw, paint, mask, and generate AI imagery on a layered canvas — without leaving your workflow. Quick Access: Click + New in the workspace tab bar and choose New image for a blank canvas, or open a sketch from the left Sketches panel. Overview The Sketch Editor is a layered raster editor with a built-in AI generation pipeline. It combines a familiar painting toolset (brush, eraser, fill, shapes, transform, selection) with the ability to generate image content directly onto a layer — bound either to a model or to one of your own workflows. Features: Layer stack with blend modes, per-layer opacity, lock, and visibility Painting tools: brush, pencil, eraser, fill, gradient, blur, clone stamp, color adjust Selection tools: rectangular marquee and AI-assisted magic wand / segmentation Shape tools: line, rectangle, ellipse, arrow Crop and free transform (scale, rotate, skew, perspective warp) Pen-pressure support, stroke stabilization, and drawing symmetry AI layers — generate a layer from a prompt or bind it to a workflow; regenerate when inputs change Unlimited undo/redo with a full history Exports a flattened image, a mask, and per-layer outputs back into your workflow Where this fits The Sketch Editor is one of NodeTool’s editing surfaces, and it sits between manual editing and workflow automation. It opens an image asset, lets you paint or generate layers by hand or bind a layer to an image workflow, then renders the result back into an asset. That asset is the same material every other surface reads — drop it back on the canvas, drag it onto a timeline, or expose the workflow behind it as a Mini-App. Every surface shares one asset store and the same model/provider system. See Key Concepts → How everything fits together for the full loop. Opening the Editor The Sketch Editor runs in three places, all backed by the same document so your work stays in sync. From the new tab button Click + New in the workspace tab bar (top of the editor). Choose New image to open a blank canvas in a new tab, or Open asset… to edit an existing image. The editor fills the tab. Use Save to image to render the composite back into the asset, then Done to return the tab to view mode. From the Sketches panel Open the Sketches panel in the left sidebar to browse documents you’ve created, grouped by date. Click one to open it, or use New Sketch to start a blank canvas. As a standalone page Every sketch has its own URL at /sketch/<documentId>. Opening a sketch in its own workspace tab gives you the full-screen editor for focused work. Changes autosave as you go. The Workspace Region What it does Toolbar (left) Pick a tool; switch foreground/background colors Tool options (top) Settings for the active tool — size, opacity, hardness, shape type, transform handles Canvas (center) Draw, paint, and composite; zoom and pan Layers panel (right) Add, reorder, blend, lock, hide, and group layers; mark a mask layer Inspector (right) Per-layer details — including AI generation controls for generated layers Press Tab to hide the panels for a clean, full-canvas view. Tools Select tools from the toolbar or by keyboard shortcut. Move — V Reposition the active layer or selection contents. Brush — B Paint freehand with the foreground color. Supports round, soft, airbrush, and spray brush styles, plus pen-pressure sensitivity. Size: [ decreases, ] increases Hardness: { decreases, } increases Opacity: press a digit 0–9 for a preset (5 = 50%, 0 = 100%) Pencil — P Paint hard-edged, aliased strokes. Eraser — E Erase pixels on the active layer. Shares size, hardness, and opacity controls with the brush. Fill — G Flood-fill contiguous regions with the foreground color, using an adjustable tolerance. Gradient — T Drag to draw a gradient between the foreground and background colors. Blur — Q Paint to soften pixels under the brush. Clone Stamp — S Alt-click to set a source point, then paint to copy detail from there — useful for retouching and removing blemishes. Color Adjust — J Paint hue, saturation, and brightness adjustments onto the layer. Eyedropper — I Click the canvas to sample a color into the foreground swatch. Crop — C Reframe the canvas. Drag the crop box, then press Enter to commit or Esc to cancel. Transform — F (or Ctrl/⌘ + T) Free-transform the active layer or selection — scale, rotate, skew, and perspective-warp with a handle box. Enter commits, Esc cancels, . resets the box to identity. Ctrl/⌘ + Shift + T repeats the last transform. Selection Tools Tool Shortcut Behavior Rectangular marquee M Drag a rectangular selection Magic wand W Click to select a region by color, with AI-assisted segmentation Once a selection is active, paint and edit operations are constrained to it. Ctrl/⌘ + D deselects. Shape Tools — U Draw vector-style shapes onto a layer. Shape Shortcut Line L Rectangle R Ellipse O Arrow A Hold Shift while dragging to constrain proportions (squares, circles, 45° lines). Layers The editor is layer-based. Stack layers, reorder them, toggle visibility, lock them, group them, and composite them with blend modes. Blend modes — choose how a layer combines with those beneath it (Normal, Multiply, Screen, Overlay, and more). In the Layers panel, ↑/↓ step through blend modes. Opacity — set per-layer transparency. Lock — protect a layer from edits. Mask layer — designate a layer as the document’s mask. It’s exported as a separate mask output for inpainting and compositing nodes. Layer via copy / cut — Ctrl/⌘ + J copies the current selection to a new layer; Ctrl/⌘ + Shift + J cuts it to a new layer. Clear layer — Delete or Backspace clears the active layer (or selection). Fill layer — Ctrl/⌘ + Backspace fills with the background color; Alt + Backspace fills with the foreground color. AI Generation What sets the Sketch Editor apart is that a layer can be generated, not just painted. Select a layer and open the Inspector to bind it to a generator: Prompt-based generation — pick a provider and model, write a prompt, and generate text-to-image, image-to-image, or inpaint results directly onto the layer. Workflow-bound generation — bind the layer to one of your own workflows and choose which output node feeds the layer. The layer becomes a live surface for any pipeline you’ve built. Each generation is tracked as a version on the layer, so you can compare results and roll back. NodeTool also tracks each layer’s inputs: if you change a prompt, parameter, or an upstream layer the generation depends on, the layer is flagged stale so you know it’s worth regenerating. Generation runs as a job over the WebSocket connection, with live progress shown on the layer. This makes the editor a natural place to sketch a rough composition, mask a region, and let a model fill it in — then keep painting on top. Colors The editor maintains a foreground and background color. X swaps the foreground and background colors. D resets them to the defaults (black / white). Ctrl/⌘ + I inverts the colors of the active layer. Symmetry & Pen Pressure Symmetry — mirror your strokes across horizontal, vertical, or radial axes (with configurable rays) for mandalas, patterns, and symmetric design. Enable it from the editor’s header controls. Pen pressure — when using a pressure-sensitive device, tune how pressure maps to brush size and opacity from the pen-pressure settings. Stroke assist — stabilize shaky strokes or snap lines to angles for cleaner linework. History & Undo The editor keeps a full history of your changes. Action Shortcut Undo Ctrl/⌘ + Z Redo Ctrl/⌘ + Shift + Z (or Ctrl/⌘ + Y) History persists while the editor is open. Large documents with extensive history use more memory. Zoom & Navigation Action Shortcut Zoom in + / = or scroll up Zoom out - or scroll down Reset zoom (fit) Ctrl/⌘ + 0 Zoom to 100% Ctrl/⌘ + 1 Pan Space + drag, or middle-click drag Toggle panels Tab Saving & Exporting Autosave — sketches save automatically as you work. Save to image — render the flattened composite back into the backing image asset, ready to use anywhere an image is. Workflow nodes — to pull a sketch into a graph, use the Render Sketch and Sketch Layers nodes, which flatten the document or expose its individual layers. Because the document is shared across the tab, the Sketches panel, and the standalone page, your edits stay in sync everywhere it’s open. Keyboard Shortcuts Tools Key Tool V Move B Brush P Pencil E Eraser G Fill T Gradient Q Blur S Clone stamp J Color adjust I Eyedropper M Rectangular marquee select W Magic wand select C Crop F Transform U Shape L Line R Rectangle O Ellipse A Arrow Edit & Selection Shortcut Action Ctrl/⌘ + Z Undo Ctrl/⌘ + Shift + Z (or Ctrl/⌘ + Y) Redo Ctrl/⌘ + C / X / V Copy / cut / paste Ctrl/⌘ + Shift + V Paste masked Ctrl/⌘ + A Select all Ctrl/⌘ + D Deselect Ctrl/⌘ + Shift + D Reselect Ctrl/⌘ + Shift + I Invert selection Ctrl/⌘ + T Free transform Ctrl/⌘ + Shift + T Repeat last transform Ctrl/⌘ + J Layer via copy Ctrl/⌘ + Shift + J Layer via cut Ctrl/⌘ + I Invert colors Delete / Backspace Clear layer/selection Ctrl/⌘ + Backspace Fill with background color Alt + Backspace Fill with foreground color ↑ ↓ ← → Nudge selection/layer by 1px Enter Commit transform / crop Esc Cancel / deselect Colors & Brush Shortcut Action X Swap foreground/background colors D Reset colors to default [ / ] Decrease / increase brush size { / } Decrease / increase hardness 0–9 Set tool opacity preset (0 = 100%) Navigation Shortcut Action + / = Zoom in - Zoom out Ctrl/⌘ + 0 Reset zoom (fit) Ctrl/⌘ + 1 Zoom to 100% Tab Toggle panels Space + drag Pan canvas Common Workflows Sketch-and-generate Open a blank canvas from + New → New image. Block in a rough composition with the brush on one layer. Add a new layer, select the region you want to generate, and bind the layer to a model or workflow in the Inspector. Write a prompt and Generate. Keep painting on top, then close the editor — the flattened result flows downstream. Masking for inpaint Open or paste an image into a sketch. Add a layer, paint over the area to change, and mark it as the mask layer. Wire the node’s image and mask outputs into an inpaint node. Quick retouch Open the image in the editor. Select the Clone Stamp (S), Alt-click clean pixels, and paint over the blemish. Adjust the layer opacity to blend, then save. Related Nodes The sketch type is also available as workflow nodes: Create Sketch — create a blank document with a chosen canvas size and background. Render Sketch — flatten layers, applying blend modes and opacity, to a single image. Sketch Layers — expose individual layer images for downstream nodes. Related Features Asset Management — organize and reuse generated images Workflow Editor — main editor documentation Last updated: June 2026--- # Storage Guide Source: https://docs.nodetool.ai/storage.html NodeTool stores user assets, workflow artifacts, and temporary files through pluggable backends defined in @nodetool-ai/storage (packages/storage/src/). The active backend is selected per execution by the runtime context. Asset Storage Backends Backend Module When it is used Notes In-memory @nodetool-ai/storage / src/memory-storage.ts Tests Keeps data in process-local dictionaries. Local filesystem @nodetool-ai/storage / src/file-storage.ts Default for development when no cloud storage is configured Stores assets under getDefaultAssetsPath() (@nodetool-ai/config / src/paths.ts), which defaults to ~/.local/share/nodetool/assets (XDG; %APPDATA%\nodetool\assets on Windows). Override with ASSET_FOLDER or STORAGE_PATH. URLs are served via the API (/storage/*). Supabase Storage @nodetool-ai/storage / src/supabase-storage.ts When SUPABASE_URL and SUPABASE_KEY are set Uses a Supabase bucket for asset storage. Public buckets are recommended for direct URLs. Amazon S3 / S3-compatible @nodetool-ai/storage / src/s3-storage.ts Production, or when S3_ACCESS_KEY_ID or S3_SECRET_ACCESS_KEY are provided Requires S3_* environment variables and optional custom endpoint for MinIO/Wasabi. Passing use_s3: true when obtaining asset storage forces S3 even in development (useful for smoke tests). Required Environment Variables Variable Description ASSET_BUCKET Bucket name (S3) or Supabase Storage bucket name, or folder name (local). ASSET_DOMAIN Public domain serving assets (S3 only). Not used for Supabase. ASSET_TEMP_BUCKET / ASSET_TEMP_DOMAIN Optional separate bucket for temporary assets. FONT_PATH, COMFY_FOLDER, VECTORSTORE_DB_PATH Additional paths for specific nodes (registered in @nodetool-ai/websocket / src/settings-registry.ts). For S3-compatible services, set: S3_ENDPOINT_URL S3_ACCESS_KEY_ID S3_SECRET_ACCESS_KEY S3_REGION (defaults to us-east-1) Temporary Storage Temp storage returns a location for scratch files, mirroring asset storage: Memory storage in tests. Local filesystem under the NodeTool data dir (~/.local/share/nodetool/temp; %APPDATA%\nodetool\temp on Windows). Supabase bucket when SUPABASE_URL/SUPABASE_KEY and ASSET_TEMP_BUCKET are set. S3 bucket when configured. Supabase Storage When SUPABASE_URL and SUPABASE_KEY are set, NodeTool prefers Supabase for asset and temp storage. Adapter: SupabaseStorage (@nodetool-ai/storage / src/supabase-storage.ts) Selection: handled by the runtime context when constructing storage backends Public URLs: get_url(key) returns a public URL. Use a public bucket or add a CDN edge. Private buckets: you can extend the adapter to use signed URLs (Supabase’s create_signed_url). Minimum configuration: SUPABASE_URL=https://your-project.supabase.co SUPABASE_KEY=your-service-role-key ASSET_BUCKET=assets # Optional temp bucket used by get_temp_storage ASSET_TEMP_BUCKET=assets-temp Recommendations: Create the buckets (assets, assets-temp) in the Supabase dashboard. Make them public to allow direct links via get_url. For private buckets, add a signing step. Scope the service role key to server-side environments only. Do not expose it to browsers. Use the temporary storage for intermediate files that do not need long-term retention. Node and Workflow Caches NodeTool caches expensive node outputs and metadata to accelerate repeated executions: @nodetool-ai/storage / src/memory-node-cache.ts – in-process dictionary, default for development. @nodetool-ai/storage / src/memory-uri-cache.ts – caches resolved asset URIs for quick lookup. Memoisation is wired up inside ProcessingContext (@nodetool-ai/runtime / src/context.ts). Accessing Storage in Workflows Workflows interact with storage through ProcessingContext (@nodetool-ai/runtime / src/context.ts): resolveAssetBytes(assetId) – loads the raw bytes for an asset. assetsToStorageUrl(value) – rewrites asset references in a value to storage URLs. downloadFile(url) – fetches file contents from a URL. NodeTool automatically mounts the asset folder into Docker and subprocess execution strategies so nodes can access assets on disk. Deployment Considerations For self-hosted deployments, mount persistent volumes for /workspace (workspace files) and the asset storage directory. In Docker-based execution (@nodetool-ai/deploy / src/docker-run.ts), the workspace path is mounted into containers, so ensure the host directory exists and has the correct permissions. When using S3, grant read/write access to the specified bucket and set NODETOOL_STORAGE_PUBLIC_URL (if exposing through a CDN). Troubleshooting Missing asset URLs – confirm ASSET_DOMAIN or NODETOOL_API_URL is set; the API uses these to build absolute URLs. S3 authentication errors – verify credentials and endpoint configuration in settings/secrets; run nodetool settings show --secrets. Local file permissions – ensure the configured asset folder is writable by the user running the service (especially in Docker). Docker jobs cannot access assets – mount the asset directory into the server container and ensure the assets path (getDefaultAssetsPath(), overridable via ASSET_FOLDER/STORAGE_PATH) points to the mounted path. Related Documentation Configuration Guide – environment variable hierarchy and secret management. Deployment Guide – configuring storage via deployment.yaml. Docker Resource Management – limiting resource usage for storage-heavy jobs.--- # Supabase Deployment Integration Source: https://docs.nodetool.ai/supabase-deployment.html Supabase provides both authentication and object storage for deployed NodeTool instances. This guide covers how to configure Supabase as your auth and storage backend. What Supabase Provides Feature What It Does Authentication User sign-up, login, and JWT-based session management Object Storage S3-compatible asset storage with public or signed URLs Row Level Security Fine-grained access control for multi-user deployments Prerequisites A Supabase project at supabase.com Your project’s URL and service role key from the Supabase dashboard A NodeTool deployment target (self-hosted Docker server) Setup 1. Create Storage Buckets In your Supabase dashboard, go to Storage and create the following buckets: Bucket Purpose Visibility assets Permanent workflow assets (images, documents, audio) Private or Public assets-temp Temporary files during workflow execution (optional) Private Public vs. Private buckets: Public buckets generate direct URLs that anyone can access – suitable for assets shared externally Private buckets require signed URLs or authenticated access – better for sensitive content 2. Configure Environment Variables Add these variables to your deployment target’s container.environment section: container: environment: # Supabase connection (presence of both enables Supabase auth) SUPABASE_URL: https://your-project.supabase.co SUPABASE_KEY: your-service-role-key # Select Supabase as the STORAGE backend (default is "file"). # Setting only SUPABASE_URL/KEY enables Supabase AUTH but NOT storage. NODETOOL_STORAGE_BACKEND: supabase # Storage buckets ASSET_BUCKET: assets TEMP_BUCKET: assets-temp # Optional Or set them as environment variables directly: export SUPABASE_URL=https://your-project.supabase.co export SUPABASE_KEY=your-service-role-key export NODETOOL_STORAGE_BACKEND=supabase export ASSET_BUCKET=assets export TEMP_BUCKET=assets-temp 3. Deploy Apply the configuration to your deployment target: nodetool deploy apply <target-name> Authentication The server enables Supabase auth automatically when both SUPABASE_URL and SUPABASE_KEY are present. There is no AUTH_PROVIDER switch read by the server entrypoint — Supabase-vs-local auth is selected purely by the presence of those two variables. When set, NodeTool uses Supabase JWTs for all API authentication: Users authenticate through Supabase (email/password, OAuth, magic link, etc.) API requests require a valid JWT in the Authorization header: Authorization: Bearer <supabase_jwt> NodeTool validates tokens against your Supabase project automatically When SUPABASE_URL/SUPABASE_KEY are not set, the server falls back to a local auth provider. Storage Behavior Backend Selection NodeTool selects the storage backend from NODETOOL_STORAGE_BACKEND (file | s3 | supabase, default file): file (default) – local filesystem under the assets path. s3 – requires ASSET_BUCKET/TEMP_BUCKET (plus S3_REGION / optional S3_ENDPOINT). supabase – requires SUPABASE_URL, SUPABASE_KEY, and ASSET_BUCKET/TEMP_BUCKET. Storage is not auto-selected from the presence of SUPABASE_URL/SUPABASE_KEY: those enable Supabase auth, but you must set NODETOOL_STORAGE_BACKEND=supabase to route storage through Supabase. The asset bucket is ASSET_BUCKET and the temp bucket is TEMP_BUCKET. Asset URLs Public buckets generate direct Supabase Storage URLs Private buckets generate time-limited signed URLs for secure access For a controlled proxy layer, configure your reverse proxy to mediate access Verification After deploying with Supabase, verify the integration: Check logs – Look for messages confirming Supabase storage is active: nodetool deploy logs <target-name> Test asset storage – Run a workflow that writes assets and verify the resulting URLs point to your Supabase storage Test authentication – Call an API endpoint with a Supabase JWT: curl -H "Authorization: Bearer <supabase_jwt>" \ https://your-deployment.example.com/api/workflows Check Supabase dashboard – Verify assets appear in your storage buckets Security Considerations Never expose your service role key in client-side code – it has full admin access Use Row Level Security (RLS) policies if multiple users share the same Supabase project Rotate your service role key periodically and update deployment configs Consider using separate Supabase projects for staging and production See Security Hardening for the full production checklist Related Deployment Guide – Overview of all deployment options Authentication – Detailed authentication configuration Storage – Storage backend options and configuration Configuration – All environment variables and settings Security Hardening – Production security checklist--- # Templates Gallery Source: https://docs.nodetool.ai/templates-gallery.html The Templates Gallery is a curated library of example workflows you can run and customize without starting from scratch. Templates surface in two places: the Templates section on the Dashboard (/dashboard), and the dedicated Examples page — open it from the Examples item in the app menu (the logo dropdown) or by navigating to /examples. What’s in the Gallery Every template is a real, runnable workflow exported from the editor. Templates ship with NodeTool and cover: Category Examples Image generation Movie Posters, Character Sheets, Product Shots Image editing Background Removal, Upscaling, Style Transfer Agents Research agents, RAG Q&A, multi-step tool runners Document intelligence Chat-with-Docs, PDF extraction, data enrichment Audio & video Transcription, summarization, text-to-speech Data pipelines CSV ingestion, chart generation, scheduled reports Realtime Voice agents, streaming transcription Browse the full list on the [Workflow Examples]({{ ‘/workflows/’ relative_url }}) page. Opening a Template Click a template tile to preview its graph. Hit Open (or double-click) to load it into the editor as a new workflow. Save a copy with Ctrl/⌘ + S — edits never modify the original template. Filtering and Searching The gallery supports: Tag filter — click any tag (e.g., “agent”, “image”) to narrow the list. Search bar — match by title, tags, or node names used inside the workflow. Sort — recently added, most popular, shortest. Anatomy of a Template Card Each tile shows: Thumbnail — a static render of the graph. Title and description — a one-line summary. Input badges — the required inputs (e.g., “Text”, “Image”). Output badges — what the workflow produces. Required models — any models you need downloaded first. If a template requires a model you don’t have, opening it shows the Recommended Models dialog so you can install them in one click. Submitting Your Own Template Community templates ship through the examples/ directory of the NodeTool repo: Save your workflow, then use nodetool workflows export-dsl <id> to export it as a TypeScript DSL file. Add it to examples/<category>/<slug>.ts in a PR. Include a short README — 1 paragraph, 1 screenshot of the graph. See the [Developer Guide]({{ ‘/developer/’ relative_url }}) for full contribution guidelines. Related Docs [Workflow Examples]({{ ‘/workflows/’ relative_url }}) — in-depth walkthroughs of individual templates [Cookbook]({{ ‘/cookbook’ relative_url }}) — reusable workflow patterns [Getting Started]({{ ‘/getting-started’ relative_url }}) — runs a template as your first workflow--- # Timeline Editor Performance Audit Source: https://docs.nodetool.ai/timeline-editor-performance-audit.html Timeline Editor Performance Audit Audited 2026-07-02 at commit 6c3004e. Scope: web/src/components/timeline/, web/src/stores/timeline/, web/src/hooks/timeline/, and the render/GPU layers they drive (~28k lines). Line numbers refer to that commit. Verdict The core architecture is right, and the most expensive problem a timeline editor can have — React re-rendering at 60 fps during playback — is already solved. TimelinePlaybackStore pushes time through a transient non-React channel (setTimeMs/subscribeTime, TimelinePlaybackStore.ts:55-118), the playhead/timecode/karaoke highlight all consume it imperatively, and the compositor gates React updates behind a scene signature. Steady playback causes zero React renders. The real costs are elsewhere, in three clusters: Gestures write to the reactive document store on every pointermove. Clip drag/trim, the transform gizmo, ~40 inspector/FX sliders, EQ and compressor curves, scrubbing, and zoom all publish store updates at pointer-event rate (60–240 Hz). Identity-preserving reducers keep most re-renders in check, but every publish still runs every subscriber’s selector, and several wide subscriptions re-render the whole editor shell per tick. Sliders and the gizmo also push one undo entry per tick, so a two-second drag wipes the entire 100-entry undo history. Per-frame playback work that scales with project size. The rAF tick re-derives the full scene model (sort + Map build + per-word caption resolution) every frame just to detect “nothing changed”, and the GPU effects chain re-processes static layers 60×/s while any video plays. Unbounded scaling. No virtualization (every clip on every track is always mounted, with thumbnail/waveform fetches), play-gesture audio decoding of the entire timeline, and several unbounded or full-frame-sized caches. Fixes are listed per finding; a priority table is at the end. The single highest-leverage theme: the repo already contains the right tools — useTimelineHistoryBatch, the transient time channel, stable generatingClipIds membership arrays — several hot paths just don’t use them. Tier 1 — interactive gestures (highest user-visible impact) 1.1 Inspector/FX sliders and curves: per-tick store writes, no history batching NodeSlider is a MUI Slider whose onChange fires per pointermove. Every call site pipes it straight into the zundo-wrapped TimelineStore: Inspector/InspectorPrimitives.tsx:641-643 (InspectorSliderRow — used by all ~30 ClipAdjustments rows: opacity 167-169, color 441-444, blur 533, anchor 338/348, radius 357-359) Tracks/TrackEffectsPanel.tsx:250 (ParamRow → updateTrackEffect, TimelineStore.ts:896-907) Tracks/TrackEffectsPanel.tsx:520-540 and 985-1020 (EQ / compressor SVG curve drags patch per pointermove; wheel handlers at 560-575, 1039-1054 patch per wheel tick) TimelineStore.ts:1498-1507 (setClipPrompt via DirectGenClipPanel.tsx:146-151 — one undo entry per keystroke) Arrow-key nudge (TracksRegion.tsx:504-518) — one entry per key repeat Each tick: full tracks.map/clips.map allocation, zundo partializedEqual scan over all clips/tracks (TimelineStore.ts:489-497), one undo entry (limit is 100 — a single knob drag evicts all prior history), and a publish to every doc-store subscriber (compositor, tracks region, transcript panel, autosave, attachUiPruning). Clip drag/trim and track resize already solve this with useTimelineHistoryBatch (Clip.tsx:496, TrackHeader.tsx:323); nothing in the inspector or FX panel uses it. Fix: in the two shared choke points (InspectorSliderRow, ParamRow), render the drag from local state and commit on onChangeCommitted, or keep live writes for preview but wrap the gesture in useTimelineHistoryBatch.begin()/mark()/end() and rAF-throttle the writes. Same treatment for the EQ/compressor pointerdown/up handlers. Pause the temporal store during focused prompt typing; treat held key-repeat as one gesture. 1.2 Transform gizmo: per-pointermove patchClip, no batching, forced layout PreviewCompositor.tsx:928: onChange={(id, next) => patchClip(id, { transform: next })} Every gizmo move (60–240 Hz) allocates a new clips array (TimelineStore.ts:1278-1282), pushes an undo entry, and re-renders every clips subscriber — compositor scene memo, video-pool layout effect, tracks region, inspector. The pointer handler also calls getBoundingClientRect() per move (TransformGizmoOverlay.tsx:330-333). Fix: wire useTimelineHistoryBatch into the gizmo gesture, or drive the live transform from a ref straight into the compositor and commit one patchClip on pointerup. Cache the SVG rect at pointerdown (a ResizeObserver already invalidates on resize). 1.3 Editor-root subscriptions re-render the whole shell per drag tick Three subscriptions sit at TimelineEditorBody — the root above TopBar, preview, inspector, tracks, and transcript: useTimelineExport (useTimelineExport.ts:57-66, hosted at TimelineEditor.tsx:352-359) reactively selects tracks + clips that are only read inside the click-time exportVideo callback. clips gets a new identity per drag/trim/gizmo/slider tick → whole-shell re-render per tick. During export, onProgress: setProgress (line 119) re-renders the shell per encoded frame. Fix: read store.getState() inside exportVideo; throttle progress to ~4 Hz or move it to a leaf component. useTimelineGenerationSubscriptions (useGenerateClip.ts:318-354, hosted at TimelineEditor.tsx:335) selects the whole clipJobs record. updateJobProgress (TimelineGenerationStore.ts:347-365) replaces the map per WebSocket progress message, so the shell re-renders and the subscription-reconcile effect re-runs per message for the entire life of every generation. The store maintains progress-stable generatingClipIds membership arrays for exactly this purpose (TimelineGenerationStore.ts:51-63) — this hook bypasses them. Fix: key the effect on generatingClipIds (or a useShallow projection of id/status/workflow) and read job details via getState() inside the effect. msPerPx (TimelineEditor.tsx:338) — every zoom tick re-renders the shell; only BottomStatusBar needs it. The tracks-height resize drag (TimelineEditor.tsx:397-405) sets React state per mousemove with the same fanout. Fix: push the zoom wiring into a wrapper around BottomStatusBar; apply resize height via ref during the drag, commit on mouseup. Also stabilize TopBar props — onOpenSettings={() => ...} and activitySlot={<ActivityIndicator />} (TimelineEditor.tsx:495-498) defeat its memo every render. 1.4 O(N²) selector churn during drags Every doc-store publish runs every subscriber’s selector. During a drag that is per-pointermove: Every mounted Clip selects s.clips.find((c) => c.id === clipId) (Clip.tsx:396-398) — N clips × O(N) = O(N²) per tick. Same pattern in TimelineInspector.tsx:110, GeneratedClipPanel.tsx:88, ClipVersionHistory.tsx:199, ClipActions.tsx:41, useGenerateClip.ts:375. At 300–500 clips this is tens of millions of comparisons/second mid-drag. Every TrackLane re-filters all clips per lane (TrackLane.tsx:113-122); the custom equality then discards the arrays. attachUiPruning (TimelineInstance.tsx:80-116) builds new Set(state.clips.map(...)) per tick whenever a selection exists — and dragging a selected clip is the common case. TracksRegion.contentEndMs rescans all clips per tick (TracksRegion.tsx:204-223; the cache key is clips identity). Fix: maintain a Map<string, TimelineClip> keyed on clips array identity (module-level WeakMap<TimelineClip[], Map> used inside selectors, or a clipsById field rebuilt in the same set). Turns every id lookup O(1). Defer attachUiPruning to a microtask/idle callback — pruning only matters after removals. 1.5 Transcript projections rebuilt per drag tick, ×3 consumers clips identity changes per drag tick; three consumers rebuild the full transcript projection from it, each O(words · log): ScriptLane.tsx:126-136 — buildTranscriptDoc(clips) per tick, then reconciles a <span> per word TranscriptPanel.tsx:41-42 — same rebuild plus a filler-count reduce TranscriptEditor.tsx SyncPlugin (206, 233-240) — transcriptSignature (80-87) runs buildTranscriptDoc and builds one giant joined string over every word, per clips change, just to detect external edits With the script feature on, a drag pays this 3–5× per pointermove. Dragging a B-roll clip with no captions pays it too. Fix: share one projection via a module-level WeakMap<TimelineClip[], TranscriptDoc>, and gate on transcript-relevant change — select the caption-bearing clip subset with an equality that compares member identities, or bump a transcriptRev counter from caption-touching mutations and key the SyncPlugin check on it. 1.6 Scrub fanout Ruler scrub and playhead drag write reactive currentTimeMs per pointermove (TimeRuler.tsx:457-465, Playhead.tsx:200-214). Repainting the preview per event is intended; the incidental subscribers are not: TimeRuler.tsx:266 subscribes to currentTimeMs solely for aria-valuenow (482) — re-renders the ruler per tick. Set it imperatively via subscribeTime, as Playhead already does. ScriptLane re-renders every phrase chip and word span per tick (ScriptLane.tsx:130, 186-199). Subscribe transiently and toggle classes. While playing, each scrub tick bumps seekNonce, whose effect runs graph.stopAll() + a full async handlePlay() — asset resolution, audio re-scheduling, clock restart — per pixel (PreviewArea.tsx:337-345). Debounce the restart (~100 ms trailing). The paused one-shot composite runs twice per scrub tick — eagerly with a stale frame and again on seeked (PreviewCompositor.tsx:811-828). Skip the eager pass when all video layers are seeking. 1.7 Zoom: per-wheel-event store publish, full re-layout, listener churn TracksRegion.tsx:375-419: every wheel/pinch event calls setZoom — one store publish per event (trackpads deliver 60–120+ Hz), re-rendering every clip (new leftPx/widthPx), lane, ruler, and scrollbar; the scroll correction (444-465) then triggers a second full region render via setScrollLeftPx. The effect re-attaches the wheel listener on every msPerPx change (dep at 419), so events landing before the next render compute from a stale scale (zoom skips). getBoundingClientRect() runs per event (389). Fix: rAF-coalesce setZoom with the latest wheel state in a ref; read msPerPx from a ref inside a stable listener; cache the container rect. For pinch smoothness, apply a temporary transform: scaleX() during the gesture and commit one re-layout at the end. Tier 2 — per-frame playback costs 2.1 Scene model fully recomputed every rAF tick The playback loop (PreviewCompositor.tsx:855-887) calls sceneSignature(liveMs) per frame, which runs computeActiveLayers (sceneModel.ts:226-312): [...tracks].sort, a Map rebuild over all clips, per-track filter().sort(), a ResolvedCaption object per word of every captioned active clip, plus signature string concatenation — usually to conclude nothing changed. On a 6-track/200-clip timeline that is thousands of allocations per second of GC churn inside the playback loop. The scene is also computed twice per actual bump (signature + the useMemo at 434-509). Fix: compute a nextChangeMs horizon with each scene (min of active clip ends, upcoming starts, caption word boundaries) and skip signature work while liveMs < nextChangeMs — steady-state playback becomes one float compare per frame. Or cache sortedTracks/clipsByTrackId in a WeakMap keyed on the arrays. 2.2 GPU effects re-process static layers every frame gpu/compositor.ts:230-244 runs the full compute chain (chroma key → grade → 2× blur → sharpen → vignette) unconditionally per render(). The rAF loop marks frames dirty whenever any video plays, so a static image with a heavy blur under a playing video is re-blurred 60×/s. Fix: cache the processed output per layer keyed on (source upload key, effects array identities — they are reference-stable until edited); return the cached texture on match. Videos self-invalidate via currentTime. 2.3 WebGPU micro-churn per frame setLayers re-sorts a copied array and rebuilds a prune Set every tick while video plays (gpu/compositor.ts:129-143) — skip when layer ids and order are unchanged. renderBlendPass creates a fresh GPUBindGroup + three createView() calls per layer per frame (packages/gpu/src/compositor/compositor.ts:340-353, plus blit 367-370). Views are immutable per texture; cache them and key bind groups on (source texture, ping-pong read texture). 2.4 Playhead writes style.left and churns attributes per frame Playhead.tsx:152-166: left invalidates layout each frame where transform: translateX() would stay on the compositor; aria-valuenow and the pill textContent mutate 60×/s. Throttle both to ~10 Hz; switch to transform. 2.5 Karaoke plugins re-query the word DOM per time tick TranscriptEditor.tsx — ActiveWordPlugin (330-355) and ScriptCaretPlugin (382-430) each run querySelectorAll(".transcript-word") + dataset string parsing + a linear scan per transient time tick (×2, 60 Hz), and the caret plugin interleaves style writes with offsetLeft/offsetWidth reads — forced reflow per frame. Both re-run their full setup on every clips identity change (deps at 354/430). Fix: build one shared sorted {el, startMs, endMs, offsets} cache per reseed (offsets refreshed on resize), binary-search the active word per tick, and reuse the cache in SelectionHighlightPlugin (445-455) and arrow-key stepping (621-637). Tier 3 — scaling limits (large projects) 3.1 No virtualization TrackLane.tsx:476-478 mounts every clip on every track regardless of the visible window. Each clip carries ~8–12 DOM nodes, three asset-URL effects (Clip.tsx:963-1015), a 24-frame thumbnail request per video URL (useClipThumbnails.ts:12), and a fetch+decode for audio waveforms — for clips hours off-screen. Every per-tick cost in Tier 1 scales with this total. Fix: window horizontally from scrollLeftPx/msPerPx (both already in the UI store — the visibility predicate is trivial from startMs/durationMs) with one viewport of overscan; window tracks vertically; gate thumbnail/peaks requests on visibility. 3.2 Play gesture decodes every future audio clip before the clock starts PreviewArea.tsx:261-304 filters all remaining audio clips and scheduleClips Promise.alls a fetch + decodeAudioData for each (AudioGraph.ts:406-414) before clock.start. Play latency grows with timeline length; peak memory is the whole timeline’s decoded PCM; and with more than BUFFER_CACHE_MAX = 16 distinct assets (AudioGraph.ts:43) the LRU evicts and re-decodes on every play/seek gesture. Seek-while-playing repeats everything. Fix: schedule only clips starting within a lookahead window (~30 s, matching PRELOAD_LOOKAHEAD_MS), top up as the playhead advances, and start the clock once the currently audible clips are ready. 3.3 Caption raster cache: full-frame bitmaps, up to ~530 MB captionRender.ts:19, 138-166: each caption state rasterizes at full sequence resolution (a lower-third strip inside a 1920×1080 bitmap); MAX_CACHE_ENTRIES = 64 × 8 MB ≈ 530 MB worst case (4K: ~2 GB). A new OffscreenCanvas is allocated per miss, and on the GPU side each new bitmap destroys and re-creates a full-frame texture per word change (gpu/compositor.ts:155-160) instead of re-uploading into the same-sized one. Fix: rasterize at the caption bounding box and position via the layer transform; reuse one OffscreenCanvas; re-upload into the existing texture when dimensions match; cap the cache by bytes. 3.4 Unbounded caches and export memory clipThumbnails.ts:31, 197-215: module-level Map of 24 base64 JPEGs per video URL, never evicted; failed entries permanently poisoned. Use blob URLs + LRU; allow retry. OffscreenVideoPool.ts:29, 92-113: export accumulates one <video> per video clip, never released until dispose — 50-clip exports risk hitting browser media-element and hardware-decoder caps. Release entries behind the export playhead. TimelineRenderer.ts:179-182, 303: BufferTarget holds the whole MP4 in memory. Fine today; a streaming target bounds it. ResultsStore.ts:704-722 (setProgress): growing string concat + whole record spread per WebSocket chunk message. Accumulate chunks in an array; coalesce progress writes to ~10 Hz. 3.5 Export frame loop: serialized seeks, main-thread TimelineRenderer.ts:248-266 awaits videoPool.seek() sequentially per layer; overlapping videos pay seek round-trips back-to-back. Issue seeks with Promise.all. Longer term, move the loop to a Worker with OffscreenCanvas. Tier 4 — smaller wins Filmstrip cells hash multi-KB data URLs through emotion per render (Clip.tsx:337-345, 1090): css({ backgroundImage: url(<base64>) }) per cell per render — emotion hashing is O(string length) and every distinct URL inserts a permanent CSSOM rule. Runs per pointermove during trims. Use style={{ backgroundImage }} exactly as the image path at 1060 does. TracksRegion subscribes to scrollLeftPx and selectedClipIds (TracksRegion.tsx:227, 231): whole-region re-render per pan frame and per rubber-band selection change; selection is only used in event handlers — read getState() there; let the memoized TimelineScrollbar subscribe to scroll itself. Every Clip subscribes to the entire ErrorStore.errors record (Clip.tsx:457-475): any error anywhere re-renders every clip and re-scans all error keys per clip. Narrow to a per-run derived lookup. Inspector sections stay mounted when folded (CollapsibleSection.tsx:118-120, MUI Collapse without unmountOnExit): ~30 control rows render per inspector re-render regardless of fold state. Add unmountOnExit/lazy children. ClipAdjustments passes fresh lambdas to every memoized row (ClipAdjustments.tsx:167-169, 441-444 and the IIFE blocks at 252, 396, 501, 572): dragging one slider re-renders all rows. Give rows (clipId, field) props or useCallback per field. DirectGenClipPanel subscribes to the whole clips array (DirectGenClipPanel.tsx:78, 104-117) to build a dropdown list; recomputed per doc change even outside image-to-image mode. Select {id,name} pairs with useShallow, gated on mode. ClipListPopover subscribes to clips while closed (ActivityIndicator.tsx:83, 205-239): during generation, every clip patch re-renders two closed popovers. Render only when open. Right-edge drags churn totalWidthPx (TracksRegion.tsx:204-223, 320-323, 735): dragging the last clip re-layouts the scroll area per move. Quantize the width (e.g. 256-px steps) or freeze during gestures. contentEndMs/boundary work in PreviewArea (PreviewArea.tsx:201-207, 374-385): re-render + O(n log n) boundary sort per drag tick; clips is otherwise only used in click handlers. Select primitives; read getState() in handlers. Cold-pool video preload never fires mid-clip (PreviewCompositor.tsx: 553-683): the effect’s time dep advances only on scene bumps, so the next clip isn’t preloaded until its boundary; upcomingVideoClips picks clips in array order, not soonest-first; and activation reloads the element from scratch, discarding the warm decoder. Re-evaluate on a coarse timer, sort by startMs, and swap the cold element into the hot slot. Autosave can flush mid-gesture (useTimelineAutosave.ts:211-216): holding a drag still for >750 ms fires a wasted full-document PATCH (which serializes every caption word). Gate the flush on “no history batch open”. Related: useTimelineSave.ts:33-38 omits transcript from the manual-save PATCH while autosave includes it — a correctness risk if the server replaces the whole document. Rubber-band and gizmo read getBoundingClientRect per pointermove (TrackLane.tsx:394, TransformGizmoOverlay.tsx:330-333): cache at pointerdown. TimelineListPanel: inline onCancelRename defeats item memo (:572); sort comparator allocates two Date objects per comparison (:374-376). useCallback + precompute getTime(). instanceStoreHook.ts:18-20 footgun: the wrapper silently drops a second (equality) argument — useTimelineStore(sel, shallow) compiles and ignores shallow. Forward it. el.src = "" teardown (PreviewCompositor.tsx:307-308) resolves to the document URL and can fire a spurious request; use removeAttribute("src"); load() as OffscreenVideoPool.dispose does. AudioContext instantiated on first play even with zero audio clips (PreviewArea.tsx:257-258). Gate on activeAudioClips.length > 0. What is already done well (do not “fix”) Transient playhead channel + PlaybackClock: zero React work per frame; Playhead, control-bar timecode, and karaoke highlighting are imperative. Store split (document / UI / playback, per-instance via TimelineInstance) with a memoized provider bundle — disjoint subscriber sets, no context churn. Identity-preserving reducers everywhere (patchClip/patchById/moveClip return same state on no-ops; untouched clips keep identity) — memoized siblings don’t re-render during another clip’s drag. Undo: zundo partialize stores references (structural sharing, no deep clones); useTimelineHistoryBatch collapses drag/trim/resize gestures. Autosave: transient store.subscribe, O(1) reference dirty-check, 750 ms debounce, single-flight — zero React renders. Clip.tsx gesture internals: snap candidates snapshotted per gesture, elementsFromPoint rAF-coalesced, waveform redraw rAF-coalesced with whole-pixel gating; TimeRuler draws a viewport-sized canvas from a latest-inputs ref. Compositor: scene-signature gating, dirty-flag composite skip for static scenes, stable clip→slot binding, LRU image cache, asset-URL cache with failed states, caption bitmap WeakMap signature memo. GPU: pipelines/samplers cached (no per-frame shader compilation anywhere), uniform-buffer ring, effects intermediate-texture pooling, seek-guard upload keying. AudioGraph: in-place param updates when the chain shape matches, ref-equality short-circuits, click-free ramps, in-flight decode dedupe. TimelineGenerationStore deliberately keeps generatingClipIds stable across progress ticks; TrackEffectsPanel mounts lazily; AddClipMenu mounts on demand with enabled-gated queries. No JSON.parse(JSON.stringify), no polling loops, TanStack Query keys stable with enabled guards throughout. Priority order # Fix Files Effort Payoff 1 Commit-on-release / history-batch for InspectorSliderRow + ParamRow + EQ/compressor + gizmo InspectorPrimitives, TrackEffectsPanel, TransformGizmoOverlay S–M Ends undo-history destruction; removes the densest store-write path 2 useTimelineExport → getState() reads; throttle export progress useTimelineExport S Stops whole-shell re-render per drag tick and per encoded frame 3 useTimelineGenerationSubscriptions → key on generatingClipIds useGenerateClip S Stops whole-shell re-render per progress message during generations 4 clipsById map (WeakMap-keyed) for all id selectors TimelineStore + call sites S O(N²)→O(N) selector work per drag tick 5 Shared WeakMap transcript projection + transcript-relevant gating ScriptLane, TranscriptPanel, TranscriptEditor M Removes O(words) rebuild ×3 per drag tick 6 Scene nextChangeMs horizon in the rAF tick PreviewCompositor, sceneModel M Removes per-frame scene recompute + GC churn during playback 7 rAF-coalesced zoom with ref-read scale TracksRegion S One React commit per displayed frame while zooming 8 Scrub hygiene: ruler aria via subscribeTime, debounced seekNonce audio restart, ScriptLane transient highlight TimeRuler, PreviewArea, ScriptLane S–M Cheap scrubbing, no audio-graph rebuild per pixel 9 Windowed audio scheduling PreviewArea, AudioGraph M Play latency stops scaling with timeline length 10 Effects output cache for static layers gpu/compositor, effectsProcessor M Removes 60 fps GPU work on static effected layers 11 Caption strip-sized rasterization + texture reuse captionRender, gpu/compositor M ~10–20× caption memory/upload reduction 12 Clip virtualization + visibility-gated media fetches TrackLane, TracksRegion, clipThumbnails L Caps all per-tick costs at viewport size; biggest ceiling 13 Tier-4 batch (filmstrip inline style, selector narrowing, memo fixes, cache bounds) various S each Broad small wins Items 1–4 are small, independent, and remove the worst interactive behavior; they are the right first PR. Item 12 is the largest single ceiling-raiser for big projects but should land after 4 so the windowing predicate can reuse the id map.--- # Tips and Tricks Source: https://docs.nodetool.ai/tips-and-tricks.html Shortcuts, hidden features, and workflow efficiency tips. Essential Shortcuts Shortcut Action Space Open node menu Ctrl/⌘ + Enter Run workflow Ctrl/⌘ + S Save F Fit view Ctrl/⌘ + Z Undo Alt/⌘ + K Command menu Quick Node Actions Adding Nodes Faster Search by typing: Press Space, then just type what you need (“agent”, “image”, “whisper”) Smart connect: Drag a connection to empty space → get compatible node suggestions Duplicate quickly: Ctrl/⌘ + D copies selected nodes horizontally Model Selection Recommended Models: Click the “Recommended Models” button on AI nodes to see compatible options Quick switch: Use the Model dropdown to change models without rewiring Node Organization Drag from header: Move nodes by grabbing the header bar (top of node) Group related nodes: Select multiple, press Ctrl/⌘ + G Align selection: Press A to align, Shift + A to align and distribute evenly Canvas Navigation Action How Pan around Space + drag, or right-click drag Zoom Ctrl/⌘ + scroll wheel Fit to screen Press F Focus on selection Select nodes, then press F Reset zoom Ctrl/⌘ + 0 Snap to grid Enable in View menu Connections Making Connections Type matching: Colors show compatible connections Quick connect: Drop on empty space for auto-suggestions Multi-output: One output can connect to multiple inputs Connection Tips Preview intermediate data: Add Preview nodes between connections Disconnect: Right-click connection or drag it away Re-route: Delete and recreate, or drag to a new target Workflow Management Organization Save often: Ctrl/⌘ + S – your work auto-saves, but manual saves create versions Use descriptive names: Rename workflows and nodes for clarity Templates: Save common patterns as templates History & Undo Action Shortcut Undo Ctrl/⌘ + Z Redo Ctrl/⌘ + Shift + Z Full history Available in Edit menu Layout Recovery Lost panels? View → Reset Layout restores default Auto layout: Click the Auto Layout button to tidy up Debugging Workflows Finding Problems Add Preview nodes between steps to see data at each stage Check node errors – red borders or icons indicate issues Verify connections – ensure types match Test incrementally – run partial workflows first Bypass nodes – right-click → Bypass to skip suspicious nodes Using Bypass for Debugging Isolate issues: Bypass nodes one at a time to find the problem Compare outputs: Toggle bypass to see before/after results Skip slow steps: Temporarily bypass heavy processing during testing Common Fixes Problem Solution “Missing Model” Click the indicator to install Wrong output Check input data and node settings Workflow won’t run Look for disconnected required inputs Slow execution Try cloud providers for heavy tasks Node causing errors Bypass it to test downstream nodes Power User Features Command Menu Press Alt/⌘ + K to open the command menu – the fastest way to: Open any workflow by name Switch views and panels Access settings Search for anything Multi-Tab Workflow Multiple workflows: Open in tabs, switch with Ctrl/⌘ + 1-9 Reference between: Copy nodes from one workflow to another Side by side: Drag tabs to split view Keyboard Navigation Shortcut Action 1-5 Switch left panel i Toggle Inspector (right panel) Ctrl/⌘ + F Search nodes on canvas by label Arrow keys Nudge selected nodes Shift + ? Open node documentation AI Model Tips Choosing Models Local for privacy: Use local models for sensitive data Cloud for speed: API models are faster Mix both: Local preprocessing → cloud generation → local post-processing Performance Smaller models first: Test with fast models, upgrade for quality Quantized models: Smaller files, similar quality (Q4, Q8) Streaming nodes: See progress during execution Asset Management Working with Files Drag and drop: Drop files directly onto the canvas Asset panel: Press 3 to open (or click the Assets icon) Preview: Click any asset to preview it Organizing Create folders: Keep projects organized Name clearly: Descriptive names save time later Clean up: Delete unused assets to save space Collaboration Tips Sharing Workflows Export workflow: Use the workflow export feature Mini-Apps: Share as simplified interfaces Screenshots: Document your workflows visually Working with Teams Consistent naming: Agree on conventions Document: Add comment nodes to explain complex sections Template library: Build shared templates Troubleshooting Quick Fixes Issue Try This Layout broken View → Reset Layout Node menu won’t open Refresh page, check for modals Connection won’t attach Check type compatibility Workflow stuck Press Esc to stop, check error messages Getting Help Node docs: Hover ? on any node, or press Shift + ? Discord: Ask the community GitHub Issues: Report bugs Cheat Sheet Most Used Shortcuts Windows/Linux Mac Action Space Space Node menu Ctrl + Enter ⌘ + Enter Run workflow Ctrl + S ⌘ + S Save Ctrl + Z ⌘ + Z Undo Ctrl + D ⌘ + D Duplicate F F Fit to screen A A Align nodes Alt + K ⌘ + K Command menu i i Toggle Inspector Next Steps Workflow Editor – Full editor documentation Sketch Editor – Image editing guide Cookbook – Workflow patterns Keyboard Shortcuts – Complete list--- # Troubleshooting Guide Source: https://docs.nodetool.ai/troubleshooting.html Step-by-step troubleshooting for common NodeTool issues. For installation-specific problems (GPU drivers, CUDA, platform issues), see Installation Troubleshooting. Jump to: Quick Diagnostic Checklist — Start here Common Issues & Solutions Workflow stuck or slow Type mismatch errors Poor LLM results RAG returns irrelevant documents Model download fails Deployment failures Empty preview output Performance Optimization Debugging Techniques Getting Help Quick Diagnostic Checklist When a workflow isn’t working as expected, work through this checklist systematically: Check node connections – Are all required inputs connected? Verify data types – Do output types match input requirements? Inspect Preview nodes – What do intermediate results show? Review error messages – Click failed nodes to see error details Check model availability – Are required models installed/configured? Verify file paths – Do referenced files exist and have correct permissions? Check API keys – Are cloud provider credentials configured in Settings? Review logs – Check console/terminal for detailed error messages Common Issues & Solutions Issue: “My workflow is stuck or very slow” Symptoms Progress bar doesn’t move No output after several minutes CPU/GPU usage is low Browser/app becomes unresponsive Diagnostic Steps Check Preview nodes – Add Preview nodes after each major step to identify where execution stalls Review node status – Look at node border colors: Gray = Not started Yellow = Running Green = Completed Red = Failed Inspect system resources – Open Task Manager (Windows) or Activity Monitor (macOS) to check: CPU usage RAM usage GPU utilization Disk I/O Common Causes & Fixes Cause 1: Model not downloaded Fix: Open Models → Model Manager and install the required model Prevention: Always install models before running workflows that use them Cause 2: Large file processing Fix: Use batch processing or chunk data into smaller pieces Example: Split large PDFs before indexing Cause 3: Insufficient memory Symptoms: Workflow crashes or hangs at specific nodes Fix: Close other applications Reduce batch size Use smaller/quantized models (e.g., Q4 instead of Q8) Increase system swap/page file Cause 4: Network timeout (cloud models) Fix: Check internet connection, verify API keys, increase timeout settings Alternative: Switch to local models for offline operation Cause 5: Infinite loop in workflow Symptoms: Workflow never completes, continuously processes Fix: Review workflow logic for circular dependencies Note: NodeTool prevents circular connections but complex conditional logic can create infinite loops Issue: “Node shows error: Type mismatch” Symptoms Red connection lines between nodes Error message: “Cannot connect [Type A] to [Type B]” Workflow won’t run Diagnostic Steps Hover over connection – Tooltip shows expected vs actual types Check node documentation – Click node and read input/output type requirements Use Preview nodes – Verify what data type is actually being produced Common Causes & Fixes Cause 1: Wrong data type Example: Connecting List[String] to a node expecting String Fix: Add conversion node (e.g., GetElement to extract single item from list) Cause 2: Null/empty output Example: Previous node returned null, next node expects data Fix: Add validation or default value nodes Cause 3: Image format mismatch Example: Node outputs PIL image, next expects Tensor Fix: Use appropriate conversion nodes (usually automatic) Type Conversion Quick Reference From Type To Type Solution Node List[T] T GetElement or SelectElements String List[String] Split or wrap in [string] Image Tensor Automatic in most cases Audio AudioSegment Automatic conversion Path String PathToString Dict String DictToJson or FormatText Issue: “LLM generates poor quality results” Symptoms Irrelevant responses Hallucinations or factually incorrect outputs Repetitive text Incomplete responses Diagnostic Steps Check prompt template – Review prompt in FormatText or Agent nodes Verify model selection – Ensure appropriate model for task Test with different temperature – Adjust creativity vs accuracy tradeoff Add examples – Provide few-shot examples in prompt Use Preview nodes – Inspect intermediate reasoning steps Common Causes & Fixes Cause 1: Vague or ambiguous prompt Fix: Be specific and explicit in instructions Bad: “Summarize this” Good: “Summarize this document in 3 bullet points, focusing on action items” Cause 2: Wrong model for task Fix: Match model to task: Creative writing → Higher parameter models (13B+) Factual Q&A → Models fine-tuned for chat/instruct Code generation → Code-specialized models Fast prototyping → Smaller models (7B) for iteration Cause 3: Temperature too high/low Fix: Adjust temperature setting: 0.0-0.3 → Deterministic, factual (Q&A, extraction) 0.5-0.7 → Balanced (general chat) 0.8-1.2 → Creative (story writing, brainstorming) Cause 4: Context window exceeded Symptoms: Truncated responses, “forgot” earlier context Fix: Reduce input length Use RAG to retrieve relevant snippets instead of full documents Switch to model with larger context window Cause 5: Missing retrieval context (RAG workflows) Fix: Verify vector database is populated Increase number of retrieved documents Check hybrid search parameters Ensure embeddings are generated correctly Issue: “RAG workflow returns irrelevant documents” Symptoms Retrieved documents don’t match query Low relevance scores Answers don’t address question Diagnostic Steps Check collection status – Verify documents are indexed in SQLite-vec (or your configured backend) Inspect embeddings – Use Preview nodes to see what’s being embedded Test search directly – Run search node separately with known queries Review chunk size – Check if documents are split appropriately Common Causes & Fixes Cause 1: Documents not indexed Fix: Run an indexing workflow first to populate the collection Verify: Check collection count, should match number of chunks Cause 2: Poor chunking strategy Symptoms: Retrieved text is too short/long or cuts off mid-sentence Fix: Use nodetool.document.SplitRecursively (or SplitDocument / nodetool.text.RegexSplit) for semantic chunking Adjust chunk size (typical: 500-1000 tokens) Add overlap between chunks (typical: 50-100 tokens) Cause 3: Embedding model mismatch Fix: Use same embedding model for indexing and retrieval Example: If you indexed with text-embedding-ada-002, query with the same Cause 4: Query too vague Fix: Reformulate queries to be more specific Technique: Use LLM to expand/rephrase query before search Cause 5: Low top_k value Fix: Increase number of retrieved documents (try 5-10 initially) Tradeoff: More documents = better recall but slower inference Issue: “Model download fails or stalls” Symptoms Download progress bar stops moving “Connection timed out” or “HTTP error” messages Model appears partially downloaded “Model not found” error even after downloading Diagnostic Steps Check disk space — models can be 4–20 GB each. Run df -h (macOS/Linux) or check This PC (Windows) Check internet connection — try accessing huggingface.co in a browser Check download progress — look at the Models panel for status indicators Common Causes & Fixes Cause 1: Insufficient disk space Fix: Free up disk space. Models are stored in ~/.cache/huggingface/ by default Check space: You need at least 2x the model size free (for download + extraction) Move cache: Set HF_HOME environment variable to a drive with more space Cause 2: Network interruption Fix: Retry the download — NodeTool resumes partial downloads automatically If behind a proxy: Configure HTTP_PROXY and HTTPS_PROXY environment variables Slow connection: Large models (12 GB+) may take 30+ minutes on slower connections Cause 3: HuggingFace rate limiting Fix: Wait a few minutes and retry For frequent downloads: Add a HuggingFace token in Settings → Providers to increase rate limits Get a token: Create one at huggingface.co/settings/tokens Cause 4: Corrupted download Symptoms: Model appears downloaded but fails to load, or shows unexpected errors Fix: Delete the model from the Models panel and re-download it Manual cleanup: Remove the model folder from ~/.cache/huggingface/hub/ Issue: “Deployment fails or service won’t start” Symptoms nodetool deploy apply fails with errors Container exits immediately Health check fails 503 Service Unavailable Diagnostic Steps Check deployment logs – nodetool deploy logs <name> Verify configuration – Review deployment.yaml for typos/errors Test locally first – Ensure workflow runs in desktop app Check resource limits – Verify target has enough CPU/RAM/GPU Verify credentials – Ensure API keys and tokens are set Common Causes & Fixes Cause 1: Invalid deployment.yaml Fix: Validate configuration with nodetool deploy plan <name> Check: Required fields (type, image, host for self-hosted) Cause 2: Missing environment variables Fix: Add required vars to env section: env: PORT: "7777" DB_PATH: "/workspace/nodetool.db" HF_HOME: "/hf-cache" Cause 3: Volume mount errors (self-hosted) Symptoms: Container logs show “Permission denied” or “No such file” Fix: Verify host paths exist Check permissions (user running Docker must have access) Use absolute paths Cause 4: Docker not running (self-hosted) Fix: Start Docker daemon: sudo systemctl start docker Cause 5: Port conflicts Symptoms: “Port already in use” error Fix: Change container.port in deployment.yaml Stop conflicting service: docker ps to find, docker stop <id> Cause 6: Image not found Fix: Build image first: docker build -t nodetool:latest . Verify: docker images | grep nodetool Cause 7: RunPod/Cloud Run credential issues Fix: RunPod: Set RUNPOD_API_KEY environment variable Cloud Run: Authenticate with gcloud auth login Issue: “Preview node shows empty or null output” Symptoms Preview panel is blank Shows “null” or “undefined” No error message Diagnostic Steps Check upstream nodes – Work backwards to find where data is lost Add intermediate Previews – Place Preview after each node to narrow down issue Review node errors – Even if node appears green, check for warnings Inspect connections – Ensure correct output is connected to correct input Common Causes & Fixes Cause 1: Node failed silently Fix: Check node logs/console for hidden errors Example: API call returned 404 but node didn’t show error Cause 2: Conditional logic filtered out data Fix: Review filter conditions, adjust criteria Example: Filter node removed all items Cause 3: Async timing issue Fix: Ensure upstream nodes complete before downstream starts Note: NodeTool handles this automatically, but custom nodes may have issues Cause 4: Path/file doesn’t exist Fix: Verify file paths, use absolute paths Check: File permissions, spelling, case-sensitivity (Linux/macOS) Performance Optimization Workflow runs slowly Diagnosis: Profile with Preview nodes to identify bottlenecks Optimization strategies: Use local models for fast tasks Example: Local Whisper for transcription instead of API Benefit: No network latency Batch processing Process multiple items together Use nodes like Extend or Collect to batch operations Parallel execution Split workflow into independent branches NodeTool automatically parallelizes non-dependent nodes Cache results Save intermediate outputs to files Reuse across workflow runs Use SaveText, SaveImage, etc. Right-size models Don’t use GPT-4 when GPT-3.5 suffices Use quantized models (Q4) for faster inference Balance quality vs speed Optimize chunking Smaller chunks = faster processing but may lose context Larger chunks = slower but better context Typical sweet spot: 500-1000 tokens High memory usage Diagnosis: Monitor system resources during workflow execution Fixes: Use smaller models – Switch to quantized versions (Q4 instead of FP16) Process in batches – Don’t load all data into memory at once Clear caches – Restart NodeTool periodically to clear accumulated memory Close Preview panels – Preview nodes keep data in memory for display Limit parallel execution – Reduce concurrent node execution Debugging Techniques Using Preview Nodes Effectively Strategy: Add Preview nodes after every major transformation Example workflow: Input → Preview(1) → Transform → Preview(2) → LLM → Preview(3) → Output Benefits: See exact data at each step Identify where data is corrupted/lost Verify transformations are correct Understand intermediate results Enable Verbose Logging Desktop app: Open Settings → Advanced Enable Debug Logging Check console/logs for detailed messages CLI/Server: nodetool serve only accepts --host and --port. Set the log level via the NODETOOL_LOG_LEVEL (or LOG_LEVEL) environment variable: NODETOOL_LOG_LEVEL=debug nodetool serve Isolate Problem Nodes Strategy: Test nodes individually Create new workflow with just the problem node Provide known good inputs Verify outputs If works in isolation, issue is with upstream data Use Reproducible Inputs Strategy: Save test inputs as files Create test-inputs/ folder Save sample images, text, audio Use FileInput nodes for consistent testing Share test cases with teammates Getting Help Quick Self-Help Checklist Before reaching out, try these steps — they resolve most issues: Read the error message — it often contains the solution or a clear hint Check the Quick Diagnostic Checklist above Search this page — use Ctrl/Cmd+F to find your error message or symptom Try a simpler workflow — isolate the problem by testing with fewer nodes Restart NodeTool — clears cached state and frees memory Check Workflow Debugging — step-by-step debugging guide Where to Get Help Channel Best For Discord Real-time help, community tips, sharing workflows GitHub Issues Bug reports, feature requests, reproducible problems Documentation Guides, API reference, node documentation How to Ask Effectively Include these details when asking for help: NodeTool version — from Help → About Operating system — macOS/Windows/Linux + version What you’re trying to do — describe the goal, not just the error Full error message — copy the complete text, not just “it doesn’t work” Steps to reproduce — exact sequence that causes the issue Screenshots — especially of error messages or unexpected behavior Workflow file — export the workflow JSON if possible Good example: I’m running NodeTool 1.5.0 on macOS 14.1. When I run the Chat with Docs workflow (attached JSON), I get “Collection not found: docs”. I’ve already run the Index PDFs workflow and can see the collection in the SQLite-vec store. Screenshots attached. Poor example: Chat with docs doesn’t work, help! Related Documentation Workflow Debugging Guide – Step-by-step debugging with Preview nodes and logs Installation Troubleshooting – Hardware, CUDA, and installation issues Getting Started – Basics of running workflows Workflow Editor – Using the visual editor Key Concepts – Understanding nodes and connections Deployment Guide – Production deployment troubleshooting Self-Hosted Deployment – Self-hosted specific issues--- # Tutorials Source: https://docs.nodetool.ai/tutorials.html Short, beginner-friendly walkthroughs. Each plays a real workflow on the canvas and zooms into every node so you can see what it does and what to fill in. Build your first workflow A complete AI pipeline, end to end: type a prompt, an LLM rewrites it into something richer, and a Text To Image node turns it into a picture — all from connected nodes, no code. You’ll learn how nodes pass data through their handles, how to read live status (running rings, streaming text, the progress bar), and where generated output appears on the canvas. Connect & run The core loop, one step at a time. Add a node, wire its output into the next node’s input, press Run, and read the result. You’ll learn what inputs, outputs, and handles are, how to run a graph and watch nodes complete, and how to find a node’s result in a Preview. Generate a list One prompt, many results. A single LLM node turns a topic into a structured list, streaming each item as it arrives — the pattern behind batching and loops. You’ll learn how to drive an LLM node from an input, watch multi-item output stream in, and pass a list into the rest of a workflow. Ask the AI The simplest chat-style graph. Type a question, send it to an LLM node, and watch the answer stream in phrase by phrase before it lands in a Preview. You’ll learn how to feed a question into an LLM node, watch the answer stream as it generates, and reuse it downstream. Combine two inputs The first graph that branches in. Two text inputs flow into one Format Text node that fills a template, composing a single result from reusable parts. You’ll learn how to wire several inputs into one node, compose text with {{ placeholders }}, and build prompts from reusable parts. Summarize a document Long text in, key points out. A single Summarizer node condenses an article, transcript, or any block of text into a short summary, streaming it as it writes — the pattern behind the Meeting Transcript Summarizer example. You’ll learn how to feed a long passage into a Summarizer, watch the summary stream as it generates, and pass the result into the rest of a workflow. Describe an image The first multimodal graph. Drop a picture into an Image Input, wire it into an Agent, and watch the model look at the image and describe it in words — the vision pattern behind captioning and alt-text workflows. You’ll learn how to bring an image into a graph, send it to a vision model, and reuse the streamed description in any downstream text node. Ask the chat agent A different surface: Chat. A question goes straight to the agent, which calls a web-search tool in the open — no hidden spinner — then streams its answer back token by token. You’ll learn how to send a message from Chat, watch a tool call run in the open, and read a streamed answer as it arrives. Cut a scene together The timeline (video) editor: trim a clip, drag in another shot, drop in a caption that syncs word-by-word with the audio, then scrub the finished cut. You’ll learn how to trim and arrange clips on real tracks, add a caption synced to the audio, and scrub and preview a cut without leaving the browser. Ready to build your own? Start with Quick Start, then browse the [Examples]({{ ‘/workflows/’ | relative_url }}).--- # ui_primitives overhaul — implementation brief Source: https://docs.nodetool.ai/ui-primitives-overhaul.html ui_primitives overhaul — implementation brief Audit finding: web/src/components/ui_primitives/ is a naming layer over MUI, not a design system. The import policy is enforced (never import raw MUI) but the visual decisions were never made, so screens built 100% from primitives still look amateur by default. This brief lists the verified defects with file:line evidence, then the fixes in priority order. Ground rules for the implementing agent: All work in web/. After changes: cd web && npm run typecheck && npm run lint && npm test. Keep the 4px grid (spacing.ts), design tokens (docs/DESIGN.md), and the primitives-first policy (web/src/components/ui_primitives/STRATEGY.md). Fix defaults in the primitives, not by adding props at call sites. The measure of success: a bare <Panel><FlexColumn><TextInput/><SelectField/></FlexColumn></Panel> with no styling props should look like a professional form. Reference implementation of the target form look: the storyboard header in web/src/components/storyboard/StoryboardBoard.tsx (local Field wrapper, labels above controls, uniform 36px control heights, bordered panel). That file hand-fixes everything this brief systematizes; once the primitives are fixed, migrate it to the shared versions and delete the local workarounds. Defect inventory (verified) 1. No control-level tokens tokens.ts exports FONT_WEIGHT, FONT_SIZE_*, TYPOGRAPHY, MOTION, Z_INDEX, BORDER_RADIUS — but no control height, control padding, or field radius. Every primitive invents its own: Component Value Source EditorButton 24 / 28 raw px editor_ui/EditorButton.tsx:55 Editor selects 28px / 32px theme.editor.heightNode/heightInspector, themes/components/editorControls.ts:104,124 TagInput 48px ui_primitives/TagInput.tsx:51 SearchInput radius 8px ui_primitives/SearchInput.tsx:46 Editor control radius 6px themes/ThemeNodetool.tsx:76 MuiButton radius theme.rounded.buttonLarge themes/ThemeNodetool.tsx:241 Consequences: a compact EditorButton (24px) next to an editor select (28px) misaligns by 4px; next to an inspector select by 8px. Four independent radius sources. A second, parallel control scale lives on theme.editor that EditorButton does not read. Off-grid values inside the primitives whose own docs ban them (spacing.ts:23 forbids 0.25/0.75/1.25 steps): Slider.tsx:45 — 13px padding NodeSlider.tsx:56 — 3px margin Panel.tsx:39 — comfortable: 2.5 units; Panel.tsx:151 divides padding to 1.25 units (5px) 2. Unrelated defaults across primitives TextInput.tsx:52 → variant="outlined", size="medium" SelectField.tsx:98 → size="medium", variant="standard" A no-props TextInput above a no-props SelectField renders a boxed field above an underline-only field — the default composition is broken. Surfaces: Panel.tsx:88-90 → background.default, bordered=false. Dark background.default is #08090A (themes/paletteDark.ts:268) — same as the page. A default Panel is invisible. Surface.tsx:64-68 → background.paper, bordered=false, elevation=0, explicit boxShadow: "none". Different background and radius source (theme.rounded[...]) than Panel (theme.shape.borderRadius). size semantics diverge: TextInput maps compact→MUI small; SelectField’s standard variant barely responds to size; EditorButton.tsx:65 does height: size ? undefined : height — passing size silently discards the density height. Flex containers: FlexRow.tsx:55, FlexColumn.tsx:53, Stack.tsx:61 all default gap/spacing to 0, so naive composition glues controls together. 3. Four label treatments, none shared MUI floating label — TextInput, hand-correcting MUI’s transform (TextInput.tsx:96 translate(14px, 17px)), forced to 15px, opacity: 0.6. MUI InputLabel — SelectField.tsx:123-129, 15px, no opacity, no transform fix. Renders differently from TextInput’s at rest. Label.tsx:79-90 — Typography component="label", 13px, weight 500, text.secondary, baked marginBottom: 2px. FormField.tsx:85-95 — reimplements the same thing (does NOT use Label), duplicating the required-asterisk logic (FormField.tsx:98-108 vs Label.tsx:16). The theme sets a fifth treatment on MuiFormLabel (ThemeNodetool.tsx:249-259): capitalize, 13px, padding: 0 0 8px 0 — which TextInput/SelectField then override back to 15px, defeating the theme’s label token exactly where labels matter. The floating-label choice is also a bug class: the shrunk label renders above the input box, and Panel.tsx:136 is overflow: hidden, so a TextInput at the top of a Panel gets its label clipped. (This shipped: the storyboard title. Worked around in StoryboardBoard.tsx with overflow: visible.) 4. No usable form composition primitive FormField exists but per STRATEGY.md:47 is used in 1 file, with 14+ places doing label+input+helper by hand. FormField double-labels: it renders its own label while TextInput/SelectField render theirs. SelectField has hideLabel (SelectField.tsx:48); TextInput has no escape hatch. No FormRow, FormSection, FieldGroup, or grid primitive. Only flex helpers. Spacing ownership is incoherent: SelectField.tsx:117 comments “spacing is the parent’s job” while Label.tsx:88, FormField.tsx:120, AutocompleteTagInput.tsx:110, EmptyState, ProgressBar, SectionHeader, and Panel bake their own margins. Stacking three fields yields four different gaps. TextInput additionally inherits MUI’s default helper-text margin (3px 14px 0) which nothing overrides — off-grid and different from SelectField’s mt: 0.5 (SelectField.tsx:158). 5. The type scale is fake Text.tsx:14 advertises 8 sizes; ThemeNodetool.tsx:40-46 collapses them to ~4 real values: fontSizeBigger = fontSizeBig = 18px; fontSizeSmaller = fontSizeTiny = fontSizeTinyer = 11px. <Text size="tiny"> and <Text size="smaller"> are indistinguishable. 6. Zero-opinion passthroughs masquerading as primitives Box.tsx — 2-line re-export. muiReexports.ts:108-137 re-exports Tabs, ToggleButton, Accordion, LinearProgress, Modal etc. with a comment claiming “ThemeNodetool already styles them” — but the theme has no MuiTabs, MuiToggleButton, MuiAccordion, or MuiLinearProgress overrides (only MuiButton, MuiFormLabel, MuiFormControl, MuiPopover, MuiModal, MuiToolbar, MuiDialogTitle, MuiTooltip, MuiDivider — ThemeNodetool.tsx:234-324). Those re-exports render stock MUI. Contrast: MuiTooltip (ThemeNodetool.tsx:301-312) gets backdrop blur, a divider border, and a layered shadow — more surface craft than any panel, card, or input receives. Inputs have no default field background: only editor-scoped controls get Paper.overlay (editorControls.ts:23, gated on a marker class). SearchInput.tsx:48 gives itself action.hover — so search boxes read as fields and plain text inputs don’t. Fixes, in priority order Do them as separate commits/PRs in this order. 1–3 are mechanical and improve every existing screen without call-site changes. Fix 1: control tokens Add to tokens.ts: export const CONTROL = { height: { compact: 28, normal: 36, comfortable: 44 }, // px paddingX: { compact: 8, normal: 12 }, // px, on-grid radius: BORDER_RADIUS.sm, // one field radius } as const; Wire it into: EditorButton (replace raw 24/28), SelectField, TextInput, SearchInput (replace 8px radius), TagInput (replace 48px), and theme.editor.heightNode/heightInspector/controlRadius (derive from CONTROL so the two scales can’t drift). Fix the off-grid values in Slider.tsx:45, NodeSlider.tsx:56, Panel.tsx:39/151 while touching them. Acceptance: a compact EditorButton, a small SelectField, a small TextInput, and the ModelSelectButton in one FlexRow all share one height, one radius, one horizontal padding. Fix 2: one label treatment Standardize on label above the control: 13px (fontSizeSmall), weight 500, text.secondary, no text-transform, 4px gap to the control. Remove the floating-label path: TextInput no longer forwards label to MUI; it renders the standard label above (via the shared Label). Delete the transform-correction sx (TextInput.tsx:88-101). SelectField same: drop InputLabel, render the shared label above. FormField uses Label instead of its own Typography block; delete the duplicated asterisk logic. Remove textTransform: "capitalize" and the 8px label padding from MuiFormLabel in ThemeNodetool.tsx:249-259 (labels are no longer MUI-managed). This kills the clipped-label bug class; the overflow: visible workaround in StoryboardBoard.tsx styles can then go. Acceptance: TextInput, SelectField, and a labelled ModelSelectButton render pixel-identical labels; no label ever renders outside the control’s box. Fix 3: coherent defaults SelectField: default variant="outlined" (matches TextInput). Audit the ~few call sites that relied on standard. Inputs get a default field background (Paper.overlay in dark; verify light) so fields read as fields outside the editor scope, matching SearchInput. Panel: default background="paper" and bordered — a default Panel must be visible on the page background. Align its radius source with Surface (pick one: theme.rounded or shape.borderRadius). FlexRow/FlexColumn/Stack: keep gap=0 default (too many call sites assume it) — instead fix via Fix 4’s form primitives which own their gaps. EditorButton: passing size must not discard density height — make size map onto the CONTROL.height scale or remove the prop. Acceptance: the bare no-props composition in the ground rules renders as a visible bordered panel containing two identically-styled fields. Fix 4: real form primitives Rework FormField: it owns the label (children render label-less), the 4px label gap, and the helper/error line with a fixed on-grid margin. Ensure TextInput/SelectField compose without double labels (either via context flag or by convention: never pass label to a control inside FormField). Add FormSection (optional uppercase group label — see .group-label in StoryboardBoard.tsx — plus a FlexColumn gap={3} body) and FormGrid (n-column CSS grid with a stack breakpoint, like .header-grid there). Migrate StoryboardBoard.tsx to these, deleting its local Field and grid CSS. Migrate 2–3 of the 14+ hand-rolled label+input sites listed in STRATEGY.md as proof. Fix 5: honest type scale Either give the collapsed sizes real values (a modular scale: e.g. 11/12/13/15/18/22) or remove the aliased size names from Text and fix call sites. Don’t leave 8 names for 4 values. Fix 6: passthrough honesty For each raw re-export in muiReexports.ts that the theme does not style (Tabs, ToggleButton, Accordion, LinearProgress at minimum): either add the theme override or move it out of the “themed” comment block so the file stops claiming coverage it doesn’t have. Low priority; do last. Verification cd web && npm run typecheck && npm run lint && npm test Visual: storyboard header (/workspace storyboard tab), node inspector, settings dialogs — before/after screenshots for each fix. Grep gates after Fix 1: no raw minHeight: "2[48]px" in primitives; no borderRadius: "8px"/"6px" literals in ui_primitives/ or editor_ui/; no 13px/3px/5px paddings in primitives.--- # Use Cases Source: https://docs.nodetool.ai/use-cases.html These are the showcase workflows from nodetool.ai. Each one starts from a handful of inputs and ends with a finished result — a trailer, a batch of video ads, a set of poster concepts — built on a single canvas. Nothing is locked: swap a model, rewrite an agent’s prompt, or drop in a new brief and run it again. The workflow is the reusable part, not the example output. Every model runs with your own keys. The bill comes from the provider, not from us, and you can switch any model for a better one the day it ships. Film Movie Trailer Generator Type one logline and the canvas builds a cinematic teaser: a showrunner agent writes the treatment, a list generator breaks it into shots, each shot is rendered as key art, animated, and cut into a finished trailer. LoglineTreatmentShotsKey artTrailer Marketing Product Video Generator Turn a campaign brief and a single product photo into a cinematic 16:9 product video. Your inputs feed a prompt, an agent directs the shot, and a text-to-video model renders it. BriefPromptAgentText-to-Video Advertising Ad Creative Factory One product photo and one offer become a batch of ready-to-test vertical video ads. A strategist agent plans a persona × angle test matrix, and every cell becomes a spoken hook, a staged scene, an animated clip, and a voiceover. OfferMatrixHooksScenesAds Design Movie Poster Generator From a title, genre, and audience, the canvas writes a creative strategy and renders a batch of cinematic poster concepts — title, tagline, billing block and all. BriefStrategyConceptsKey art Creators Podcast Repurposing Studio Drop in one episode and ship the whole content pack: titles and show notes, a newsletter edition, five social posts, and quote cards rendered as square images. Transcribed once, written four ways. EpisodeTranscriptNotesNewsletterCards Content SEO Content Engine One topic in, a keyword-targeted article batch out. A strategist agent plans the cluster, and every brief becomes a full Markdown article with an editorial hero image — a cluster per run, not an article per invoice. TopicClusterBriefsArticlesHeroes Build your own These are starting points, not limits. The same building blocks — inputs, prompts, agents, and image/video/audio models on one canvas — cover far more ground. For smaller, single-purpose examples, see the [Workflow Gallery]({{ ‘/workflows/’ | relative_url }}) and the [Cookbook]({{ ‘/cookbook’ | relative_url }}). New to the canvas? Start with [Getting Started]({{ ‘/getting-started’ relative_url }}). Want the building blocks? See [Key Concepts]({{ ‘/key-concepts’ relative_url }}) and [Nodes]({{ ‘/nodes/’ relative_url }}). Bringing your own keys? See [Models & Providers]({{ ‘/models-and-providers’ relative_url }}). </content> </invoke>--- # Ad Creative Factory Source: https://docs.nodetool.ai/use-cases/ad-creative-factory.html Use case · Advertising Turn one product photo and one offer into a batch of ready-to-test vertical video ads. A strategist agent plans the test, and every variant comes out staged, animated, and voiced — what an agency bills per video, this graph produces by the batch at provider cost. Since ad platforms took over targeting, creative volume is the lever that’s left: accounts win by testing many angles and refreshing them before they fatigue. That makes ad variants a recurring line item — bought monthly, per video. This workflow moves that line item onto your canvas. How it works Four inputs, one fan-out. The strategist plans once; everything after the list generator runs once per variant. graph LR brief["Product & offer (String)"] audience["Audience (String)"] count["Variant count (Integer)"] photo["Product photo (Image)"] strategist["Strategist (Agent)"] hooks["Hooks (ListGenerator)"] scene["Stage scene (ImageToImage)"] clip["Animate (ImageToVideo)"] vo["Voiceover (TextToSpeech)"] mix["Mix (AddAudio)"] brief --> strategist audience --> strategist count --> strategist photo --> strategist strategist --> hooks hooks --> scene photo --> scene scene --> clip hooks --> vo clip --> mix vo --> mix Feed in the campaign. The product and offer, the audience, a variant count, and one clean product photo — the same shot anchors every variant. (e.g. “Aurora Trail smart fitness watch · 20% off launch week · weekend hikers · 6 variants”) Plan the test. A strategist agent studies the photo and brief, then writes a test matrix: buyer personas × message angles (pain point, bold claim, social proof, offer urgency), and picks the strongest pairings to test. Write the hooks. A list generator turns each matrix cell into one spoken hook — 18 words or fewer, plain language, said in the first three seconds. Stage every scene. For each hook, an image-to-image model places the product from your photo into that hook’s world: native UGC framing, natural light, 9:16, upper third kept clear for caption text. Animate and voice. An image-to-video model adds a subtle handheld push-in; a text-to-speech model reads the hook. Mix and ship. The voiceover lands on the clip, and the finished ads collect in a preview grid — one ready-to-upload vertical ad per matrix cell. One photo in, a test cell out Every variant isolates one persona × angle pairing, so when one ad wins in the ad account you know why it won — and the next run can breed from it. Paste last week’s winning hooks into the audience or offer input, raise the variant count, and the strategist plans the next generation around them. Make it yours Swap any model. Language, image, video, and voice are each one node. Trade the video model up for hero variants, down for cheap broad tests. Retune the strategist. The persona × angle matrix is a prompt you can read and edit — push it toward founder-story ads, comparison ads, or a single angle across ten personas. Change the format. Aspect ratio, duration, and resolution live on the image and video nodes. Go 1:1 for feed placements or 16:9 for YouTube in one click. Localize the batch. Point the voiceover at another voice or language and re-run — the same matrix ships in every market. Models in this workflow The template ships with no models selected — pick one per role when you open it. Called with your own keys; the bill comes from the provider. Role Node Works well with Test-matrix strategist Agent Any strong language model Hook writer List Generator Any language model Scene staging Image To Image An image-editing model that keeps the product intact (Nano Banana, FLUX) Ad animation Image To Video Veo, Kling, Seedance Voiceover Text To Speech OpenAI TTS, ElevenLabs See Models & Providers to set up keys. Next steps Open the template: Templates → Ad Creative Factory in the dashboard Product Video Generator — one cinematic hero video instead of a test batch Movie Trailer Generator — the same fan-out pattern, pointed at film All use cases--- # Movie Poster Generator Source: https://docs.nodetool.ai/use-cases/movie-poster.html Use case · Design From a title, genre, and audience, the canvas writes a creative strategy and renders a batch of cinematic poster concepts — title, tagline, billing block and all. One run gives you a spread of directions, not a single guess. How it works Three text inputs and a strategy-then-render pipeline turn a one-line brief into a batch of theatrical posters. {% mermaid %} graph LR title[“Title (String)”] genre[“Genre (String)”] audience[“Audience (String)”] strategy[“Strategy (Prompt)”] strategist[“Strategist (Agent)”] plots[“Concepts (ListGenerator)”] poster[“Poster prompt”] render[“Render (TextToImage)”] title –> strategy genre –> strategy audience –> strategy strategy –> strategist strategist –> plots plots –> poster title –> poster genre –> poster poster –> render {% endmermaid %} Set the brief. Three text inputs hold the film’s title, its genre, and the audience you’re selling to. That’s the entire creative input. (e.g. “Singularity · Sci-Fi · AI Enthusiasts”) Write the strategy. A Prompt node frames the brief and an agent returns a real creative strategy: positioning, audience insight, a core visual concept, and a color palette. Spin up concepts. A list generator turns the strategy into a batch of distinct plot angles, so one run gives you a spread of directions. Render the key art. Each concept is templated into a full theatrical poster prompt, then an image model renders it — title, tagline, billing block and all. Concepts from one run "The end of time isn't the end." "The end of limits. The beginning of everything." "The future is not ours to control." "The end of man is only the beginning." "The future doesn't evolve. It accelerates." Make it yours Swap the image model. GPT Image-2, Flux, Nano Banana, Ideogram. Change one node and the whole batch re-renders in a new look. Shift the tone. Change the genre or audience and the strategy, palette, and posters all follow — no manual re-briefing. Edit the poster recipe. The poster Prompt node owns the composition, typography, and billing-block layout. Tune it once, every render obeys. Batch any title. Drop in a new title and run it again. The workflow is the reusable part; the posters are just this run’s output. Models in this workflow Called with your own keys. The bill comes from the provider, and you can switch any of them for a better model the day it ships. Model Role Provider Gemini 3.1 Pro Preview Writes the strategy and plot concepts Gemini GPT Image-2 Renders the poster key art kie See [Models & Providers]({{ ‘/models-and-providers’ relative_url }}) to set up keys. Next steps [Movie Posters workflow]({{ ‘/workflows/movie-posters’ relative_url }}) — the build-it-yourself version in the gallery [Movie Trailer Generator]({{ ‘/use-cases/movie-trailer’ relative_url }}) — from one logline to a cut teaser [All use cases]({{ ‘/use-cases’ relative_url }}) </content> </invoke>--- # Movie Trailer Generator Source: https://docs.nodetool.ai/use-cases/movie-trailer.html Use case · Film Type one logline and the canvas builds a cinematic teaser — a treatment, a shot list, key art for every beat, then animated and cut into a finished trailer. No editor, no studio, one canvas you can re-run for any story. How it works A handful of nodes do the work. One line becomes a treatment, the treatment becomes shots, the shots become a trailer. {% mermaid %} graph LR logline[“Logline (String)”] vstyle[“Visual style (String)”] treatment[“Treatment (Prompt)”] showrunner[“Showrunner (Agent)”] shotlist[“Shot list (ListGenerator)”] keyart[“Key art (TextToImage)”] i2v[“Animate (ImageToVideo)”] concat[“Concat → Trailer”] logline –> treatment vstyle –> treatment treatment –> showrunner showrunner –> shotlist shotlist –> keyart keyart –> i2v i2v –> concat {% endmermaid %} Start with one line. Type the logline. Two more inputs set the visual style and the shot count — that’s the entire brief. (e.g. “A getaway driver outruns a collapsing bridge · gritty daylight · 6 shots”) Treatment, then shots. A Prompt frames the brief, a showrunner agent returns a teaser treatment, and a list generator breaks it into concrete, one-line shots. Render the key art. Each shot is templated into a key-art prompt, then a text-to-image model renders it as a cinematic 16:9 frame. Animate and cut. An image-to-video model animates every frame, then a Concat node stitches them into one finished trailer. Six shots, one trailer Every beat is rendered as its own cinematic frame, then animated and cut together. Here are all six frames from a single run, straight off the canvas. Blown supercharger spits fire down the straight A raider hauls the war-rig in by the chain Tires tear through the canyon floor A lone rider guns it through the ruins Last repairs before the run The getaway car breaks loose across the flats Make it yours Nothing here is locked. Swap models, change the tone, or point it at a different story. Swap the video model. Veo, Seedance, Kling, Runway. Change one node and the treatment, shots, and key art stay exactly the same. Redirect the showrunner. Rewrite the agent’s system prompt for horror, comedy, or arthouse without touching the pipeline. Restyle every shot. The visual-style input flows into each key-art prompt. Change one line and the whole trailer shifts mood. Re-run any story. Drop in a new logline and run it again. The workflow is the reusable part, not this trailer. Models in this workflow Called with your own keys. The bill comes from the provider, and you can switch any of them for a better model the day it ships. Model Role Provider Gemini 3.1 Pro Preview Writes the treatment and shot list Gemini GPT Image-2 Renders each shot’s key art kie Veo 3.1 Preview Animates the frames into video Gemini See [Models & Providers]({{ ‘/models-and-providers’ relative_url }}) to set up keys. Next steps [Product Video Generator]({{ ‘/use-cases/product-video’ relative_url }}) — one product photo to a cinematic clip [All use cases]({{ ‘/use-cases’ relative_url }}) </content> </invoke>--- # Podcast Repurposing Studio Source: https://docs.nodetool.ai/use-cases/podcast-repurposing-studio.html Use case · Creators Drop in one episode and ship the whole content pack: titles and show notes, a newsletter edition, five social posts, and quote cards rendered as square images. This is the work podcasters rent per-seat repurposing tools for, or skip entirely — here it’s one canvas that already knows your show’s voice. An episode’s value doesn’t stop at the RSS feed: the episode page, the email list, and the social feeds each need their own version of it, every week. That recurring rewrite is exactly what this graph automates — transcribe once, fan out to every format. How it works One transcription, four writer branches. Everything downstream of the transcript runs in parallel. {% mermaid %} graph LR audio[“Episode audio (Audio)”] show[“Show context (String)”] count[“Quote count (Integer)”] asr[“Transcribe (ASR)”] notes[“Show notes (Agent)”] news[“Newsletter (Agent)”] posts[“Social posts (ListGenerator)”] quotes[“Quotes (ListGenerator)”] cards[“Quote cards (TextToImage)”] audio –> asr asr –> notes asr –> news asr –> posts asr –> quotes show –> notes show –> news show –> posts count –> quotes quotes –> cards {% endmermaid %} Feed in the episode. The recording, a line of show context (name, host, voice, call to action), and how many quote cards you want. Transcribe once. A speech-recognition model turns the episode into a transcript that every branch reads from. Write the episode page. An agent drafts three title options, a summary, chapter-style show notes, a resource list, and the closing CTA. Draft the newsletter. A second agent writes the email edition in the host’s voice — subject-line options and a body that teases instead of summarizes. Generate the posts. A list generator writes five standalone social posts in mixed formats: a bold take, a question, a mini-story, a stat, a lesson. Render the quote cards. Another list generator pulls the most quotable verbatim lines, and each one becomes a square typographic card via a text-to-image model. One recording, a week of content Everything reads from the same transcript, so the pack is consistent: the newsletter teases what the show notes explain, and the quote cards say it in the host’s own words. Run it as the last step of every edit session and publishing day becomes an upload, not a writing shift. Make it yours Tune each voice separately. Show notes, newsletter, and posts each have their own prompt and system prompt — make the newsletter personal and the posts punchy without compromise. Swap the transcription model. Whisper variants, local or hosted — the branch prompts don’t change. Restyle the cards. The quote-card prompt controls typography, palette, and format. Match your cover art, or go 9:16 for stories. Add branches. A YouTube description, a LinkedIn essay, a blog post — each is one more Prompt → Agent pair reading the same transcript. Models in this workflow The template ships with no models selected — pick one per role when you open it. Called with your own keys; the bill comes from the provider. Role Node Works well with Transcription Automatic Speech Recognition Whisper (hosted or local) Show notes, newsletter Agent Any strong language model Posts, quotes List Generator Any language model Quote cards Text To Image A model with reliable text rendering (Nano Banana, GPT Image) See [Models & Providers]({{ ‘/models-and-providers’ relative_url }}) to set up keys. Next steps Open the template: Templates → Podcast Repurposing Studio in the dashboard [Ad Creative Factory]({{ ‘/use-cases/ad-creative-factory’ relative_url }}) — the same fan-out, pointed at paid social [Transcribe Audio]({{ ‘/workflows/transcribe-audio’ relative_url }}) — the transcription building block on its own [All use cases]({{ ‘/use-cases’ relative_url }})--- # Product Video Generator Source: https://docs.nodetool.ai/use-cases/product-video.html Use case · Marketing Turn a campaign brief and a single product photo into a cinematic 16:9 product video. Your inputs feed a prompt, an agent directs the shot, and a text-to-video model renders a ready-to-post clip. How it works Four inputs and four nodes: a brief, an audience, a feature list, and the hero photo go in; a finished clip comes out. {% mermaid %} graph LR brief[“Brief (String)”] audience[“Audience (String)”] features[“Features (String)”] photo[“Product photo (Image)”] prompt[“Prompt”] agent[“Director (Agent)”] i2v[“Render (ImageToVideo)”] brief –> prompt audience –> prompt features –> prompt prompt –> agent photo –> agent agent –> i2v photo –> i2v {% endmermaid %} Feed in the campaign. Three text inputs hold the brief, the target audience, and the key features. A product photo goes in alongside them as the hero shot. (e.g. “Aurora Trail smart fitness watch · active millennials who hike · GPS, heart-rate, adaptive coaching, water resistance”) Assemble the prompt. A Prompt node templates those inputs into a precise, rule-based request: one camera move, one subject action, lighting, color anchors, and hard constraints (no on-screen text, logos, or watermarks). Direct the shot. An agent turns the request into a concrete, cinematic prompt with framing, lens, and motion cues — the part most people get wrong by hand. Render the video. A text-to-video model animates the product photo from the agent’s prompt into a finished, ready-to-post clip. One photo in, a clip out The whole pipeline runs from a single product render. Same graph, any product. Input · product photo Output · cinematic clip Make it yours Swap the video model. Veo, Seedance, Kling, Runway. Change one node, the rest of the graph stays exactly the same. Retune the agent. Rewrite the system prompt for a luxury, playful, or hyper-technical tone without touching the pipeline. Change the format. Aspect ratio, resolution, and duration all live on the video node. Go vertical for social in one click. Reuse for any product. Drop in a new photo and brief, run it again. The workflow is the reusable part, not the output. Models in this workflow Called with your own keys. The bill comes from the provider, and you can switch any of them for a better model the day it ships. Model Role Provider Gemini 3.1 Pro Preview Writes the video prompt Gemini Veo 3.1 Preview Renders the clip from the product photo Gemini See [Models & Providers]({{ ‘/models-and-providers’ relative_url }}) to set up keys. Next steps [Movie Trailer Generator]({{ ‘/use-cases/movie-trailer’ relative_url }}) — one logline to a cut teaser [Color Boost Video]({{ ‘/workflows/color-boost-video’ relative_url }}) — polish footage on the canvas [All use cases]({{ ‘/use-cases’ relative_url }}) </content> </invoke>--- # SEO Content Engine Source: https://docs.nodetool.ai/use-cases/seo-content-engine.html Use case · Content One topic in, a keyword-targeted article batch out. A strategist agent plans the cluster, and every article arrives written, structured for search, and paired with an editorial hero image — the deliverable content agencies bill per article, produced by the batch at provider cost. Ranking rarely comes from one article. Search engines reward clusters — several pieces that own a topic together, each answering a different intent. Producing a cluster is what makes content expensive; this graph makes it one run. How it works The strategist plans once; everything after the list generator runs once per article. {% mermaid %} graph LR business[“Business & site (String)”] seeds[“Audience & seeds (String)”] count[“Article count (Integer)”] strategist[“Strategist (Agent)”] briefs[“Briefs (ListGenerator)”] writer[“Writer (Agent)”] hero[“Hero image (TextToImage)”] business –> strategist seeds –> strategist count –> strategist strategist –> briefs briefs –> writer briefs –> hero {% endmermaid %} Feed in the campaign. Who’s publishing and which pages the articles support, who searches and the seed topics, and how many articles to produce. (e.g. “ultralight hiking shop · beginner hikers · trail runners vs boots · 4 articles”) Plan the cluster. A strategist agent picks the topic cluster, assigns one primary keyword per article with its search intent, and divides coverage so the articles don’t compete with each other. Write the briefs. A list generator turns the plan into one brief per article: title, keyword, intent, H2 sections, and what the reader must be able to do afterward. Write the articles. For each brief, a writer agent produces the full Markdown piece — meta description, answer-first opening, the brief’s section structure, an FAQ, and one unforced CTA. Render the heroes. In parallel, each brief becomes a 16:9 editorial hero image with room left for a headline overlay. One run, one cluster Because every article comes from the same plan, the batch behaves like a cluster instead of a pile: keywords don’t cannibalize, internal logic is consistent, and the voice section of the plan keeps ten articles sounding like one publication. Point the seed inputs at the next cluster and run it again. Make it yours Retune the writer. The article rules — length, structure, point of view, FAQ — are a prompt, not a setting. Make it terse and technical or warm and story-led. Raise the stakes per article. Swap the writer’s model up for money pages, down for supporting pieces. Change the imagery. The hero prompt controls the visual language — photography, flat illustration, product-in-context. Chain the next step. Add a Save Text node per article, or a translation branch to ship each cluster in every market’s language. Models in this workflow The template ships with no models selected — pick one per role when you open it. Called with your own keys; the bill comes from the provider. Role Node Works well with Cluster strategist Agent Any strong language model Brief writer List Generator Any language model Article writer Agent A long-output model you trust with prose Hero images Text To Image FLUX, Imagen, Nano Banana See [Models & Providers]({{ ‘/models-and-providers’ relative_url }}) to set up keys. Next steps Open the template: Templates → SEO Content Engine in the dashboard [Ad Creative Factory]({{ ‘/use-cases/ad-creative-factory’ relative_url }}) — paid acquisition from the same brief-style inputs [Podcast Repurposing Studio]({{ ‘/use-cases/podcast-repurposing-studio’ relative_url }}) — the content pack for what you’ve already recorded [All use cases]({{ ‘/use-cases’ relative_url }})--- # NodeTool User Interface Source: https://docs.nodetool.ai/user-interface.html Tour of the interface — Dashboard, Canvas, Chat, Mini-Apps, Assets. Same on desktop and in the browser. New here? Start with Getting Started, then come back. Main Views The primary destinations across the web app and desktop. Each entry links to its detailed docs page. Dashboard — /dashboard The home screen. Search, recent workflows, templates, and quick chat. Docs: Getting Started Workflow Editor — /workspace The main visual editor. Build workflows by connecting nodes on an infinite canvas, with panels on every edge: left drawer, right inspector, bottom diagnostics, floating toolbar, node menu, and tabs. (The legacy /editor/:workflow URL redirects into this unified /workspace shell.) Docs: Workflow Editor · Editor Panels Chain Editor — /chain/:workflowId? A linear, card-based alternative to the node graph. Better for simple pipelines and guided authoring. Docs: Chain Editor Chat — /chat/:thread_id? Conversational AI with multi-thread history, an always-on agent loop, tools, and workflow integration. Docs: Chat Mini-Apps — /apps/:workflowId? Run saved workflows through simplified form UIs. Mini-apps can also be launched as standalone frameless windows from the desktop tray. Docs: Mini-Apps · Electron Mini-App Window Asset Explorer — /assets Browse, search, organize, and tag every file used in your workflows. Opens the full-featured Sketch Editor for image assets. Docs: Asset Management · Sketch Editor Video Editor — /timeline/:sequenceId A generation-aware timeline editor. Sequence video, audio, and image clips across multiple tracks — clips can be imported media or live workflow outputs that regenerate when you change their parameters. Composite a preview in real time and export the whole timeline to MP4. Docs: Video Editor Collections — /collections Group related documents into indexable collections for RAG workflows. Docs: Collections · Indexing Examples / Templates — /examples Ready-to-use example workflows organized by tag and use case. Templates also surface on the Dashboard. Docs: Templates Gallery Models Manager — /models Find, install, filter, and manage local and cloud AI models. Docs: Models Manager Settings — Dialog Central configuration surface: general preferences, provider API keys, folders, secrets, remote, and about. Docs: Configuration · Models & Providers Mobile Touch-optimized Dashboard, Chat, and Graph Editor. See Mobile App for the full set. Desktop (Electron) The desktop app shares all the views above, plus an Install Wizard, System Tray, and frameless mini-app windows. See Desktop App Views. At a glance Five workspaces: Workspace Purpose Dashboard Home, templates, recent projects Canvas Build and run workflows Chat Chat with models, run agents Mini-Apps Run workflows behind a simple UI Assets Files and media The App Menu The logo at the top of the left rail opens the app menu — your navigation hub: Dashboard – Home screen Examples – Browse ready-to-run example workflows Costs – Usage and cost tracking Model Manager – Manage your AI models (Flux, Qwen Image, etc.) Collections – RAG document collections Workspaces – Configure workspace folders (when enabled) Settings – Configure API keys, preferences, account Help – Documentation and support Downloads – Model/asset download progress Dashboard The Dashboard is the home screen. Contents Your Workflows – Saved projects Templates – Ready-to-use workflows Recent Chats – Past conversations Getting Started Panel – Interactive onboarding guide for new users with step-by-step instructions Common Actions Task How to Do It Start a new creation Click “New Workflow” button Open a saved workflow Click any workflow card Try a template Browse Templates, click to open Continue a chat Click a recent chat thread Workflow Canvas Build workflows visually by connecting nodes. The Canvas The center area is an infinite canvas for arranging workflows. Navigation: Pan: Hold Space and drag, or right-click drag Zoom: Ctrl/⌘ + scroll wheel Fit all: Press F Adding Nodes To add a node: Press Space anywhere on canvas, OR Double-click an empty area This opens the Node Library: Search by typing (e.g., “generate image”, “transform video”) Browse categories on the left (Image, Video, Audio, Text) Click a node to add it Connecting Nodes Connections show data flow between nodes: Find the circles on nodes (outputs on right, inputs on left) Click and drag from an output circle Release on an input circle of another node Tip: Drop a connection on empty space to see compatible nodes. Properties Panel When you select a node, the right panel shows its configuration: Inputs – What the node needs Settings – Configuration options Output – What the node produces Chat AI assistant built into NodeTool. Features Chat with AI models Run workflows from conversation Agent loop for autonomous, multi-step task execution File sharing – images, audio, documents Chat Features Feature Description Threads Multiple conversations, each with its own history Model Selector Choose which AI model to chat with Workflow Menu Attach and run your saved workflows Agent Loop The AI uses tools and modifies your canvas as each task needs Standalone Chat Window Access chat directly from the system tray for quick conversations: Quick Access: Click the tray icon → Chat to open a dedicated chat window Focused Interface: Chat without the full NodeTool interface Background Access: Start conversations while other apps are open Thread Persistence: All threads sync with the main application Mini-Apps Convert workflows into simple apps. Purpose Hide complexity – Users see only inputs and outputs Share easily – No NodeTool knowledge required Focused interface – Just what users need How It Works Build your workflow in the Editor Click Mini-App in the top-right See a clean interface with just: Input fields (from your Input nodes) Run button Output results Standalone Mini-App Windows Launch mini-apps in dedicated windows from the system tray: Quick Launch: Right-click the tray icon to see available mini-apps Independent Windows: Run mini-apps without opening the main editor Background Execution: Keep mini-apps running while working on other tasks Assets The Asset Explorer manages all your files. Supported Files Images: PNG, JPG, GIF, WebP Audio: MP3, WAV, M4A Video: MP4, MOV, WebM Documents: PDF, TXT, Markdown Working with Assets Action How Upload files Drag & drop into Assets panel Use in workflow Drag asset onto the canvas Preview Click any asset to preview Organize Create folders, rename files Audio Player The built-in audio player features waveform visualization: Visual Waveforms: See the audio shape with WaveSurfer.js integration Playback Controls: Play, pause, and seek through audio files Preview in Workflows: Audio results display with interactive waveforms Models Manager Download, organize, and configure AI models. Finding Models Search by name or task type Filter by provider (local, OpenAI, etc.) Sort by size or popularity Managing Downloads One-click install for any supported model Progress tracking in the header Space usage shown per model Easy uninstall to free space Panels and Layout NodeTool’s interface is flexible – customize it to your workflow. Rearranging Panels Move: Drag a panel tab to a new location Split: Drag a tab to the edge of another panel Resize: Drag the borders between panels Close: Click the X on any tab Restore: Use View menu to add closed panels back Saving Layouts Your layout is saved automatically. To reset: View → Reset Layout restores defaults Command Menu Press Ctrl+K (Windows/Linux) or ⌘+K (Mac) to open the Command Menu. This is the fastest way to: Open any workflow Switch between sections Search for anything Access settings Just start typing what you want! Keyboard Shortcuts Essentials Shortcut Action Space Open node menu Ctrl/⌘ + Enter Run workflow Ctrl/⌘ + S Save Ctrl/⌘ + Z Undo F Fit view Esc Stop workflow All Shortcuts Global Shortcut Action Ctrl+K / ⌘+K Command Menu Ctrl/⌘+N New workflow Ctrl/⌘+O Open workflow Ctrl/⌘+S Save Ctrl/⌘+Z Undo Ctrl/⌘+Shift+Z Redo Ctrl/⌘+1…9 Switch tabs Editor Shortcut Action Space + Drag Pan Ctrl/⌘ + Scroll Zoom F Fit to screen Ctrl/⌘+Enter Run workflow Esc Stop workflow Ctrl/⌘+D Duplicate Ctrl/⌘+G Group A Align nodes Delete / Backspace Delete selection Chat Shortcut Action Enter Send message Shift+Enter New line Esc Stop generation Next Steps Workflow Editor deep dive – Master the canvas Editor Panels – Left, right, bottom, and floating panels Tips & Tricks – Power user secrets Cookbook – Learn workflow patterns--- # Vector Storage Source: https://docs.nodetool.ai/vector-storage.html NodeTool stores embeddings through a pluggable provider abstraction in @nodetool-ai/vectorstore. The same VectorProvider / VectorCollection interface backs the local SQLite-vec store, Supabase/pgvector, and (stubbed) Pinecone, so every caller — base nodes, the tRPC collections router, the file-upload REST handler, and agent tools — works identically against any backend. Choosing a Backend Backend When to use Setup sqlite-vec (default) Local-first, single-machine deployments. Embedded, zero-config. None — first use creates vectorstore.db in the NodeTool data dir. supabase Hosted multi-user deployments backed by a Supabase / Postgres+pgvector project. Install packages/vectorstore/sql/supabase-migration.sql once on the project, then set SUPABASE_URL + SUPABASE_KEY. pinecone Stub — surfaces a clear notImplemented error today. — Selection happens in createVectorProviderFromEnv() (packages/vectorstore/src/provider-factory.ts), which is called by getDefaultVectorProvider() on first use. Configuration The active backend is chosen by NODETOOL_VECTOR_PROVIDER. Defaults to sqlite-vec if unset. NODETOOL_VECTOR_PROVIDER=sqlite-vec (default) Env var Purpose VECTORSTORE_DB_PATH Override the SQLite database file. Defaults to vectorstore.db in the NodeTool data dir (getDefaultVectorstoreDbPath() in @nodetool-ai/config). The provider re-uses the process-wide getDefaultStore() so any code that touches the lower-level SqliteVecStore API and any code that goes through getDefaultVectorProvider() share one connection. NODETOOL_VECTOR_PROVIDER=supabase Env var Purpose SUPABASE_URL Supabase project URL (e.g. https://xyz.supabase.co). SUPABASE_KEY or SUPABASE_SERVICE_ROLE_KEY API key. Service-role is recommended server-side; anon-key requires RLS policies that grant the role access to the registry/records tables. NODETOOL_VECTOR_SCHEMA Optional Postgres schema; defaults to public. These match the env vars the Supabase storage backend already uses, so a single Supabase project can back both file storage and vector data. One-time SQL install The provider does no DDL itself — it expects the schema to exist already. Run packages/vectorstore/sql/supabase-migration.sql once via the Supabase SQL editor or supabase migration. It creates: nodetool_vec_collections — registry table (name PK, metadata jsonb, dimension int, metric text). nodetool_vec_records — shared records table (collection FK, id, document, embedding vector, uri, metadata jsonb). The FK has ON UPDATE CASCADE so renaming a collection follows automatically — no record rewrite. nodetool_vec_match — RPC for similarity search. Required because PostgREST cannot use pgvector operators (<=>, <->, <#>) directly. Once the dimension is fixed for a collection, you can add an ivfflat index for faster search — the migration file documents the exact statement. Filter Support Matrix VectorFilter is a MongoDB-style predicate language. Adapters translate the subset they can support and must throw UnsupportedFilterError for anything else, rather than silently dropping conditions. Operator sqlite-vec supabase {field: scalar} (equality) — yes (via jsonb @>) {field: { $eq: v }} — yes {field: { $ne | $gt | $gte | $lt | $lte: v }} — — {field: { $in: [...] }} — — {$and: [...]} — yes (flattens into the metadata predicate) {$or: [...]} — — {$document: { $contains: "text" }} yes yes {$document: { $or: [{$contains}, ...] }} yes — If you need a richer filter on Supabase, extend the SQL function and the splitFilter() translator in packages/vectorstore/src/supabase-provider.ts together — the JS-side translator is the contract. Programmatic API import { getDefaultVectorProvider, type VectorCollection } from "@nodetool-ai/vectorstore"; const provider = getDefaultVectorProvider(); // Create / get a collection. const docs = await provider.getOrCreateCollection({ name: "support_articles", dimension: 1536, metric: "cosine", metadata: { embedding_model: "text-embedding-3-small" } }); // Upsert records — embeddings can be pre-computed or generated by the // collection's embedding function. await docs.upsert([ { id: "kb-101", document: "How to reset your password", uri: "https://…" } ]); // Vector + metadata + document filtering. const matches = await docs.query({ text: "I forgot my password", topK: 5, filter: { $document: { $contains: "password" } } }); For tests or migrations, swap the active provider with setDefaultVectorProvider(...) (and resetDefaultVectorProvider() to release it). When constructing SupabaseProvider directly, pass a pre-built client to mock the network layer: import { SupabaseProvider } from "@nodetool-ai/vectorstore"; const provider = new SupabaseProvider({ client: fakeSupabaseClient // any object satisfying SupabaseClient }); Where it’s used Workflow nodes — every node under vector.* in @nodetool-ai/core-nodes (packages/core-nodes/src/nodes/vector.ts) goes through the default provider. Collections REST/tRPC API — packages/websocket/src/trpc/routers/collections.ts and packages/websocket/src/collection-api.ts. Agent tools — VecTextSearchTool, VecHybridSearchTool, VecRecursiveSplitAndIndexTool, etc. in packages/agents/src/tools/vector-tools.ts. Adding a New Backend Implement VectorProvider and VectorCollection (packages/vectorstore/src/provider.ts) in a new <backend>-provider.ts. Translate VectorFilter to the backend’s query language. Throw UnsupportedFilterError for predicates you can’t express — never silently drop. Wire it into createVectorProviderFromEnv() in provider-factory.ts under a new kind. Add a tests file under tests/ that injects a mock client (no live backend). Re-export the provider class and its options interface from src/index.ts. The existing SqliteVecProvider and SupabaseProvider are good references — keep adapter state mutable so modify({ name, metadata }) is observable on the same handle, and surface uri on every read path (it’s part of the VectorRecord / VectorMatch contract).--- # Video Editor Source: https://docs.nodetool.ai/video-editor.html Cut a sequence on a multi-track timeline, bind any workflow to a clip, and generate the footage in place. NodeTool’s Video Editor is a non-linear editor where clips can be imported media or live workflow outputs that regenerate when you change their parameters. Quick Access: Open a timeline at /timeline/:sequenceId, or add a timeline tab from the workspace. Create a new sequence from the timeline list panel or the Asset Explorer. Overview The Video Editor (timeline editor) is a non-linear, multi-track surface for assembling video, audio, and image clips. What sets it apart from a conventional NLE is that clips can be bound to NodeTool workflows — a clip can be the output of a text-to-image, image-to-video, or text-to-speech pipeline, and it stays editable: change a parameter and regenerate just that clip. Features: Multi-track timeline — stack video, audio, and overlay tracks Imported clips (drag media from the Asset Explorer) and AI-generated clips side by side Authored text and shape clips with preset-based motion Clip editing — move, trim, split, duplicate, delete, with snap-to-playhead and snap-to-clip alignment Real-time preview compositing with a GPU compositor (WebGPU, Canvas2D fallback) and WebAudio mixing Frame-accurate transport — play/pause, stop, frame-step, skip to clip boundaries, timecode readout Generated clips bound to any workflow, with parameters exposed in the Inspector Staleness tracking — edit a bound workflow and the clip is flagged for regeneration Version history per clip — keep the last N successful generations and restore them Offline export — render the whole timeline to MP4 at the sequence resolution Full undo/redo and debounced autosave Where this fits The Video Editor is the surface where outputs become finished, time-based media. It pulls assets in as clips, binds workflows to generated clips so footage regenerates when parameters change, and exports the sequence back to a video asset — the same material a sketch, another workflow, or a Mini-App can pick up. Every surface shares one asset store and the same model/provider system. See Key Concepts → How everything fits together for the full loop. Opening the Video Editor There are a few ways in: Direct route — navigate to /timeline/:sequenceId. Workspace tab — open a timeline as a tab alongside your workflows in /workspace. New sequence — create one from the timeline list panel or the Asset Explorer. Each timeline (whether a standalone page or a workspace tab) runs in its own isolated set of stores, so multiple timelines can be open at once without interfering. Anatomy of the Editor Region What it does Top bar Sequence name, Save, Export, and an activity indicator for running generations Preview The composited video with transport controls, timecode, and FPS readout Tracks The multi-track timeline — a canvas-rendered ruler, the playhead, and the clips Inspector Per-clip controls for transforms, motion, authored content, and generated clip nodes Tracks & Clips The timeline holds multiple tracks. Video and overlay tracks composite top-down; audio tracks mix together. Clip operations: Add — drag media from the Asset Explorer onto a track, or add a generated clip from the add menu. Select — click a clip; Shift/Ctrl-click to multi-select. Move / trim — drag the body to move, drag the edges to trim the in/out points. Split — position the playhead and press S to cut a clip in two. Duplicate / delete — Ctrl/⌘ + D to duplicate, Delete to remove. Snapping — clips snap to the playhead and to neighbouring clip boundaries for clean alignment. Linked clips that share a link (for example a video and its extracted audio) move and trim together. Preview & Playback The preview composites every visible, unmuted track in real time. Transport — play/pause, stop, step one frame forward/back, and skip to the previous/next clip boundary. Timecode — frame-accurate HH:MM:SS:FF readout, with the sequence FPS shown alongside. Scrubbing — drag the playhead across the ruler to scrub. Zoom — Ctrl/⌘ + scroll zooms the timeline anchored at the cursor; a zoom slider sits in the status bar. Audio — per-clip gain and fades, per-track mute/solo and volume, mixed down to a master output. Audio clips render a waveform in the clip body. Video and audio are kept in sync by driving playback against the audio clock. Generated Clips A clip doesn’t have to be a file — it can be the output of a workflow. Generation modes: Text-to-Image — generate a still from a prompt. Image-to-Video — animate an image. Text-to-Speech / Text-to-Audio — synthesize an audio clip. Workflow — bind any NodeTool workflow to the clip. When you bind a workflow, NodeTool clones it into a clip-private variant and exposes the workflow’s Input* nodes as editable parameters in the Inspector — so you can tweak the prompt, seed, or any input and regenerate without leaving the timeline. Staleness & versions: Each clip tracks a content hash of the bound workflow plus its parameter overrides. Edit the workflow (or its inputs) and the clip is flagged stale. A clip moves through states: draft → queued → generating → generated, plus stale, failed, locked, and missing. Successful generations are kept as versions — restore, favourite, or delete previous results per clip. Generation runs on the standard WorkflowRunner; status streams back over the WebSocket connection keyed by job, so the timeline only listens to the jobs it cares about. Round-tripping to the node editor: from a generated clip you can Open in Node Editor to edit the bound workflow on the full canvas, then return — the clip auto-marks stale so you can regenerate with the new logic. Inspector The Inspector swaps based on what’s selected: Imported clip — asset info, in/out points, transform, opacity, speed, and volume controls, plus a Replace Media action. Generated clip — a vertical node stack of the bound workflow’s nodes, each with its parameters (reusing NodeTool’s property fields) and a per-node status indicator. A clip-actions menu covers Generate, Regenerate, Generate Stale, Duplicate as Variation, Open in Node Editor, Lock, and Revert, alongside the version list. Text or shape clip — content and appearance controls for text, fill, stroke, geometry, and corner radius. Every visual clip includes an Animate section. Add an entrance, exit, emphasis, or loop preset, then adjust its timing, easing, and preset parameters. Effects Clips and tracks carry an effects chain that runs in the same compositor as the live preview: Video — color correction, blur, sharpen, vignette, chroma key. Audio — gain, 3-band EQ, filter (lowpass/highpass/bandpass), compressor. Clip transforms — opacity, blend mode (normal, screen, multiply, add, overlay), speed, and fade-in/out. Animations Motion presets add animation without a keyframe editor. Select a visual clip, open Animate in the Inspector, choose a role and preset, then adjust duration or period, delay, easing, and preset parameters. In / Out — fade, slide, pop, spin, wipe, blur, and colorFade at a clip boundary. Wipe reveals the clip behind a directional mask with an adjustable feathered edge; blur racks from soft to sharp; colorFade blooms grayscale into full color. Emphasis / Loop — pulse, flash, shake, bounce, float, breathe, spin, or Ken Burns motion while the clip is active. Word stagger — on text clips, any animation can run once per word, each word offset in time from the previous (offset and on/off in the inspector; stagger on ui_timeline_animate_clip for the agent). Words pop, slide, or fade in sequence — motion typography without keyframes. See plans/motion-typography.md. Track markers — shaded edge wedges show entrance and exit windows; a loop icon marks emphasis or repeating motion. Select a marker to open its controls. Agent authoring — timeline tools can list presets, apply or clear motion, and inspect fixed-time frames before export. Preview and export sample the same animation resolver, so transforms and opacity match at each frame. Export Export renders the entire timeline to an MP4 at the sequence resolution. Rendering is frame-by-frame through the same GPU compositor used for preview, so the export matches what you see. Audio is mixed offline and muxed with the encoded video. A progress dialog reports the phase (audio mix → video encode → finalize) and the export can be cancelled. Keyboard Shortcuts Shortcut Action Space Play / pause S Split clip at playhead Delete / Backspace Delete selected clip(s) Ctrl/⌘ + C / X / V Copy / cut / paste clips Ctrl/⌘ + D Duplicate selected clip(s) Ctrl/⌘ + Z Undo Ctrl/⌘ + Shift + Z Redo Ctrl/⌘ + scroll Zoom timeline (anchored at cursor) Common Workflows Assemble a rough cut from your assets Open a new sequence and drag clips from the Asset Explorer onto the video track. Trim and reorder clips; use S to split and Delete to remove. Add an audio track for music or narration and adjust per-clip volume and fades. Scrub the preview to check timing, then Export to MP4. Generate a shot in place Add a generated clip and choose Image-to-Video (or bind a custom workflow). In the Inspector, set the prompt and inputs exposed from the workflow. Generate — the clip streams through queued → generating → generated. Not quite right? Tweak a parameter and Regenerate, or Duplicate as Variation to compare versions. Iterate on a bound workflow Select a generated clip and Open in Node Editor. Edit the workflow on the full canvas and save. Return to the timeline — the clip is flagged stale. Generate Stale to refresh it with the new logic. Related Features Workflow Editor — build the workflows you bind to generated clips Asset Management — organize the media you import as clips Sketch Editor — edit stills before or after they land on the timeline User Interface — tour of the main NodeTool views Next Steps Bind one of your existing workflows to a clip and generate a shot. Build a multi-track sequence mixing imported footage with generated clips. Explore the Cookbook for workflow patterns you can drop onto the timeline. Last updated: July 2026--- # WebSocket API Source: https://docs.nodetool.ai/websocket-api.html This document describes the WebSocket API used to run workflows, stream results, and receive real-time updates from the NodeTool backend. Overview NodeTool exposes a single WebSocket endpoint for workflow execution and live updates. Clients connect once and multiplex commands and responses for many concurrent workflows over that connection. Endpoint: ws(s)://<host>/ws Auth: Include a bearer token as a query parameter (?api_key=<token>) or rely on the same auth flow used by REST endpoints. Tokens are optional in local/none auth modes. Protocol: Binary (MessagePack) or Text (JSON) frames. The server auto-detects frame type. See examples/workflow_runner/js/workflow-runner.js for a complete client implementation used by the bundled runner UI. Encoding The protocol supports two encoding modes on the same connection: Mode Frame type When to use Binary (default) Binary WebSocket frame MessagePack-encoded. Preferred for production use — compact format and native binary data (images, audio) without Base64 overhead. Text Text WebSocket frame JSON-encoded. Useful for debugging, curl/websocat testing, and lightweight clients that don’t need binary payloads. Binary frames are decoded with MessagePack; text frames are decoded as JSON. Both modes are interchangeable — the server accepts either and responds in the same format. Every message is a map/object with at least a type field. Connection Lifecycle Connect — open a WebSocket to ws://<host>/ws (or wss:// for TLS). For binary mode set binaryType = "arraybuffer". Ready — the onopen event fires; the client can now send commands. Reconnect — if the connection drops, retry with exponential backoff (the reference client retries up to 10 times starting at 1 s). Close — call socket.close() or let the server close the connection. Client → Server Commands All client messages contain command and data fields. run_job Start a new workflow execution. { "command": "run_job", "data": { "type": "run_job_request", "api_url": "http://localhost:7777/api", "workflow_id": "<uuid>", "job_type": "workflow", "auth_token": "<token>", "params": { "<input_name>": "<value>" }, "job_id": "<uuid>", "user_id": "<user_id>", "graph": { "nodes": [], "edges": [] }, "explicit_types": false, "execution_strategy": "threaded", "resource_limits": null } } Field Type Description type "run_job_request" Constant discriminator api_url string Base URL of the REST API workflow_id string UUID of the workflow to run job_type string Always "workflow" for workflow runs auth_token string Bearer token for authentication params object Input parameter values keyed by input name job_id string \| null Optional client-generated UUID to track the job user_id string User identifier graph object \| null Optional graph override ({ nodes, edges }) explicit_types boolean When false, let the server infer types execution_strategy string "threaded" (default) or "subprocess" resource_limits object \| null Optional resource constraints cancel_job Cancel a running job. { "command": "cancel_job", "data": { "job_id": "<uuid>", "workflow_id": "<uuid>" } } resume_job Resume a paused or suspended job. { "command": "resume_job", "data": { "job_id": "<uuid>", "workflow_id": "<uuid>" } } reconnect_job Reconnect to an in-flight job (e.g. after a page reload). The server replays any missed updates. { "command": "reconnect_job", "data": { "job_id": "<uuid>", "workflow_id": "<uuid>" } } stream_input Push a value into a streaming input node while a job is running. { "command": "stream_input", "data": { "job_id": "<uuid>", "workflow_id": "<uuid>", "input": "<input_name>", "value": "<any>", "handle": "<handle_name | null>" } } end_input_stream Signal that a streaming input is complete. { "command": "end_input_stream", "data": { "job_id": "<uuid>", "workflow_id": "<uuid>", "input": "<input_name>", "handle": "<handle_name | null>" } } update_node_properties Update a node’s properties on a running or pending job. { "command": "update_node_properties", "data": { "job_id": "<uuid>", "workflow_id": "<uuid>", "node_id": "<uuid>", "properties": { "<name>": "<value>" } } } clear_models Ask the server to unload cached models and free memory. { "command": "clear_models", "data": {} } Server → Client Messages Every server message contains a type field used for dispatch. Messages also include routing fields (workflow_id, job_id, or thread_id) so the client can multiplex updates across concurrent workflows. job_update Reports overall job status changes. { "type": "job_update", "status": "running", "job_id": "<uuid>", "workflow_id": "<uuid>", "message": "optional status text", "error": null, "traceback": null, "result": null, "duration": null, "run_state": null } Field Type Description status string "queued", "running", "completed", "failed", "timed_out", "cancelled", "suspended", or "paused" job_id string \| null Job UUID workflow_id string \| null Workflow UUID for routing message string \| null Human-readable status message error string \| null Error description on failure traceback string \| null Python traceback on failure result object \| null Final result map on completion duration number \| null Execution duration in seconds run_state object \| null Extended state info (e.g. suspension reason) node_update Reports per-node status changes during execution. { "type": "node_update", "node_id": "<uuid>", "node_name": "GenerateImage", "node_type": "nodetool.image.Generate", "status": "running", "error": null, "result": null, "properties": null, "workflow_id": "<uuid>" } Field Type Description node_id string Node UUID node_name string Display name of the node node_type string Fully qualified node type status string "booting", "starting", "running", "completed", or "error" error string \| null Error message if the node failed result object \| null Node output on completion properties object \| null Updated node properties workflow_id string \| null Workflow UUID for routing node_progress Reports progress for long-running nodes (e.g. image generation steps). { "type": "node_progress", "node_id": "<uuid>", "progress": 3, "total": 20, "chunk": "", "workflow_id": "<uuid>" } Field Type Description node_id string Node UUID progress number Current step total number Total steps chunk string Optional text chunk for streaming output workflow_id string \| null Workflow UUID for routing output_update Delivers a final output value from an output node. { "type": "output_update", "node_id": "<uuid>", "node_name": "ImageOutput", "output_name": "image", "value": { "type": "image", "data": "<binary>" }, "output_type": "image", "metadata": {}, "workflow_id": "<uuid>" } Field Type Description node_id string Node UUID node_name string Display name output_name string Name of the output handle value any The output value (see Value Types) output_type string Type descriptor (e.g. "image", "string") metadata object Additional metadata workflow_id string \| null Workflow UUID for routing edge_update Reports data flow status on a connection between nodes. { "type": "edge_update", "workflow_id": "<uuid>", "edge_id": "<edge_id>", "status": "active", "counter": 5 } Field Type Description workflow_id string Workflow UUID edge_id string Edge identifier status string Edge status counter number \| null Number of items that have passed through log_update Streams log output from a running node. { "type": "log_update", "node_id": "<uuid>", "node_name": "RunModel", "content": "Loading model weights...", "severity": "info", "workflow_id": "<uuid>" } Field Type Description node_id string Node UUID node_name string Display name content string Log text severity string "info", "warning", or "error" workflow_id string \| null Workflow UUID for routing notification Server-initiated notification to display to the user. { "type": "notification", "node_id": "<uuid>", "content": "GPU memory low", "severity": "warning", "workflow_id": "<uuid>" } Field Type Description node_id string Originating node UUID content string Notification text severity string "info", "warning", or "error" workflow_id string \| null Workflow UUID for routing planning_update Reports agent planning phases. { "type": "planning_update", "phase": "analyzing", "status": "running", "node_id": "<uuid>", "content": "Determining approach...", "workflow_id": "<uuid>" } task_update Reports agent task progress. { "type": "task_update", "task": { "...task object..." }, "event": "started", "node_id": "<uuid>", "workflow_id": "<uuid>" } tool_call_update Reports when an agent node invokes a tool. { "type": "tool_call_update", "name": "web_search", "args": { "query": "example" }, "node_id": "<uuid>", "tool_call_id": "<id>", "workflow_id": "<uuid>" } tool_result_update Delivers the result of a tool call. { "type": "tool_result_update", "node_id": "<uuid>", "result": { "...result data..." }, "workflow_id": "<uuid>" } prediction Reports prediction/inference status from an external provider. { "type": "prediction", "id": "<prediction_id>", "node_id": "<uuid>", "status": "running", "user_id": "<user>", "workflow_id": "<uuid>", "logs": "Downloading model...", "error": null, "duration": null } chunk Streams incremental text/media content from a node. { "type": "chunk", "content": "Hello ", "content_type": "text", "content_metadata": {}, "done": false, "thinking": false, "node_id": "<uuid>", "workflow_id": "<uuid>" } Field Type Description content string Content fragment content_type string "text", "audio", "image", "video", or "document" content_metadata object Extra metadata for the content done boolean true on the final chunk thinking boolean true when the model is in reasoning/thinking mode node_id string \| null Node UUID workflow_id string \| null Workflow UUID for routing Value Types Output values are typically objects with a type discriminator: Type Shape Description Image { "type": "image", "data": <binary> } Raw image bytes (PNG) Audio { "type": "audio", "data": <binary> } Raw audio bytes (MP3/WAV) Video { "type": "video", "data": <binary> } Raw video bytes (MP4) Text "plain string" Simple string value Number 42 or 3.14 Numeric value Boolean true / false Boolean value Object { ... } Arbitrary JSON-like object Binary data in MessagePack frames is transmitted as raw byte arrays, avoiding Base64 overhead. In JSON/text mode, binary payloads are Base64-encoded strings. Message Routing The server tags each message with one or more routing keys: workflow_id — primary key for workflow execution updates. job_id — fallback when workflow_id is not present. thread_id — used for chat/conversation streams. The GlobalWebSocketManager in the main web app multiplexes a single connection and dispatches messages to per-workflow handlers based on these keys. The standalone workflow runner uses a simpler approach, handling all messages in a single callback. Typical Message Sequence Client Server | | |--- run_job ---------------------->| | | |<------------- job_update (queued) | |<------------ job_update (running) | | | |<---- node_update (node A running) | |<--- node_progress (node A 1/10) | |<--- node_progress (node A 5/10) | |<-- node_update (node A completed) | | | |<---- node_update (node B running) | |<-------------- output_update (B) | |<-- node_update (node B completed) | | | |<---------- output_update (final) | |<-------- job_update (completed) | | | Error Handling Connection errors trigger automatic reconnection with exponential backoff. A 120-second timeout is applied to each run_job call; if no terminal job_update arrives the promise is rejected. Server errors are delivered as job_update messages with status: "failed" and an error field, or as standalone error type messages. Per-node errors arrive via node_update with an error field set. Quick Start Examples Binary mode (MessagePack) const socket = new WebSocket("ws://localhost:7777/ws"); socket.binaryType = "arraybuffer"; socket.onopen = () => { socket.send( msgpack.encode({ command: "run_job", data: { type: "run_job_request", workflow_id: "YOUR_WORKFLOW_ID", auth_token: "YOUR_TOKEN", job_type: "workflow", params: { prompt: "hello world" } } }) ); }; socket.onmessage = (event) => { const data = msgpack.decode(new Uint8Array(event.data)); console.log(data.type, data); }; Text mode (JSON) const socket = new WebSocket("ws://localhost:7777/ws"); socket.onopen = () => { socket.send( JSON.stringify({ command: "run_job", data: { type: "run_job_request", workflow_id: "YOUR_WORKFLOW_ID", auth_token: "YOUR_TOKEN", job_type: "workflow", params: { prompt: "hello world" } } }) ); }; socket.onmessage = (event) => { const data = JSON.parse(event.data); console.log(data.type, data); };--- # Worker Deployment Source: https://docs.nodetool.ai/worker-deployment.html NodeTool runs most graphs on your machine. When a node needs a GPU you don’t have — large image/video models, HuggingFace pipelines — you can rent one for the duration of the work instead of buying hardware. A worker is a remote box running the lean NodeTool Python worker image; your local NodeTool instance attaches to it, runs the Python nodes there, and tears it down when you’re done. This is a different subsystem from server deployment. A server is long-lived infrastructure that humans connect into. A worker is an ephemeral, billing-sensitive box that one NodeTool instance connects out to. Because GPUs bill by the minute, teardown is the headline feature — see the cost guard below. The model: profiles → instances Two concepts, deliberately split:   Worker profile Worker instance What A declarative, reusable preset A live, running worker Lifetime Permanent until you delete it Ephemeral — spin up, attach, tear down Holds target, image, GPU/vCPU spec, token policy, idle timeout, max lifetime the provider’s pod/instance id, the wss:// URL, the bearer token, status, cost Stored worker_profiles table (DB) worker_instances table (DB) A profile is the recipe (“an A40 RunPod pod running the HuggingFace worker image, idle-stop after 15 minutes”). Provisioning a profile launches an instance. Instances are never written to deployment.yaml — nothing should ever be able to resurrect a torn-down GPU pod from declarative config. Both tables live in NodeTool’s SQLite DB so the UI and CLI share one source of truth. Prerequisites A RunPod account and API key (runpod.io console → settings), or a Vast.ai account and API key. A worker container image — the published NodeTool worker image, or your own built from a NodeTool Python package. It must run python -m nodetool.worker on port 7777 (msgpack RPC, bearer-token auth). Worker images Image Contents Use for ghcr.io/nodetool-ai/nodetool-worker:latest The lean Python worker Python nodes, HuggingFace pipelines, LLM providers ghcr.io/nodetool-ai/nodetool-worker-comfy:latest Worker + a co-located, loopback-only ComfyUI Everything above plus the Run ComfyUI Workflow (Worker) node A worker started from the ComfyUI image fronts ComfyUI over the worker bridge (ComfyUI itself is never exposed outside the container) and reports worker.status.comfy.enabled: true. The Run ComfyUI Workflow (Worker) node runs an API-format ComfyUI workflow on such a worker. Pick the ComfyUI image from the Worker image preset dropdown in the Profiles editor, or pass --image ghcr.io/nodetool-ai/nodetool-worker-comfy:latest on the CLI. Store the API key in the secret store so the manager can read it: nodetool secrets store RUNPOD_API_KEY # prompts for the value nodetool secrets store VAST_API_KEY # for Vast.ai If the secret store is unreachable (headless/sandboxed), the manager falls back to the RUNPOD_API_KEY / VAST_API_KEY environment variables. Supported targets Target Provider URL form Teardown runpod RunPod pod (REST rest.runpod.io/v1/pods) wss://<podid>-7777.proxy.runpod.net deletes the pod vast Vast.ai instance ws://<ip>:<port> destroys the instance Both run the same worker image — there is no per-provider image work. Local or LAN workers are also supported, but unmanaged: run the worker container yourself and point NODETOOL_WORKER_URL (and NODETOOL_WORKER_TOKEN) at it. There is no provisioning provider for local Docker — you start and stop it. Quick start (CLI) # 1. Store your provider API key nodetool secrets store RUNPOD_API_KEY # 2. Create a reusable profile nodetool worker profile add hf-a40 \ --target runpod \ --image ghcr.io/nodetool-ai/nodetool-worker:latest \ --gpu "NVIDIA A40" \ --idle-timeout 15 \ --max-lifetime 120 # 3. Provision an instance from it and attach in one step nodetool worker create --profile hf-a40 --attach # 4. Watch what's live (and what it's costing) nodetool worker list # 5. Tear it down when you're done nodetool worker stop <instance-id> # or, the panic button: nodetool worker stop --all worker create prints the new instance id, its wsUrl, a redacted bearer token, and its status. With --attach it also points your bridge at the worker and prints export NODETOOL_WORKER_URL=…, followed by a commented hint for the token (the raw token is never printed to stdout): export NODETOOL_WORKER_URL=wss://… # NODETOOL_WORKER_TOKEN was redacted; set it with: # export NODETOOL_WORKER_TOKEN=$(nodetool worker token <instance-id>) Retrieve the full token on demand with nodetool worker token <id>, which prints only the token so it pipes cleanly into NODETOOL_WORKER_TOKEN. CLI reference nodetool worker profile add <name> --target <runpod|vast> --image <img> \ [--gpu <type>] [--vcpu <n>] \ [--token-policy <generate|fixed>] \ [--idle-timeout <minutes>] [--max-lifetime <minutes>] nodetool worker profile list [--json] nodetool worker profile rm <name> nodetool worker create --profile <name> [--attach] nodetool worker create --target <t> --image <img> [--gpu <g>] [--attach] # inline, one-off nodetool worker list [--json] nodetool worker status <id> # refresh status from the provider nodetool worker token <id> # print the decrypted bearer token (pipeable) nodetool worker stop <id> nodetool worker stop --all The inline form of create synthesises a throwaway profile from the flags, so you don’t have to define one first. Attaching from the UI The Workers panel (in Settings) is the desktop-first surface: Profiles editor — pick a target, image, GPU, idle timeout, and token policy, and save a reusable profile. Provision — “Start” launches an instance and shows live progress (provisioning → running → attached). Live-instances table — every running worker with its status, uptime, and estimated cost, plus attach/detach and stop actions. Status-bar indicator — when a worker is attached, a status-bar badge shows it and offers a one-click quick-stop. Attaching re-points NodeTool’s Python bridge at the worker’s wss:// URL and bearer token without a restart; detaching reverts to the local stdio worker. The active worker is a single DB pointer (active_worker_instance_id in the settings table) that any NodeTool instance reads — which is why a self-hosted Docker server can adopt a worker by the same mechanism. Cost guard GPU pods and Vast instances bill continuously — there’s no scale-to-zero in pod mode. The laptop sleeps, the app crashes, a tab gets forgotten. Four mechanisms keep a stray worker from quietly billing for days: Guard What it does Real teardown Every stop issues the provider’s true delete/destroy — never just a status flip. Idle auto-stop The reaper stops an instance after its profile’s idle_timeout_minutes of bridge inactivity (the bridge tracks last_activity_at). Hard TTL Optional max_lifetime_minutes — an absolute kill switch regardless of activity. Orphan reconcile On startup/refresh, NodeTool diffs the DB’s running instances against the provider’s live list. Workers killed out-of-band are marked stopped; provider-live boxes the DB doesn’t track are surfaced as orphans with a “N workers live, ~$X/hr” summary and a one-click stop-all. Set both --idle-timeout and --max-lifetime on every profile you provision from. A profile that sets neither opts its instances out of the reaper entirely — do that only deliberately. How provisioning works WorkerManager.provision(profileName) looks up the profile and resolves the target’s provider (RunPod or Vast). If the profile’s token_policy is generate, it mints a high-entropy bearer token for the worker. The provider launches the image on the chosen GPU/spec, polls until the box is running, and derives the WebSocket URL. A worker_instances row is written and transitioned provisioning → running. On attach, the manager writes the active-worker pointer, marks the instance attached, and hands the { wsUrl, token } to the bridge. All state is persisted through the DB — the manager never holds instance state only in memory, so a forgotten pod is always recoverable from the registry. Related Self-Hosted Deployment — run the NodeTool server on your own infra with Docker. Deployment Guide — overview of server self-hosting and GPU workers. CLI Reference — full nodetool command reference. </content> </invoke>--- # Workflow API Guide Source: https://docs.nodetool.ai/workflow-api.html NodeTool exposes workflow REST endpoints under /api/workflows from a single server (nodetool serve): /api/workflows for CRUD and query operations, and POST /api/workflows/{id}/run to run a workflow. This page collects the basics from the project README. See API Reference for the canonical endpoint list and auth requirements. When AUTH_PROVIDER is static or supabase, include Authorization: Bearer <token>; tokens are optional only in local/none modes. Loading Workflows const response = await fetch("http://localhost:7777/api/workflows/"); const workflows = await response.json(); Running a Workflow HTTP API curl -X POST "http://localhost:7777/api/workflows/<workflow_id>/run" \ -H "Authorization: Bearer YOUR_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "params": { "param_name": "param_value" } }' const response = await fetch( "http://localhost:7777/api/workflows/<workflow_id>/run", { method: "POST", headers: { "Content-Type": "application/json", "Authorization": "Bearer YOUR_TOKEN", }, body: JSON.stringify({ params: params, }), } ); const body = await response.json(); // body has the shape: // { // "job_id": "<uuid>", // "workflow_id": "<uuid>", // "status": "completed" | "cancelled" | "failed", // "outputs": { /* one property per output node, keyed by node name */ }, // "error": null, // "message_count": 42, // "background": false // } // outputs values can be a string, image, audio, etc. POST /api/workflows/{id}/run runs the workflow to completion and returns a single JSON response — it does not stream. For real-time progress (job and node updates, incremental output), run the workflow over the WebSocket endpoint instead. WebSocket API For real-time streaming and job control over WebSocket, see the dedicated WebSocket API page. The reference client in examples/workflow_runner/js/workflow-runner.js shows how to consume job_update, node_update, and node_progress messages. API Demo Grab the example runner (examples/workflow_runner). Open index.html in a browser locally. Select the endpoint (local or api.nodetool.ai for alpha users). Enter an API token from the NodeTool settings dialog. Select a workflow and run it.--- # Workflow Debugging Guide Source: https://docs.nodetool.ai/workflow-debugging.html This guide teaches you how to debug NodeTool workflows when they don’t work as expected. You’ll learn to inspect intermediate data, read error logs, and systematically isolate problems. Quick Debugging Checklist When a workflow isn’t working: Check node status colors (gray/yellow/green/red) Click on failed (red) nodes to see error messages Add Preview nodes after suspicious nodes Verify all required inputs are connected Check data types match between connections Review the console/logs for detailed errors Test problem nodes in isolation Understanding Node Status Nodes display their execution status through border colors: Color Status Meaning Gray Not started Node hasn’t been processed yet Yellow Running Node is currently executing Green Completed Node finished successfully Red Failed Node encountered an error Blue outline Selected Node is currently selected When a node fails (red), click it to see the error message in the properties panel. Using Preview Nodes Preview nodes are your primary debugging tool. They show exactly what data is flowing through your workflow at any point. Adding Preview Nodes Press Space to open the node search Type “Preview” and select the Preview node Connect the output you want to inspect to the Preview node Run the workflow Preview Node Strategies After every major transformation: Input → Process → Preview(1) → Transform → Preview(2) → LLM → Preview(3) → Output Before and after suspicious nodes: ... → Preview(before) → SuspiciousNode → Preview(after) → ... At branch points: → Preview(A) → ProcessA Input → → Preview(B) → ProcessB What Preview Shows Preview nodes display data based on type: Data Type Preview Display Text/String Full text content, scrollable Image Rendered image with dimensions Audio Playable audio widget List JSON array with items Object/Dict Formatted JSON Number Numeric value Boolean True/False Inspecting Workflow JSON Every NodeTool workflow is stored as JSON. Inspecting this JSON can help debug complex issues. Exporting Workflow JSON Open your workflow in the editor Open the floating toolbar’s ⋮ menu and choose Download JSON (also available in the command menu) Save the .json file Open in any text editor or JSON viewer Workflow JSON Structure { "id": "workflow_abc123", "name": "My Workflow", "description": "...", "nodes": [ { "id": "node_1", "type": "nodetool.input.StringInput", "data": { "value": "Hello world" }, "position": { "x": 100, "y": 100 } }, { "id": "node_2", "type": "nodetool.agents.Agent", "data": { "prompt": "...", "model": { "provider": "openai", "id": "gpt-4" } }, "position": { "x": 300, "y": 100 } } ], "edges": [ { "id": "edge_1", "source": "node_1", "sourceHandle": "output", "target": "node_2", "targetHandle": "text" } ] } What to Look For Missing connections: Check that edges correctly connect outputs to inputs Incorrect values: Look at data for each node to verify settings Node types: Ensure type matches expected node (typos happen in programmatic workflows) Position issues: If nodes overlap or are off-canvas, check position values Reading Error Logs Desktop App Logs Enable debug logging: Open Settings → Advanced Enable Debug Logging Logs appear in the console panel (View → Developer Tools) Log location: Windows: %USERPROFILE%\.nodetool\logs\ macOS: ~/Library/Logs/NodeTool/ or ~/.nodetool/logs/ Linux: ~/.nodetool/logs/ CLI/Server Logs When running NodeTool from command line: # Verbose logging nodetool serve --verbose # Or set log level nodetool serve --log-level debug Understanding Error Messages Common error patterns: Error: Type mismatch - cannot connect 'List[String]' to 'String' → Use a node to extract a single item from the list (e.g., GetElement) Error: Required input 'prompt' is not connected → Connect something to the prompt input, or set a default value Error: Model not found: 'gpt-4' → Check API key configuration in Settings → Providers Error: CUDA out of memory → Use a smaller model, reduce batch size, or close other GPU applications Error: Connection refused on localhost:7777 → NodeTool server isn’t running, or firewall is blocking Debugging Specific Issues LLM Not Responding Check model availability Open Models → Model Manager Verify model is installed (local) or API key is set (cloud) Test with Preview Add Preview after the Agent/LLM node Run and check if any output appears Check prompt Is the prompt template valid? Are all template variables being filled? Verify connectivity For cloud models: check internet connection For local models: check if Ollama/llama.cpp is running Image Generation Fails Check model installation Open Model Manager Ensure Flux/Qwen Image/etc. is downloaded Verify VRAM Check GPU memory with nvidia-smi Close other GPU apps if memory is low Inspect parameters Valid dimensions? (usually multiples of 8 or 64) Reasonable step count? (20-50 typical) Valid CFG scale? (5-15 typical) Add Preview before generation Confirm prompt text is correct Check that conditioning inputs are valid RAG/Search Returns Nothing Verify collection exists Check that index workflow was run first Collection name matches between index and search Check embedding model Same embedding model for index and search? Model available and running? Test query Try a simpler query Increase top_k to retrieve more documents Check if documents were chunked appropriately Inspect indexed content Use Preview to see what was indexed Check chunk sizes aren’t too small/large Workflow Runs Forever Check for loops While NodeTool prevents circular connections, complex logic can create infinite loops Look for conditional nodes that might never exit Monitor node status Which node is stuck on “running” (yellow)? That node is the bottleneck Check network calls API timeouts can appear as hangs Add timeout parameters where available Resource constraints Is the system running out of memory? Is disk full (for large file operations)? Isolating Problems Binary Search Debugging When you have a complex workflow and something’s wrong: Disable half the workflow Disconnect nodes in the middle Run first half only Check results Working? Problem is in second half Broken? Problem is in first half Repeat Continue halving until you find the problem node Testing Nodes in Isolation Create a minimal test workflow: New workflow with just the suspicious node Add input nodes with known good test data Add Preview/Output to see results Run and verify If the node works in isolation, the problem is with the data it receives in the full workflow. Comparing Working vs Broken If a workflow used to work: Export both versions (working and broken) as JSON Diff the files to see what changed Focus on changed nodes/edges diff working_workflow.json broken_workflow.json Debugging Tools Summary Tool When to Use How to Access Preview nodes See intermediate data Space → search “Preview” Node click See error messages Click red (failed) nodes Console/DevTools View detailed logs View → Developer Tools JSON export Inspect workflow structure ⋮ menu → Download JSON –verbose flag CLI debugging nodetool serve --verbose Log files Historical debugging ~/.nodetool/logs/ Common Fixes Problem Solution “Type mismatch” Add conversion node between incompatible types “Not connected” Wire up all required inputs “Model not found” Install model in Model Manager or configure API key “Out of memory” Use smaller model, reduce batch size, close apps “Timeout” Check internet, increase timeout setting “Empty output” Add Preview to find where data is lost “Wrong format” Use FormatText or conversion nodes “Permission denied” Check file paths and permissions Getting More Help If you’re still stuck after debugging: Export your workflow as JSON Take screenshots of error messages Note your system info (OS, GPU, RAM, NodeTool version) Ask on Discord with these details File a GitHub issue for bugs Related Documentation Troubleshooting Guide – Broader troubleshooting for common issues Key Concepts – Understanding nodes, edges, and data flow Workflow Editor – Using the visual editor effectively Cookbook – Working workflow patterns to learn from--- # Workflow Editor Source: https://docs.nodetool.ai/workflow-editor.html The canvas: place nodes, connect ports, run, debug. Covers basic navigation through node bypass and auto layout. New here? Start with Getting Started, then come back. For panel-by-panel detail, see Editor Panels. Where this fits The workflow editor is NodeTool’s automation layer. A workflow reads assets, calls models, and writes new assets — which flow on into the Sketch Editor and Video Editor for hand editing, come back as fresh assets, or ship to non-technical users as a Mini-App. Every surface shares this graph’s outputs through one asset store and the same model/provider system. See Key Concepts → How everything fits together for the full loop. Editor Layout Area Where What It Does Canvas Center Place and connect nodes Side Panels Left Workflows, nodes, assets, timelines, sketches, favorites Composer Bottom Chat, run, save, auto-layout Canvas Basics Your infinite workspace. Navigate: Do This How Pan Space + drag, or right-click drag Zoom Ctrl/⌘ + scroll Fit everything F Reset zoom (to 50%) Ctrl/⌘ + 0 The grid helps align nodes. Turn on Snap to Grid in View menu. Working with Nodes Each node does one thing. Add Nodes Space bar: Press Space anywhere Type what you want (“image”, “text”) Click to add Double-click: Double-click empty space Opens node menu Smart connect: Drag from a node’s output Drop on empty space See compatible nodes Node Structure Header (top) - Name, drag to move Inputs (left circles) - Data in Outputs (right circles) - Data out Properties - Settings panel Select Nodes Do This How One Click it Multiple Shift + click, or drag box All Ctrl/⌘ + A None Click canvas Move Nodes Drag header to move Arrow keys to nudge Auto Layout button to organize Bypass Nodes Skip temporarily without deleting: Right-click node Select Bypass Node Node dims, data passes through Good for: Testing - Compare with/without Debugging - Isolate problems A/B testing - Toggle effects Re-enable: Right-click → Enable Node Connections Connections are the lines between nodes that show how data flows through your workflow. Data always flows left to right — from output ports (right side of a node) to input ports (left side of another node). Make Connections Click output circle (right side) Drag the line to an input circle (left side of another node) Release to connect Connection Rules Types must match: You can only connect compatible types (text to text, image to image) One input, multiple outputs: Each input accepts one connection; outputs can connect to many Color coding: Connection colors indicate data type Removing Connections Click a connection line, then press Delete Right-click a connection for options Drag the connection away from its target and release Smart Connections When you drag a connection and release on empty space, the Connection Menu appears: Auto-create common nodes for that data type Browse compatible nodes filtered by what can receive the data Cancel by pressing Esc Running Workflows Starting a Run Method How Button Click Run in the bottom toolbar Keyboard Ctrl/⌘ + Enter Watching Progress Streaming nodes show output as it’s generated Preview nodes display intermediate results Node borders indicate status (running, complete, error) Edge animations show data flowing between nodes Stopping a Run Method How Button Click Stop (enabled when running, paused, or suspended) Keyboard Esc Organizing Your Workflow Auto Layout Click the Auto Layout button in the floating toolbar to automatically arrange your nodes in a clean, readable layout. The editor also auto-arranges nodes when Chat creates or modifies workflows. (There is no keyboard shortcut for auto-layout — it’s a toolbar button only.) Grouping Nodes Select multiple nodes and press Ctrl/⌘ + G to group them. Groups: Keep related nodes together Can be collapsed to save space Move as a unit Aligning Nodes Shortcut Action A Align selected nodes Shift + A Align and distribute evenly Left Panel Access these views by clicking icons on the left rail: Nodes, Workflows, Sketches, Timelines, Settings, History, Favorites, Assets, and Agent. See Editor Panels → Left Panel for details on each. Right Panel (Inspector) Detailed properties for selected nodes Input/output documentation Validation errors and warnings The right panel hosts only the Inspector. Logs, Queue, Trace, Version History, and Workspace live in the Bottom Panel. Finding Nodes The Node Menu Press Space to open, then: Search: Just start typing (“whisper”, “image”, “agent”) Browse: Explore the category tree on the left Filter: Click the filter icon to show only nodes with specific input/output types Move: Drag the menu to reposition it Close: Esc or click outside Node Documentation Get help on any node: In the Node Menu: Hover over a node to see its description Inspector: Select a node and view full documentation in the right panel Context Menus Right-click for options anywhere: Location Options Canvas Add node, paste, select all Node header Copy, duplicate, delete, group, bypass Input/Output Disconnect, add compatible node Connection Delete, add node in middle Built-in Editors NodeTool includes professional editing tools for creative work. Sketch Editor Open a blank canvas from + New → New image in the workspace tab bar, or edit an existing image asset, to use the full layered editor: Layers: Blend modes, per-layer opacity, lock, and visibility Painting: Brush, pencil, eraser, fill, gradient, blur, clone stamp Shapes & transform: Rectangle, ellipse, line, arrow, crop, free transform AI generation: Generate a layer from a prompt or bind it to a workflow History: Unlimited undo/redo 📖 Full Guide: See Sketch Editor for complete documentation with tool reference, shortcuts, and workflows. Color Picker The color picker appears when selecting colors in properties: Visual Selection: Saturation/brightness picker with hue slider Multiple Formats: Enter values as HEX, RGB, or HSL Harmony Modes: Complementary, triadic, analogous color suggestions Gradient Builder: Create and edit color gradients Swatches: Save and reuse favorite colors Contrast Checker: Verify accessibility compliance Eyedropper: Pick colors from anywhere on screen Keyboard Shortcuts Essential Shortcuts Shortcut Action Space Open node menu Ctrl/⌘ + Enter Run workflow Ctrl/⌘ + S Save Ctrl/⌘ + Z Undo F Fit view Esc Stop / Cancel All Editor Shortcuts Shortcut Action Ctrl/⌘ + C Copy Ctrl/⌘ + V Paste Ctrl/⌘ + X Cut Ctrl/⌘ + D Duplicate horizontally Ctrl/⌘ + Shift + D Duplicate vertically Ctrl/⌘ + G Group selection Ctrl/⌘ + 0 Reset zoom to 50% Ctrl/⌘ + 1-9 Switch to tab 1-9 A Align selected nodes Shift + A Align and distribute Arrow keys Nudge selected nodes Delete / Backspace Delete selection i Toggle Inspector Tips Design Principles Left to right — Arrange nodes so data flows left to right across the canvas for readability Preview often — Add Preview nodes after each major step to inspect intermediate results Name clearly — Rename nodes (double-click the header) to describe their purpose, e.g., “Resize to 512px” instead of “Resize” Group logically — Keep related nodes together and use Groups (Ctrl/⌘ + G) to visually organize complex workflows Debugging Add Preview nodes between steps to see exactly what data each node produces Check connections — verify data types match (connection colors indicate type) Look at node borders — red = error, yellow = running, green = completed Test incrementally — bypass downstream nodes and run partial workflows to isolate problems Use the Inspector — press i to see detailed error messages and validation warnings Performance Local models — slower but work offline and are free to use Cloud models — faster response times, require internet and API keys Streaming nodes — show progress during long-running operations (look for the streaming indicator) Parallel branches — NodeTool automatically runs independent branches in parallel for faster execution Next Steps Cookbook – Workflow patterns and best practices Workflow Examples – Ready-to-use workflows Tips & Tricks – Power user features Node Reference – All available nodes--- # Workflow Graph View Source: https://docs.nodetool.ai/workflow-graph-view.html The Workflow Graph View is a read-only rendering of a saved workflow. It’s useful for sharing a visual snapshot, embedding workflow diagrams in documentation, and giving stakeholders a look without handing them the editor. Opening the View The graph view lives at /graph/:workflowId. You can link directly to it from the workflow’s Share menu, or paste the URL into a Markdown file — Jekyll will render the image when the page is exported statically. Unlike the [Workflow Editor]({{ ‘/workflow-editor’ relative_url }}), the graph view: Does not load the Node Menu, Inspector, or the panel drawers. Does not allow editing — nodes can’t be added, moved, or deleted. Does not require authentication on localhost deployments. This makes it lightweight (~5× smaller bundle) and safe to expose behind a read-only proxy. Interactions The graph view is fully static — it’s a snapshot, not an interactive canvas. Nodes can’t be dragged, connected, or selected; panning, scroll-zoom, pinch-zoom, and double-click-zoom are all disabled. The view auto-fits the whole graph to the viewport once when it loads, so the entire workflow is visible without any interaction. Running the workflow is not possible from this view — open it in the editor first. To pass a workflow inline (without a stored ID), supply base64-encoded workflow JSON via the ?data= query parameter. Optional ?bg= and ?padding= parameters control the background color and fit padding. Embedding in Docs You can embed the graph view inside another page with an iframe: <iframe src="https://your-nodetool-host/graph/WORKFLOW_ID" width="100%" height="540" loading="lazy" referrerpolicy="no-referrer"> </iframe> For stored Markdown docs, take a screenshot instead so the graph is captured even when the server is offline. Access Control The graph view inherits the workflow’s own visibility: Private — requires the user to be signed in with access to the workflow. Shared — accessible to anyone with the link. Public — indexed and crawlable. For localhost deployments, all workflows are effectively local and the view is open to anyone who can reach your server. Next Steps [Workflow Editor]({{ ‘/workflow-editor’ relative_url }}) — the full editor where you author workflows [Chain Editor]({{ ‘/chain-editor’ relative_url }}) — the linear alternative [Authentication]({{ ‘/authentication’ relative_url }}) — how access is enforced--- # Workflow Gallery Source: https://docs.nodetool.ai/workflows/ Ready-to-use workflow examples. Each includes explanations and visual diagrams. For the flagship, end-to-end showcases, see Use Cases. See Workflow Patterns for reusable techniques. 🎨 Start Here: Beginner-Friendly Workflows Good starting points for learning NodeTool: Creative Story Ideas - Generate story concepts and creative prompts Image Enhance - Polish photos with sharpening and auto-contrast Transcribe Audio - Convert speech to text with AI 🖼️ Visual Creation & Image Workflows Create and transform images: Movie Posters - AI-generated poster designs from creative briefs Image Enhance - Image enhancement pipeline Image To Audio Story - Turn images into narrated stories 🎬 Video & Motion Workflows Create and enhance video content: Color Boost Video - AI-powered color enhancement for video 🎵 Audio & Voice Workflows Work with sound, music, and voice: Transcribe Audio - Speech-to-text with word-level timestamps Image To Audio Story - Generate narrated stories from visuals ✍️ Content Creation & Writing Generate and transform written content: Creative Story Ideas - Brainstorming for writers and creators Flashcard Generator - Turn content into study materials 📚 Document Workflows Work with documents and knowledge: Chat with Docs - Ask questions about your documents with AI Fetch Papers - Retrieve and process academic papers 🤖 Productivity & Automation Save time with these workflows: Meeting Transcript Summarizer - Auto-summarize meetings Categorize Mails - Organize emails automatically Summarize RSS - Stay updated with feed summaries 📊 Data & Visualization Generate and visualize data: Data Generator - Create synthetic datasets All Workflows (Alphabetical) Categorize Mails Chat with Docs Color Boost Video Creative Story Ideas Data Generator Fetch Papers Flashcard Generator Image Enhance Image To Audio Story Meeting Transcript Summarizer Movie Posters Preview Subgraph Summarize RSS Transcribe Audio Workflow Workflow Gallery nodetool.workflows Nodes nodetool.workflows.base_node Nodes nodetool.workflows.subgraph Nodes nodetool.workflows.workflow_node Nodes How to Use These Workflows Option 1: One-Click Import (Easiest) Open NodeTool Click Templates in the dashboard Browse and click any workflow Start creating immediately! Option 2: Build Manually (Learning) View any workflow page Follow the visual diagram Add nodes and connect them yourself Great for understanding how it works! Share Your Creations Created a workflow? Share it on the NodeTool repository.--- # Categorize Mails Source: https://docs.nodetool.ai/workflows/categorize-mails.html Overview Classifies emails into predefined categories (Newsletter, Work, Family, Friends) using an LLM and applies matching Gmail labels. Demo Workflow Steps Gmail Search - Fetches up to 10 recent emails using the specified filters (date, subject, sender). Template - Formats each email into a structured prompt with subject, sender, and body snippet. Classifier - Uses an LLM to classify the email into one or more categories. Add Label - Applies the determined label(s) to each email in Gmail. Tags email, start Workflow Diagram graph TD template_29a39f["Template"] classifier_a6df08["Classifier"] addlabel_663354["AddLabel"] gmailsearch_b776a8["GmailSearch"] template_29a39f --> classifier_a6df08 classifier_a6df08 --> addlabel_663354 gmailsearch_b776a8 --> addlabel_663354 gmailsearch_b776a8 --> template_29a39f--- # Chat with Docs Source: https://docs.nodetool.ai/workflows/chat-with-docs.html Overview Document retrieval and question-answering using vector search and local LLMs. This tutorial does not ship as an importable template. Build it manually by following the steps below. Demo Tags chat, rag Workflow Diagram graph TD formattext_9["FormatText"] hybridsearch_8e9b81["HybridSearch"] query_2cff53["Query"] agent_d83380["Agent"] query_2cff53 --> hybridsearch_8e9b81 hybridsearch_8e9b81 --> formattext_9 query_2cff53 --> formattext_9 formattext_9 --> agent_d83380--- # Color Boost Video Source: https://docs.nodetool.ai/workflows/color-boost-video.html Overview Extracts frames from a video, amplifies color intensity, and reassembles them into a modified video. Video Input - A video is loaded. Frame Iteration (nodetool.video.ForEachFrame) - Frames are extracted. Color Enhancement - Each frame’s exposure and saturation are amplified using lib.image.color_grading.Exposure and lib.image.color_grading.SaturationVibrance. Reassembly (nodetool.video.FrameToVideo) - Enhanced frames are recompiled into a new video stream. Demo Tags video, start Workflow Diagram graph TD video_7074f1["Video"] foreachframe_abb8ee["ForEachFrame"] exposure_b14f76["Exposure"] saturation_b14f77["SaturationVibrance"] frametovideo_4ca887["FrameToVideo"] video_7074f1 --> foreachframe_abb8ee foreachframe_abb8ee --> exposure_b14f76 exposure_b14f76 --> saturation_b14f77 saturation_b14f77 --> frametovideo_4ca887 foreachframe_abb8ee --> frametovideo_4ca887 foreachframe_abb8ee --> frametovideo_4ca887 foreachframe_abb8ee --> frametovideo_4ca887--- # Creative Story Ideas Source: https://docs.nodetool.ai/workflows/creative-story-ideas.html Overview A beginner-friendly template for combining inputs with an AI agent to generate creative story ideas. How it works: Set inputs - Define genre, character type, and setting Format template - Combine inputs into a prompt Generate ideas - AI produces multiple story concepts Key concepts: Connect nodes to create workflows Use templates with for dynamic prompts AI agents process inputs and generate results Demo Tags start, beginner, tutorial, template, creative, writing Workflow Diagram graph TD genre_input_["Genre"] character_type_input_["Character Type"] setting_input_["Setting"] string_templa["String"] formattext_format["FormatText"] listgenerator_list_g["ListGenerator"] genre_input_ --> formattext_format character_type_input_ --> formattext_format setting_input_ --> formattext_format string_templa --> formattext_format formattext_format --> listgenerator_list_g How to Use Open NodeTool and find “Creative Story Ideas” in Templates Click each input node to set your preferences: Genre: “Sci-fi”, “Fantasy”, “Mystery”, etc. Character Type: “Hero”, “Villain”, “Detective”, etc. Setting: “Space station”, “Medieval castle”, “Future city”, etc. Press Ctrl/⌘ + Enter or click Run View the generated ideas in the Preview node Tips: Run multiple times to get different ideas Try unusual combinations for unique results Next Steps Movie Posters - Generate images from ideas Image Enhance - Chain image transformations--- # Data Generator Source: https://docs.nodetool.ai/workflows/data-generator.html Overview Generate synthetic structured data using an AI agent. Describe the data with a prompt and add columns to specify its shape. Demo Tags agents Workflow Diagram graph TD datagenerator_2ed27b["DataGenerator"]--- # Fetch Papers Source: https://docs.nodetool.ai/workflows/fetch-papers.html Overview Fetches and downloads research papers from the Awesome Transformers GitHub repository. Extracts paper links from the README, filters for actual papers, and downloads them to a specified folder. Demo Tags automation Workflow Diagram graph TD fromlist_5["FromList"] filter_6["Filter"] extractcolumn_25["ExtractColumn"] extractlinks_32["ExtractLinks"] downloadfiles_21698c["DownloadFiles"] getrequest_422102["GetRequest"] fromlist_5 --> filter_6 extractlinks_32 --> fromlist_5 filter_6 --> extractcolumn_25 extractcolumn_25 --> downloadfiles_21698c getrequest_422102 --> extractlinks_32--- # Flashcard Generator Source: https://docs.nodetool.ai/workflows/flashcard-generator.html Overview Generate study flashcards using AI and store them in a database. Enter any topic and get instant flashcards saved for future review. Demo Tags education, database, ai, flashcards, learning Workflow Diagram graph TD topic_topic_["topic"] createtable_create["CreateTable"] formattext_format["FormatText"] datagenerator_genera["DataGenerator"] insert_insert["Insert"] query_query_["Query"] topic_topic_ --> formattext_format formattext_format --> datagenerator_genera datagenerator_genera --> insert_insert createtable_create --> query_query_ createtable_create --> query_query_ createtable_create --> insert_insert createtable_create --> insert_insert createtable_create --> query_query_--- # Image Enhance Source: https://docs.nodetool.ai/workflows/image-enhance.html Overview A simple image enhancement pipeline: sharpen for crisp details, then auto-contrast for better exposure balance. The Pipeline: Image Input (nodetool.input.ImageInput) - Your photo or artwork Unsharp Mask (lib.image.filter.UnsharpMask) - Enhance edges and fine details Auto Contrast (lib.image.enhance.AutoContrast) - Optimize brightness and contrast Image Output - Your enhanced result For more options: Replicate.Image.Upscale for AI upscaling, Nodetool.Image.Transform for color and brightness controls. Demo Tags image, start, photography, enhancement Workflow Diagram graph TD enhanced_97877["enhanced"] image_97879["image"] sharpen_9d8850["UnsharpMask"] autocontrast_4c9c6b["AutoContrast"] image_97879 --> sharpen_9d8850 sharpen_9d8850 --> autocontrast_4c9c6b autocontrast_4c9c6b --> enhanced_97877 How to Use Open NodeTool and find “Image Enhance” in Templates Click the Image Input node and upload a photo (JPG, PNG, WebP) Optionally click Unsharp Mask to adjust intensity Press Ctrl/⌘ + Enter or click Run Tips: More sharpness isn’t always better Works well on batches Next Steps Movie Posters - Combine enhancement with AI generation Color Boost Video - Apply similar enhancements to video--- # Image To Audio Story Source: https://docs.nodetool.ai/workflows/image-to-audio-story.html Overview Transforms images into narrated stories by combining vision AI with language generation and speech synthesis. How it works: Image Input - Your photo, artwork, or any visual AI Vision + Story - AI analyzes the image and writes a narrative Text-to-Speech - Converts the story to spoken audio Audio Output - Narration you can save or share Prompt examples: “Describe this image as if you’re a museum curator” “Write a short poem inspired by this artwork” “Create a brief backstory for this scene” Demo Tags start, multimodal, creative, audio, storytelling Workflow Diagram graph TD image_1["Image"] agent_77a9cf["Agent"] texttospeech_ffb9de["TextToSpeech"] image_1 --> agent_77a9cf agent_77a9cf --> texttospeech_ffb9de How to Use Open NodeTool and find “Image to Audio Story” template Load your image Customize the AI prompt in the Agent node (default: “Create a story inspired by this image”) Choose your voice in the TextToSpeech node Press Ctrl/⌘ + Enter to run Related Workflows Movie Posters - Create visual content Creative Story Ideas - Generate story concepts--- # Meeting Transcript Summarizer Source: https://docs.nodetool.ai/workflows/meeting-transcript-summarizer.html Overview Transcribes a meeting recording with OpenAI Whisper, then condenses the transcript into concise notes. Audio Input - Load a meeting recording. Transcription - Speech is transcribed with Whisper. Summarization - Transcript is condensed into key points. Output - A text node displays the final summary. Demo Tags audio, llm Workflow Diagram {% mermaid %} graph TD audio_1[“Audio”] summarizer_3[“Summarizer”] summary_4[“summary”] automaticspeechrecognition_a2e093[“AutomaticSpeechRecognition”] automaticspeechrecognition_a2e093 –> summarizer_3 audio_1 –> automaticspeechrecognition_a2e093 summarizer_3 –> summary_4 {% endmermaid %}--- # Movie Posters Source: https://docs.nodetool.ai/workflows/movie-posters.html Overview An AI-powered poster generator: a strategy agent plans the visual concept, then an image model renders poster variations. How it works: Input - Movie title, genre, and target audience Strategy Agent - Creates a marketing approach and visual concept Prompt Generator - Converts strategy into image prompts for multiple variations Image Generator - Renders 512×768px posters with Stable Diffusion/Flux Preview - Shows the strategy and all generated posters Demo Tags start, image, creative, design, posters Workflow Diagram {% mermaid %} graph TD agent_strate[“Agent”] string_strate[“String”] formattext_strate[“FormatText”] movie_title_movie_[“Movie Title”] genre_genre_[“Genre”] primary_audience_audien[“Primary Audience”] listgenerator_prompt[“ListGenerator”] string_design[“String”] texttoimage_d31191[“TextToImage”] string_strate –> formattext_strate formattext_strate –> agent_strate agent_strate –> listgenerator_prompt string_design –> listgenerator_prompt primary_audience_audien –> formattext_strate movie_title_movie_ –> formattext_strate genre_genre_ –> formattext_strate listgenerator_prompt –> texttoimage_d31191 {% endmermaid %} How to Use Open NodeTool and find “Movie Posters” in Templates Fill in your movie details: Movie Title: “Quantum Horizon” (or your movie name) Genre: “Sci-Fi Thriller” (choose any genre) Primary Audience: “Adults 25-40 who love space mysteries” Press Ctrl/⌘ + Enter or click Run View the AI strategy in the top preview, posters in the bottom gallery Tips: Try different genres for the same title to see how styles change Be specific about audience to get more targeted designs Next Steps Image Enhance - Enhance your poster designs Creative Story Ideas - Generate movie concepts--- # Summarize RSS Source: https://docs.nodetool.ai/workflows/summarize-rss.html Overview Fetches an RSS feed, collects all entries, and generates a summary. Demo Tags rss, llm Workflow Diagram {% mermaid %} graph TD fetchrssfeed_3fd0dc[“FetchRSSFeed”] collect_98cc0c[“Collect”] summarizer_065961[“Summarizer”] fetchrssfeed_3fd0dc –> collect_98cc0c collect_98cc0c –> summarizer_065961 {% endmermaid %}--- # Transcribe Audio Source: https://docs.nodetool.ai/workflows/transcribe-audio.html Overview Convert speech to text using the Whisper model with word-level timestamps. Audio Input - Record your voice or upload an audio file Automatic Speech Recognition - Processes audio through Whisper String Output - Displays the transcribed text Demo How to Use Record your voice or upload a file using the audio input Click Run Read the transcription in the output Tags start, audio, huggingface Workflow Diagram {% mermaid %} graph TD audio_7[“audio”] transciption_8[“transciption”] automaticspeechrecognition_c51917[“AutomaticSpeechRecognition”] audio_7 –> automaticspeechrecognition_c51917 automaticspeechrecognition_c51917 –> transciption_8 {% endmermaid %}--- # Workspaces Source: https://docs.nodetool.ai/workspaces.html A workspace is a folder that NodeTool can safely access for reading and writing files during workflow runs, agent tasks, and chat sessions. Why workspaces exist Workspaces make local file access explicit and controlled: Keep project files organized under known roots Limit file operations to approved directories Share the same folder roots across workflows and agent tools Manage workspaces in the app Workspaces have their own full-screen page at /workspaces (not a Settings tab): Open the app menu (click the logo at the top of the left rail). Choose Workspaces (shown when workspaces are enabled). Add one or more workspace directories on the Workspaces page. After adding a workspace, NodeTool can browse files, list folders, and read/write files inside the configured root. Where workspaces are used Agents and chat tools for file operations Workflow nodes in the lib.os namespace (file read/write, path, and filesystem operations) Project organization when working with local assets and generated outputs Related docs Workflow Editor Storage Guide Node Reference: lib.os