Replicate: Add Models & Nodes

Runs: cloud. Needs: REPLICATE_API_TOKEN. The provider constructor throws REPLICATE_API_TOKEN is required without it. Codegen also needs the token.

TL;DR

Add the model’s "owner/name" entry to the matching config file in packages/replicate-codegen/src/configs/.
Export REPLICATE_API_TOKEN and run npm run generate:replicate.
Run npm run build:packages (replicate-nodes loads from dist/).
Verify with npm run dev:nodetool -- node run replicate.<module>.<ClassName> --props '{...}'.

Do not hand-edit packages/replicate-nodes/src/replicate-manifest.json — it is generated output.

Where things live

Path	Purpose
`packages/replicate-codegen/src/configs/`	One TypeScript file per module (e.g. `image-generate.ts`). Edit these to add models.
`packages/replicate-codegen/src/configs/index.ts`	Imports every module config and exports `allConfigs`. Register new config files here.
`packages/replicate-codegen/src/generate.ts`	CLI entry point for the generator. Fetches Replicate schemas and writes the manifest.
`packages/replicate-codegen/src/schema-fetcher.ts`	Fetches/caches OpenAPI schemas from `api.replicate.com`. Cache lives in `.schema-cache/replicate/`.
`packages/replicate-codegen/src/schema-parser.ts`	Parses the OpenAPI schema into a `NodeSpec`.
`packages/replicate-codegen/src/node-generator.ts`	Applies `NodeConfig` overrides (`className`, `returnType`, `fieldOverrides`) onto a `NodeSpec`.
`packages/replicate-codegen/src/types.ts`	`ModuleConfig`, `NodeConfig`, `NodeSpec`, `FieldDef`, `EnumDef`.
`packages/replicate-nodes/src/replicate-manifest.json`	Generated. Node definitions consumed at runtime. Never edit by hand.
`packages/replicate-nodes/src/replicate-factory.ts`	Builds `NodeClass` instances from manifest entries at runtime.
`packages/replicate-nodes/src/replicate-base.ts`	Node helpers: `assetToUrl` (uploads through the provider), the capability for each output type, output converters.
`packages/replicate-nodes/src/index.ts`	Exports `REPLICATE_NODES` (array of `NodeClass`) and `registerReplicateNodes`.
`packages/runtime/src/providers/replicate-provider.ts`	`ReplicateProvider` — chat, image, video, audio, TTS, ASR, embeddings, and the endpoint calls and file uploads the nodes use.
`packages/runtime/src/providers/manifest-models.ts`	Reads the manifest to surface image/video/music model lists to the UI.
`packages/runtime/src/providers/index.ts`	Registers `ReplicateProvider` under `PROVIDER_IDS.REPLICATE`.

How Replicate nodes are generated

The codegen pipeline runs in four steps:

Config lookup — generate.ts reads allConfigs from configs/index.ts. Each config maps a Replicate model id ("owner/name") to a NodeConfig (class name, return type, field overrides).
Schema fetch — SchemaFetcher calls https://api.replicate.com/v1/models/<owner>/<name> to get the model’s OpenAPI input/output schema. Results are cached under .schema-cache/replicate/ (keyed by SHA-256 of the model id). Pass --no-cache to force a fresh fetch.
Parsing and override — SchemaParser converts the OpenAPI schema to a NodeSpec. NodeGenerator.applyConfig() then merges the NodeConfig overrides onto it (renames class, sets outputType, fixes propType for image/audio/video fields that Replicate types as string).
Manifest write — The generator writes every NodeSpec as a ManifestEntry to packages/replicate-nodes/src/replicate-manifest.json. At runtime, replicate-factory.ts reads this file and produces NodeClass objects via createReplicateNodeClass().

The node type for a manifest entry follows the pattern replicate.<moduleName>.<className> where moduleName uses dots instead of dashes (e.g. replicate.image.generate.Flux_Schnell).

Add a new Replicate model

Step 1 — Choose the right config file

Pick the config whose module matches the model’s task:

File	Module key	Covers
`image-generate.ts`	`image.generate`	Text-to-image, image-to-image
`image-analyze.ts`	`image.analyze`	Captioning and image analysis
`image-ocr.ts`	`image.ocr`	OCR
`image-face.ts`	`image.face`	Face restoration and swap
`image-process.ts`	`image.process`	Other image processing
`image-3d.ts`	`image.3d`	Image to 3D (`returnType: "str"`)
`image-enhance.ts`	`image.enhance`	Enhancement, retouching
`image-upscale.ts`	`image.upscale`	Upscaling
`image-background.ts`	`image.background`	Background removal
`video-generate.ts`	`video.generate`	Text/image to video
`video-enhance.ts`	`video.enhance`	Video enhancement
`video-face.ts`	`video.face`	Video face swap and restoration
`video-process.ts`	`video.process`	Other video processing
`audio-generate.ts`	`audio.generate`	Music, sound generation
`audio-enhance.ts`	`audio.enhance`	Audio enhancement
`audio-separate.ts`	`audio.separate`	Stem separation
`audio-speech.ts`	`audio.speech`	TTS
`audio-transcribe.ts`	`audio.transcribe`	ASR
`text-generate.ts`	`text.generate`	Text LLMs via Replicate
`embedding.ts`	`embedding`	Embedding models

If none fit, create a new file (see Step 1b below).

Step 2 — Add the model entry

Add one key to the configs object in the chosen file. The key is the Replicate model id exactly as it appears in the URL (replicate.com/<owner>/<name>).

Minimal entry (no image inputs):

"acme-org/my-flux-variant": {
  className: "MyFluxVariant",
  returnType: "image"
}

Entry with image inputs (Replicate types these as string; NodeTool needs image/video/audio):

"acme-org/my-img2img": {
  className: "MyImg2Img",
  returnType: "image",
  fieldOverrides: {
    image: { propType: "image" },
    mask: { propType: "image" }
  }
}

Full NodeConfig shape (all fields optional):

interface NodeConfig {
  className?: string;           // PascalCase; if omitted the parser derives one
  docstring?: string;           // Override the model description
  tags?: string[];
  useCases?: string[];
  returnType?: string;          // "image" | "video" | "audio" | "str"
  fieldOverrides?: Record<string, Partial<FieldDef>>;   // propType corrections
  enumOverrides?: Record<string, string>;               // rename enum classes
  enumValueOverrides?: Record<string, Record<string, string>>;
}

returnType is required for media-producing models. Without it the factory cannot map the output to the right AssetRef type. Use "str" for outputs that are not an asset, such as a 3D mesh URL.

Asset input propType rules — Replicate schemas type all URLs as string. Override these manually:

Field carries	Set `propType` to
Single image	`"image"`
Single video	`"video"`
Single audio	`"audio"`
List of images	`"list[image]"`

Step 1b (optional) — Create a new config file

If no existing module fits, create packages/replicate-codegen/src/configs/<domain>-<task>.ts:

// packages/replicate-codegen/src/configs/image-depth.ts
import type { ModuleConfig } from "../types.js";

export const imageDepthConfig: ModuleConfig = {
  configs: {
    "acme-org/depth-map": {
      className: "DepthMap",
      returnType: "image"
    }
  }
};

Then register it in packages/replicate-codegen/src/configs/index.ts:

import { imageDepthConfig } from "./image-depth.js";
// ...
export const allConfigs: Record<string, ModuleConfig> = {
  // ...existing entries...
  "image.depth": imageDepthConfig,
};

The module key ("image.depth") becomes the moduleName in the manifest (stored as "image-depth") and the middle segment of every node’s type string (replicate.image.depth.<ClassName>). --module <name> writes a manifest that contains only that module and overwrites replicate-manifest.json, so use it only to inspect output. To update the shipped manifest, always run npm run generate:replicate (--all).

Verify

Run these in order after adding the config entry:

# 1. Generate the manifest (fetches from Replicate API; uses cache by default)
export REPLICATE_API_TOKEN="r8_..."
npm run generate:replicate

# 2. Regenerate without cache if the model schema was recently updated
REPLICATE_API_TOKEN="r8_..." npm run generate --workspace=packages/replicate-codegen -- --all --strict --no-cache

# 3. Build — replicate-nodes loads from dist/, so rebuild is required
npm run build:packages

# 4. Type-check the codegen package (the script runs tsc --noEmit)
npm run lint --workspace=packages/replicate-codegen

# 5. Run codegen tests
npm run test --workspace=packages/replicate-codegen

# 6. Smoke-test the node (confirm it appears in the registry and properties look right)
npm run dev:nodetool -- node run replicate.image.generate.MyFluxVariant \
  --props '{"prompt": "a red fox"}' --no-secrets --json

# 7. Affected tests, lint, and the harness gate
npm run test:affected
npm run lint
npm run dev:nodetool -- harness gate --base origin/main

If the model id is wrong or the model is private/unavailable, the generator exits with an error listing the failures. Fix the id or remove the entry.

Edit this page on GitHub