Together AI: Add Models & Nodes

Runs: cloud. Needs: TOGETHER_API_KEY. The provider constructor throws TOGETHER_API_KEY is required without it.

TL;DR

Edit IMAGE_MODELS, VIDEO_MODELS, TTS_MODELS, or ASR_MODELS in scripts/generate-manifest.mjs.
Run npm run gen:manifest — rewrites src/together-manifest.json.
Run npm run build in the package, then npm run lint && npm run test (lint is tsc --noEmit).
For TTS and ASR models, also update the matching static array in packages/runtime/src/providers/together-provider.ts. Embedding models exist only there.
Verify with nodetool node run together.<module>.<ClassName>.

Where things live

What	Path
Manifest generator (source of truth)	`packages/together-nodes/scripts/generate-manifest.mjs`
Generated manifest (do not hand-edit)	`packages/together-nodes/src/together-manifest.json`
Node factory (manifest → `BaseNode` subclasses)	`packages/together-nodes/src/together-factory.ts`
Asset resolution and per-modality request building	`packages/together-nodes/src/together-base.ts`
Node registration entry point	`packages/together-nodes/src/index.ts`
Provider class (chat, image, video, TTS, ASR, embeddings, and the endpoint calls the nodes use)	`packages/runtime/src/providers/together-provider.ts`
Manifest→model-list bridge	`packages/runtime/src/providers/manifest-models.ts`
Provider registration	`packages/runtime/src/providers/index.ts` (`registerBuiltinProvider(PROVIDER_IDS.TOGETHER, ...)`)
Pack registration (`registerTogetherNodes`)	`packages/base-nodes/src/builtin-packs.ts`
Pack catalog entry	`packages/protocol/src/builtin-packs.ts`
`PROVIDER_IDS.TOGETHER` constant	`packages/protocol/src/api-types.ts`

How Together models and nodes are defined

Image and video nodes — generated from the manifest

src/together-manifest.json is generated code. The source of truth is scripts/generate-manifest.mjs, which defines four model catalogs (IMAGE_MODELS, VIDEO_MODELS, TTS_MODELS, ASR_MODELS) and expands each (model × task) pair into a manifest entry (image, video, audio/text_to_speech, and transcription/automatic_speech_recognition entries).

Each entry looks like this:

{
  "className": "FLUX1SchnellTextToImage",
  "moduleName": "image",
  "modality": "text_to_image",
  "modelId": "black-forest-labs/FLUX.1-schnell",
  "outputType": "image",
  "title": "FLUX.1 Schnell — Text to Image",
  "description": "FLUX.1 Schnell — text to image via Together AI serverless.",
  "fields": [ ... ],
  "supportedTasks": ["text_to_image"]
}

The factory (together-factory.ts) turns every entry into a BaseNode subclass at runtime. The node type is together.<moduleName>.<className>. Fields declared in "fields" become node properties; the "modality" value routes execution to the matching executor in together-base.ts. The executor builds Together’s request body and sends it with context.callProviderEndpoint. TogetherProvider.callEndpoint holds the API key, posts to the endpoint family named by route (images, audio/speech, audio/transcriptions or videos), polls video jobs, and writes the generation record. Result URLs download through context.downloadProviderMedia. Nodes never read the API key.

The same manifest is read by manifest-models.ts (via loadImageModels / loadVideoModels) to populate TogetherProvider.getAvailableImageModels() and getAvailableVideoModels(). This means the node catalog and the provider’s model catalog stay in sync automatically — there is no second list to update for image and video models.

Chat / language models — fetched dynamically

TogetherProvider.getAvailableLanguageModels() calls https://api.together.xyz/v1/models at runtime and filters entries where type === "chat" || type === "language" || type === "". No static list exists for chat models; Together’s live catalog drives what appears in the model picker. No code change is required when Together adds a new chat model.

TTS, ASR, and embedding models — static arrays in the provider

These are hardcoded in packages/runtime/src/providers/together-provider.ts:

TOGETHER_TTS_MODELS — 3 entries (Orpheus 3B, Kokoro 82M, Cartesia Sonic), each with a voices list.
TOGETHER_ASR_MODELS — 3 entries (Whisper Large v3, Voxtral Mini 3B, Parakeet TDT 0.6B v3).
TOGETHER_EMBEDDING_MODELS — 1 entry (Multilingual E5 Large, 1024 dims).

TTS and ASR models are also in the manifest, so they get workflow nodes (together.audio.<Model>TextToSpeech and together.transcription.<Model>Transcribe). Embedding models appear only in the provider array and have no workflow node.

Add a new model or node

The steps differ slightly by modality.

Image or video model

1. Open the generator and add an entry to the right catalog.

// packages/together-nodes/scripts/generate-manifest.mjs

const IMAGE_MODELS = [
  // existing entries ...
  {
    id: "black-forest-labs/FLUX.3-ultra",     // Together API model id
    name: "FLUX.3 Ultra",                     // display name → drives className
    tasks: ["text_to_image", "image_to_image"] // which node(s) to generate
  }
];

For video, append to VIDEO_MODELS with tasks: ["text_to_video", "image_to_video"] (or just one if the model is task-specific).

2. Regenerate and build.

cd packages/together-nodes
npm run gen:manifest        # rewrites src/together-manifest.json
npm run build               # tsc + copies manifest into dist/
npm test

That’s all. The factory auto-generates the node class, and manifest-models.ts picks up the new model for the provider’s model lists.

TTS model

TTS models need entries in both the manifest generator and the provider array.

1. Add to the generator’s TTS_MODELS.

// packages/together-nodes/scripts/generate-manifest.mjs

const TTS_MODELS = [
  // existing entries ...
  {
    id: "vendor/new-tts-model",
    name: "New TTS Model",
    voices: ["voice_a", "voice_b"]
  }
];

2. Regenerate and build (same as above).

3. Add the provider-side entry.

// packages/runtime/src/providers/together-provider.ts

const TOGETHER_TTS_MODELS: TTSModel[] = [
  // existing entries ...
  {
    id: "vendor/new-tts-model",
    name: "New TTS Model",
    provider: "together",
    voices: ["voice_a", "voice_b"]
  }
];

ASR model

ASR models also need entries in both places. Add { id, name } to ASR_MODELS in the generator, regenerate and build, then add the provider entry:

// packages/runtime/src/providers/together-provider.ts

const TOGETHER_ASR_MODELS: ASRModel[] = [
  // existing entries ...
  { id: "vendor/new-asr-model", name: "New ASR Model", provider: "together" }
];

Embedding model

Update only the provider array. The generator has no embedding catalog:

// packages/runtime/src/providers/together-provider.ts

const TOGETHER_EMBEDDING_MODELS: EmbeddingModel[] = [
  // existing entries ...
  {
    id: "vendor/new-embedding-model",
    name: "New Embedding Model",
    provider: "together",
    dimensions: 1024
  }
];

Verify

1. Build and type-check.

npm run build --workspace=packages/together-nodes   # tsc, so it type-checks

2. Run package tests.

npm run test --workspace=packages/together-nodes

3. Exercise the node in isolation (requires TOGETHER_API_KEY in secrets or env).

For a text-to-image node the type is together.image.<ClassName>:

npm run dev:nodetool -- node run together.image.FLUX1SchnellTextToImage \
  --props '{"prompt":"a red panda on a skateboard","width":512,"height":512}'

For a video node:

npm run dev:nodetool -- node run together.video.Veo30TextToVideo \
  --props '{"prompt":"timelapse of clouds","aspect_ratio":"16:9","resolution":"720p","duration":6}'

For a TTS node:

npm run dev:nodetool -- node run together.audio.Orpheus3BTextToSpeech \
  --props '{"text":"Hello from Together AI","voice":"tara","format":"mp3"}'

4. Static workflow check (no API call).

npm run dev:nodetool -- validate workflow.json

5. Affected tests, lint, and the harness gate.

npm run test:affected
npm run lint
npm run dev:nodetool -- harness gate --base origin/main

Edit this page on GitHub