OpenAI-Compatible Providers: Add a Provider or Models

How to add a new OpenAI-compatible cloud provider (Groq, Mistral, DeepSeek, etc.) or adjust an existing one's model list.

Audience: Coding agents and contributors who want to add a new OpenAI-compatible cloud provider or change how an existing one exposes models.

This guide covers eight providers that share one implementation pattern: Groq, Mistral, DeepSeek, Moonshot (Kimi), Cerebras, Alibaba Cloud (Qwen), Cohere, and OpenRouter.

No code needed: add an endpoint from Settings

Most OpenAI-compatible endpoints — a proxy, a company gateway, a self-hosted router — need nothing from this guide. Settings → Models & Providers → OpenAI-compatible endpoints → Add endpoint takes a name, a slug, a base URL and an optional API key, and the provider appears in the model menu straight away. Add as many as you like.

The slug becomes the wire id: myproxy → custom_myproxy, which is what lands on model objects, spans and saved workflows. It is fixed once created, because renaming it would orphan every model reference in a saved graph.

The endpoint and key are stored per account and encrypted, and they also read from the environment under the same names — so a server can be configured with no database writes at all:

CUSTOM_MYPROXY_BASE_URL=https://proxy.example.com/v1
CUSTOM_MYPROXY_API_KEY=sk-…

Include the version path the endpoint serves (usually /v1) in the URL, the same way you would set baseURL on an OpenAI client. Models come from GET <base_url>/models. Endpoints without that route take a hand-written list of model ids instead.

An aggregator often lists chat, image and video models side by side. Each listed model goes to the matching node pickers:

  • A vendor field decides first. A type, model_type, task, category or mode naming image or video counts, as does OpenRouter-style output_modalities. A model whose output is image and text stays a chat model.
  • Otherwise the id decides. Family names such as flux, dall-e, seedream and imagen mean image, and kling, veo, sora and seedance mean video. A model classified only by its id also stays in the chat picker, so a misread chat model can still be picked.
  • The Image models and Video models fields in the endpoint dialog override both, for ids the detection misses. They are stored in the catalog as image_models and video_models.

Image models call POST <base_url>/images/generations and /images/edits with the requested size passed through unchanged. Video models call the OpenAI video API: POST <base_url>/videos, then poll until the clip is done. No resolution or duration limits are declared, so the node’s own options apply. Test reports how many chat, image and video models the endpoint serves, and runs on its own after each save.

What Path
Ids, secret names, validation packages/protocol/src/custom-providers.ts
Chat, image and video classification packages/runtime/src/providers/custom-model-kinds.ts
The provider packages/runtime/src/providers/custom-openai-provider.ts
Registration packages/runtime/src/providers/custom-provider-registry.ts
Storage + registry sync packages/websocket/src/custom-providers.ts
API packages/websocket/src/trpc/routers/custom-providers.ts (customProviders.list\|save\|delete\|test)
UI web/src/components/menus/CustomProvidersSection.tsx

Write a provider file instead when the endpoint needs more than the OpenAI routes over a base URL: its own media API, non-standard request fields, per-model tool support or size limits, or a curated model list that ships with NodeTool. That is the rest of this guide.

TL;DR

  • Models appear automatically: each provider fetches its /models endpoint at runtime, so new upstream models show up in the model picker without a code change.
  • Adding a whole new provider is three small edits: one constant in @nodetool-ai/protocol, one ~90-line file in packages/runtime/src/providers/, one registerBuiltinProvider call in the runtime index.
  • Cohere is the only exception: it subclasses BaseProvider directly (embeddings only, no chat).
  • Moonshot is another exception: it subclasses AnthropicProvider (Kimi exposes an Anthropic-compatible endpoint, not OpenAI).

Where things live

What Path
Provider ID constants packages/protocol/src/api-types.ts (PROVIDER_IDS const, line 858)
Provider source files packages/runtime/src/providers/<name>-provider.ts
Registration + exports packages/runtime/src/providers/index.ts
OpenAI base class packages/runtime/src/providers/openai-provider.ts
Base class for all providers packages/runtime/src/providers/base-provider.ts
Provider types (LanguageModel, etc.) packages/runtime/src/providers/types.ts

The shared pattern

DeepSeek is the canonical example. The full 93-line file is annotated below.

// packages/runtime/src/providers/deepseek-provider.ts

import OpenAI from "openai";
import { OpenAIProvider } from "./openai-provider.js";        // (1) inherit chat, streaming, tools
import type { LanguageModel } from "./types.js";

const DEEPSEEK_BASE_URL = "https://api.deepseek.com/v1";     // (2) provider's OpenAI-compatible base

interface DeepSeekProviderOptions {
  client?: OpenAI;
  clientFactory?: (apiKey: string) => OpenAI;
  fetchFn?: typeof fetch;                                     // (3) injected for testability
}

export class DeepSeekProvider extends OpenAIProvider {
  // (4) requiredSecrets() names the env/DB key the registry resolves at call time
  static override requiredSecrets(): string[] {
    return ["DEEPSEEK_API_KEY"];
  }

  private _deepseekFetch: typeof fetch;

  constructor(
    secrets: { DEEPSEEK_API_KEY?: string },
    options: DeepSeekProviderOptions = {}
  ) {
    const apiKey = secrets.DEEPSEEK_API_KEY;
    if (!apiKey) {
      throw new Error("DEEPSEEK_API_KEY is required");
    }

    const fetchFn = options.fetchFn ?? globalThis.fetch.bind(globalThis);

    // (5) pass key as OPENAI_API_KEY; supply a clientFactory that sets baseURL
    super(
      { OPENAI_API_KEY: apiKey },
      {
        client: options.client,
        clientFactory:
          options.clientFactory ??
          ((key) => new OpenAI({ apiKey: key, baseURL: DEEPSEEK_BASE_URL })),
        fetchFn
      }
    );

    // (6) override the `provider` field so spans and model objects carry the right id
    (this as { provider: string }).provider = "deepseek";
    this._deepseekFetch = fetchFn;
  }

  // (7) export the provider-specific key to Docker/subprocess environments
  override getContainerEnv(): Record<string, string> {
    return { DEEPSEEK_API_KEY: this.apiKey };
  }

  // (8) true when the API supports function/tool calling; override per-model if mixed
  override async hasToolSupport(_model: string): Promise<boolean> {
    return true;
  }

  // (9) fetch the live model list; return [] on failure (graceful degradation)
  override async getAvailableLanguageModels(): Promise<LanguageModel[]> {
    const response = await this._deepseekFetch(
      `${DEEPSEEK_BASE_URL}/models`,
      { headers: { Authorization: `Bearer ${this.apiKey}` } }
    );
    if (!response.ok) return [];

    const payload = (await response.json()) as {
      data?: Array<{ id?: string; name?: string }>;
    };
    const rows = payload.data ?? [];
    return rows
      .filter(
        (row): row is { id: string; name?: string } =>
          typeof row.id === "string" && row.id.length > 0
      )
      .map((row) => ({ id: row.id, name: row.name ?? row.id, provider: "deepseek" }));
  }
}

Groq, Mistral, Cerebras, Alibaba Cloud, and OpenRouter follow this pattern exactly. Alibaba Cloud reads DASHSCOPE_API_KEY and points at Model Studio’s international DashScope endpoint, https://dashscope-intl.aliyuncs.com/compatible-mode/v1. OpenRouter adds a few extras: defaultHeaders on the OpenAI client (HTTP-Referer, X-Title), textToImage, and a hasToolSupport override that returns false for o1/o3 model IDs.

Add a brand-new OpenAI-compatible provider

Step 1 — Add the provider ID to @nodetool-ai/protocol

Open packages/protocol/src/api-types.ts and add a key to PROVIDER_IDS:

// packages/protocol/src/api-types.ts  (around line 858)

export const PROVIDER_IDS = {
  // ... existing entries ...
  ACME: "acme",                   // wire id — used in model objects, spans, settings
} as const;

The comment above the const (line 855) says: “Adding a provider? Add it here first, then register it in @nodetool-ai/runtime’s provider index.” Follow that order.

Build the protocol package so downstream packages see the new constant:

cd packages/protocol && npm run build

Step 2 — Write acme-provider.ts

Create packages/runtime/src/providers/acme-provider.ts. Copy the DeepSeek template above and substitute:

Placeholder Replace with
DeepSeek Acme
DEEPSEEK_API_KEY ACME_API_KEY
DEEPSEEK_BASE_URL / https://api.deepseek.com/v1 ACME’s base URL
"deepseek" (the wire id string) "acme"
this._deepseekFetch this._acmeFetch

If the provider’s /models response uses a different shape (not { data: [{ id, name }] }), adjust the getAvailableLanguageModels parser accordingly.

If the provider does not support tool calling for some or all models, implement hasToolSupport to return false where appropriate (see the OpenRouter implementation for a model-name heuristic).

Step 3 — Export and register in packages/runtime/src/providers/index.ts

Two lines:

// near the other thin-provider imports
import { AcmeProvider } from "./acme-provider.js";

// near the other exports
export { AcmeProvider };

And one registerBuiltinProvider call in the registration block (around line 199):

registerBuiltinProvider(PROVIDER_IDS.ACME, AcmeProvider, { ACME_API_KEY: "" });

The empty string for ACME_API_KEY is intentional — it forces the registry to resolve the key from the DB or env on every getProvider() call rather than baking a stale value at module load.

Adjust an existing provider’s models

Chat models are fetched dynamically — no change needed

If Groq, DeepSeek, Mistral, Cerebras, Alibaba Cloud, or OpenRouter adds a new chat model, it appears in the model picker automatically on the next call to getAvailableLanguageModels(). No code change is required.

Tool-support overrides

hasToolSupport(model: string) controls whether the UI offers tool/function calling for a given model. All five OpenAI-subclass providers currently return true unconditionally. OpenRouter overrides per-model-id:

// packages/runtime/src/providers/openrouter-provider.ts
override async hasToolSupport(model: string): Promise<boolean> {
  const lower = model.toLowerCase();
  if (lower.includes("o1") || lower.includes("o3")) return false;
  return true;
}

Apply the same pattern to any other provider where some models lack tool support.

Embedding models (Mistral)

Mistral is the only OpenAI-subclass provider that also exposes embeddings. The model list is static (one entry, mistral-embed) and lives in getAvailableEmbeddingModels() in packages/runtime/src/providers/mistral-provider.ts. To add an embedding model, append to that array.

Cohere — embeddings-only, different base class

Cohere subclasses BaseProvider directly, not OpenAIProvider. It has no chat support. Its embedding model list is the static COHERE_EMBEDDING_MODELS array at the top of packages/runtime/src/providers/cohere-provider.ts. Append there to add a new Cohere embedding model.

Moonshot — Anthropic-compatible endpoint

Moonshot subclasses AnthropicProvider and uses a hardcoded knownModels list in getAvailableLanguageModels() (not a live fetch). To add a Kimi model, add its id to that array in packages/runtime/src/providers/moonshot-provider.ts.

Verify

After any change:

1. Build protocol if you changed api-types.ts.

cd packages/protocol && npm run build

2. Type-check.

npm run typecheck

3. Confirm the provider returns models (requires the API key in secrets or env).

npm run dev:nodetool -- info --json | grep acme

Or run a single chat to exercise the full path:

npm run dev:chat -- --provider acme --model <model-id>

4. Single-node smoke test (chat node via the provider).

npm run dev:nodetool -- node run nodetool.agents.Agent \
  --props '{"prompt":"hello","model":{"type":"language_model","provider":"acme","id":"<model-id>"}}'

5. Full suite.

npm run check

How past PRs did it

The seven OpenAI-subclass providers arrived across four commits, not one:

  • 4469fd80 (“Add TypeScript backend packages from nodetool-core/ts”) — cerebras-provider.ts, groq-provider.ts, mistral-provider.ts, openrouter-provider.ts
  • 377d2304 (“feat: add Moonshot AI (Kimi) provider via Anthropic-compatible endpoint”) — moonshot-provider.ts
  • e2118f8e (“feat(runtime): add DeepSeek and xAI (Grok) providers”) — deepseek-provider.ts
  • a21a5131 (“feat(runtime): add Cohere, Voyage AI, and Jina AI embedding providers”) — cohere-provider.ts

Each wired its provider into index.ts in the same commit — that pairing, not the batch, is the pattern to copy.

Commit 34599547 (“refactor(providers): centralize provider IDs in protocol”) moved provider id strings out of scattered literals and into PROVIDER_IDS in @nodetool-ai/protocol. Any new provider follows the post-refactor convention: constant in protocol, PROVIDER_IDS.X everywhere else.

Contributing

Edit this page on GitHub