Ollama: Add Models

How to add local Ollama models to NodeTool and how to change provider-level behavior such as tool-support detection.

The Ollama provider connects NodeTool to a local Ollama daemon. Models are discovered dynamically from the running daemon — no code change is needed to add a new model. Provider-level changes (tool detection, embedding behavior, URL configuration) require editing the provider source.

Runs: locally. Needs: no API key, only a running Ollama daemon. OLLAMA_API_URL overrides the default http://127.0.0.1:11434.

TL;DR

ollama pull llama3.2          # makes the model appear in NodeTool automatically
ollama pull nomic-embed-text  # embedding models work the same way

No code change. No restart required (NodeTool queries the daemon on every model-list call). If you need to change tool-support detection, the default URL, or keep_alive behavior, see Provider-level changes below.

Where things live

Concern Path
Provider implementation packages/runtime/src/providers/ollama-provider.ts
Default daemon URL packages/runtime/src/providers/defaults.ts (OLLAMA_DEFAULT_URL)
Registration + optionalKwargs packages/runtime/src/providers/index.ts (the PROVIDER_IDS.OLLAMA block)
Provider registry resolution logic packages/runtime/src/providers/provider-registry.ts

How Ollama models are discovered

OllamaProvider.getAvailableLanguageModels() fetches GET {apiUrl}/api/tags from the local daemon on every call. It maps each entry’s model field (falling back to name) to a LanguageModel record:

// packages/runtime/src/providers/ollama-provider.ts
async getAvailableLanguageModels(): Promise<LanguageModel[]> {
  const response = await this._fetch(`${this.apiUrl}/api/tags`);
  if (!response.ok) return [];
  const payload = (await response.json()) as {
    models?: Array<{ name?: string; model?: string }>;
  };
  ...
}

getAvailableEmbeddingModels() calls getAvailableLanguageModels() and re-wraps every result as an EmbeddingModel with dimensions: 0. Ollama does not advertise context size or dimensions over /api/tags, so both methods surface all pulled models regardless of type. The caller (RAG pipeline, agent config) is responsible for picking an appropriate model.

URL resolution order (highest priority first):

  1. Secret store, key OLLAMA_API_URL (set via nodetool secrets store OLLAMA_API_URL or Settings → Integrations → Local Model Servers)
  2. Environment variable — OLLAMA_API_URL
  3. Registered default — http://127.0.0.1:11434 (OLLAMA_DEFAULT_URL in defaults.ts)

The registration in index.ts uses an optionalKwarg for OLLAMA_API_URL so the registry always considers Ollama “configured” (no key required) while still resolving a user-set URL on every getProvider() call without a restart.

// packages/runtime/src/providers/index.ts
registerBuiltinProvider(
  PROVIDER_IDS.OLLAMA,
  OllamaProvider,
  {},
  {
    OLLAMA_API_URL: OLLAMA_DEFAULT_URL,
    OLLAMA_CONTEXT_LENGTH: "",
    OLLAMA_KEEP_ALIVE: ""
  },
  { access: "local_service", displayName: "Ollama" }
);

Ollama is registered only when the cloud profile is inactive. The cloud profile is active when NODETOOL_NODE_PROFILE=cloud, or when NODETOOL_NODE_PROFILE is unset and NODETOOL_ENV=production. Self-hosted production servers that set NODETOOL_NODE_PROFILE=full keep Ollama, so they can point it at a sidecar container.

The constructor also reads OLLAMA_KEEP_ALIVE from the secret store, then process.env. Default is "10m". Set it to "-1" to keep models resident indefinitely, or "0" to unload immediately after each request.

OLLAMA_CONTEXT_LENGTH (secret store, then env) sets num_ctx for every chat request. When it is unset, the provider uses the Modelfile’s num_ctx, else the model’s trained context length capped at OLLAMA_DEFAULT_NUM_CTX (32768). See resolveNumCtx in ollama-provider.ts.

Add a model (user path)

Pull the model with the Ollama CLI:

# Language models
ollama pull llama3.2
ollama pull gemma3:27b
ollama pull phi4
ollama pull deepseek-r1:8b

# Embedding models
ollama pull nomic-embed-text
ollama pull mxbai-embed-large

# Vision models (multimodal)
ollama pull llava:13b
ollama pull minicpm-v

NodeTool picks up the model on the next call to getAvailableLanguageModels() or getAvailableEmbeddingModels() — no restart needed. The model ID passed to the provider must match the tag exactly as shown by ollama list (e.g. llama3.2:latest, gemma3:27b).

To point NodeTool at a non-default Ollama instance (remote host, different port):

npm run dev:nodetool -- secrets store OLLAMA_API_URL
# Paste the URL when prompted, e.g.: http://192.168.1.10:11434

Or set the environment variable before starting the server:

OLLAMA_API_URL=http://192.168.1.10:11434 npm run dev

Provider-level changes (dev path)

Edit packages/runtime/src/providers/ollama-provider.ts. The main extension points:

Tool-support detection

hasToolSupport(model) queries POST {apiUrl}/api/show and checks for "tools" in the capabilities array. It caches results in _modelInfoCache (a Map<string, Record<string, unknown>>). When the /api/show call fails or the model has no capabilities field, it defaults to true.

If a model reports no tool support, generateMessage and generateMessages fall back to _injectToolEmulationPrompt + _parseEmulatedToolCalls, which inject tool descriptions into the system prompt and parse function_name(param='value') patterns from the output.

To override detection for a specific model (e.g. a model that advertises tools but calls them incorrectly), add a guard at the top of hasToolSupport:

if (model.startsWith("my-broken-model")) return false;

Embedding dimensions

getAvailableEmbeddingModels() returns dimensions: 0 for every model because Ollama’s /api/tags response does not include that field. If you need accurate dimensions, query POST /api/show for the pulled model and read model_info["embedding_length"]. The _modelInfoCache already stores the full /api/show response, so the call is free on the second access.

Keep-alive

Change the default (currently "10m") by editing the fallback in the constructor:

// packages/runtime/src/providers/ollama-provider.ts
const keepAlive = (
  secrets.OLLAMA_KEEP_ALIVE || process.env.OLLAMA_KEEP_ALIVE
)?.trim();
this.keepAlive = keepAlive && keepAlive.length > 0 ? keepAlive : "10m";

Default URL

Change OLLAMA_DEFAULT_URL in packages/runtime/src/providers/defaults.ts. The constant is re-exported from packages/runtime/src/providers/index.ts so callers import it as @nodetool-ai/runtime.

Verify

# 1. Confirm the daemon is running and a model is present
curl http://127.0.0.1:11434/api/tags

# 2. List the models NodeTool sees
npm run dev:nodetool -- models by-provider ollama --kind llm

# 3. Run a single Ollama-backed node (replace model with one you have pulled)
npm run dev:nodetool -- node run nodetool.agents.Agent \
  --props '{"prompt":"hello","model":{"type":"language_model","provider":"ollama","id":"llama3.2:latest"}}'

# 4. Affected tests
npm run test:affected

If the daemon is not running, getAvailableLanguageModels() returns [] silently (the provider guards if (!response.ok) return []). Check that ollama serve (or the Ollama desktop app) is running before debugging further.

Edit this page on GitHub