---
title: "Tokenize"
description: "Local-first visual environment for building and running AI workflows. Build agents visually, deploy anywhere, privacy by design."
canonical: https://docs.nodetool.ai/nodes/lib/nlp/tokenize
markdown: https://docs.nodetool.ai/nodes/lib/nlp/tokenize.md
product: NodeTool
source: https://github.com/nodetool-ai/nodetool/blob/main/docs/nodes/lib/nlp/tokenize.md
---

# Tokenize

**Type:** `lib.nlp.Tokenize`

**Namespace:** `lib.nlp`

## Description

Tokenizes text into words or sentences.
    tokenize, split, words, sentences, NLP

    Use cases:
    - Break text into individual words for analysis
    - Split text into sentences for processing
    - Prepare text for further NLP operations
    - Count words or sentences in text

## Properties

| Property | Type | Description | Default |
|----------|------|-------------|---------|
| text | `str` | The text to tokenize | `` |
| mode | `enum` | Tokenization mode: split into words or sentences | `word` |

## Outputs

| Output | Type | Description |
|--------|------|-------------|
| output | `list` |  |
| count | `int` |  |

## Related Nodes

Browse other nodes in the [lib.nlp](./) namespace.