How to Use TOON with ChatGPT, Claude, and Other LLMs

If you're working with large datasets in your LLM prompts, you're probably paying more than you need to. This guide shows you how to use TOON (Token-Oriented Object Notation) to cut your token usage by 30-60% while keeping—or even improving—response accuracy.

Why Bother?

The math is simple. When you're sending hundreds or thousands of API requests with large payloads:

• GPT-4 at $0.03 per 1K tokens → 60% savings means you're paying $0.018 instead
• Claude 3 at $0.015 per 1K tokens → 60% savings brings it down to $0.009
• Scale that across thousands of requests, and you're looking at real money

Beyond cost, there's a practical benefit: TOON's structure actually helps models understand data better. In our testing, TOON achieved 73.9% accuracy compared to JSON's 69.7%. The tabular format with explicit headers makes it easier for models to parse relationships.

And when you're bumping against context window limits, fitting 2.5x more data in the same space means you can include more context without truncation.

Quick Example

Here's the difference in practice:

The JSON way (85 tokens)

const data = {
  "users": [
    { "id": 1, "name": "Alice", "role": "admin" },
    { "id": 2, "name": "Bob", "role": "user" }
  ]
};

const prompt = `
Analyze this user data:
${JSON.stringify(data, null, 2)}

Which users are admins?
`;

The TOON way (34 tokens)

import { encode } from '@toon-format/toon';

const toonData = encode(data);
// users[2]{id,name,role}:
//   1,Alice,admin
//   2,Bob,user

const prompt = `
Analyze this user data (TOON format):
\`\`\`
${toonData}
\`\`\`

Which users are admins?
`;

That's 60% fewer tokens for the exact same information. Scale that up to 1000 user records, and the savings compound.

Using TOON with Different LLMs

ChatGPT (OpenAI API)

The simplest approach is to just include TOON data directly in your prompt:

import OpenAI from 'openai';
import { encode } from '@toon-format/toon';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const data = await fetchUsersFromDatabase();
const toonData = encode(data);

const response = await openai.chat.completions.create({
  model: "gpt-4",
  messages: [{
    role: "user",
    content: `
Here's a dataset in TOON format (field names are declared once in the header):

\`\`\`
${toonData}
\`\`\`

Find all users in the Engineering department.
    `.trim()
  }]
});

For better results, add a system message that explains the format once:

const response = await openai.chat.completions.create({
  model: "gpt-4",
  messages: [
    {
      role: "system",
      content: `When you receive data in TOON format, understand that field names
are declared once in headers like: items[N]{field1,field2}: followed by value rows.
This is more token-efficient than JSON where field names repeat for every object.`
    },
    {
      role: "user",
      content: `\`\`\`\n${toonData}\n\`\`\`\n\nAnalyze the data above.`
    }
  ]
});

Claude (Anthropic API)

Claude handles TOON well with minimal explanation:

import Anthropic from '@anthropic-ai/sdk';
import { encode } from '@toon-format/toon';

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

const data = await fetchEmployees();
const toonData = encode(data);

const message = await anthropic.messages.create({
  model: "claude-3-opus-20240229",
  max_tokens: 1024,
  messages: [{
    role: "user",
    content: `
Employee data in TOON format (fields declared once in header):

\`\`\`
${toonData}
\`\`\`

Calculate the average salary by department.
    `.trim()
  }]
});

Gemini (Google AI)

import { GoogleGenerativeAI } from '@google/generative-ai';
import { encode } from '@toon-format/toon';

const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);
const model = genAI.getGenerativeModel({ model: "gemini-pro" });

const data = await fetchProducts();
const toonData = encode(data);

const prompt = `
Product catalog in TOON format:

\`\`\`
${toonData}
\`\`\`

Find the top 5 most expensive products.
`;

const result = await model.generateContent(prompt);
console.log(result.response.text());

Writing Better TOON Prompts

Use code blocks for clarity

Wrap your TOON data in code blocks to separate it from instructions:

// Do this
const prompt = `
Here's the data:
\`\`\`
${toonData}
\`\`\`

Find all active users.
`;

// Not this
const prompt = `
Here's the data:
${toonData}

Find all active users.
`;

Code blocks make it unambiguous where the data starts and ends, especially when you have multi-line values or special characters.

Add brief context on first use

The first time you use TOON in a conversation, mention it briefly:

const prompt = `
Here's employee data in TOON format (a compact, tabular format):
\`\`\`
${toonData}
\`\`\`
...
`;

After that, models generally remember the format within the conversation.

Handle commas in data

If your data contains commas in the values (like addresses or descriptions), use tab or pipe delimiters:

const toonData = encode(data, {
  delimiter: '\t'  // tab delimiter
});

// Result:
// addresses[2]{street	city	country}:
//   123 Main St, Suite 100	Boston	USA
//   456 Oak Ave, Apt 5B	Seattle	USA

Or use pipe delimiters for maximum clarity:

const toonData = encode(data, {
  delimiter: '|'
});

// addresses[2]{street|city|country}:
//   123 Main St, Suite 100|Boston|USA

Collapse nested keys when appropriate

For deeply nested single-key chains, key folding can help:

const toonData = encode(data, {
  keyFolding: 'safe'
});

// Instead of:
// user:
//   profile:
//     settings:
//       theme: dark

// You get:
// user.profile.settings.theme: dark

This works well for configuration objects but can be harder to read for complex nested structures. Use it when the nesting adds more noise than value.

Advanced Patterns

Teaching models to generate TOON

You can ask models to output data in TOON format:

const prompt = `
Generate 10 sample users in TOON format. Use this structure:

users[N]{id,name,email,role}:
  1,Alice,alice@example.com,admin
  2,Bob,bob@example.com,user
  ...

Requirements:
- [N] should match the actual number of rows
- All rows must have the same number of fields
- Use 2-space indentation
`;

This is useful when you want consistent, structured output that you can parse programmatically.

Multi-step data processing

For complex analysis, break it into steps:

const prompt = `
Step 1: Here's the raw data in TOON format:
\`\`\`
${toonData}
\`\`\`

Step 2: Filter for records where department='Engineering'

Step 3: Calculate average salary for those records

Step 4: Return the results in TOON format
`;

Explicit steps help models follow your logic and make debugging easier.

Data transformation workflows

You can use TOON as an intermediate format:

const prompt = `
Input (JSON):
${JSON.stringify(jsonData)}

Transform this to TOON format following the pattern:
items[N]{field1,field2,...}:
  value1,value2,...

Then analyze the transformed data for anomalies.
`;

Real-World Examples

E-commerce recommendations

import { encode } from '@toon-format/toon';
import OpenAI from 'openai';

async function getRecommendations(userId, products, userHistory) {
  const openai = new OpenAI();

  const productsToon = encode({ products });
  const historyToon = encode({ history: userHistory });

  const response = await openai.chat.completions.create({
    model: "gpt-4",
    messages: [{
      role: "user",
      content: `
User ID: ${userId}

Product Catalog:
\`\`\`
${productsToon}
\`\`\`

User's Purchase History:
\`\`\`
${historyToon}
\`\`\`

Based on the history, recommend 5 products. Return as:
recommendations[5]{product_id,reason}:
      `.trim()
    }]
  });

  return response.choices[0].message.content;
}

Log analysis

async function analyzeLogs(logs) {
  const toonLogs = encode({
    logs: logs.map(l => ({
      timestamp: l.timestamp,
      level: l.level,
      message: l.message,
      user_id: l.user_id
    }))
  });

  const analysis = await callClaude(`
    Analyze these application logs:
    \`\`\`
    ${toonLogs}
    \`\`\`

    Identify:
    1. Error patterns
    2. Suspicious activities
    3. Performance bottlenecks

    Format your findings as a structured report.
  `);

  return analysis;
}

Database query explanation

async function explainQueryResults(sqlQuery, results) {
  const toonResults = encode({ results });

  const explanation = await callGemini(`
    SQL Query:
    \`\`\`sql
    ${sqlQuery}
    \`\`\`

    Results:
    \`\`\`
    ${toonResults}
    \`\`\`

    Explain these results in plain English for non-technical users.
  `);

  return explanation;
}

Common Issues

"The model doesn't understand TOON"

Add explicit format instructions:

const prompt = `
TOON format explanation:
- Field names are declared once in headers: items[count]{field1,field2,...}:
- Followed by indented rows with values only
- Example: users[2]{id,name}: → "1,Alice" → "2,Bob"

Now analyze this data:
\`\`\`
${toonData}
\`\`\`
`;

Most modern LLMs handle TOON well, but being explicit helps with edge cases.

"Token count is still high"

Try these optimizations:

1. Use tab delimiter: encode(data, { delimiter: '\t' })
2. Enable key folding: encode(data, { keyFolding: 'safe' })
3. Remove unnecessary fields before encoding
4. For pure tabular data, plain CSV might be even better

"Model generates invalid TOON"

Provide examples in your prompt:

const prompt = `
Generate employee data in TOON format. Example:

employees[3]{id,name,dept}:
  101,Alice,Engineering
  102,Bob,Sales
  103,Carol,Marketing

Now generate 5 employees following the same format:
`;

Measuring Your Savings

Here's a simple calculator to measure the impact:

import { encode as tokenize } from 'gpt-tokenizer';
import { encode } from '@toon-format/toon';

function calculateSavings(data, pricePerThousand = 0.03) {
  const jsonStr = JSON.stringify(data);
  const toonStr = encode(data);

  const jsonTokens = tokenize(jsonStr).length;
  const toonTokens = tokenize(toonStr).length;

  const savedTokens = jsonTokens - toonTokens;
  const savedPercentage = (savedTokens / jsonTokens * 100).toFixed(1);
  const savedCost = (savedTokens / 1000 * pricePerThousand).toFixed(4);

  return {
    jsonTokens,
    toonTokens,
    savedTokens,
    savedPercentage: `${savedPercentage}%`,
    savedCost: `$${savedCost} per request`
  };
}

// Example with 100 user records
const data = { users: [/* ... 100 users ... */] };
console.log(calculateSavings(data));
// { jsonTokens: 3245, toonTokens: 1298, savedTokens: 1947,
//   savedPercentage: '60.0%', savedCost: '$0.0584 per request' }

Run this on your actual data to see the real-world impact.

Getting Started

The workflow is straightforward:

1. Install the package: npm install @toon-format/toon
2. Convert your data: const toonData = encode(data)
3. Include it in your prompts with code blocks
4. Measure the token savings

For tabular data with repeated structures (user lists, logs, product catalogs), you'll typically see 50-60% reduction. For nested objects, 30-40% is common.

Try our free converter to test it with your own data, or check the TOON documentation for advanced features like custom delimiters and key folding.