How to Reduce Token Usage When Sending JSON Data to LLMs in Java

Mar 25, 2026

Problem

When I send structured data to LLMs through tool calls or RAG payloads, I waste tokens on JSON syntax—braces, brackets, quotes, and repeated key names. These overhead costs accumulate quickly, eating into my context window budget and increasing API costs.

For example, sending 20 employee records as JSON:

[
  {"name": "Alice", "age": 28, "department": "Engineering"},
  {"name": "Bob", "age": 34, "department": "Marketing"},
  {"name": "Carol", "age": 31, "department": "Sales"}
  // ... 17 more records
]

This JSON format costs 412 tokens using OpenAI’s o200k_base encoding. That’s a lot of overhead just for syntax.

Environment

Java 8-24
json-io 4.98.0
OpenAI o200k_base token encoding

What happened?

I was building an agentic workflow where my Java application sends structured data to an LLM for analysis. The tool call results were eating up most of my context window.

Here’s my original code:

public String getEmployeesAsJson() {
    List<Employee> employees = employeeRepository.findAll();
    return JsonIo.toJson(employees, null);  // Standard JSON serialization
}

The JSON output worked, but I realized I was paying for:

Braces {} and brackets []
Repeated key names on every object
Quotes around every string

When I measured the token count:

EncodingRegistry registry = Encodings.newDefaultEncodingRegistry();
Encoding encoding = registry.getEncoding(EncodingType.O200K_BASE);

String json = JsonIo.toJson(employees, null);
int tokens = encoding.countTokens(json);

// For 20 employees: 412 tokens

That’s when I discovered TOON (Token-Oriented Object Notation).

How to solve it?

I switched from JSON to TOON format using the same json-io library:

public String getEmployeesForLLM() {
    List<Employee> employees = employeeRepository.findAll();
    return JsonIo.toToon(employees, null);  // TOON serialization
}

The TOON output looks like this:

[20]{name,age,department}:
  Alice,28,Engineering
  Bob,34,Marketing
  Carol,31,Sales
  ...

Now let me measure the token savings:

String json = JsonIo.toJson(employees, null);
String toon = JsonIo.toToon(employees, null);

int jsonTokens = encoding.countTokens(json);  // 412 tokens
int toonTokens = encoding.countTokens(toon);  // 231 tokens

double savings = 100.0 * (jsonTokens - toonTokens) / jsonTokens;
// 44% savings!

You can see that I succeeded to reduce token consumption by 44% with a single method change.

The reason

TOON achieves these savings by eliminating redundant syntax:

Header declares field names once - Instead of repeating "name":, "age":, "department": on every object, TOON lists them in a header row.
No punctuation overhead - No braces, brackets, or excessive quotes. Just indentation and commas between values.
Tabular format for collections - Arrays of uniform objects become compact tables with one field header and value rows.

Here’s the token savings breakdown from my benchmarks:

Items    JSON tokens    TOON tokens    Savings
3        46             35             24%
10       206            121            41%
20       412            231            44%

The larger your dataset, the bigger the savings.

When to use TOON vs JSON

TOON is not a universal replacement. Here’s when to use each:

// Use TOON for LLM-bound data
String llmPayload = JsonIo.toToon(employees, null);

// Use JSON for REST APIs, message queues, config files
String apiResponse = JsonIo.toJson(employees, null);

Common mistakes to avoid:

Using pretty-print - JsonIo.toToon(employees, new WriteOptions().prettyPrint(true)) reverts to verbose format and loses token savings
Applying to single objects - For a single object, savings are only ~5%; TOON shines with collections
Using TOON for service-to-service APIs - External services expect JSON

Summary

In this post, I showed how to reduce LLM token consumption by 30-44% using TOON format. The key point is switching from JsonIo.toJson() to JsonIo.toToon() for data bound to LLMs, while keeping JSON for traditional API communication.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

👨‍💻 TOON Format Specification
👨‍💻 json-io GitHub Repository

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!