Token cost is one of the most controllable variables in LLM applications. Most prompts sent to production APIs contain 20-40% redundant tokens — filler phrases, verbose instructions, and unnecessary examples that do not change the model's output. Removing them reduces cost and often improves clarity. Here is a systematic approach.
Why Prompt Compression Matters
At scale, token costs add up quickly. If your application sends 1,000 prompts per day with an average prompt length of 500 tokens, and you can reduce that to 300 tokens, you save 200,000 tokens per day. At GPT-4o pricing of approximately $2.50 per million input tokens, that is $0.50 per day, or $182 per year — from one prompt optimization. Applications with higher volume or longer prompts see proportionally larger savings.
Beyond cost, shorter prompts often produce better outputs. Models have finite attention. Long prompts with padding dilute the signal of the important instructions. A focused 300-token prompt tends to produce more on-target responses than a 500-token version with the same core instructions buried in filler.
Technique 1: Remove Filler Phrases
Filler phrases are sentences that sound polite or professional but add no information for the model. Compare:
Before:
Please make sure to carefully review the following text and provide a comprehensive and detailed summary that captures all of the main points. It's very important that you don't miss any key information. The summary should be thorough and accurate.
After:
Summarize the following text. Include all key points.
The second version is 12 tokens. The first is 52. The output quality is identical or better with the shorter version because the instruction is clearer.
Common filler phrases to remove:
- "Please make sure to..."
- "It's very important that..."
- "I would like you to..."
- "Carefully and thoroughly..."
- "Comprehensive and detailed..."
- "Make sure you don't forget to..."
These phrases are patterns humans use in spoken language to soften requests. Models do not need social softening — they respond to direct instructions.
Technique 2: Use Bullet Points for Instructions
When you have multiple constraints, paragraphs use more tokens than bullet points for the same information. Compare:
Before (paragraph form):
When writing the email, make sure to use a professional tone and avoid using slang or casual language. The email should be no longer than 150 words. You should start with a direct statement of the purpose of the email and end with a clear call to action.
After (bullet form):
Write a professional email. Requirements:
- Tone: formal, no slang
- Length: max 150 words
- Structure: purpose statement first, call to action last
The bullet version is approximately 35% shorter and scans faster for both the model and any human reviewing the prompt.
Technique 3: Define Terms Once, Then Abbreviate
If your prompt uses a long term repeatedly, define it once and use an abbreviation for subsequent mentions.
Before:
Analyze the customer support conversation below. Identify each instance where the customer expressed frustration. For each instance of customer frustration, note the cause of the frustration and how the support agent responded to the frustration...
After:
Analyze the customer support conversation below. Identify each instance where the customer expressed frustration (CF). For each CF, note: the cause and the agent's response...
This works especially well in long prompts with repeated domain-specific terms. Define the abbreviation in parentheses on first use and use it consistently afterward.
Technique 4: Remove Examples That Are Not Needed
Few-shot examples are one of the biggest sources of token cost in prompts. Each example adds 50-200 tokens. Before including an example, ask: does the model need this to understand the task, or am I adding it out of habit?
For tasks with clear, unambiguous instructions (translate this text, classify this sentiment as positive/negative/neutral, extract the date from this text), zero-shot prompts often perform as well as few-shot prompts. Test with and without examples. If the output quality is the same, remove them.
When examples are needed, use the shortest example that demonstrates the pattern. An example that covers the format but not every edge case is sufficient. You do not need five examples to demonstrate a simple transformation — one or two are usually enough.
Technique 5: Use Structured Formats to Compress Information Density
Prose descriptions of structured data use more tokens than the data itself. If you are passing a schema, use a compact representation.
Before:
The user object has a field called "name" which is a string containing the user's full name. It also has a field called "email" which is a string containing the user's email address. There is also an "age" field which is a number representing the user's age in years.
After:
User schema: { name: string, email: string, age: number }
The compressed version is approximately 80% shorter and is clearer. Models are trained on code and structured formats — they understand JSON schemas, TypeScript interfaces, and table formats natively.
Token Counting Tools
Before and after compressing a prompt, count the tokens to measure the actual reduction.
For GPT models: Use the tiktoken Python library. It implements the exact tokenization used by OpenAI's models.
import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o")
tokens = enc.encode("your prompt here")
print(len(tokens))
For Claude: Use the token counting endpoint in the Anthropic API:
import anthropic
client = anthropic.Anthropic()
response = client.messages.count_tokens(
model="claude-opus-4-5",
messages=[{"role": "user", "content": "your prompt here"}]
)
print(response.input_tokens)
Rule of thumb: English text averages approximately 4 characters per token. A 2,000-character prompt is roughly 500 tokens. Use this to estimate before measuring precisely.
Before and After: A Real Example
Here is a 487-token prompt reduced to 201 tokens with the same output quality.
Before (487 tokens):
You are a helpful assistant that is going to help me analyze customer feedback. I would like you to carefully read through the customer feedback that I provide and identify the main themes that appear across the feedback. It's really important that you identify all of the themes and don't miss any. For each theme that you identify, please provide a brief description of what the theme is about and give me some specific examples of the customer feedback that relates to that theme. Please make sure that your response is well-organized and easy to read. The customer feedback I would like you to analyze is as follows: [feedback text]
After (201 tokens):
Analyze customer feedback below. For each theme:
1. Name the theme
2. Describe it in one sentence
3. List 2-3 verbatim examples
Feedback: [feedback text]
The output from the second version is better organized because the model follows the explicit numbered structure rather than interpreting a verbose prose description.
When Compression Hurts
Not every part of a prompt can be compressed. Remove words cautiously in these situations:
Complex multi-step instructions. When each step depends on the previous one and the sequence matters, abbreviated instructions sometimes cause the model to skip steps or perform them in the wrong order. If you see regression in output quality after compressing multi-step instructions, restore the fuller version.
Edge case specifications. "Do not include values that are null" is 8 tokens. Removing it to save tokens means the model will sometimes include null values. If an edge case matters for correctness, keep the instruction.
Ambiguous domain terms. In specialized domains (legal, medical, financial), abbreviating terminology can introduce ambiguity. If "TPA" could mean "third-party administrator" or "tissue plasminogen activator" depending on context, spell it out.
Summary
Prompt compression is one of the highest-leverage prompt engineering techniques because it directly reduces cost and often improves quality. Remove filler phrases, convert paragraphs to bullets, abbreviate repeated terms, eliminate unnecessary examples, and use compact structured formats. Count tokens before and after to measure actual savings. Test compressed prompts against your baseline output quality before deploying to production — regression in quality is rare but worth checking.
Keep Reading
- Prompt Versioning Guide — managing compressed and uncompressed versions in production
- Prompt Testing Methodology Guide — how to verify that compression does not reduce output quality
- Structured Output Prompting Guide — compact formats that improve both compression and parsability
Pristren builds AI-powered software for teams. Zlyqor is our all-in-one workspace — chat, projects, time tracking, AI meeting summaries, and invoicing — in one tool. Try it free.
