I've been stress-testing Kimi's API for a new ingestion pipeline, and I keep hitting a wall with the token limit. The docs say the `max_tokens` for the input is 128K, but I'm getting `'input too long'` errors when my payload is well under that.
Here's the kicker: I'm counting tokens client-side using `tiktoken` (same encoding they claim to use, `cl100k_base`), and my counts are consistently 20-30% lower than what their API rejects. We're talking submissions around 100K tokens getting bounced.
This isn't a fuzzy "feels long" situation. My payloads are clean JSON with structured text. I've ruled out the obvious:
* The count includes the entire request JSON (system prompt, user message, etc.). Yes, I know.
* No off-by-one errors or miscalculated arrays.
* Tried both `messages` and `raw_text` endpoints, same result.
My working theory is one of two things:
1. Their token counting happens *after* some internal serialization or formatting, adding significant overhead.
2. The 128K limit isn't a pure token limit, but a raw character/byte limit they're approximating poorly.
Has anyone else run this to ground? I need a reliable, deterministic way to know if a payload will pass *before* I call the API. Guessing based on a fuzzy margin isn't acceptable for a production pipeline.
My test script snippet:
```python
import tiktoken
encoder = tiktoken.get_encoding("cl100k_base")
def count_tokens_in_messages(messages):
# Format as per API spec, then count
text = ""
for msg in messages:
text += msg["role"] + ": " + msg["content"] + "n"
return len(encoder.encode(text))
# Example structure that fails at ~100k tokens by my count
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": massive_text_string}
]
my_token_count = count_tokens_in_messages(messages)
print(f"My count: {my_token_count}") # e.g., 102,400
# API call fails with "input too long"
```
What's the real safe ceiling? 90K? 80K? And more importantly, **why**?
garbage in, garbage out