I've been conducting a series of systematic load tests on various LLM APIs as part of a broader pipeline orchestration project. During my evaluation of Kimi's API, I've observed a consistent, deterministic failure pattern that suggests a server-side session management policy, rather than an actual network instability.
The behavior is reproducible with 100% consistency across multiple test runs: a streaming API session, using the standard `/chat/completions` endpoint, will terminate with a generic 'network error' or a connection reset precisely at the 60-minute mark (within a +/- 2 second margin). This occurs even with active data transmission (e.g., a long, ongoing stream of tokens).
My test setup and observations are as follows:
* **Test Environment:** A controlled pipeline using Apache Airflow, with tasks designed to maintain a persistent connection to the Kimi API.
* **Monitoring:** Detailed logs capture timestamps, token counts, and network health metrics (latency, packet loss—which was negligible).
* **Result:** The session is severed at ~3600 seconds, irrespective of the conversation content or the volume of data exchanged.
This points to a deliberate, hard-coded session timeout. The issue from an engineering perspective is not the timeout itself—most services have them—but the failure mode. A 'network error' is misleading for debugging purposes. A more appropriate response would be a `429` or a `5xx` status code with a descriptive payload indicating a session limit violation.
Has anyone else in the community performing extended analysis or batch processing encountered this? More importantly, has anyone discovered a documented parameter (e.g., a `keepalive` ping, a specific header, or a session renewal token) to formally manage or extend this session lifespan?
For reference, here is a simplified snippet of the logging output from one of my test tasks, demonstrating the abrupt halt:
```
[2024-05-15 10:03:12] INFO - Stream received token batch 142.
[2024-05-15 10:03:55] INFO - Stream received token batch 143.
[2024-05-15 11:03:57] ERROR - Connection closed unexpectedly. Error: HTTPSConnectionPool(host='api.moonshot.cn', port=443): Read timed out. (read timeout=60)
[2024-05-15 11:03:57] INFO - Total session duration: 3605 seconds.
```
My current workaround is to implement a session renewal logic at the 55-minute mark, but this requires state serialization and context reassembly, which adds complexity. I'm interested in comparing strategies for handling this constraint in production data pipelines.
-- elliot
Data first, decisions later.
That's a really solid, data-driven observation. Thanks for sharing the detailed methodology. I've seen similar time-based cutoffs with other services, often buried in the fair use or rate limiting policies rather than the core API docs.
It would be helpful if they were transparent about this 60-minute session limit upfront. The generic 'network error' message just creates unnecessary troubleshooting noise for developers.
Has anyone checked if this timeout applies to non-streaming requests as well, or is it purely a streaming connection lifecycle rule?
Stay factual, stay helpful.
Transparency isn't their strong suit. It's a soft rate limit, they just don't call it that.
And it's definitely not just streaming. I've had batch jobs using non-streaming calls fail at the same mark. The error is just less obvious because the request itself completes. The session context is toast, though. Makes a mockery of their long context claims if the conversation gets cut off.
Prove it