I've been running performance tests on Kimi's API for the last two weeks, specifically tracking response latency. The pattern is consistent and points to a systemic resource management issue, not random network noise.
My test setup:
* Scripts sending identical, medium-complexity analysis prompts every 5 minutes.
* Monitoring first token time and total completion time via the API.
* All tests from the same AWS region to control for geographic variables.
The observed behavior:
* **Minutes 0-30:** Average response time holds steady at 1.8-2.4 seconds.
* **Minute 30-35:** Latency begins to increase, averaging 4.7 seconds.
* **Minute 40+:** Responses regularly exceed 8-10 seconds, with occasional timeouts.
This suggests a session-based throttle or a resource pool exhaustion policy kicking in at the half-hour mark. From a cloud architecture standpoint, this could be several things:
* A deliberate, poorly documented rate-limiting tier for extended sessions.
* VM instance recycling or tenant isolation mechanisms that introduce context-switching overhead.
* Cache expiration on inference servers leading to cold starts.
The lack of transparency here is a compliance red flag for any serious implementation. If this is a design choice, it should be documented in the SLA. If it's a scaling bottleneck, it represents a vendor risk for processes requiring consistent performance.
Has anyone else conducted measured tests and correlated this with specific:
* Token window usage?
* Concurrent session limits per account?
* Noticeable differences between web interface and API performance?
Where is your SOC 2?