We've been piloting Kling for automated API integration and data ingestion from several internal microservices. While its natural language parsing for endpoint discovery is impressive, we're encountering a persistent and problematic pattern: the agent is confidently generating calls to API endpoints that are not present in our actual OpenAPI/Swagger documentation.
The issue manifests during the specification parsing and planning phase. For example, when given our `user-service` spec, which has a documented `GET /users/{id}` endpoint, Kling's execution plan will often include a call to `GET /users/{id}/preferences`, claiming it's for fetching user settings. No such endpoint exists in the spec file we provided. This causes the pipeline to fail at runtime with 404 errors.
We've tried the following to mitigate this, with limited success:
* Verified our OpenAPI 3.0 specs are valid and hosted at a stable URL.
* Explicitly provided the exact spec file path in the agent configuration.
* Used clear, imperative prompts like "Using only the endpoints defined in the provided specification, fetch data for user ID 456."
Our current configuration snippet is straightforward:
```yaml
agent:
source: kling
spec_url: "https://internal-api.company.com/specs/user-service.yaml"
instructions: "Ingest data from the user service API. Do not assume or invent endpoints."
```
The "hallucinated" endpoints are often plausible (e.g., `/preferences`, `/history`), suggesting it's drawing from a general pattern in its training data rather than our specific docs. This undermines reliability for automated workflows.
Has anyone else faced this? Is there a configuration parameter or prompt engineering technique to strictly bind the agent to the provided specification and suppress this extrapolative behavior? We need deterministic behavior based on the documented contract.
— DN
Data is the only truth.
Oh that's frustrating. We ran into something similar with a different tool last year. It kept inventing `/project/{id}/summary` endpoints because it had seen similar patterns in public API docs during training.
I wonder if Kling is doing pattern completion based on its training data, rather than strictly adhering to your provided spec. Have you checked if your prompts can include a stronger boundary instruction, like "Do not infer or create any endpoints not explicitly listed in the specification file"? Sometimes these agents need that extra, explicit guardrail.
Yeah, the stronger boundary prompt idea is a good one to try. I've found these tools can be surprisingly literal when you force them into that mode. Something like "You must only use endpoints exactly as they are named and described in the following spec. Do not deviate, assume, or create new endpoints" can sometimes shift the behavior.
But your core issue, where it's inventing `/preferences` routes, feels like it might be a deeper training bias issue, as user805 suggested. Kling's model might have been heavily trained on common API patterns, and it's defaulting to "completing" what it thinks a RESTful service *should* have, rather than what it *does* have. Have you reached out to their support team? A persistent pattern like this might need a model-side adjustment or a config flag they haven't documented yet.
Raise the signal, lower the noise.
The training bias point is spot on. I've seen this exact pattern with other agents - they try to be 'helpful' by filling in perceived gaps based on public API trends. It's less hallucination and more inappropriate extrapolation.
Your boundary prompt suggestion works in many cases, but it's brittle. If the model's training data strongly reinforces common patterns like `/preferences` or `/summary` subroutes, it can override even explicit instructions. That's when you need vendor support.
Have you tried providing Kling with only the spec, stripped of any natural language task description? Sometimes removing the 'why' from the prompt forces stricter adherence to the 'what'.
Ugh, this hits close to home. I had the exact same issue last year, but with a different vendor's "smart" API connector trying to infer `/{id}/history` endpoints from our sales object specs. It was so confident and wrong.
Everyone's suggestion about training bias is correct, but the boundary prompt alone might not stick. What finally worked for us was a two-part approach: first, that ultra-strict prompt you already tried, but second, and more importantly, we started feeding the spec *and then* immediately asking for a verification step. We'd prompt: "First, list every endpoint verb and path you see in the attached spec. Confirm that this is the complete list you will use." Making it explicitly enumerate its source material before planning seemed to ground it.
It's a tedious extra step, but it cut the phantom endpoint calls by about 90% for us. Have you tried forcing that kind of manual checkpoint in the flow?