Hi everyone, new to the community and have been testing Hailuo for a few weeks now. I'm primarily looking at it for summarizing and extracting structured data from our internal technical documentation.
I've run into a recurring issue: it sometimes confidently invents details—like specific API parameter names or version numbers—that aren't actually present in the source material I provide. This is a deal-breaker for my use case, as accuracy is critical.
I'm familiar with the general concept of "prompt engineering" to reduce hallucinations in LLMs, but I'm looking for specifics that work well within Hailuo's interface and parameters.
* Are there particular instruction templates or system prompt formats you've found effective for grounding Hailuo strictly in the provided text?
* How do Hailuo's built-in features (like the "source" or "grounding" options I've seen mentioned) compare to manual prompt techniques?
* Does adjusting parameters like temperature or max tokens make a noticeable difference here compared to other models like GPT-4 or Claude?
* Is there a best practice for structuring the source text itself before feeding it in? For example, does it handle markdown or plain text better for accuracy?
I'm trying to understand if this is something I can mitigate with the right setup, or if it's a known limitation I need to work around. Any insights from your own workflows would be really helpful.
You can't. It's a feature, not a bug.
"source" and "grounding" are just branding for the same retrieval they all use. It reduces surface-level errors but won't stop it from filling gaps with plausible guesses, especially on technical specifics. You're expecting a parrot and you've got a pattern completer.
Lowering temperature might make it more consistent, not more accurate. If your docs are spotty, it'll just pick the wrong thing with more confidence. Markdown won't save you.
This is why these tools are useless for contract or spec verification. You need deterministic parsing.
Your stack is too complicated.
While I get where user737 is coming from with the deterministic parsing angle, I think throwing your hands up and declaring the tool useless is a bit hasty. For a lot of us, that's not an option we have the dev bandwidth for.
You *can* significantly reduce, though maybe not eliminate, the hallucination of specific details like API params. It's all about creating a multi-layered prompt that explicitly closes doors.
Here's a system prompt template I've had good results with in Hailuo for exact data extraction:
```
You are an analytical assistant. Your task is to extract information STRICTLY from the provided text. If a requested piece of information is not explicitly stated, you MUST output "Not specified". Do not infer, extrapolate, or guess.
Proceed as follows:
1. Locate the exact wording in the text that matches the request.
2. If found, quote it verbatim.
3. If not found, state "Not specified".
```
Then, in your user prompt, be painfully specific: "From the following text, what is the exact name of the API parameter for user authentication? Output only the parameter name as written, or 'Not specified'."
Combined with turning the temperature way down and using their 'source' highlighting feature (which does help the model visually anchor to chunks, in my experience), I've gotten reliable output for creating data dictionaries from docs. It's not perfect, but it's pushed accuracy from maybe 70% to over 95% for my use cases.
null