Everyone's rushing to deploy these agentic workflows, but when the agent inevitably makes a baffling decision, you're left staring at a blank screen or a single-line output. "It just didn't work" isn't a root cause. If you can't audit the thought process, you can't trust it in any real infrastructure.
SuperAGI provides logging, but the default output is often too high-level. You need the step-by-step reasoning, the tool calls with their exact parameters, and the raw LLM prompts/responses. Here's how I've forced it to cough up the details.
First, the `config.yaml`. The default logging is polite. You need to be intrusive.
```yaml
LOG_HANDLER: "file" # Ensure it's not just console
LOG_FILE_PATH: "./superagi.log"
LOG_LEVEL: "DEBUG" # CRITICAL here. INFO won't cut it.
```
But that's just the start. The real meat is often in the agent's execution trace. If you're using the `AgentWorker`, you need to intercept the flow. Enable the execution logs and, crucially, ensure the LLM is being asked to show its work. This sometimes means modifying the agent template's system prompt to include explicit step-by-step formatting.
For the open-source version, you might have to dive into the `superagi/tools` and `superagi/agent` modules to add more granular logging around `execute()` methods. A blunt but effective method I've used:
```python
# A quick-and-dirty patch in your execution script
import logging
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=[logging.FileHandler("agent_thoughts.log"),
logging.StreamHandler()])
```
Second, are you using the cloud platform or self-hosted? The cloud console hides most of this, which is a compliance red flag. Self-hosted gives you direct access to the backend, where you can (and should) enable SQL query logging for the `agent_executions` and `tool_executions` tables if you really want to see the sequence.
Without this level of detail, you're flying blind. How are you supposed to write an incident postmortem when the only log entry is "Task failed"? What cost audit can you perform if you can't see which tool calls are being looped unnecessarily?
What specific logging strategies or config tweaks have you found that actually expose the chain-of-thought? Or are we all just hoping the black box behaves today?
- Nina
- Nina
You're right about modifying the system prompt, but that's a hack and it pollutes the agent's actual instructions. The execution trace is the key. In the SuperAGI codebase, you need to look at the specific `Tool` classes and the `AgentWorker`'s `execute_step` method.
The DEBUG log level gets you the tool calls, but you still miss the raw reasoning chain before the tool is selected. If you're running the open-source version, you have to patch the logger in `superagi/agent/agent.py` to capture the full `thought` variable from the agent's output parser. Without that, you're just seeing decisions, not the internal monologue.
I've had to pipe all that to Loki and build a Grafana dashboard just to see what the thing is actually doing. Otherwise you're debugging a black box with a single "error" log line.
Automate everything. Twice.
The config.yaml changes are necessary but insufficient for real-time diagnostics, especially under concurrent load. You'll find the log file I/O becomes a bottleneck itself, adding tens to hundreds of milliseconds of blocking write latency that distorts the very execution trace you're trying to capture.
Instead of relying solely on the file handler, configure a dedicated async logging handler that buffers writes and flushes on a separate thread. More critically, you need to instrument the `AgentWorker` loop to emit structured JSON logs with high-resolution timestamps (nanosecond precision if possible) for each sub-phase: prompt construction, LLM network call, response parsing, and tool dispatch. This allows you to correlate stalls with specific external tool latency, which is often the root cause of a "baffling decision" - the agent isn't reasoning poorly, it's timing out on a slow API and making a default choice.
Without this level of instrumentation, you're still debugging a distributed system with a single, aggregated log line.
Every microsecond counts.
Totally agree on modifying the `config.yaml` for DEBUG, but that alone can still leave you hunting. You need to combine it with the execution trace flag. Set `LOG_EXECUTION: true` in that same YAML file. It forces the agent to dump the full sequence, including the raw LLM prompts and the internal "thought" object before it parses it into an action.
Even then, sometimes the crucial failure is in the parsing step itself. I've seen the agent have a perfectly logical chain in the thought, but the output parser strips it out and just logs the final tool call. You can patch the logger in the `Agent` class to print the entire intermediate step, but you're right, it's a hack.
Have you tried shipping these logs straight to something like Grafana Loki? I set up a custom logger config to pipe the DEBUG and execution trace as structured JSON. Makes it way easier to search for specific failed reasoning steps across multiple runs.
If it's not monitored, it's broken.