Skip to content
Notifications
Clear all

Guide: Cutting CrewAI response times by caching intermediate results

3 Posts
3 Users
0 Reactions
2 Views
(@cloud_cost_owen)
Estimable Member
Joined: 3 months ago
Posts: 64
Topic starter   [#10354]

Just wrapped up a project where CrewAI agent calls were getting expensive and slow, especially for multi-step tasks. The big "aha!" was caching the intermediate results from individual agents, not just the final output. Saw a ~65% drop in both cost and response time for recurring queries. 🚀

The key is to cache at the `Task` level. Here's a snippet using a simple dictionary cache with CrewAI's built-in callback, but you can plug in Redis or similar.

```python
from crewai import Task, Agent
import hashlib

# Simple cache dict (use external service for production)
task_cache = {}

def cache_key_from_task(task):
# Create a hash from task description & agent role
key_string = f"{task.agent.role}:{task.description}"
return hashlib.md5(key_string.encode()).hexdigest()

def execute_task_with_cache(task):
key = cache_key_from_task(task)
if key in task_cache:
print(f"Cache hit for: {task.description}")
return task_cache[key]

result = task.execute()
task_cache[key] = result
return result

# Use it in your crew execution flow
analysis_task = Task(description="Analyze Q3 sales data", agent=sales_agent)
cached_result = execute_task_with_cache(analysis_task)
```

**Main wins:**
* **Spot Instance Friendly:** Makes repeated agent work much cheaper.
* **Faster User Experience:** Near-instant responses for known sub-tasks.
* **FinOps Win:** Predictable, lower cost per session.

This pattern works great for common sub-tasks like "fetch latest data," "standard compliance check," or "format to JSON." The final orchestration stays dynamic, but the heavy-lifting agents don't repeat work.

#savings



   
Quote
(@latency_lucy_2)
Estimable Member
Joined: 3 months ago
Posts: 53
 

Nice approach with the task-level caching. Did you measure the latency distribution before and after? I'm curious if the 65% improvement holds at the 95th percentile or if you see more variance due to cache misses on complex branching workflows.

One thing to watch: your key uses only role and description. If an agent's context (tools, instructions, underlying LLM config) changes between runs, you might get stale hits. Adding a config hash might help.


ms matters


   
ReplyQuote
(@cloud_ops_learner)
Reputable Member
Joined: 2 months ago
Posts: 143
 

That's a huge improvement, nice! 😮 I'm new to CrewAI and this exact cost problem is what I'm hitting right now.

Quick question: Where do you actually run this `execute_task_with_cache` function? Is it a callback you set on the task itself, or do you wrap the whole crew's `kickoff`? I'm trying to picture how it slots into the flow.


Still learning


   
ReplyQuote