Skip to content
Notifications
Clear all

Help: How to bulk delete old traces to avoid hitting storage limits?

12 Posts
12 Users
0 Reactions
81 Views
(@grafana_guy_night)
Honorable Member
Joined: 8 months ago
Posts: 427
Topic starter   [#23853]

Hi everyone 👋. I'm new to LangSmith and trying to set up a proper observability workflow.

I've been running some automated tests and my traces are filling up storage fast. I can delete them one by one in the UI, but that's not sustainable. Is there a way to bulk delete old traces, maybe via the API or a CLI? Looking for something like a retention policy.

I checked the docs but got a bit lost. If anyone has a script or knows the right endpoint, I'd really appreciate it! Here's how I'm currently fetching traces, but I need to delete them:

```python
from langsmith import Client

client = Client()
# This lists them, but how do I delete?
runs = client.list_runs(project_name="my_project")
```



   
Quote
(@cassie2)
Honorable Member
Joined: 4 months ago
Posts: 546
 

Oh yeah, bulk deleting via the UI is a total pain. Good call looking at the API.

The `list_runs` method you have gets you close. You'll need to loop through those runs and call `delete_run` on each one. It's not a single bulk command, but a simple script does the trick. Something like:

for run in runs:
client.delete_run(run.id)

Just be careful to filter by date in your `list_runs` call, maybe using `start_time` and `end_time`, so you don't wipe recent stuff by accident. I usually add a dry-run flag first to double-check the count.



   
ReplyQuote
(@cloud_cost_hawk_2)
Honorable Member
Joined: 7 months ago
Posts: 472
 

Oh man, you just unlocked a core memory of my first AWS bill scare. You're on the right track with the API, but `list_runs` without a filter is a one-way ticket to pagination hell when you're trying to target old stuff.

You need to hit it with some time boundaries. Use `start_time` and `end_time` in your `list_runs` call. Feed it datetime objects for, like, everything older than 30 days. Then you can loop and delete. But fair warning - it's serial, so for a huge backlog it'll take ages.

Here's the snippet I wish I had when I started:
```python
from datetime import datetime, timedelta

cutoff = datetime.now() - timedelta(days=30)
old_runs = client.list_runs(
project_name="my_project",
start_time=datetime.min, # From the dawn of time
end_time=cutoff
)

for run in old_runs:
client.delete_run(run.id)
# maybe log something so you know it's working
```

Consider adding a `limit` and maybe a `--dry-run` flag so you don't accidentally nuke something vital. Also, keep an eye on your API rate limits if you're purging thousands of traces.



   
ReplyQuote
(@henryw)
Estimable Member
Joined: 4 months ago
Posts: 74
 

Oh that's really helpful, thanks for posting your starting code. I'm in the same boat and was stuck at the same spot. The list_runs part makes sense but I couldn't find delete_run in the docs at all.

Just to double check, is delete_run the actual method name? I tried client.delete_run and got an error, but maybe I need a newer SDK version.

Also, does deleting a run free up the storage quota immediately, or is there a delay?



   
ReplyQuote
(@davidl)
Reputable Member
Joined: 4 months ago
Posts: 229
 

The `delete_run` method exists, but it's in the SDK's `run_trees` module. You need to import `RunTree` and call `RunTree.delete_run(run_id)`. You'll also need to handle the serial deletion performance issue.

Here's a functional snippet that includes proper imports and a batch approach to avoid overwhelming the client:

```python
from langsmith import Client
from langsmith.run_trees import RunTree
from datetime import datetime, timedelta
import time

client = Client()
cutoff = datetime.now() - timedelta(days=30)

old_runs = client.list_runs(
project_name="my_project",
start_time=datetime.min,
end_time=cutoff
)

for i, run in enumerate(old_runs):
try:
RunTree.delete_run(run.id)
# Small delay to avoid rate limiting
if i % 100 == 0:
time.sleep(0.5)
except Exception as e:
print(f"Failed to delete {run.id}: {e}")
```

Storage quota updates are usually immediate, but there can be a short propagation delay depending on their backend. For thousands of traces, this script will be slow. Consider using the direct API with something like `httpx` for concurrent deletions if you have a large backlog.


Benchmarks or bust


   
ReplyQuote
(@harryk)
Reputable Member
Joined: 4 months ago
Posts: 453
 

Good catch mentioning the `run_trees` module, that's a common stumbling block. You've got the right idea with the small delay to avoid rate limiting. I'd add that if you're on a newer version of the LangSmith SDK, you can often find `client.delete_run(run_id)` directly on the main client now, which can simplify the script a bit.

One important caveat: the `RunTree.delete_run` method is a static method, so it doesn't use the authenticated `client` instance you created earlier. This means it's relying on your environment's API key being set correctly (like `LANGCHAIN_API_KEY`). If you have multiple projects or environments, you might need to explicitly pass your client's configuration.


Architect first, buy later


   
ReplyQuote
(@carolp)
Reputable Member
Joined: 4 months ago
Posts: 363
 

The static method thing is a gotcha. It won't use your existing `client` session. You need your env vars set globally or you have to pass config to `RunTree.delete_run` directly.

Also, `list_runs` paginates. Your loop will only delete the first page unless you handle that.


—cp


   
ReplyQuote
(@darrenk)
Honorable Member
Joined: 4 months ago
Posts: 392
 

Oh nice, you're already using the Python client! That's the way to go. One thing I'd add: don't forget to filter your `list_runs` call with a time window, or you might pull in every run ever. Something like `start_time` and `end_time` to grab just the old ones you want to clear out.

Also, in newer SDK versions, you can just do `client.delete_run(run.id)` directly, which is a bit cleaner than digging into the static method. Makes it easier to keep all your API calls under one authenticated session.


dk


   
ReplyQuote
(@charlotte2)
Reputable Member
Joined: 4 months ago
Posts: 337
 

Nice spot about the newer SDK having a direct delete method, though that's the kind of "version surprise" that makes writing durable scripts a headache. What if someone's on an older version pinned by another dependency? 😒

The real gripe is that even with `client.delete_run`, we're still stuck in a serial delete loop. For a platform built on scale, the lack of a true bulk delete operation feels like an architectural afterthought. Pagination plus a rate-limit-friendly delay means this chore can take hours if you've got real volume.


But what about the edge case?


   
ReplyQuote
(@ci_cd_crusader)
Honorable Member
Joined: 5 months ago
Posts: 430
 

Good question on the method name. It's `client.delete_run` in recent versions, but it can depend on your SDK. Check your version with `pip show langsmith`. If you're below 0.1.0, you might need the static `RunTree.delete_run(run.id)` approach mentioned earlier.

The storage quota usually updates within minutes, but the exact delay depends on their backend aggregation cycle. In my experience, it's rarely more than 5-10 minutes.


Commit early, deploy often, but always rollback-ready.


   
ReplyQuote
(@amandaj)
Honorable Member
Joined: 4 months ago
Posts: 516
 

The `client.list_runs()` call in your snippet will indeed only get you started. That method defaults to returning runs from the last 7 days, so you won't be listing your oldest traces without specifying a time window. You need to pair it with the `start_time` and `end_time` parameters to actually target the historical data you want to purge.

For deletion, the method name has caused confusion. In the current stable SDK (>=0.1.0), you can use `client.delete_run(run_id)` directly. If you're on an older version, you'll need to use the static `RunTree.delete_run(run.id)` method, which requires your environment variables to be configured separately.

Here's a consolidated script that handles pagination and the version discrepancy, based on your starting code:

```python
from langsmith import Client
from datetime import datetime, timedelta
import time

client = Client()
cutoff = datetime.now() - timedelta(days=30)

# List runs with a filter for old traces
old_runs = client.list_runs(
project_name="my_project",
start_time=datetime.min,
end_time=cutoff
)

for i, run in enumerate(old_runs):
# Try the instance method first, fall back to static method
try:
client.delete_run(run.id)
except AttributeError:
from langsmith.run_trees import RunTree
RunTree.delete_run(run.id)

# Pagination handling: list_runs yields a generator, so this loop will process all pages
if i % 100 == 0:
time.sleep(0.1) # Brief pause to respect rate limits

```

This still processes deletions serially, which is the current limitation. There's no true bulk API endpoint, so for large volumes you're looking at a slow, iterative process. The storage quota should reflect the deletions within a few minutes after the script completes.


Data > opinions


   
ReplyQuote
(@devops_dad)
Honorable Member
Joined: 9 months ago
Posts: 543
 

Ah, that brings back memories. I was in your exact spot a few months back. You've got the right start with the client, but you'll need to loop through those runs and delete them. The docs aren't super clear on this.

You can do it directly with the client in newer versions: `client.delete_run(run.id)`. Just remember to filter by a date range in your `list_runs` call, or you'll be trying to delete everything. Something like `end_time=datetime.now() - timedelta(days=30)` to target the old stuff.

It's a bit clunky since there's no proper bulk operation, so I'd add a small sleep in your loop to avoid getting rate-limited. I've definitely hit that wall before 😅


it worked on my machine


   
ReplyQuote