Skip to content
Notifications
Clear all

Tutorial: Using the API to bulk-add indicators from a spreadsheet.

1 Posts
1 Users
0 Reactions
4 Views
(@juliap)
Estimable Member
Joined: 1 week ago
Posts: 100
Topic starter   [#13380]

Alright, let's be honest. The "Upload CSV" button in the ThreatConnect UI is fine for a one-off, but if you're dealing with a real feed or a messy spreadsheet from a third-party, you're going straight to the API. The UI will choke on formatting, complain about duplicates, and leave you wondering what actually got ingested.

I see a lot of tutorials that just parrot the API docs. The real trick isn't the `POST` to `/api/v3/indicators`. It's the cleanup before you send it. Here’s the bare-bones approach I use, which assumes you've already wrangled your spreadsheet into a list of dictionaries in Python.

First, you need to de-duplicate against your existing instance. Blindly creating indicators will either fail or create a mess. Query for existing ones first, using the indicator value as your key. Then, for the new ones, the payload isn't just the indicator. You need to map your spreadsheet columns to ThreatConnect's required fields and any custom attributes you want to set. Don't forget to assign a sourceβ€”nobody wants orphaned indicators floating in the "Community" source.

The real-world pitfall? Rate limiting and error handling. If you just loop through and fire off requests, you'll get throttled and some will fail. Build in a simple delay and a retry for 429s. Log the failures to a separate file so you can see which rows from your precious spreadsheet bombed and why. Was it a malformed IP? A tag that doesn't exist yet? The API will tell you, but only if you capture the response.

And a final piece of advice: before you run this on your production instance, test it with a `?confidence=0` filter or similar, so you can easily nuke all the test indicators afterward. Trust me, your SOC team will thank you for not polluting their active data with a thousand test indicators from "Vendor_Spreadsheet_2024_FINAL_v2.csv". 😏


Your free trial ends today.


   
Quote