Ran into memory limits processing 10k+ text entries with Cartesia's Python SDK. The default `generate` call tries to load everything at once. Here's how to stream large jobs without blowing up your instance.
Key is to batch and use the async client. Don't send your entire dataset.
```python
from cartesia import AsyncCartesia
import asyncio
async_client = AsyncCartesia(api_key="your_key")
async def process_batch(texts):
# Process in manageable chunks
results = await async_client.generate(
model="sonic",
prompt=texts,
voice_id="your_voice",
stream=False
)
return results
# Split your dataset
batch_size = 50 # Adjust based on your memory
all_texts = [...] # Your large dataset
all_results = []
for i in range(0, len(all_texts), batch_size):
batch = all_texts[i:i+batch_size]
results = await process_batch(batch)
all_results.extend(results)
# Force cleanup if needed
del results
```
Also, set `stream=False` if you don't need real-time playback. Streaming (`stream=True`) holds connections open and can pile up. Monitor your Python process RSS; if it climbs, reduce batch size.
Ship fast, review slower
Great point about batching with the async client! I'd also suggest looking at `asyncio.as_completed` for your loop if you're okay with out-of-order processing - you can fire off multiple batches concurrently without waiting for each to finish, which can really speed things up on a beefy instance.
One caveat: even with `stream=False`, the SDK might buffer the full audio response in memory before returning it. For really large batches, you might want to check if there's a way to write the output directly to disk as it arrives. I've had to do that with other TTS APIs before.
What's your memory ceiling? I found that keeping batches under 100 entries usually keeps things stable on a 4GB machine, but it depends heavily on text length and the audio format.
Data nerd out
Batch size is critical, but 50 might still blow up depending on audio duration. For me, anything over 15-20 seconds per file means dropping to batch size 10.
Also, that `del results` does almost nothing if you're just reassigning the variable next loop. You need to explicitly call `gc.collect()` after if you're truly memory-bound.
Benchmark or bust