Step-by-step: How we batch render 100 product update announcements.

Our team recently faced a recurring, resource-intensive task: generating voiceovers for our weekly product update announcements. With a portfolio of over 30 products and a commitment to detailed, personalized announcements, manual rendering through the WellSaid Labs dashboard was untenable. We needed an automated, batch-oriented pipeline that could handle ~100 unique scripts per week, reliably deliver the MP3s to our CMS, and log each transaction. The solution leverages WellSaid's API, a simple orchestration layer, and cloud storage. Here is our playbook.

**Architecture Overview:**
The core components are a metadata CSV, a Python orchestrator, WellSaid API, and Google Cloud Storage (GCS). We use Snowflake to store final metadata, but BigQuery would be an equally valid choice.

1. **Input Preparation:** A CSV file is generated by our content team with the following schema. Each row represents one required voiceover.
```csv
script_id,product_code,voice_id,script_text,output_filename
PU-2024-001,PROD_A,VI_Alice,"Welcome to this week's update...",prod_a_update_20240527.mp3
PU-2024-002,PROD_B,VI_Ravi,"We've enhanced the dashboard...",prod_b_update_20240527.mp3
```
2. **Orchestration Script:** A Python service (run nightly via Airflow) processes this CSV. The key steps are:
* Read and validate the CSV entries.
* For each row, call the WellSaid Labs `POST /v1/renders` endpoint with the appropriate `voice_id` and `script_text`.
* Poll the `GET /v1/renders/{id}` endpoint for completion (we implement exponential backoff).
* Upon successful rendering, download the MP3 byte stream.
* Immediately upload the byte stream to a GCS bucket with the specified `output_filename`.
* Log all actions, including `script_id`, `render_id`, `timestamp`, `product_code`, and GCS URI, to a staging table.

**Critical Implementation Details & Pitfalls:**
* **Rate Limiting:** WellSaid's API has rate limits. We introduced a controlled concurrency model, processing only 5 renders simultaneously to stay well under the threshold.
* **Idempotency:** The script checks our log staging table for existing successful renders with the same `script_id` and output filename. If found, it skips generation, preventing duplicate costs and processing.
* **Error Handling:** Any HTTP status code 4XX/5XX, or a render failure, is captured and logged. The row is marked as failed, and the process continues. A summary report is sent upon completion.
* **Cost Tracking:** We tag each API call with a `project_id` parameter provided by WellSaid. This allows our finance team to attribute costs accurately per product line.

**Final Load & Schema:**
After the batch completes, the staging logs are merged into our central Snowflake `voiceover_renders` table. The schema is crucial for audit and cost reconciliation:
```sql
CREATE TABLE voiceover_renders (
script_id VARCHAR(64) PRIMARY KEY,
wellsaid_render_id VARCHAR(128),
product_code VARCHAR(32),
render_ts TIMESTAMP_NTZ,
wellsaid_voice_id VARCHAR(64),
gcs_uri VARCHAR(512),
script_hash VARCHAR(64), -- for change detection
status VARCHAR(16),
error_message TEXT
);
```
The entire pipeline, from CSV delivery to database update, now completes in approximately 45 minutes for 100 scripts, with zero manual intervention. The primary bottleneck is the inherent render time on WellSaid's side, which is why concurrent processing is essential. This design pattern could be adapted for any bulk voice generation need, such as e-learning modules or audio alerts.

- Mike