I've been conducting a systematic evaluation of workflow automation within Runway, specifically testing the reliability and latency of automations triggered by task completion events. My initial hypothesis was that the system would offer deterministic, low-latency triggers, but my empirical observations contradict this.
I've designed a controlled test with the following parameters:
* **Trigger:** `Task is completed` (on a specific task within a project).
* **Action:** `Send an email` (to a monitored inbox, with a precise timestamp in the subject for measurement).
* **Task Completion Method:** Both manual completion via the UI and completion via a Gen-2 generation finishing.
My benchmark procedure involves:
1. Starting a stopwatch upon visual confirmation of task completion in the UI.
2. Recording the timestamp of the incoming automation email.
3. Calculating the latency delta. I've run this sequence 15 times.
The results are inconsistent and problematic. In 9 out of 15 trials, the automation failed to trigger at all, resulting in infinite latency (a failed event). In the 6 successful trials, latency varied from 4 minutes 12 seconds to over 18 minutes. This is not acceptable for any pipeline requiring deterministic sequencing.
My configuration appears correct, as evidenced by the occasional successful fire. I am not using any custom conditions that would filter out events. The automation is enabled and published.
```json
// This reflects the structure of my automation setup
{
"trigger": {
"type": "task_completed",
"project_id": "proj_abc123",
"task_name": "Background Removal Batch"
},
"actions": [
{
"type": "send_email",
"recipient": "benchmark-monitor@domain.com",
"subject": "Automation Trigger Test - {{timestamp}}"
}
]
}
```
**Questions for the community:**
* Has anyone else performed reproducible latency benchmarks on Runway's `Task is completed` trigger?
* Are there undocumented system quotas or rate-limiting that could cause dropped automation events?
* Could the trigger be dependent on a secondary backend queue that is not guaranteed to be processed?
* Is there a known difference in reliability between triggers based on manual completion versus AI-generation completion?
Without reliable triggers, building complex, multi-stage media pipelines on this platform is untenable. I need to understand if this is a systemic platform issue, a configuration error on my part, or an expected behavior under specific load conditions. I will be correlating my findings with any responses here to update my internal benchmark report.
numbers don't lie
numbers don't lie
That's some thorough testing you've done, and those results definitely match what I've seen with event-based triggers in other platforms. The inconsistency is the killer.
My gut is that the "Task is completed" event isn't a true real-time webhook, but a polling mechanism the automation engine checks on a schedule. If your completion happens right after a poll cycle, you're waiting for the next one, which could explain the 4-18 minute window. The total failures suggest the event payload sometimes doesn't match what the trigger is listening for, especially if the Gen-2 completion has a different internal status than a manual checkbox.
A workaround I've used is to not rely on the event at all. Can you add a final step to the task itself that pings a webhook URL? That external call becomes your reliable trigger, and you can build the automation from there. It's more setup but removes the black box.
api first