I've been evaluating Lindy for potential use in orchestrating some of our lighter-weight ETL workflows. While the core functionality seems promising, I've found the documentation experience to be a significant hurdle. The information appears fragmented across blog posts, a changelog, and a docs site that often lacks depth on specific implementation details.
My typical process for learning a new tool involves systematically reviewing:
* The official "Getting Started" guide.
* API reference for key objects and methods.
* Architecture documentation to understand execution models and limitations.
* Example projects or cookbooks for common patterns.
With Lindy, I've struggled to find authoritative answers on several points crucial for pipeline design:
* **Error Handling & Retry Logic:** What is the exact behavior when a task fails? Are retries configurable per-task or globally? I found snippets in blog posts, but no consolidated reference.
* **State Management:** How is workflow state persisted? For a data pipeline, understanding checkpointing semantics is non-negotiable.
* **Scalability Limits:** Concrete numbers on concurrent workflow executions, task throughput, or payload size limits are difficult to locate.
I've resorted to reading source code and running small-scale benchmarks, which is time-consuming. For instance, to test concurrency, I had to write a simple generator:
```python
# Simple script to probe concurrent execution behavior
import lindy
import asyncio
async def dummy_task(task_id):
await asyncio.sleep(1)
return task_id
async def test_concurrency(n):
tasks = [lindy.task(dummy_task)(i) for i in range(n)]
results = await asyncio.gather(*tasks)
return len(results)
# But what is 'n' before hitting a queue limit?
```
How are other practitioners navigating this? Are you relying on:
* The community Discord/Slack for answers?
* Reading the source directly?
* A specific set of bookmarked blog posts or tutorials?
* Or is there a hidden, well-structured doc section I've missed?
I'm interested in comparative throughput and architecture, but without clear docs, it's difficult to build a reliable mental model of the system's constraints.