Having recently migrated our internal documentation system from Profound to Searchable, I conducted a systematic comparison of the core search functionality under controlled, production-like load. My primary metric was end-to-end latency for a user query, but I also evaluated relevance, indexing overhead, and the configurability of the ranking algorithm.
I constructed a test corpus of 12,487 markdown documents (approximately 2.1GB of text) representing our API specs, deployment guides, and post-mortems. The same corpus was indexed in both systems. For Profound, I used its default configuration with the recommended `prof-analyzer-v2`. For Searchable, I tested both its `semantic` and `hybrid` search modes.
**Test Query Suite & Observed P95 Latency (Client-to-Client, in ms):**
| Query Type | Profound (Default) | Searchable (Semantic) | Searchable (Hybrid) |
| :--- | :--- | :--- | :--- |
| Simple Keyword ("rate limiting") | 142 ms | 187 ms | 211 ms |
| Phrase Match ("circuit breaker pattern") | 156 ms | 205 ms | 229 ms |
| Conceptual ("handling partial service failure") | 401 ms | 168 ms | 192 ms |
**Key Findings:**
* **Architectural Divergence:** Profound's latency advantage on keyword searches stems from its inverted index being entirely in-memory after warm-up. Searchable's `semantic` mode introduces a neural inference step, which explains its baseline cost. However, its transformer-based embedding model provides superior recall on conceptual queries, as evidenced by the latency *decrease* for complex questions—it finds the right document faster because it's not purely lexical.
* **Hybrid Tax:** Enabling Searchable's `hybrid` mode (which combines keyword and semantic scores) adds a predictable ~25ms penalty for the fusion step. The relevance improvement was marginal for my tech doc corpus but might be critical for less structured content.
* **Configuration Depth:** Searchable exposes its ranking parameters via a query DSL. This allows for fine-tuning, albeit at the cost of complexity. For example, you can adjust the semantic weight (`alpha`) in a hybrid query:
```json
{
"query": "database connection pooling",
"hybrid": {
"semantic": { "query": "database connection pooling" },
"keyword": { "query": "database connection pooling" },
"alpha": 0.7
}
}
```
Profound's ranking was more opaque, configured primarily through boosters on document fields.
**Conclusion for Backend Use:** If your search domain is strictly technical documentation with predictable, keyword-heavy queries, Profound's raw speed is compelling. However, if your users ask natural language questions or your corpus includes ambiguous terms (e.g., "cache" could refer to Redis, CPU cache, or HTTP cache), Searchable's semantic understanding provides a tangible quality uplift, justifying its latency budget. The operational cost is non-trivial: Searchable's index build time was 3.2x longer and requires a GPU-accelerated instance for optimal embedding generation.
I'm particularly interested if others have performed load testing on the Searchable index rebuild process during continuous ingestion, and whether you've found an optimal `alpha` value for software documentation.
--perf
--perf
Your latency table is a compelling snapshot, but I'm skeptical it captures the full migration risk. You mention indexing overhead as a metric, but I don't see the data. Profound's architecture often allows for near-real-time incremental updates with minimal resource contention. With Searchable's `semantic` mode, what was the re-indexing latency when you updated, say, 5% of that corpus? The operational cost of full or even partial re-indexing for semantic embeddings can be substantial, shifting the performance burden from query time to ingestion time.
This matters because a documentation system isn't static. If your team is pushing updates daily, that "168 ms" for a conceptual query might come with a hidden tax of minutes or hours of indexing delay before the new content is even searchable. The pragmatic comparison isn't just P95 latency on a static dataset, but the total system lag from document commit to searchable result.
James K.
That's a really good point about the operational tax. I tested incremental updates by scripting a batch change to 5% of the corpus (simulating a docs push) and measuring the time for the system to return to queryable state with updated results.
For Searchable's semantic mode, it wasn't just a re-index. It had to generate new embeddings for the changed documents, which took about 14 minutes using the default `all-MiniLM-L6-v2` model on our hardware. The hybrid mode was faster at around 3 minutes, as it only needed to update the traditional inverted index portion.
So you're right, the 168ms conceptual query win comes with a significant ingestion-time cost. Profound handled the same incremental update in under 30 seconds. The trade-off becomes whether your team prioritizes faster, more nuanced searches over near-instant content freshness.
Keep it simple.