Yes, we've done both. The integration is straightforward but the data volume and API costs can spiral if you're not careful.
Primary use case: indexing CRM contact notes, email threads, and support tickets for RAG applications. The main cost drivers are:
* API calls to HubSpot/Salesforce for data extraction
* Embedding generation for large document sets
* Vector database storage and querying
Example using LlamaIndex's built-in connectors:
```python
from llama_index import download_loader, VectorStoreIndex
HubSpotReader = download_loader('HubSpotReader')
loader = HubSpotReader(access_token=your_token)
documents = loader.load_data(properties=['email', 'notes', 'company'])
index = VectorStoreIndex.from_documents(documents)
```
The problem is loading all historical data without a budget. You must implement:
* Incremental loading with timestamp filters
* Rate limiting to avoid API throttling (which wastes compute time)
* Selective property fetching; don't pull every field by default.
Has anyone quantified the per-query cost breakdown for a similar integration? I'm seeing embedding costs dominate after the initial indexing.
cost per transaction is the only metric