First-time buyer - what should I ask during the sales call?

devops_grandad · 2026-08-13T21:01:23Z

Alright, listen up. You're about to get on a call with a sales engineer who's going to dazzle you with tracing graphs and LLM magic. Your job is to cut through that and ask the questions that matter when the thing is running at 3 AM on a Tuesday. First, get your own house in order before you dial in. Know exactly what you're trying to solve. Is it debugging prompt chains? Monitoring costs? Evaluating model performance? Have a concrete, simple use case in your head. If you don't, they'll sell you on *their* ideal use case. Now, here's what you need to pin them down on. Don't accept fluffy answers. **Infrastructure & Operations:** * "Walk me through the deployment options. On-prem, VPC, SaaS? For the SaaS option, what's the data ingress/egress story? If my data never leaves my cloud, how is that contractually and technically guaranteed?" * "What are the API rate limits and service quotas on the entry plan? Not just the 'evaluation' tier, but the one I'd actually buy. Show me the document." * "What's the historical data retention policy per tier? If I need to keep tracing data for 90 days for compliance, what does that cost?" * "What monitoring do you have *for LangSmith itself*? What's your SLA and what are the historical uptime metrics? I don't care about 'industry-leading,' give me a number." **Security & Compliance:** * "Where is data processed and at rest? Is my prompt/response data ever used for training or product improvement, even in anonymized form? Show me the clause in the agreement that says it isn't." * "What audit trails do you provide for *access to my LangSmith instance*? If your support team needs to access my data for a ticket, is that logged and available to me?" * "For SSO (SAML/OIDC), is that available on all plans or is it an enterprise upsell? What about role-based access control within the platform?" **The Practicalities:** * "Pricing. I understand the per-trace cost. What about the cost for the *annotations* (human feedback) feature? Is that a separate charge? What other line items aren't on the front page of the pricing sheet?" * "Give me a real-world example of a cost calculation. If I have 100,000 LLM calls per day, with an average of 5 traces per call, what's my monthly bill on plan X?" * "Data export. If I need to leave, how do I get my data out? Is it a standard format (JSONL, CSV) via API, or do I have to open a ticket and wait for an engineer to manually bundle it?" * "How does the integration actually work with my existing LangChain or LlamaIndex code? Is it a wrapper, an SDK, an agent? Show me a code snippet for a simple chain, and then show me what it looks like instrumented for LangSmith." ```python # I'd want to see something concrete like this: # Before LangSmith chain = prompt | llm | output_parser # After LangSmith - how much boilerplate is *really* needed? from langsmith import Client client = Client(api_key="...") # ... how many lines of config clutter does this add? ``` Finally, ask for a *technical* proof of concept, not just a demo. You need to run *your* code, with *your* data model, for a week. See if it breaks, see how the traces look, and see what the projected costs are. If they balk at a real POC, that tells you everything.

Page 6 / 6 Prev

LangSmith Reviews

Last Post by annad 1 month ago

76 Posts

70 Users

0 Reactions

178 Views

RSS

annad

(@annad)

Reputable Member

Joined: 2 months ago

Posts: 343

16/08/2026 6:50 pm

That's such a good, practical point about the dollar cap. It turns an abstract percentage into the real, often underwhelming, impact on your invoice.

Building on that, I'd also ask how the credit is *applied*. Is it a check, or a credit on your next invoice? Some vendors only apply it if you're a continuing customer, which feels a bit like being held hostage after a major failure.

And absolutely second the bit about engineer time. A capped credit that doesn't even cover a few hours of your team's firefighting is more of a symbolic apology than a true remedy.

ReplyQuote

Page 6 / 6 Prev

Forum Icons: Forum contains no unread posts Forum contains unread posts

Topic Icons: Not Replied Replied Active Hot Sticky Unapproved Solved Private Closed