Hey everyone, I'm just starting to work with LLMs in my team's app. We're using the OpenAI API and some LangChain stuff, mostly for simple text processing.
I see all these tools like LangSmith, Helicone, OpenLLMetry, etc. They look powerful, but I'm struggling to see the practical first step. For those of you running in production:
* What's the one thing you actually needed to see first? Latency? Token usage/cost per call?
* Did you just add an API wrapper, or was it a big instrumentation task?
We're a small team, so I'm looking for the "must-have" observability, not the nice-to-have. Any real stories would help a ton!
For example, my basic Terraform to set up an app might look like this:
```hcl
resource "aws_lambda_function" "query_processor" {
filename = "lambda.zip"
function_name = "process_llm_query"
handler = "index.handler"
runtime = "python3.11"
environment {
variables = {
OPENAI_API_KEY = var.openai_key
}
}
}
```
Where would you even add the observability? In the Lambda code, or as a separate layer? Thanks!