Hi everyone! 👋 I'm just starting my DevOps journey and I've been trying to integrate some basic monitoring into my LLM workflows.
I finally managed to set up Slack alerts for prompt failures in PromptLayer using their webhooks. It was a bit tricky for a beginner like me, so I wanted to share my simple steps in case it helps others!
First, I created a Slack incoming webhook in my workspace. Then, in my Python script where I call the LLM, I wrapped the call in a try-except block. On an exception, I send a POST request to the webhook URL. Here's the basic snippet I used:
```python
import requests
import openai
from promptlayer import track_prompt
openai.api_key = "YOUR_OPENAI_KEY"
promptlayer_api_key = "YOUR_PROMPTLAYER_KEY"
slack_webhook_url = "YOUR_SLACK_WEBHOOK_URL"
try:
response = openai.Completion.create(
engine="davinci",
prompt="Hello world",
max_tokens=5
)
track_prompt(
request_id=response["id"],
prompt_name="my_prompt",
prompt_input_variables={"text": "Hello world"},
api_key=promptlayer_api_key,
)
except Exception as e:
slack_message = {"text": f"❌ Prompt failed: {str(e)}"}
requests.post(slack_webhook_url, json=slack_message)
raise e
```
It's probably super basic, but it works! Does anyone have a better or more robust way to do this? I'd love to learn. Thanks in advance for any tips! 😊