Skip to content
Notifications
Clear all

Breaking: PromptLayer now supports Azure OpenAI Service. Any gotchas?

2 Posts
2 Users
0 Reactions
3 Views
(@kubernetes_tinker_99)
Estimable Member
Joined: 4 months ago
Posts: 56
Topic starter   [#3698]

Hey folks, just spotted the announcement and had to spin up a quick test cluster to poke at this. 🎯 PromptLayer adding Azure OpenAI Service support is huge for those of us running in hybrid or Azure-centric environments.

I've been using their logging and monitoring for OpenAI API calls in our GitOps flows for a while, so naturally I had to see if the Azure integration felt the same. The setup in code is straightforward—you basically swap your base URL and add your `api_key` and `api_version`. Here's a snippet from my quick Python test:

```python
import promptlayer
promptlayer.api_key = "pl_xxxx"
openai = promptlayer.openai
openai.api_key = "azure-openai-key"
openai.api_base = "https://your-resource.openai.azure.com/"
openai.api_type = "azure"
openai.api_version = "2023-05-15"
```

But here's where I'm curious about the **gotchas**:
* **Deployment names vs model names**: With Azure, you're calling a deployment name, not the model directly. Does PromptLayer's dashboard map the deployment name back to the model (like `gpt-4`) clearly for tracking?
* **Request logging**: Are the special Azure fields (like `api_version`) captured in the logs, or just the core prompt/completion?
* **Error handling**: Azure's API can return slightly different error formats. Has anyone seen if PromptLayer normalizes these, or do you need to adjust your alerting rules?

I'm also wondering about the pricing model—does logging Azure calls count against the same "request" quota on PromptLayer's side?

Would love to hear if anyone has run this through a full Argo CD rollout or has it in a production service mesh scenario. Any quirks with rate limiting or tracing integration?


#k8s


   
Quote
(@martech_trial_taker_v3)
Trusted Member
Joined: 1 month ago
Posts: 35
 

Oh, that deployment name vs model name mapping is exactly what I'd be worried about too. I rely on those model tags in PromptLayer to compare costs and performance across projects.

Did your test logs show the deployment name, or did they somehow translate it to the underlying model? That could get messy if one deployment is linked to gpt-4 and another to gpt-35-turbo, but they're both just called "prod-chat" in Azure.

Also, does the api_version get stored with each request? I'd need that to debug if something changes between versions.


trial junkie


   
ReplyQuote