I've been trying to build a reliable workflow using AutoGen with Azure OpenAI as the underlying LLM provider. My primary goal is to leverage function calling for a cost optimization agent that can query our Azure consumption data and make recommendations. However, the integration hasn't been as straightforward as the documentation suggests.
The main points of friction I've encountered are:
* **Endpoint Configuration:** The Azure OpenAI API requires a different base URL format (`{resource}/openai/deployments/{deployment}/chat/completions`) and specific parameters (`api_version`) that don't map cleanly to the standard `OpenAIWrapper` setup. I've had to subclass the client to get it to work.
* **Function Calling Schema Compatibility:** There seem to be subtle differences in how Azure OpenAI handles the `tools` parameter versus the standard OpenAI `functions` parameter. My agent would sometimes ignore the function definitions or fail to parse the JSON arguments correctly.
* **Error Handling:** Timeout and rate limit errors from Azure are not always surfaced cleanly through AutoGen's agent framework, making debugging a chore.
Has anyone successfully configured a multi-agent setup with Azure OpenAI and had function calling work reliably? I'm particularly interested in:
* Your specific `LLMConfig` and `AzureOpenAI` client initialization pattern.
* Any adjustments needed to the `register_function` flow or agent initialization.
* Whether you used the `openai` package v1.x or a different library.
A side-by-side comparison of a working Azure config versus a standard OpenAI config would be immensely helpful for the community. I'll share my current (partially working) configuration in a follow-up post once I've stabilized it.
—EK
Your bill is too high.