Guide: Reducing ChatGPT API errors by implementing a retry-with-fallback logic.

Another day, another "revolutionary" AI API guide. The hype train insists you must use ChatGPT for everything, but the reality is you'll spend half your time handling its flakiness. Rate limits, overloaded endpoints, random 500 errors. It's a distributed system, and like all distributed systems, it will fail in exciting new ways.

Instead of crossing your fingers, write some actual code. Here's a boring, robust retry-with-fallback pattern in Python. It tries the primary model, and if that fails too many times, falls back to a cheaper/more reliable one. Keep your service running instead of crying to support.

```python
import openai
from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type

client = openai.OpenAI()

@retry(
stop=stop_after_attempt(3),
wait=wait_exponential(multiplier=1, min=2, max=10),
retry=retry_if_exception_type((openai.RateLimitError, openai.APITimeoutError, openai.InternalServerError))
)
def chat_completion_with_retry(messages, model="gpt-4"):
try:
response = client.chat.completions.create(
model=model,
messages=messages
)
return response
except (openai.RateLimitError, openai.APITimeoutError, openai.InternalServerError) as e:
raise e
except openai.APIError as e:
# For other API errors (e.g., 429, 5xx), we also retry
raise e

def get_completion_with_fallback(messages):
primary_model = "gpt-4"
fallback_model = "gpt-3.5-turbo"

try:
return chat_completion_with_retry(messages, primary_model)
except Exception as e:
print(f"Primary model {primary_model} failed: {e}. Falling back to {fallback_model}.")
# No retry on fallback, just let it fail fast if it's down too.
return client.chat.completions.create(
model=fallback_model,
messages=messages
)
```

Wrap your calls in `get_completion_with_fallback`. It's not fancy, but it keeps the lights on when OpenAI's latest wonder-model decides to take a nap.