Don't store them in your code. Don't put them in environment variables passed directly to your app. That's amateur hour for something meant to be production-ready.
The standard is to use a secret manager (Vault, AWS Secrets Manager, GCP Secret Manager) and inject them at runtime. For local dev, use a `.env` file loaded by something like `python-dotenv`, but never commit it. Your orchestration layer (e.g., your main chain assembly) should fetch the keys and pass them as parameters to the components.
Basic pattern using a config object fetched from a secure source:
```python
# config_loader.py
import os
from dataclasses import dataclass
from my_secret_manager import get_secret
@dataclass
class LLMConfig:
api_key: str
model: str
def load_llm_config(provider: str) -> LLMConfig:
# Fetches from secure manager in prod, .env in dev
secret = get_secret(f"llm/{provider}")
return LLMConfig(
api_key=secret["api_key"],
model=secret.get("model", "default")
)
# chain_builder.py
from langchain_openai import ChatOpenAI
from config_loader import load_llm_config
openai_config = load_llm_config("openai")
llm = ChatOpenAI(api_key=openai_config.api_key, model=openai_config.model)
```
Key points:
* Single source of truth for each key.
* Audit trail on secret access.
* Keys can be rotated without redeploying code.
* Different keys per environment (dev/staging/prod) are managed by the secret manager, not your application logic.
If you're not at the scale for a secret manager, at least use a centralized config service. Hardcoded keys are a guaranteed incident.
—DD
Metrics don't lie.