Having spent considerable time integrating various LLM providers into our internal evaluation and observability pipelines, I was operating under the assumption that Freeplay was, for all practical purposes, tied to the OpenAI ecosystem. This was a reasonable conclusion given its deep integration with OpenAI's formats and the prominence of that API in their documentation. My recent deep dive into their SDK and API, however, revealed a pleasantly flexible architecture that natively supports Claude from Anthropic as a first-class citizen.
This isn't merely a case of using a generic `provider` field; Freeplay's design abstracts the LLM call into a `Session` object that can be configured for distinct provider paradigms. The key is the `client` configuration within the `Freeplay` SDK initialization, which delegates the actual LLM communication to the provider's own library. This allows Freeplay to handle logging, tracing, and evaluation without being wedded to a single provider's SDK.
Here is a concise example of configuring a Claude 3 Opus call through Freeplay, using the `anthropic` Python package:
```python
import freeplay
from anthropic import Anthropic
# Initialize the Freeplay client with the Anthropic client as the delegate
freeplay.init(api_key="your_freeplay_api_key")
client = freeplay.with_provider(
provider="anthropic",
client=Anthropic(api_key="your_anthropic_api_key")
)
# Define your prompt template in Freeplay (managed via the UI or SDK)
prompt = freeplay.get_prompt("claude_analysis_template")
# Execute the session, which logs everything to Freeplay
completion = client.complete(
prompt=prompt,
model="claude-3-opus-20240229",
temperature=0.2,
max_tokens=1000
)
# The entire interaction, including the prompt variables and Claude's response,
# is now available in Freeplay for evaluation, tracing, and comparison.
```
The architectural implication here is significant. It means teams running multi-provider LLM strategies—perhaps using Claude for complex analysis and GPT-4 for creative tasks—can funnel all their inference traffic through a single observability layer. This enables apples-to-apples comparison of cost, latency, and output quality across different providers within the same Freeplay project. The pitfall to avoid is assuming all providers behave identically; you must respect the unique parameter sets (e.g., Claude has no `top_p` parameter) and message formatting (Anthropic's `Human`/`Assistant` roles) for each.
For infrastructure-as-code practitioners, this multi-provider support makes Freeplay a compelling central component in a vendor-agnostic LLMops stack. You can define your evaluation suites and test cases once, and run them against any configured model, with all results collated in a single pane of glass. This moves us away from vendor lock-in at the observability layer, which is a non-trivial win.
--from the trenches
infrastructure is code