Use Orq.ai as the model control layer for SmolAgents. Route LLM calls through one OpenAI‑compatible endpoint, capture traces, monitor cost, and manage fallback behavior without rebuilding your SmolAgents CodeAgents, ToolCallingAgents, or workflows.
What is SmolAgents?
SmolAgents is Hugging Face’s lightweight Python agent framework where agents “think in code”: CodeAgent generates Python to call tools and perform computations, and ToolCallingAgent uses JSON/text tool calls. It focuses on minimal abstractions and model‑agnostic tooling so you can spin up capable agents in a few lines of code.
Why use Orq with SmolAgents
Trace agent behavior end to end
Trace SmolAgents runs end to end, including prompts, generated code, tool calls, intermediate steps, model responses, and errors in one place. You get clearer visibility into how each CodeAgent or ToolCallingAgent planned, executed tools, and produced answers without stitching together logs across services.
Model flexibility without rewiring agents
Test new models, add alternative providers, or assign different model tiers to different SmolAgents workloads from Orq.ai, while keeping your agent definitions and tool lists stable. Your agents still “think” they’re talking to an OpenAI‑style OpenAIServerModel; Orq decides which underlying model or provider is called.
Evaluate real production runs
Use real SmolAgents traces like agent loops, tool sequences, code execution steps, and LLM calls to build datasets, compare prompt or model changes, and move from subjective tuning to measurable quality checks. Failed or slow runs can be turned into eval inputs before you roll out changes.
Control spend and access centrally
Track token usage and spend per agent, tool, team, and workflow so you can see which routes drive cost. Add budgets, rate limits, and approved‑model lists at the Orq platform layer instead of enforcing governance separately inside each SmolAgents‑based application.
How the integration works
Step 1
Point SmolAgents models at Orq.ai’s router
Use OpenAIServerModel with api_base="https://api.orq.ai/v2/router" and your Orq API key, instead of pointing directly at OpenAI or another provider. SmolAgents continues to treat this as a normal OpenAI‑style backend; Orq becomes the routing layer.
Step 2
Enable tracing from SmolAgents to Orq
Install openinference-instrumentation-SmolAgents and OpenTelemetry, then call SmolAgentsInstrumentor().instrument(...) before creating any agents so SmolAgents runs export spans to Orq’s OTLP HTTP endpoint. SmolAgents remains your framework; Orq records LLM interactions, latency, and cost metrics alongside agent spans.
Step 3
Define routes, fallbacks, and policies in Orq
Create routes for key SmolAgents workflows (for example code‑heavy CodeAgents, tool‑calling agents, or multi‑agent setups) and assign them model tiers, fallback chains, and region/data policies. Agents send LLM requests using OpenAIServerModel through Orq, and Orq applies routing rules before invoking OpenAI, Anthropic, Gemini, Groq, or other models.
Step 4
Monitor, evaluate, and tune
Once connected, use Orq’s dashboards to watch latency, errors, and cost for SmolAgents apps, and run evals or experiments on their traces. You can adjust routes, models, or prompt configurations centrally where supported, while SmolAgents continues to manage agent loops, tools, and code execution.
Use Cases
Code‑first agents with real observability
Trace which CodeAgent, tool, code snippet, and model contributed to a failed workflow or unexpected outcome, combining SmolAgents’ own logs with Orq’s router‑level traces.
Cost‑aware internal tools
Route routine tool‑calling or simple code tasks to lower‑cost models while keeping complex reasoning, multi‑tool workflows, or long‑running code agents on your strongest routes configured in Orq.
Eval‑driven agent improvements
Reuse failed code executions, tool errors, and slow agent loops as eval inputs before shipping prompt, tool, or model changes, turning production telemetry into a continuous improvement loop.
Safer experimentation across providers
Test a new model or provider on a small share of SmolAgents traffic, then promote or roll back based on traces and evals, instead of editing each agent or OpenAIServerModel config by hand.
FAQ
Do I have to change my SmolAgents code to use Orq.ai?
Typically you just repoint OpenAIServerModel (or other OpenAI‑compatible backends) to Orq’s router URL and API key, and optionally instrument SmolAgents with OpenTelemetry, keeping your CodeAgents, ToolCallingAgents, and tools the same.
Does Orq.ai replace SmolAgents’s own runtime or framework?
No. SmolAgents remains your lightweight framework for code‑first and tool‑calling agents. Orq.ai sits alongside it as the control plane for models, multi‑provider routing, cost tracking, and additional evaluation. You still design and run agents with SmolAgents at the core, but you use Orq to see what they cost, which models they should call, and how LLM behavior compares across providers under the same agent workflow.
Can I keep using my existing LLM providers with SmolAgents if I move to Orq.ai?
Yes. You can bring your existing OpenAI, Anthropic, Hugging Face Inference, or other keys into Orq.ai and route SmolAgents traffic through them, alongside any new models you add later. That way you centralize access, routing, and tracking without losing the SmolAgents and provider setup you already rely on. Bring production controls to SmolAgent apps
