Use Orq.ai as the model control layer for DSPy. Route LLM calls through one OpenAI‑compatible endpoint, capture traces, monitor cost, and manage fallback behavior without rebuilding your DSPy program, signatures, or optimization pipeline.
What is DSPy?
DSPy (Declarative Self‑improving Python) is Stanford’s framework for programming, rather than prompting, language models, using typed signatures and modular programs that can be automatically optimized against a metric. It helps teams focus on program structure and evaluation instead of hand‑tuning every prompt string.
Why use Orq with DSPy
Trace program behavior end to end
Trace DSPy programs end to end, including signatures, module calls, retrieval steps, optimizer runs, model responses, and errors in one place. You get clearer visibility into pipeline behavior without stitching together custom logs across multiple services.
Model flexibility without rewiring programs
Test new models, add alternative providers, or assign different model tiers to different DSPy modules from Orq.ai, while keeping your DSPy signatures, modules, and program structure stable. Your DSPy code still “thinks” it’s talking to a standard LM; Orq handles which underlying model is called.
Evaluate real production runs
Use real DSPy traces, compiled programs, predictions, and metrics to build datasets, compare prompt or model changes, and move from subjective tuning to measurable quality checks. Failed or slow runs can be turned into eval inputs before you roll out changes.
Control spend and access centrally
Track token usage and spend per program, team, and workflow so you can see which routes drive cost. Add budgets, rate limits, and approved‑model lists at the Orq platform layer instead of enforcing governance separately inside each DSPy project.
How the integration works
Step 1
Point DSPy traffic at Orq.ai’s router
Configure DSPy’s language model setup (which uses LiteLLM‑style adapters) so its LM calls go to Orq.ai’s base URL instead of calling each provider directly. This gives Orq the context it needs to apply routing rules, capture usage, and enforce fallback behavior.
Step 2
Enable tracing from DSPy workloads to Orq
Keep DSPy’s own debugging / logging patterns while routing LLM calls through Orq’s AI Gateway / Router. DSPy continues to emit information about signatures, modules, and optimizer runs, and Orq records LLM interactions, latency, and cost metrics for those requests, so you can see both program‑level and model‑level traces.
Step 3
Define routes, fallbacks, and policies in Orq
Create routes for key DSPy workflows (for example RAG pipelines, ReAct agents, or evaluators) and assign them model tiers, fallback chains, and region/data policies. Your DSPy program sends the LM request to the configured Orq route, and Orq applies the routing rules you define before invoking OpenAI, Claude, Groq, or other models.
Step 4
Monitor, evaluate, and tune
Once connected, use Orq’s dashboards to watch latency, errors, and cost for DSPy programs, and run evals or experiments on their traces. You can then adjust routes, models, or prompt configurations centrally where supported, while DSPy continues to handle signatures, metrics, and optimizers like MIPROv2 or GEPA.
Use Cases
Multi‑agent products with real observability
Trace which DSPy module (Predict, ChainOfThought, ReAct, Retrieve), tool call, and model contributed to a failed workflow or unexpected outcome, combining DSPy’s debugging views with Orq’s router‑level traces.
Cost‑aware internal tools
Route routine classification or summarization steps to lower‑cost models while keeping complex reasoning, RAG, or evaluation modules on your strongest routes configured in Orq.
Eval‑driven agent improvements
Reuse failed predictions, optimizer logs, and DSPy traces as eval inputs before shipping program, metric, or model changes, turning production telemetry into a continuous improvement loop.
Safer experimentation across providers
Test a new provider or model on a small share of DSPy traffic, then promote or roll back based on traces and evals, instead of editing each module or LM adapter config by hand.
FAQ
Do I have to change my DSPy code to use Orq.ai?
In many cases, you can repoint DSPy’s LM configuration (base URL, model string, and key) to Orq’s router and keep your existing signatures, modules, and optimization code as‑is, adding only minimal configuration instead of rewriting programs. The program structure and metrics stay in DSPy.
Does Orq.ai replace DSPy’s own runtime or framework?
No. DSPy remains your framework for programming and optimizing language‑model pipelines; you keep using its signatures, metrics, and optimizers like MIPROv2 and GEPA. Orq.ai sits alongside it as the control plane for models, multi‑provider routing, cost tracking, and additional evaluation. You still design and run programs in DSPy, but you use Orq to see what they cost, which models they should call, and how LLM behavior compares across providers under the same DSPy workflow.
Can I keep using my existing LLM providers with DSPy if I move to Orq.ai?
Yes. You can bring your existing provider keys (OpenAI, Anthropic, Groq, local Llama, etc.) into Orq.ai and route DSPy traffic through them, alongside any new models you add later. That way you centralize access, routing, and tracking without losing the providers and DSPy LM adapters you already rely on. Bring production controls to DSPy agents
