Use Orq.ai as the model control layer for Instructor. Route LLM calls through one OpenAI‑compatible endpoint, capture traces, monitor cost, and manage fallback behavior without rebuilding your Instructor schemas, validation logic, or application code.
What is Instructor?
Instructor is a Python and TypeScript library for “structured prompting”: you define typed schemas (for example Pydantic models) and it handles calling the LLM and validating responses against those types. It helps teams focus on data contracts and business logic instead of hand‑parsing and validating JSON by themselves.
Why use Orq with Instructor
Trace structured responses end to end
Trace Instructor‑backed runs end to end, including prompts, schema definitions, tool invocations, validated model responses, and errors in one place. You get clearer visibility into how each typed result was produced, without stitching together logs across multiple services.
Model flexibility without rewiring schemas
Test new models, add alternative providers, or assign different model tiers to different schemas and endpoints from Orq.ai, while keeping your Instructor type definitions and validation logic stable. Your Instructor code still “thinks” it’s calling a standard OpenAI‑style client; Orq handles which underlying model is used.
Evaluate real production runs
Use real Instructor traces like requests, typed outputs, and validation errors to build datasets, compare prompt or model changes, and move from subjective tuning to measurable quality checks. Failed or malformed responses can be turned into eval inputs before you roll out changes.
Control spend and access centrally
Track token usage and spend per schema, endpoint, team, and workflow so you can see which routes drive cost. Add budgets, rate limits, and approved‑model lists at the Orq platform layer instead of enforcing governance separately inside each Instructor‑based application.
How the integration works
Step 1
Point Instructor’s client at Orq.ai’s router
Configure the OpenAI‑style client that Instructor uses so its HTTP calls go to Orq.ai’s base URL and key instead of calling each model provider directly. This gives Orq the context it needs to apply routing rules, capture usage, and enforce fallback behavior.
Step 2
Enable tracing from Instructor workloads to Orq
Keep your existing logging and monitoring in place while routing LLM calls through Orq’s AI Gateway / Router. Your app continues to emit logs for schema use and validation, and Orq records LLM interactions, latency, and cost metrics for those requests, so you can see both schema‑level and model‑level traces.
Step 3
Define routes, fallbacks, and policies in Orq
Create routes for key Instructor workflows (for example extraction endpoints, classification APIs, or agents that return typed outputs) and assign them model tiers, fallback chains, and region/data policies. Your code sends the request to the configured Orq route, and Orq applies routing before invoking OpenAI, Gemini, Claude, or other models.
Step 4
Monitor, evaluate, and tune
Once connected, use Orq’s dashboards to watch latency, errors, and cost for Instructor‑based endpoints, and run evals or experiments on their traces. You can then adjust routes, models, or prompt configurations centrally where supported, while Instructor continues to handle schemas, parsing, and validation.
Use Cases
Multi‑agent products with real observability
Trace which Instructor schema, prompt, and model contributed to a failed workflow or unexpected output, combining your own logs with Orq’s router‑level traces.
Cost‑aware internal tools
Route routine structured tasks (for example simple extraction or tagging) to lower‑cost models while keeping complex reasoning or safety‑critical typed responses on your strongest routes configured in Orq.
Eval‑driven agent improvements
Reuse failed validations, malformed JSON, and Instructor traces as eval inputs before shipping schema, prompt, or model changes, turning production telemetry into a continuous improvement loop.
Safer experimentation across providers
Test a new model or provider on a small share of Instructor traffic, then promote or roll back based on traces and evals, instead of editing each endpoint or client config by hand.
FAQ
Do I have to change my Instructor code to use Orq.ai?
In many cases, you can repoint the OpenAI‑style client Instructor uses (base URL, key, and model string) to Orq’s router and keep your existing schemas and validation logic as‑is, adding only minimal configuration instead of rewriting endpoints. The type definitions and business logic stay in Instructor.
Does Orq.ai replace Instructor’s own runtime or framework?
No. Instructor remains your framework for structured prompting and typed responses; you keep using its schema definitions and validation. Orq.ai sits alongside it as the control plane for models, multi‑provider routing, cost tracking, and evaluation. You still design and run typed endpoints with Instructor at the core, but you use Orq to see what they cost, which models they should call, and how LLM behavior compares across providers under the same structured workflow.
Can I keep using my existing LLM providers with Instructor if I move to Orq.ai?
Yes. You can bring your existing provider keys into Orq.ai and route Instructor traffic through them, alongside any new models you add later. That way you centralize access, routing, and tracking without losing the providers and Instructor client setup you already rely on. Bring production controls to Instructor-based apps
