VS Code

Use VSCode as an IDE interface for your Orq.ai workspace. Query traces, run evals, inspect deployments, and debug production LLM behavior without leaving your editor, with the Orq.ai dashboard available for deeper drill‑down

VS Code

Use VSCode as an IDE interface for your Orq.ai workspace. Query traces, run evals, inspect deployments, and debug production LLM behavior without leaving your editor, with the Orq.ai dashboard available for deeper drill‑down

MCP

Skills

Natural language

Local-first

Get your API key

View setup docs

Field

Value

Integration type

MCP server (remote HTTP / Streamable HTTP / SSE)

Setup time

Quick setup once VS Code’s AI/agent features and MCP are enabled, with an Orq API key configured in your MCP settings.

Auth

Orq.ai API key (workspace‑ or project‑level) passed as a bearer token via environment variables or inline config in .vscode/mcp.json, depending on how you configure the server.

Skills support

VS Code’s agents (for example GitHub Copilot Agent or AI Toolkit agents) can call Orq MCP tools when the Orq server is registered and enabled in the MCP configuration.

Cloud-based

VS Code runs locally as your IDE, while MCP requests from the AI agents call Orq’s cloud APIs for workspace data.

Multi‑workspace

Define multiple Orq entries in user MCP config (global) and workspace .vscode/mcp.json with different API keys to point VS Code at different Orq workspaces or environments.

Vendor

Microsoft

Pricing

Included with supported Orq.ai workspaces. MCP features in VS Code are available with supported AI/agent configurations; confirm availability in both your VS Code setup and Orq plans.

Why Connect VSCode to Orq.ai?

Keep your team in one editor

Stop switching between VS Code, the Orq.ai dashboard, and separate eval scripts. Query traces, run experiments, and inspect deployments from the same IDE your team already uses every day.

Ask operational questions in natural language

Ask questions like “Show me yesterday’s failed agent runs grouped by error type” and let your VS Code agent turn that intent into Orq MCP tool calls. No SDKs to learn and no API URLs to memorize.

Connect evals to your development workflow

Use Orq.ai inside your existing VS Code workflows. From chat/agent mode, pull trace data, design and run evals, and kick off experiments as part of your normal coding sessions.

Keep production behavior visible

Orq.ai gives teams visibility into MCP‑driven activity, including which tools ran, when they ran, and which key or workspace triggered them. VS Code brings that visibility into the same environment where you write and review code.

Keep your team in one editor

Stop switching between VS Code, the Orq.ai dashboard, and separate eval scripts. Query traces, run experiments, and inspect deployments from the same IDE your team already uses every day.

Ask operational questions in natural language

Ask questions like “Show me yesterday’s failed agent runs grouped by error type” and let your VS Code agent turn that intent into Orq MCP tool calls. No SDKs to learn and no API URLs to memorize.

Connect evals to your development workflow

Use Orq.ai inside your existing VS Code workflows. From chat/agent mode, pull trace data, design and run evals, and kick off experiments as part of your normal coding sessions.

Keep production behavior visible

Orq.ai gives teams visibility into MCP‑driven activity, including which tools ran, when they ran, and which key or workspace triggered them. VS Code brings that visibility into the same environment where you write and review code.

Keep your team in one editor

Stop switching between VS Code, the Orq.ai dashboard, and separate eval scripts. Query traces, run experiments, and inspect deployments from the same IDE your team already uses every day.

Ask operational questions in natural language

Ask questions like “Show me yesterday’s failed agent runs grouped by error type” and let your VS Code agent turn that intent into Orq MCP tool calls. No SDKs to learn and no API URLs to memorize.

Connect evals to your development workflow

Use Orq.ai inside your existing VS Code workflows. From chat/agent mode, pull trace data, design and run evals, and kick off experiments as part of your normal coding sessions.

Keep production behavior visible

Orq.ai gives teams visibility into MCP‑driven activity, including which tools ran, when they ran, and which key or workspace triggered them. VS Code brings that visibility into the same environment where you write and review code.

Keep your team in one editor

Stop switching between VS Code, the Orq.ai dashboard, and separate eval scripts. Query traces, run experiments, and inspect deployments from the same IDE your team already uses every day.

Ask operational questions in natural language

Ask questions like “Show me yesterday’s failed agent runs grouped by error type” and let your VS Code agent turn that intent into Orq MCP tool calls. No SDKs to learn and no API URLs to memorize.

Connect evals to your development workflow

Use Orq.ai inside your existing VS Code workflows. From chat/agent mode, pull trace data, design and run evals, and kick off experiments as part of your normal coding sessions.

Keep production behavior visible

Orq.ai gives teams visibility into MCP‑driven activity, including which tools ran, when they ran, and which key or workspace triggered them. VS Code brings that visibility into the same environment where you write and review code.

Setup

1: Install Codex

Make sure MCP servers and agents are enabled in your VS Code installation.

  • Open VS Code.

  • Ensure you have an AI/agent experience installed (for example GitHub Copilot Agent or Microsoft’s AI Toolkit).

  • Check AI/MCP settings (for enterprise, ChatMCP policies may control whether MCP is allowed).

2: Create an Orq.ai API key

In Orq.ai, create an API key for the workspace or project you want VS Code to access.

Keep this key handy; you’ll reference it in MCP config or via environment variables, for example:

bash

export ORQ_API_KEY="<your-orq-api-key>"

3: Add the Orq MCP server to Codex

VS Code supports MCP servers via a JSON configuration file, typically .vscode/mcp.json in your workspace or a user/global configuration.

You can configure Orq in two main ways:

Option A – Via Command Palette (wizard)

Use the built‑in MCP wizard to create or edit your MCP config.

  1. Open the Command Palette (Ctrl+Shift+P / Cmd+Shift+P).

  2. Run “MCP: Add Server” (or “MCP: Add server”).

  3. Choose HTTP (server‑sent events / HTTP) when asked how to connect to the MCP server.

  4. Paste the Orq MCP URL:https://my.orq.ai/v2/mcp

  5. Give the server a name like orq.

  6. VS Code will create or update your MCP config file (often .vscode/mcp.json in the workspace) with a servers entry for Orq.

  7. Reload VS Code or the agent to ensure your changes are picked up.

Option B – Edit .vscode/mcp.json directly

If you prefer explicit JSON, or want multiple Orq environments, edit the config file directly.

  1. Open your project folder in VS Code.

  2. Create a .vscode folder at the root if it doesn’t exist.

  3. Inside .vscode, create or open mcp.json.

  4. Add an Orq entry to the servers object. A conceptual example:

json

{

 "inputs": [],

 "servers": {

   "orq": {

     "type": "http",

     "url": "https://my.orq.ai/v2/mcp",

     "env": {

       "ORQ_API_KEY": "<your-orq-api-key>"

     }

   }

 }

}
json

{

 "inputs": [],

 "servers": {

   "orq": {

     "type": "http",

     "url": "https://my.orq.ai/v2/mcp",

     "env": {

       "ORQ_API_KEY": "<your-orq-api-key>"

     }

   }

 }

}
json

{

 "inputs": [],

 "servers": {

   "orq": {

     "type": "http",

     "url": "https://my.orq.ai/v2/mcp",

     "env": {

       "ORQ_API_KEY": "<your-orq-api-key>"

     }

   }

 }

}
json

{

 "inputs": [],

 "servers": {

   "orq": {

     "type": "http",

     "url": "https://my.orq.ai/v2/mcp",

     "env": {

       "ORQ_API_KEY": "<your-orq-api-key>"

     }

   }

 }

}

This mirrors how other remote MCP servers (like GitHub’s server) are configured in .vscode/mcp.json, using env to pass tokens securely.

  1. Save the file. VS Code’s agent mode will use this configuration for MCP tools in that workspace.

4: Start using Orq.ai tools from VSCode

With Orq configured:

  1. Open the AI/Agent chat in VS Code (for example, Copilot Agent or AI Toolkit Chat).

  2. Optionally, use the Tools panel / “Select tools” UI to verify that orq is listed as an available MCP server with tools.

  3. In chat, ask:

  • What MCP tools do you have access to?”

  • “Use Orq to list recent failed agent runs grouped by error type.

If everything is configured correctly, the agent will call Orq’s MCP tools and show results inline.


What Can You Do with Orq.ai + VSCode

Query observability data in natural language

Use VS Code’s agent to “talk” to your Orq.ai traces. Ask for failed agent runs, slowest requests over the last 24 hours, or errors grouped by model, then apply those insights directly in your editor.

Design and run evaluations

Describe the behavior you want to test, let the agent and Orq.ai scaffold evaluators and datasets, then run evals against your deployments without leaving VS Code.

Compare prompts, models, and configs

From VS Code, create experiments that compare prompts, models, or configurations, run them on real or synthetic datasets, and inspect results in Orq.ai when you need deeper drill‑down.

Generate reusable synthetic datasets

Ask the agent to create challenging synthetic test cases for workflows such as contract analysis or support tickets, and save them as reusable Orq.ai datasets.

Debug production regressions as a team

When something breaks, stay in VS Code. Pull recent traces for a deployment via Orq, filter by failure pattern, and use those examples to guide prompt or model changes backed by experiments and evals.

Query observability data in natural language

Use VS Code’s agent to “talk” to your Orq.ai traces. Ask for failed agent runs, slowest requests over the last 24 hours, or errors grouped by model, then apply those insights directly in your editor.

Design and run evaluations

Describe the behavior you want to test, let the agent and Orq.ai scaffold evaluators and datasets, then run evals against your deployments without leaving VS Code.

Compare prompts, models, and configs

From VS Code, create experiments that compare prompts, models, or configurations, run them on real or synthetic datasets, and inspect results in Orq.ai when you need deeper drill‑down.

Generate reusable synthetic datasets

Ask the agent to create challenging synthetic test cases for workflows such as contract analysis or support tickets, and save them as reusable Orq.ai datasets.

Debug production regressions as a team

When something breaks, stay in VS Code. Pull recent traces for a deployment via Orq, filter by failure pattern, and use those examples to guide prompt or model changes backed by experiments and evals.

Query observability data in natural language

Use VS Code’s agent to “talk” to your Orq.ai traces. Ask for failed agent runs, slowest requests over the last 24 hours, or errors grouped by model, then apply those insights directly in your editor.

Design and run evaluations

Describe the behavior you want to test, let the agent and Orq.ai scaffold evaluators and datasets, then run evals against your deployments without leaving VS Code.

Compare prompts, models, and configs

From VS Code, create experiments that compare prompts, models, or configurations, run them on real or synthetic datasets, and inspect results in Orq.ai when you need deeper drill‑down.

Generate reusable synthetic datasets

Ask the agent to create challenging synthetic test cases for workflows such as contract analysis or support tickets, and save them as reusable Orq.ai datasets.

Debug production regressions as a team

When something breaks, stay in VS Code. Pull recent traces for a deployment via Orq, filter by failure pattern, and use those examples to guide prompt or model changes backed by experiments and evals.

Query observability data in natural language

Use VS Code’s agent to “talk” to your Orq.ai traces. Ask for failed agent runs, slowest requests over the last 24 hours, or errors grouped by model, then apply those insights directly in your editor.

Design and run evaluations

Describe the behavior you want to test, let the agent and Orq.ai scaffold evaluators and datasets, then run evals against your deployments without leaving VS Code.

Compare prompts, models, and configs

From VS Code, create experiments that compare prompts, models, or configurations, run them on real or synthetic datasets, and inspect results in Orq.ai when you need deeper drill‑down.

Generate reusable synthetic datasets

Ask the agent to create challenging synthetic test cases for workflows such as contract analysis or support tickets, and save them as reusable Orq.ai datasets.

Debug production regressions as a team

When something breaks, stay in VS Code. Pull recent traces for a deployment via Orq, filter by failure pattern, and use those examples to guide prompt or model changes backed by experiments and evals.

VSCode direct vs with Orq.ai MCP

Capability

VSCode alone

VSCode + Orq.ai MCP

Query production LLM traces

No built‑in view into Orq.ai’s observability data.

Ask VS Code’s agent to list, filter, and group Orq.ai traces (failures, slow runs, agent tool calls, etc.) from inside the IDE.

Run experiments on prompts

Teams can iterate on prompts manually in chat, but no native experiment tracking in Orq.ai.

Create and run Orq.ai experiments comparing prompts, models, or configs against datasets, directly from your VS Code sessions.


Generate synthetic eval data

You can prompt the agent to generate examples, then copy/paste them elsewhere.

Generate synthetic test cases and save them as reusable Orq.ai datasets for evals and experiments.

Pull cost and usage analytics

No view into Orq router or deployment analytics.

Query Orq.ai’s cost, usage, and performance metrics for models and deployments via MCP tools, then inspect them alongside your code in VS Code.

Run evaluators on datasets

No built‑in concept of Orq evaluators or datasets.

Work with Orq evaluators and datasets from VSCode, depending on the MCP tools enabled.


FAQs

Do I have to use VS Code to get value from Orq.ai?

No. Orq.ai works on its own through the UI and API. VS Code is an optional IDE front‑end for your workspace. You get the same experiments, evals, and observability in Orq; VS Code simply lets teams drive them from their editor using natural language.

What can VS Code see in my Orq.ai workspace, and how is access controlled?

VS Code’s agents only see what the Orq API key you configure is allowed to access. If you use a project‑level key scoped to a specific workspace or environment, they can only query traces, experiments, datasets, and deployments inside that scope. Rotate or revoke the key in Orq to instantly cut off access.

Can I point VS Code at different Orq environments (dev, staging, prod)?

Yes. You can create separate Orq entries in user MCP configuration and/or .vscode/mcp.json (for example orq-dev, orq-staging, orq-prod) with different API keys or URLs, then select the one you need per workspace. That way, you can run evals and inspect traces in dev or staging first, then switch the same VS Code setup to the production environment.

Does VS Code connect to Orq directly from my machine?

Yes. VS Code runs as a local IDE and connects to remote MCP servers over the network using HTTP/SSE. Your Orq MCP endpoint must be reachable over the public internet (or via your network/VPN), and Orq handles authentication and scoping via your API key.

Create an account and start building today.

Book a demo

Explore docs

Create an account and start building today.

Book a demo

Explore docs

Create an account and start building today.

Book a demo

Explore docs

Create an account and start building today.

Book a demo

Explore docs