Research
From experiments to insights: Orq.ai research hub
Filter by theme:
Large Language Models
Generative AI
Prompt Engineering
RAG-as-a-Service
Working safely with PII in the AI Gateway
The Orq.ai PII plugin is now in open beta. It masks personal data before a request leaves the AI Gateway, and restores the real values before the response reaches your user.
orq launch, running coding agents with Orq using one command
orq launch starts a coding agent already routed through the Orq.ai Gateway, with the workspace tools and skills borrowed for the session and nothing left on disk.
Introducing the Orq.ai CLI
The Orq.ai CLI covers the entire Orq.ai API, and ships the Orq.ai Skills inside the binary so your coding agent knows how to use it.
LLM-as-a-judge Alignment with Humans
Ranking evaluation data by how often an LLM judge flips its own answer gives you a review queue that is two to three times better than random sampling, and one blind spot you have to plan around.
PII redaction is cheap for RAG, until your users search for people
Redacting PII from a RAG query costs a few points of recall for semantic search, but knocks the right document out of the top result about 60% of the time for person queries.
Three Red-Teaming Frameworks, One Judge Panel, 5,624 Attacks
What 5,624 adversarial attacks across Evaluatorq, PromptFoo, and DeepTeam tell us about attack quality, judge calibration.
Weak judges, strong panel: an ensemble approach to LLM eval
Your eval pipeline has a judge. That judge has biases. Adding two more judges fixes that, but only if they disagree on the right things.
Copy-Pasting Your Prompt Twice Makes LLMs Smarter
The cheapest accuracy upgrade in LLM engineering still works on GPT-5.4, Claude Sonnet 4.6, and Gemini 3 Pro. Here is what 334 runs taught us.
System Prompt Placement for LLM-as-a-Judge
Every team building an LLM-as-a-judge pipeline faces the same question: does it matter where you put your instructions?
Prompt Optimization for Smaller, Higher-Performing Models
We explored using prompt optimization to make cheaper, smaller language models perform as well as expensive, more capable ones.
Can a 14B Model Match a 100B+ Model? We Fine-Tuned 8
Key takeways from fine-tuning 8+ language models on a text classification task, from tiny 0.6B models to 14B behemoths.
Filter by theme:
Large Language Models
Generative AI
Prompt Engineering
RAG-as-a-Service
Working safely with PII in the AI Gateway
The Orq.ai PII plugin is now in open beta. It masks personal data before a request leaves the AI Gateway, and restores the real values before the response reaches your user.
orq launch, running coding agents with Orq using one command
orq launch starts a coding agent already routed through the Orq.ai Gateway, with the workspace tools and skills borrowed for the session and nothing left on disk.
Introducing the Orq.ai CLI
The Orq.ai CLI covers the entire Orq.ai API, and ships the Orq.ai Skills inside the binary so your coding agent knows how to use it.
LLM-as-a-judge Alignment with Humans
Ranking evaluation data by how often an LLM judge flips its own answer gives you a review queue that is two to three times better than random sampling, and one blind spot you have to plan around.
PII redaction is cheap for RAG, until your users search for people
Redacting PII from a RAG query costs a few points of recall for semantic search, but knocks the right document out of the top result about 60% of the time for person queries.
Three Red-Teaming Frameworks, One Judge Panel, 5,624 Attacks
What 5,624 adversarial attacks across Evaluatorq, PromptFoo, and DeepTeam tell us about attack quality, judge calibration.
Weak judges, strong panel: an ensemble approach to LLM eval
Your eval pipeline has a judge. That judge has biases. Adding two more judges fixes that, but only if they disagree on the right things.
Copy-Pasting Your Prompt Twice Makes LLMs Smarter
The cheapest accuracy upgrade in LLM engineering still works on GPT-5.4, Claude Sonnet 4.6, and Gemini 3 Pro. Here is what 334 runs taught us.
System Prompt Placement for LLM-as-a-Judge
Every team building an LLM-as-a-judge pipeline faces the same question: does it matter where you put your instructions?
Prompt Optimization for Smaller, Higher-Performing Models
We explored using prompt optimization to make cheaper, smaller language models perform as well as expensive, more capable ones.
Can a 14B Model Match a 100B+ Model? We Fine-Tuned 8
Key takeways from fine-tuning 8+ language models on a text classification task, from tiny 0.6B models to 14B behemoths.
Filter by theme:
Large Language Models
Generative AI
Prompt Engineering
RAG-as-a-Service
Working safely with PII in the AI Gateway
The Orq.ai PII plugin is now in open beta. It masks personal data before a request leaves the AI Gateway, and restores the real values before the response reaches your user.
orq launch, running coding agents with Orq using one command
orq launch starts a coding agent already routed through the Orq.ai Gateway, with the workspace tools and skills borrowed for the session and nothing left on disk.
Introducing the Orq.ai CLI
The Orq.ai CLI covers the entire Orq.ai API, and ships the Orq.ai Skills inside the binary so your coding agent knows how to use it.
LLM-as-a-judge Alignment with Humans
Ranking evaluation data by how often an LLM judge flips its own answer gives you a review queue that is two to three times better than random sampling, and one blind spot you have to plan around.
PII redaction is cheap for RAG, until your users search for people
Redacting PII from a RAG query costs a few points of recall for semantic search, but knocks the right document out of the top result about 60% of the time for person queries.
Three Red-Teaming Frameworks, One Judge Panel, 5,624 Attacks
What 5,624 adversarial attacks across Evaluatorq, PromptFoo, and DeepTeam tell us about attack quality, judge calibration.
Weak judges, strong panel: an ensemble approach to LLM eval
Your eval pipeline has a judge. That judge has biases. Adding two more judges fixes that, but only if they disagree on the right things.
Copy-Pasting Your Prompt Twice Makes LLMs Smarter
The cheapest accuracy upgrade in LLM engineering still works on GPT-5.4, Claude Sonnet 4.6, and Gemini 3 Pro. Here is what 334 runs taught us.
System Prompt Placement for LLM-as-a-Judge
Every team building an LLM-as-a-judge pipeline faces the same question: does it matter where you put your instructions?
Prompt Optimization for Smaller, Higher-Performing Models
We explored using prompt optimization to make cheaper, smaller language models perform as well as expensive, more capable ones.
Can a 14B Model Match a 100B+ Model? We Fine-Tuned 8
Key takeways from fine-tuning 8+ language models on a text classification task, from tiny 0.6B models to 14B behemoths.
