Back to Toolkit

LLM Agent Architecture

A framework for understanding the components of an agent setup. Start with the minimum — a model and a prompt. Give it context to work with (e.g. today's date, conversation history). Give it actions to reach beyond the conversation (e.g. database access, code execution). Then wrap it in infrastructure to keep it reliable in production (e.g. fallback models, cost controls).

LLM Agent Architecture

Required

Model Configuration

Model selection plus inference parameters — temperature, top-k, top-p, max tokens

Model Configuration

Model selection plus inference parameters — temperature, top-k, top-p, max tokens

Instructions / System Prompt

Agent identity, tone, operating principles, constraints — primary behavioural control

Instructions / System Prompt

Agent identity, tone, operating principles, constraints — primary behavioural control

User Prompt

The query or task that initiates each run — may itself contain pasted data, context, or live information

User Prompt

The query or task that initiates each run — may itself contain pasted data, context, or live information

Runtime Constraints

Max iterations and max execution time — defines the operational boundary of the agent run

Runtime Constraints

Max iterations and max execution time — defines the operational boundary of the agent run

Recommended

Output Format / Schema

Response structure — JSON, markdown, structured object — critical for downstream systems

Output Format / Schema

Response structure — JSON, markdown, structured object — critical for downstream systems

Optional

Knowledge Base

Vector store, RAG documents — built once, queried dynamically at runtime

Knowledge Base

Vector store, RAG documents — built once, queried dynamically at runtime

Memory Store

Persistent storage across sessions with entity-based isolation. Supports query, write, and delete.

Memory Store

Persistent storage across sessions with entity-based isolation. Supports query, write, and delete.

Output Format / Schema

Response structure — JSON, markdown, structured object — critical for downstream systems

Output Format / Schema

Response structure — JSON, markdown, structured object — critical for downstream systems

Output Format / Schema

Response structure — JSON, markdown, structured object — critical for downstream systems

Output Format / Schema

Response structure — JSON, markdown, structured object — critical for downstream systems

Common examples

Report writing

Structured notes into a formal report.

~1 hr / report

Sales assistant

Build buyer confidence and guide purchases.

Drives revenue

Email triage

Classify, summarise, route to the right team.

~5 min each, 100+ / day

Invoice processing

Extract data, match to POs, flag discrepancies.

~1 hr / report

Meeting actions

Extract action items, decisions, and owners.

~30 min / meeting

RFP responses

Assemble answers from past responses and docs.

~30 min / meeting

Common examples

Quantify the opportunity

Use the Business Case Builder to calculate the ROI and build a case for stakeholders.

Open Business Case Builder

Browse more use cases

Explore a wider set of LLM use cases across industries to spark ideas.

Read on orq.ai