Give Claude a clean ticket, repository access, and a decent setup, and it can do a lot.
It can inspect the codebase, make the change, run tests, and open a pull request.
In a demo, that can feel surprisingly close to end-to-end software automation.
Then you give the same tool to an engineering team.
Now you have different repositories, different laptops, different permissions, different ideas about what “enough context” means, and engineers who are much better at working with agents than others.
The question changes from “can the agent complete this task?” to “can we make this way of working repeatable?”
That second problem is much bigger.
A Software Factory starts from a different assumption: the agent run itself isn’t enough.
And by Software Factory, we mean the wider system around the coding agent: how work is planned, executed, checked, released, and improved.
The factory shouldn’t have to rebuild every cross-cutting control itself. Governance, observability, security and cost control are better treated as shared infrastructure underneath the factories that use them.
One team might build a vulnerability-patching factory while another builds a frontend-maintenance factory.
Both should inherit the same underlying controls.
You also have to decide what the agent knows and what happens when the result isn’t good enough.
The demo breaks at team scale
Give 50 engineers the same coding agent and you probably won’t get the same improvement 50 times.
A few people get very good with it.
They know which architecture docs Claude needs, which tools are worth exposing and when to kill a run before it produces another 500 lines of code. Someone else can use exactly the same model and mostly replace typing code with typing prompts.
We’ve seen the consequences show up in technology choices.
An enterprise may already have perfectly sensible standards for frameworks and architecture.
The coding agent doesn’t automatically know that. If a library looks like a good fit for the task in front of it, the model may use it even though the architecture team ruled it out.
Do that across enough engineers and repositories and you can actually end up with more technology sprawl after rolling out coding agents.
“We’ve actually had more technology sprawl because these coding agents just do their own thing, and people aren’t giving them the skills and context properly.” - Sohrab Hosseini, Orq.ai co-founder
Local execution adds another variable. Claude Code can see the files, tools, and configuration available on a developer’s machine. For interactive coding, that’s often exactly what the engineer wants. It may bear little resemblance to the environment the change eventually has to run in.
The same problem appears if every team builds its own factory independently. One team adds its own logging. Another invents a permission model. A third builds separate model controls and cost tracking. Pretty quickly, the enterprise has moved the sprawl up a layer.
We think those concerns belong lower in the foundation. Factories should be able to differ in the work they perform while inheriting common governance, observability, security and financial controls from the same foundation.
Good engineers usually find ways around these problems. They accumulate instructions and a feel for where the agent tends to make bad calls.
We see that happening already.
A lot of that knowledge stays with the engineer who figured it out.
A ticket hides more decisions than it looks like
Take a ticket that says “add enterprise SSO.”
Claude can start writing code from that.
That doesn’t mean it should.
What happens to existing customers?
Which service should own authentication?
Is the dependency the agent wants to use actually approved?
Are we extending the existing identity flow or creating another one?
There may be obvious answers inside the engineering team.
They’re only obvious to the agent if somebody gives it access to them.
This is why we put planning before a long implementation run.
The ticket can be triaged first, then turned into a plan engineers can inspect and change.
If the proposed architecture is wrong, changing a few lines in a plan is a lot less painful than discovering it after the agent has produced a large, internally consistent implementation.
Sometimes the ticket should go backwards.
Maybe another team owns the service. Maybe the requirement is too vague. Maybe the dependency the agent picked is off limits.
Once those questions are settled, writing the code is only part of what remains.
Tests still have to pass. Automated reviewsreview may find something. Policies may block the change.
Some work will still warrant a person looking at it before it goes any further.
A pull request isn't the finish line
A factory can produce plenty of accepted pull requests and still be doing a poor job.
Approved changes can fail in QA. They can introduce regressions later. Reviewers can repeatedly miss the same policy violation. And a factory can produce beautiful numbers simply because you keep feeding it the easiest work.
We’d want to see where humans are still correcting the output, whether the same failure comes back two weeks later, and how often somebody has to step in and rescue the run.
Cost becomes visible at a different level, too. If work runs through a common factory runtime, model and tool usage can be attributed back to the work that caused it.
Rather than only knowing the monthly AI bill, teams can start asking how much agent execution went into a PR, what it cost to fix a class of bugs, or how much AI spentspend went into a sprint.
It gives the cost numbers some context. A more expensive run may be perfectly reasonable if it resolves difficult work with little human intervention. A cheap run that repeatedly produces rejected PRs may not be cheap at all.
Look beyond PR acceptance and monitor the signals that show whether the system is actually behaving well.
The type of work matters as well.
A factory could become boringly reliable at dependency updates and still be nowhere near ready to change authentication logic without supervision.
The factory should remember what the team learns
Say Claude keeps choosing a library your architecture team doesn’t want in the stack.
You can fix the PR. Then somebody else fixes the same thing next week.
Or you can change the setup the agent works with.
“Your best engineers become factory supervisors. They’re looking at where Claude Code spirals a bit out of control: do we need to improve the skill, give it a better MCP, or give it better blueprints of the architecture?” - Sohrab Hosseini, Orq.ai co-founder
Different failures call for different fixes.
If somebody already figured this out last week, the next engineer shouldn’t have to solve it again.
The underlying technology will move around too.
A team might use Claude Code and run its agents inside its own GCP environment today. Six months from now it could be evaluating another model or another harness.
Or even both.
That shouldn’t mean recreating all of the architecture guidance and lessons the team accumulated along the way.
Fix the setup once, and later runs should benefit from it.
Start with the work your team keeps doing again and again
A vulnerability scanner raises an issue.
The affected dependency is known.
There are tests.
The remediation path is familiar and the change already has a normal review process.
Compare that with a vaguely specified feature that touches five services and needs three teams to agree on what the customer actually wants.
They can both show up as tickets. They’re very different candidates for automation.
Suppose a team spends 10% of its time patching vulnerabilities and another 25% dealing with bugs.
Some of those bugs are ugly problems that need an experienced engineer to sit with them for a while.
Others follow a path the team has walked dozens of times before.
That recurring work is where we’d look first.
The same approach can apply beyond code changes as agent security testing can become another recurring workflow.
For example, evaluatorq can target specific vulnerabilities such as prompt injection or goal hijacking:
Dependency upgrades, some maintenance work, and small fixes can have similarly recognizable paths.
The boundary will get complicated. A bug that looked trivial can turn into an architectural problem halfway through, and the factory needs a way to stop rather than stubbornly finish a task that no longer fits the workflow.
That’s also why we wouldn’t start by asking, “How much of engineering can we automate?”
Look at where the team’s time actually goes.
Find a category of work with reasonably predictable inputs and a clear idea of what “good” looks like. See where the factory struggles with it.
Fix that.
Then decide whether there’s another chunk of work worth handing over.
Better coding agents alone don't create reliable software delivery
Six months from now, there will probably be a better coding model.
An engineering team shouldn’t have to start another experiment from zero when it arrives. By then, the team should already know which work it’s comfortable handing over and where the workflow tends to fail and which checks are actually useful.
Then you can put the new model into an existing setup and see whether the work improves.
The model may write better code. It may need less supervision. It may make another category of work practical to automate. None of that requires rebuilding the operating model around it.
That’s the difference we care about.
If you want to see what that looks like in practice, explore how Orq.ai helps engineering teams build Software Factories around the stack they already use.
