Agentic AI Reliability: Two Engineering Disciplines Most Teams Skip
Agentic AIMost teams building agentic AI treat model selection as the primary reliability lever. A detailed case study from Thoughtworks, published on Martin Fowler’s site, argues the real determinants of production reliability are two disciplines most teams treat as afterthoughts.
The Problem Agentic Systems Expose
The PRINCE platform, built at Bayer’s preclinical drug discovery division, started as a metadata search tool and evolved into a multi-agent workflow system processing thousands of study reports. The build team identified three structural problems that plague most enterprise agentic deployments: data silos fragmenting context across disparate systems, keyword-based retrieval failing on technical terminology variation, and manual analysis workloads outpacing what any search layer can offload.
These aren’t infrastructure problems. They’re engineering discipline problems. Thoughtworks principal consultant Sarang Sanjay Kulkarni framed the solution around two foundational concepts.
Context Engineering: What Each Agent Gets to See
Context engineering is the discipline of deciding what information each model receives and how context flows between specialized steps — not just plugging in a vector store and assuming retrieval works.
In PRINCE, this meant a deliberate hybrid retrieval pipeline: 70% semantic search weight, 30% keyword search, with query expansion generating five semantically similar variants per user question before any retrieval begins. Initial results (~20 chunks) are reranked via a cross-encoder down to the top seven before the generation step. Every one of those decisions is a context engineering choice. Get them wrong and even the best foundation model hallucinates to fill gaps.
Harness Engineering: What Keeps the System Alive
Harness engineering covers everything that makes multi-agent systems survivable in production: orchestration logic, tool boundaries, state persistence, retries, fallbacks, validation, reflection loops, observability, and human review checkpoints.
In PRINCE, harness engineering meant PostgreSQL-backed state persistence so a failed multi-hour research run could resume from a checkpoint rather than restart from scratch. It meant LLM fallbacks across multiple providers and SQL retry loops hard-capped at three iterations to prevent runaway agent behavior. The distinction matters because these are frequently treated as DevOps concerns to bolt on after the model works — the Thoughtworks team found that treating them as first-class engineering disciplines from the start is what separated prototypes from production systems.
The Pattern
Multiple Stack Archive posts have examined the same gap from different angles over the past three months — observability built for agent workloads, data isolation for agentic pipelines, and the engineering discipline required to ship reliable agents. The PRINCE case study names the two meta-disciplines that unify all of them.
Reflection Loops as a Trust Mechanism
The most operationally interesting part of the PRINCE architecture is its three-layer reflection system. Process reflection validates whether the workflow is sound before committing compute. Data reflection assesses whether retrieved information is sufficient before synthesis. Draft reflection checks whether the output is complete before delivery.
Each layer addresses a distinct failure mode. Without process reflection, agents proceed on flawed plans. Without data reflection, systems hallucinate to fill retrieval gaps. Without draft reflection, incomplete outputs ship with apparent confidence. Crucially, these reflection loops are enforced at the harness level — not the prompt level — which makes them reliable across model and temperature changes. The result is the granular citations and intermediate step transparency that build user trust in production environments, backed by daily live-traffic evaluation using RAGAS metrics for faithfulness, relevancy, and accuracy.
The So What
Teams building agentic AI should audit which of these two disciplines they’ve actually invested in versus assumed would sort itself out. Context engineering requires deliberate retrieval architecture decisions — weighting, chunking strategy, reranking — taken before users touch the system. Harness engineering requires designing for failure modes before they appear in production: state persistence, retry caps, observable intermediate steps. The PRINCE case study is a production reference specific enough to adapt to any domain requiring multi-step agentic research workflows.
Source: Martin Fowler
Content created with AI assistance and reviewed for accuracy.
Join the conversation
Stack Insiders is our free community for readers who want to go deeper — share resources, ask questions, and connect with others across every vertical we cover.
Join Stack Insiders →