The Human-Machine Partnership: Architectures for Reliable AI

 

Agentic AI for balancing speed, resilience and human authority in automation.

The Human-In-The-Loop (HITL) concept went through its own history of becoming a milestone for humanity: from the human in the role of a controller to supervisor to model trainer, the human has lately been assigned the role of an ethical or legal safeguard and in its most recent job description it is a collaborator and system governor [Figure 1].

Figure 1: Human-In-The-Loop (HITL) History

HITL is not the destination. It is a transitional architecture until machine verification becomes sufficiently trustworthy for medium- and low-risk applications. HITL remains a legally mandated form of human oversight for high-risk applications under the EU AI Act[1].

Humans in control and oversight positions become tired and their output inconsistent. HITLs as accountability gatekeepers is ethically noble but in reality not applicable: when it boils down to responsibility and accountability of consequences from high-risk behavior that promised commensurate profit, the relevant person is either absent to begin with or conveniently fired afterwards. Furthermore, the ability and mental capability of the human in the oversight position might become a process-flow bottleneck, an insider threat or simply unreliable due to distractions.

HITL remains legally mandated for high-risk systems, but its weaknesses make it an unreliable sole safeguard. This is where automatic verification, not as a replacement, but as a complement, becomes strategically valuable. MITL and dMITL architectures can filter routine decisions, reduce human fatigue, and escalate only when human judgment is genuinely needed.

 

MITL and dMITL Architectures

In this article, MITL refers to model-based operational verification through a model-in-the-loop, while dMITL is proposed as a diverse model architecture also for operational verifications [Figure 2].

Figure 2: HITL, MITL, and dMITL Architectures

With the concept of the model-in-the-loop (MITL) certain verifications can be further automated by an independent model preventing a HITL escalation. An independent model can be a model from the same lab but with higher reasoning skills or it can be a comparable model from another lab. For simple verifications it might be easier to use models from the same lab as only one API key must be managed. For more complex situations it might be useful to use a model from another lab to prevent mutual blind spots – but it requires managing at least two API keys. Using models from different labs only reduces the risk of blind spots – it does not eliminate it[2].

The diverse Model-In-The-Loop (dMITL) concept most likely does improve resilience as teams of diverse agents show improved predictability, enhanced system-level alignment, and resilience[3]. It might also be beneficial to intentionally use models from diverse geographical and/or geopolitical regions as main models or resilient fallbacks. Privacy constraints and the decision to work on-prem or in the cloud might be the decisive factor in some cases while business continuity and resilience are the more important factors in other cases.

Diverse agents can contribute in different ways, for example by taking an average of their scores or by letting them find a consensus for the task at hand. Consensus could be defined as a 90% agreement on confidence thresholds across models, or a unanimous classification of an action as 'safe' before execution. Please observe, that dMITL always needs empirically calibrated scores, independent evidence or task-specific benchmarks. Depending on the application and its context, it might be beneficial to run simulations on which models work best together and produce the most reliable output for a specific automation process. If an agentic team is deployed consisting of diverse models with a consensus involved, it reduces the chance that consensus is reached based on correlated failure, shared bias or conformity compared to a MITL agent team approach – but it doesn’t exclude this risk[4]. Hence, consensus should never be taken for proven evidence in any process automation but it can serve as a practical indicator. If equipped with a confidence score for each consensus, decisions can even be ranked and selectively overseen by a human even below the escalation threshold, smartly reducing its workload while maintaining or even improving quality. For high-risk applications, consensus should never override deterministic safety constraints, prohibited action rules, independent evidence, minority objections, or human escalation thresholds.

An example of a fully autonomous dMITL approach is the ConceptRadar.org, a platform for knowledge representation that uses a diverse team of LLM-based agents to evaluate and validate content quality. Rather than relying on authorship status or single-model judgment, ConceptRadar deploys a consensus mechanism across geographically distributed models to assess factual accuracy, logical consistency, novelty and more. This operationalizes the dMITL principle: resilience through diversity, quality through consensus without requiring a human in the loop.

Deploying dMITL agents or agent teams also has a downside: a massively increased attack surface, several more API keys to manage and the computational cost of a consensus mechanism is high. If consensus is a hard requirement for the process, it is necessary to have a fallback for each agent team member. The dMITL provides resilience and quality but it doesn’t come for free.

The decision to deploy dMITL is therefore not a technical one; it is a governance decision. The business case must demonstrate that the increased resilience justifies the increased complexity and cost, and this trade-off must be explicitly budgeted within the organization's risk framework.

 

Controlling a multi-agent system can feel like managing a bag of fleas: each agent moving independently, with emergent behaviors that are difficult to predict. The MITL and dMITL architectures described above provide a framework for bringing order to this chaos... These architectures allow for a dynamic governance of agentic AI in diverse fields including, with the corresponding caveat, the regulated industries.

The electric grid is a good example for nested control mechanisms and how governance of agentic AI could be designed accordingly. The grid is controlled by a very fast automatic layer with pre-defined reaction mechanisms that are augmented with human supervision and organizational governance. If this simple concept is extended to the governance of agentic AI, four distinguished loops can be identified [Figure 3]:

1. Operational loop: agents act within specified boundaries.
2. Verification loop: rules, tools, models or humans check the action.
3. Escalation loop: uncertainty or disagreement triggers human intervention.
4. Governance loop: humans define boundaries, objectives and acceptable residual risk.

The four loops described below provide the 'net' and 'training' for this bag of fleas, channeling emergent behavior toward predictable outcomes.

Figure 3: Loops of Agentic AI Governance

This makes governance look simple and allows to break down even complex systems into processes that can be governed. The complexity of a system to be governed always influences the overall quality of governance: to overview exceedingly complex systems is simply not possible.

This challenge of steering complex systems is not new. Let’s look at the linguistic origin of “Government” and “Governance”. Both words come from the Greek word kybernan (to steer or pilot a ship), which passed into Latin as gubernare but they took different suffixes over time (-ment vs. -ance). The root of both words is “Steering” and implies active, continuous course correction in a dynamic environment. I leave it up to the reader to judge whether the complexity of any modern Government still can be successfully governed… let alone actively steered.

This leads to the challenge of navigating in open, complex and dynamic environments. Humans’ capability for decision making in such environments still usually exceed the according ability of agentic systems. At least in the summer of 2026. But AI is catching up as the Metagent-P shows in Minecraft[5].

The most resilient systems have been and are based on human-machine interaction. Let’s take as an example the self-drive abilities of cars: in good weather conditions the self-drive module is doing fine with minimal supervision. Research[6] found reduced situational awareness of the human pilot and slower responses under automation. Hence, if the system is in a high-confidence state, this is acceptable. But for an urgent transition, the user should receive sufficient transition time and should not face a time-critical takeover. Therefore, self-driving systems require in challenging weather that the human places its hands on the steering wheel, be attentive and ready to take over immediately - if necessary. This exemplifies well the human-machine-interaction and how the degree of interaction depends on the context: it is a fluid and adaptive mechanism. For best process automation these two diverse “computational systems” work in combination and are aware of each other’s blind spots. This boils down to cooperation, one of the most basic and most successful concepts in evolution, and is applicable also to agentic AI as explored in recent work on meta-cognition for agentic AI[7].

The above-described governance loops are indeed loops and not levels because they are interlinked and nested within a dynamic control system where each loop feeds from the previous while feeding into the consecutive, while the governance loop itself is fed and feeds into all other loops. This information exchange is not only useful for controlling the system but also for learning from the responses and behaviours of the system.

Of course…, but how can a system intentionally be designed such that any friction within the process or any escalation becomes a feature and not a flaw? Friction of any kind becomes a feature when it is predictable and actionable. Hence, the goal is to design the system in a way that escalation is not a failure mode, but a planned transition state. This requires that every escalation trigger is paired with a pre-packaged diagnostic package for the human expert, turning an interruption into a targeted intervention.

Then… this gained knowledge can be monetized in the governance level for an overall smoother process regulation: fewer false alarms, faster throughput, better and more consistent quality. Patterns in failures might reveal systemic weaknesses that competitors haven't identified yet. This enables improved risk forecasting and may even allow for less strict, and therefore more profitable, operational boundaries. This is exactly what turns governance from a cost center into a profit driver.

Let’s dive into each loop on its own to understand each loop’s core functionality.

Control systems or agents act within pre-defined boundaries and in most applications, speed is an important factor and must be balanced with quality. This loop ties closely into the governance loop where boundaries must be agreed on between relevant parties (ie business and compliance) and the playground clearly defined.

In many use cases, especially in manufacturing automation, the operational loop is based on deterministic constraints, ML or narrow AI. In more recent AI platforms though even the operational loop might involve GenAI and must be either constrained accordingly or verified. This leads to the following verification loop.

The definition of the edge of the playground of the agent(s) can depend on additional verifications and quality control mechanisms, for example that pre-defined rules are observed and all accessed tools documented. The boundaries might also depend on context or initial confidence of the operational layer executive functions.

Here comes into play that it is smart to not put all your eggs into one basket. With the MITL and especially dMITL concept described above, agentic AI presents novel ways of process control beyond the human-in-the-loop.

The approach must be in accordance with applicable governance and compliance requirements and agreed risk profile. The following illustrative decision matrix presents a guide on how different agentic architectures can be deployed.

Illustrative Decision Matrix for Agentic AI Architecture Implementation:

• Low-Risk/High-Speed: Use a MITL from the same lab.
• Medium-Risk/Medium-Speed: Use a dMITL from different labs.
• Medium-Risk/High-Speed: Use deterministic constraints plus dMITL verification and HITL if needed.
• High-Risk/Medium-Speed: Use a dMITL (agent team with consensus). HITL escalation if needed. Implement according to risk profile.
• High-Risk/High-Speed: Use a dMITL (agent team with consensus) combined with HITL escalation. For time-critical high-risk processes, the escalation loop must trigger a 'safe state' (e.g., hold, shutdown, or revert) immediately, while the human expert is consulted asynchronously for the post-mortem, rather than for real-time intervention. If the residual risk can’t be accepted, speed must be adjusted. Implement according to governance.
• Highest-Risk/Low-Speed: Use dMITL (agent team with consensus) plus HITL authorization.
• Uncertain but urgent: automated containment first, human decision second.

Design the system so that the more tolerable error is more likely. That is called asymmetric loss and fail-safe design. This asymmetry must be explicitly factored into both the verification logic and the risk profile.

Which option is finally chosen is not a technical preference; it is a risk-based governance decision, directly informed by your organization's compliance requirements, governance strategy, risk appetite and the criticality of the process for business continuity. Where possible, deploy observability tools, check possible agent drift and enforce strict alignment that prevents gaming any step of the process.

The escalation loop to a human supervisor can be triggered by exceeding a pre-defined value, bandwidth, uncertainty or disagreement within AI agents. For any automation process, this is the loop that costs the most: it brings production to a halt, it interferes with an autonomous mode and it depends on one or several humans with often specialized skills. Experts are expensive. While the business is interested in preventing escalations, the governance and compliance and possibly also quality control are very fond of such interventions. This boils down again to governance: how much risk can be accepted and where are the boundaries drawn. Governance is the back-and-forth between profit and safety: a business does not survive without profit but it also doesn’t survive if it kills itself accidentally. A healthy balance must be found between survival and quality.

Maintaining human experts who deal with the rarest and scariest exceptions and edge-cases while remaining idle (bored) the rest of the time, is another challenge that is often seen, for example in Nuclear Power Plants. Because this scenario throws the job description of the HITL back to where it was in the summer of ’69 of the previous century: a control function with the requirement to be responsible and accountable for the statistically impossible[8]. Skills must not just be learned but also maintained. How can this be achieved pleasantly? One option would be, that AI presents the human experts with interesting edge cases in the “grey zone”: this improves the system and sharpens the expert’s skills. This shifts the human experts from a simple cost-factor into a business-continuity-facilitator. The more complex and dynamic the system, the more continuous adaptation is needed—and the more a human expert can contribute.

To prevent the human expert though from becoming the new single point of failure, organizations could adopt a tiered expert model. Not every escalation requires the most expensive resource. A layered human oversight structure with generalists, specialists, and architects, ensures that the right level of expertise is applied at the right cost and speed, while also preventing expert burnout.

The introduction of a model-as-a-judge transfers the single-point-of-failure to the implemented judge. But the same applies possibly to the HITL. Hence, a diverse and democratic system would likely show the highest reliability but also the highest latency. And again, this boils down to governance and its definitions.

Here we are with business, experts, compliance and quality control all on one table discussing governance. A very theoretical situation from a real-world perspective… Here are three suggestions to make good governance a tangible result:

1. Introduce a “Boundary Negotiation Protocol” using the 5W1H-Questions with an emphasis on the “who” as governance fails when accountability is diffuse:

• Why do we need this process (business case)?
• Who is involved (human and/or AI)?
  • Who sets risk appetite?
  • Who owns the process risk?
  • Who can veto deployment?
  • Who can stop operations?
  • Who accepts residual risk?
  • Who independently audits the result?
• What is the goal or outcome (define speed, reliability, quality, other KPIs)?
• Where do the input, defining values, boundaries come from (definitions)?
• When does the process take place (triggers) and when not (escalation/risk)?
• How is the process done (details, tools, power)

For risk evaluation there must also be a matrix with “Permitted Zone” (acceptable behavior – usually high-confidence/low-consequence), “Prohibited Zone” (prohibited actions – usually low-confidence/high-consequence), and, most critical, a clearly defined “grey zone”, where escalation is required according to a predefined and exercised pathway.

2. Introduce “Residual Risk Budgeting”: Treat acceptable risk not as an abstract concept but as a quantifiable budget to be allocated across different agentic functions, similar to a financial budget. Example: if the budget for a specific process is used up, it triggers a governance overview process – hopefully resulting in an optimized process. This turns risk management from a subjective debate into a data-driven allocation problem and it makes governance and compliance more tangible and useful for the business.

3. Introduce “Continuous Improvement Rewards”: Errors and failures can be monetized if it is an ‘intelligent failure’[9]: this occurs in novel territory, provides actionable insights, and is modest in scale and can be distinguished from ‘preventable failure’, which results from negligence or inattention. Only intelligent failures should be rewarded. Preventable failures should trigger process review and governance overview.

This shows where the highest risk lingers: between the unspoken words, the unfought fights and the unresolved definitions about boundaries (like in any human relationship). The more complex a system is, the more complex is its governance.

For the governance of process automation though it is crucially important that governance is layered and each layer adjusted to the complexity and risk of the task at hand. Hence, the EU AI Act requires effective human oversight, not continuous human oversight. This gap can be strategically optimized by a governance that drives process optimization and hence, supports the business goal.

The question is not whether you will have AI agents. The question is whether you will have the governance to steer them. The frameworks are here. The technology is here. The competitive advantage will belong to those who master the interplay between human and machine, profit and safety, speed and resilience. The time to build your governance is now.