Jack of All Trades: Designing Meta-Cognition for Agentic AI

AI excels in closed systems; the real world, though, is open. When real-world data is analyzed through high-speed correlation but without anchoring the outputs in context, data driven decision-making is at risk. Humanity eliminated the "Jack of all Trades" — the integrator — and is now replicating that fragmented, multi-expert culture into AI, emphasizing silos while trying to escape them.

A possible solution is to embed an integrator directly into the architecture: a meta-cognitive core that links specialized agents, questions their assumptions, reframes problems, and acknowledges the boundaries of its own box. Such an "AI Jack of all Trades" would not only improve the reliability of current systems but could open the door to superintelligence.

With the ability of meta-cognition comes a danger as well as a choice. This is the blueprint, the warning, and the invitation to build such systems with humility and awareness because: what can be done, will be done. How we do it — and who we become in the process — is still ours to shape.

 

1. Are experts “mostly harmless”?

According to an MIT report in 2025[1], a staggering 95% of AI pilots do not generate revenue updating numbers of around 50% (Gartner)[2] and 80% (RAND)[3] reported in 2024. Conclusion: AI is not improving on ROI across industries despite the theoretical development of artificial intelligence itself is proceeding fast and the hype goes on. Where is the gap?

The gap is between AI performance in closed systems (like math) versus AI performance in open systems (like the “real world”).

AI is a master of high-speed correlation and pattern recognition. In closed systems with a clear set of rules, AI already outperforms human capabilities. But companies reside in the “real world”, in the messy, chaotic, dynamic “real world”. For AI to be reliably predictive in an open environment, it needs to anchor the correlations in context. And it needs “common sense”.

The gap has its root in the misconception that AI performing stunningly well in closed systems, but is not designed to be reliable in open environments [Figure 1]!

Figure 1: Agentic AI in closed versus open environments

Why do we need AI to perform reliable in open systems – the domain of (originally) human competence?

Most companies are fragmented and organized in silos. Process optimization. Sounds familiar? The goal was to break workflows into clear, measurable tasks with defined boundaries. It's efficient and quantifiable. Success is measurable. Failure can be tracked down. All good?

No, because we fired the person whose job it was to connect those tasks: the “Jack of all Trades”. Now a customer gets handed from silo to silo while everyone repeats, "I'm just doing my job." Remember Asterix’ and Obelix’ A38 adventure? That’s process optimization ad ultimum, but without integration. Orchestration coordinates outputs. Integration changes the process landscape[4]. And that's exactly why most AI pilots fail: they lack integration.

The integrator, the “Jack of all Trades” needs to know enough about each silo to facilitate seamless integration. “Jack” needs to envision and drive the big picture while still being able to talk to experts as well as the business and grasp all their unique requirements for a productive solution. Now, we are betting on AI to solve the integration challenge.

Currently, human and artificial systems base their input on data analytics coming from silos. This is called “data driven decisions”. No creativity. No intuition. Even though “data driven decisions” are deterministic, trackable, compliant, they might not be as harmless as they appear... Some companies already are realizing that business decisions based on data alone are not working. But they also realize that it is difficult to move beyond the concept — not because the process can't be adapted, but because the culture needs to be changed.

“Culture eats strategy for breakfast” (Peter Ducker)

Our culture rewarded experts for decades. Thinking out of the box has mostly manifested in arts and hardly found its way into the corporate environment. To build an AI that is able to connect the dots, we must acknowledge the box, its limits and use meta-cognition to generate integrated, useful and safe solutions that are grounded in the context of the “real world”.

But where is meta-cognition located in the control plane of Agentic AI and what is its primary goal [Figure 2]?

Figure 2: Agentic AI - Layers of Control

 

2. Life, the Universe and Everything: Nature’s Answer

Nature has been working on the concept of integration for billions of years - and solved it through communication and collaboration. Nature cultivates highly specialized experts – but it connects the dots.

The journey from a single cell to a highly developed multi-cell organism started with communication between single-cell organisms. Chemical signaling allowed individual cells to sense their neighbors, synchronize their behavior, and act as a coordinated whole. It allowed for cooperation and is the very basis on which evolution has been building complex organisms. The human brain, for example, is a highly specialized group of expert systems – but: they are governed by a meta-reasoning function. Let’s call this function “Jack” for now. “Jack” doesn’t know everything but enough to cross-examine, weigh confidence, reframe (if indicated), and: hesitate to evaluate the consequence of different strategies within the given context.

Let this sink in.

The leap from isolated cells to the human brain didn’t come from better cells; it came from better communication and collaboration. The integrator was the innovation and allowed the evolution of highly specialized experts functioning within an organism [Figure 3].

Figure 3: Thinking in/out of the box: experts versus Jack of all Trades

Our AI experts are highly advanced and sophisticated, they are even – sometimes – orchestrated but they are rarely integrated into a system that is able to consider context, deal with uncertainty, communicate ambiguity, and anchor results in the context of reality. Therefore, integration is the key for the next step towards more reliable artificial intelligence and it might even be the decisive step towards superintelligence.

Now might be the time to link the design of agentic AI back to Biology, where systems are built on:

0) Diversity as the foundation

1) Communication

2) Cooperation

3) Meta-Cognition

In this order. Diversity is number zero because without it, the other three pillars have nothing to work with.

• Communication is the foundation of coordination, biological signaling and regulation. It also allows for discussing contrary perspectives and a thorough exploration of the solution space from different angles.
• Cooperation keeps the development of egos in check and acknowledges that a diverse team always outperforms a specialized one – within or across species[5].
• Meta-cognition questions assumptions and opens up the box but should be designed with the intrinsic feature to mitigate the corresponding risks.

Let’s check out the technical architecture of such a system [Figure 4].

Figure 4: How is Meta-Cognition implemented in Open Systems?

The architecture can be adjusted by adding several experts assessing the challenge from their specific point of view. They all report to the meta-cognition agent that takes into the account the context of the system. There might be discussions between the experts or the meta-cognition realizes that the task is way out of the training and associated with corresponding uncertainty or it might even decide that it doesn’t have enough information to assess the situation properly. Ideally, it is able to assess the situation and communicate the assessment to the responsible agent within the system [Figure 5].

Figure 5: Assessment in Open Systems

But not just the assessment needs an overhaul, also the planning function [Figure 6]. If context is taken into consideration, there is often more than one good solution. But which one is the best? Different plans allow simulation scenarios within the given or even different future contexts. There are a lot of dynamic variables and each decision changes the outcome. This is like playing Tetris – or being CEO of a company: a decision must be taken, often under time constraints and associated with more or less uncertainty: the consequences might only appear five moves or months down the road… The meta-cognition evaluates the different plans and decides for the one with the highest chance of success. Only time will tell if the decision taken within the open and dynamic environment turns out to be right – or wrong.

Figure 6: Planning in Open Systems

This requires a rethinking about the expected accuracy and realistic reliability of AI systems operating in open environments: their decision might turn out to be wrong but it still could have been the best possible at the time.

The execution of the decision, the action, is assumed for now to not require any meta-cognition. But the learn function, again, depends on the context and must be integrated into the meta-cognition [Figure 7].

 

Figure 7: Learning depends on context.

 

Example: Blueprint of a multi-agent system without meta-cognition

Figure 8: Example of a multi-agent system without meta-cognition.

Meta-cognition for such a system would add for example:

• The question of what assumptions each agent makes.
• Checks context (-> including memory).
• Checks specifically for missing data.
• Queries an artificial SME or specialized DB if required (cave: data format compatibility).
• Checks plans for contradictions with context (possibly with trackable simulations).

We need humble and risk-aware systems that are able to ask questions and willing to test their outputs for “real-world” compatibility. The technical frontier is catching up to realistically allow the implementation of meta-cognition functions. For further information, please see “Technical Concepts” below.

Humanity, AI’s data source, is not performing extraordinarily well in the areas of communication or cooperation… But the deliberate integration of communication, cooperation and meta-cognition into agentic AI might still result in intelligent systems that are highly reliable in open environments and eventually surpass human capabilities.  

 

Example: Metagent-P

The paper frames Minecraft as an open-world environment: dynamic, partially observable, and unbounded. The tasks below are within Minecraft [Table 1] but give an indication of the potential of AI acting in open systems.

Difficulty	Human Success Rate	Metagent-P Success Rate
Basic	100%	99.67%
Medium	100%	88.33%
Hard	86%	53.00%
Complex	16.98%	20.20%

Table 1: Comparison of success rate in the open system Minecraft between humans and AI

Meta-cognition is still a new concept – but it is one that can take artificial intelligence to the point where it surpasses human capabilities: because it is the better integrator.