Architecture theory and the hourglass model
Everyone is talking about AI agent architectures, frameworks, and protocols at the moment. Let me apply John Doyle’s architecture theory1 lens to this topic, as well as Micah Beck’s hourglass model (2019).
In this first post in this series, I present the key ideas from the architecture theory and the hourglass model. In the next post, I will apply these ideas in the domain AI agents.
The distinction between system levels and layers
The two core concepts in John Doyle’s architecture theory are levels and layers. They are distinct, and it’s important not to confuse them.
Levels
Levels are the levels of modelling (coarse-graining, abstraction, renormalisation, weak emergence, interpretation) of a system. At different levels of modelling of the same system, the whole ontology, and the types of variables in the dynamical model of the system changes.
Theories, ontics, ontologies, and spatiotemporal scales are among loose synonyms for “levels” in this sense, or 1-1 associated concepts, such as, it could be said that a certain scientific or normative theory with a specific ontic/ontology describes and defines a specific system level.
Marr’s “levels of analysis” of a cognitive system (implementation, algorithm, and semantics) are the example of “Doyle’s levels”2. You can describe the human brain/mind, as a physical object, on many levels: molecular dynamics, neuronal dynamics, brain network, circuit, and processing stream dynamics, Hawkins’ reference frame (i.e., inference states) dynamics, reasoning dynamics such as internal monologue aka. chain-of-thought, psychological and developmental dynamics, etc.
In a coherent description stack of a system (such as a human brain/mind), the theories behind all the levels should be weaved together into an abstraction—grounding graph. Beware, this could be a source of confusion: the word “level” may imply that there should be a total order between the levels of analysis of a system. However, there is probably no order, i.e. asymmetric emergence relation, between the levels of brain network dynamics and psychological dynamics.
Within the context of architecture theory, it seems useless to distinguish “mere renormalisations” (such as a transition from neuronal to brain network dynamics) from model-theoretic interpretations that give rise the levels across the semantic hierarchy, such as Marr’s implementation → algorithm → semantics transitions. So, below in this post I will ignore these distinctions.
Layers
In Doyle’s terminology, layers are nested and/or compartmentalised (sub)systems (components, parts), i.e., groups of variables (atoms, elements, objects), separated by Markov blankets (boundaries).
Beware of the terminological confusion: in the fields of network systems engineering (and, more narrowly, protocol engineering) where Micah Beck’s hourglass model and related to it end-to-end principle come from, the term layer is used to refer to [Doyle’s] level. Hence, the internet protocol layers in the OSI stack would be called levels by Doyle. To minimise confusion, I’ll avoid using the term layers and will use other synonyms: subsystems, components, parts, compartments instead.
The phrase “system level” may also be confusing because this may hint at Doyle’s layer when the system boundaries fully nest within each other, such as the egg yolk within the whole egg within the egg with shell. So, I will call levels simply “levels”, or modelling/abstraction/renormalisation levels.
The way the dynamic variables at a certain level are compartmentalised into subsystems (by drawing “imaginary” system boundaries/Markov blankets) is itself a subject of inference for the observer (aka. modeller, rational agent, scientist). Different ways to “slice up” the “whole modelling field” at the given level (aka “the whole system”) into (sub)systems could be more or less useful for specific practical goals that the observer has.
For example, when analysing the brain on the level of neuronal dynamics, neuroscientists may group neurons into subsystems in different ways, such as the cortical layers, cortical columns, or neuronal circuits, and then hypothesise the emergent properties of these subsystems and thus move to a higher level of analysis.
Diversity hourglass architecture
Diversity hourglass is an architecture (a class of systems) with a particular pattern on three successive modelling levels: some model diversity on the lowest level, very little model diversity on the middle level, the most diversity on the top level.
The middle, low-diversity level is called the spanning layer in network systems and protocol engineering literature. Below, I’ll simply call it the “middle level”.
Also, the “hourglass” metaphor refers to two different aspects of this architecture (see more on this in the next section): (1) the low diversity of the alternative models/theories on the middle level, and (2) the relative weakness, simplicity, and generality of these model(s) on the middle level. As network engineering literature focuses more the second notion, they don’t use the “diversity” modifier and call this architecture simply the hourglass architecture (design, model).
The prototypical example of diversity hourglass architecture is the modern computing ecosystem (PCs, phones, servers, etc.):
The hardware level’s objects are transistors, wires, electric signals, etc. This level is diverse: there are a lot of vendors, hardware specifications, and of course specific hardware products.
The operating system level vaguely starts from “computing platforms” (such as x86, ARM, or CUDA) with all their objects such as registers, cache levels, platform events, interruptions, and execution modes. They followed by the “OS-proper” objects such as processes, threads, devices, memory maps, descriptors, and mounted file systems. The OS level is not diverse (relatively speaking): there are relatively few computing platforms and operating systems.
The software level’s objects and variables include those that the OS have decided to expose to the user space (and thus are shared with the OS level), such as processes, threads, memory maps, files, and more. This is a source of confusion. Of course, software level also includes arbitrary objects, variables, and abstractions built on top of the OS-level objects and variables, such as elements of the execution models specific to programming languages (e.g., variables, data structures, types, channels, events, etc.), process containers, communication protocols, and more. Even higher, there are domain-level abstractions for specific industries, businesses or organisations, specific software projects within an organisation, specific scopes in project’s source code, etc. Informally, the “software level” refers to all these finer levels (starting from the programming language and up) lumped together. The software level is the most diverse of all three, even more diverse than the hardware level.
The findings of the architecture and the hourglass theories
John Doyle makes statements about the diversity hourglass that could be seen as the main conclusions or “theorems” of his architecture theory. Micah Beck’s calls the main results of his hourglass model (theory) The Hourglass Theorem and The Deployment Scalability Tradeoff. In this section, I summarise these findings.
The relatively low diversity in the middle level of diversity hourglass is exactly what enables diversity on both the lower and the higher levels, provided there are evolutionary processes driving the diversity up in both the lower and higher levels.
Low diversity in the middle level enables the evolutionary processes in the lower and higher levels. I don’t remember if I saw a formal argument for this from Doyle or anyone else, but you can think of the following example: a uniform set of laws and regulations (“laws and regulations” being a level here, “a uniform set” means zero diversity) enables more businesses to evolve, apply themselves and diversify in different product lines, geographies, customer demographics, etc.
However, the low diversity in the middle level is not by itself sufficient to enable diversification in the lower and higher levels. The specific model of the middle level matters.
First, middle level’s weakness (also called genericness by Beck), as well as low complexity/high simplicity of the model/abstraction/theory/ontology of the system on this level enable more diversity on the lower levels because such a weak/simple model is simpler to implement (support, enable) by the lower levels.
Note that weakness/genericness and simplicity are closely related concepts (I’m not even sure it makes much sense to distinguish between them), but the low diversity is a categorically different thing. In the context of protocol engineering, low diversity refers to the fact that there is a single, “spanning” protocol that all other systems and protocols implement in the lower levels and use in the higher levels. Whereas weakness/genericness and simplicity are possible properties of that spanning protocol itself.
Second, the abstractions and elements defined (entailed) by the middle level for the higher level may be more or less composable and recombinable, that will the determine the “evolutionary breeding potential” on the higher level.
Beck combines composability with some extra, very informal property of “broad reach” or “broad applicability” and call this combined property generality (nb. difference with genericness mentioned above). Composability determines in part (but perhaps not in full) the “broadness of applicability” through the “computational power” reached by the middle level’s model. The ultimate ceiling here is Turing-completeness. It is reached by many programmatic abstractions, but of course not many other levels in real life, such as law.
Another important property of the interface between the middle and the higher levels that Doyle emphasises is how prone it is for hacking or hijacking by viruses, parasites, and bad actors.
Diversity hourglass’s benefits: diversity-enabled sweet spots and scalability
Doyle proposes that the diversity of level models/theories and subsystem designs at the lower and higher levels in the diversity hourglass architecture (that is enabled by the low model/theory diversity at the middle level, and by weak/generic, composable, and general model design(s) at the middle level, as discussed above) enables combining heterogeneous subsystems (components, parts, layers) to achieve optimal properties for the whole system.
I will not justify the above statement here, please refer to Matni, Ames, and Doyle, 20243.
For example, evolution of human brains have combined fast, but inaccurate “System 1” inference with slow, but accurate “System 2” reasoning to achieve optimally adaptive cognitive performance for humans in their environment. “System 1” and “System 2” here are thought to be two distinct subsystems on a certain level of brain modelling; the diversity of designs and hence operational characteristics is thought to be enabled by the low diversity and genericness of the lower “substrate” levels, such as the levels of neuronal dynamics and neuronal circuits.
Doyle calls these system designs with heterogeneous subsystems/components where the whole system “takes the best from all its components” diversity-enabled sweet spot (DeSS) [designs].
Beck and other authors in the field network systems and protocol engineering focus more on sociotechnical aspects and economic benefits of the hourglass architecture, such as protocol scalability, which in this context means the potential for broad adoption and huge economic utility to be derived from the use of a single “spanning” protocol.
This argument intersects with the informal argument for why low-diversity middle level enables evolution in the higher levels to the fullest: wide interoperability creates a “huge market” which in turn makes experimentation and bets on the higher levels more attractive due to potentially higher returns on successful experiments.
Diversity hourglass’s risks
In the context of AI agents, I’m not sure the scalability benefit of the hourglass architecture is very relevant: the utility of AI agents is probably going to be big enough, and the cost of their development low enough, that a lot of experimentation will happen even without the promises of maximally broad adoption. In fact, this “adoption amplification” effect of the hourglass architecture can be considered a downside when applied to AI agents, considering the potential societal or institutional disruption due to too quick adoption, “slide to criticality”4 and hence the idea that Short timelines and slow, continuous takeoff as the safest path to AGI.5
Other risks of the diversity hourglass architecture are also directly connected to its benefits:
Low diversity of the middle level increases the “evolutionary breeding potential” not only for “good” systems, but also for viruses, parasites, and zombies.
What’s worse, the scale of disruption (the “blast radius”) that could be caused by the viruses is additionally exacerbated by the broad adoption of the given middle level or “spanning” protocol.
Several distinct approaches for addressing these risks have been proposed:
Complete verification (proving) that the models/theories across the entire abstraction/level DAG are not hackable. In the domain of AI (agents), this approach has been called a “Guaranteed Safe AI” agenda (Dalrymple et al., 2024)6.
Multi-level and multi-component (layered) immunity system for fighting viruses and parasites. Immunity systems themselves should leverage the benefits of the hourglass architecture, namely diversity-enabled sweet spot designs and scalability. Apart from the applications of system-level synthesis (SLS) framework that underpins Doyle’s diversity enabled sweet spots theory in control theory7 and game theory8 that are not very specific to immunity domain, perhaps the closest work that I can find that takes this systems immunity perspective is (Ciaunica et al., 2023)9. Yet, in application to AI agents, this approach is even less developed: the closest idea that gained some prominence in the AI space is the Swiss Cheese Model for risk mitigation.
The resilience and safety engineering perspective: see (Dekker and Woods, 2024)10 for a recent work specifically in application to highly automated systems such as AI agents. This is also sometimes called a complex systems perspective, including in Dan Hendrycks’ AI Safety textbook.
Conclusion
Steering towards the diversity hourglass architecture of AI agents11 doesn’t is not a automatically good because the diversity hourglass architecture entails both benefits (not all of them I’ve even covered in this post, will elaborate more on those in the following post) and risks.
A thoughtful hourglass architecture for AI agents should proactively mitigate the risks through the combination of
Using provably tamper-proof models at certain levels of abstraction,
Designing “diversity-enabled sweet spot” layered immunity systems alongside the core functionality within this multi-level architecture, and
Accounting for the ideas from resilience engineering such as slide to criticality [4], robust yet fragile, graceful extensibility12, and more.
“John Doyle’s architecture theory” is primarily conveyed through multiple John Doyle’s presentations (you can find them on YouTube) between 2019 and 2022. More materials could be found in Doyle’s public Dropbox folder. For the most unifying and comprehensive published work, see footnote 3.
Or, groups of levels in the abstraction—grounding DAG of levels/models/theories, where the grouping criteria should be that each group is a connected sub-DAG. Ontological distinction between “levels proper” and “groups of levels” looks hopeless to me at the moment, and is probably not that useful anyway, so I will mostly just call both levels and groups of levels simply “levels” below.
N. Matni, A. D. Ames and J. C. Doyle, "A Quantitative Framework for Layered Multirate Control: Toward a Theory of Control Architecture", in IEEE Control Systems Magazine, vol. 44, no. 3, pp. 52-94, June 2024, doi: 10.1109/MCS.2024.3382388.
D. Alderson, J. Allspaw and D. Woods, "Re-architecting tomorrow’s internet for “survivability” (a resilience engineering perspective)", in proceedings of NSF Workshop: Towards Re-architecting Today’s Internet for Survivability, 2023.
The opposite stance on offer here is Nathan Labenz’s “adoption accelerationist, hyperscaler pauser”.
Dalrymple, David davidad, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, et al. “Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems.” arXiv.org, 2024. https://arxiv.org/abs/2405.06624.
Deglurkar, Sampada, Haotian Shen, Anish Muthali, Marco Pavone, Dragos Margineantu, Peter Karkus, Boris Ivanovic, and Claire J Tomlin. “System-Level Analysis of Module Uncertainty Quantification in the Autonomy Pipeline.” arXiv.org, 2024. https://arxiv.org/abs/2410.12019.
Neto, Michela Mulas, and Francesco Corona. “SLS-BRD: A System-Level Approach to Seeking Generalised Feedback Nash Equilibria.” arXiv.org, 2024. https://arxiv.org/abs/2404.03809.
Ciaunica, Anna, Evgeniya V. Shmeleva, and Michael Levin. “The Brain Is Not Mental! Coupling Neuronal and Immune Cellular Processing in Human Organisms.” Frontiers in Integrative Neuroscience 17 (May 17, 2023). https://doi.org/10.3389/fnint.2023.1057622.
Sidney, and David D Woods. “Wrong, Strong, and Silent: What Happens When Automated Systems with High Autonomy and High Authority Misbehave?” Journal of Cognitive Engineering and Decision Making 18, no. 4 (April 23, 2024): 339–45. https://doi.org/10.1177/15553434241240849.
Saying “designing the diversity hourglass architecture” would not be correct here because this activity is not done by a single person or organisation. The common abstractions and levels that will be eventually most widely adopted depend on a myriad of theoretic proposals, technical innovations, marketing campaigns, and political efforts done by numerous actors. Cf. the architecture in the large concept in systems engineering.
Woods, David D. “The Theory of Graceful Extensibility: Basic Rules That Govern Adaptive Systems.” Environment Systems and Decisions 38, no. 4 (September 10, 2018): 433–57. https://doi.org/10.1007/s10669-018-9708-3.