Gaia Network: a practical, incremental pathway to Open Agency Architecture
Co-written with Rafael Kaufmann. This article has been originally published on LessWrong.
Introduction
The Open Agency Architecture (OAA) proposal by Davidad has been gaining traction as a conceptual architecture to allow humanity to benefit from the nearly boundless upsides of AI while rigorously limiting its risks. As quoted here, the tl;dr is to “utilise near-AGIs to build a detailed world simulation, train and formally verify within it that the AI adheres to coarse preferences and avoids catastrophic outcomes.”
We think this traction is well-deserved: the only "Safe AGI" plans that stand a chance of feasibility will operationalise outer alignment through negotiation and model-based decision-making and coordination1. We also agree with most of Davidad’s takes on other hard problems in AI Safety and how to tackle them with OAA.
However, if we are going to achieve the OAA goals, we must go beyond its current presentation – as idealised formal desiderata for the components of the system envisioned and a general shape of how they all fit together – into an actual detailed design and plan. This design and plan must be based on existing technologies, but even more critically, it must not rely on a technocratic deus ex machina: instead, it needs to follow from proven incentive systems and governance institutions that can bring together the vast masses of knowledge and adoption muscle required, and tie them together in a stable and resilient way.
Here we present a specific design and plan that fulfills these requirements: the Gaia Network, or simply Gaia. This design has been in the works since 2018, it follows directly from first principles of cybernetics and economics, and it has been developed in collaboration with leading experts in collective intelligence and active inference. However, more importantly, it is a practical design that leverages a proven software stack and proven economic mechanisms to solve the real-world problems of accelerating scientific sensemaking and connecting it to better business and policy decisions. It is designed for ease of adoption, taking a “plug-and-play” approach that integrates with modeling and decision-making frameworks that are already widely used in the real world. Finally, we know it works because we’ve built a prototype (described in the appendix) expressly to solve specific real-world problems and we’ve learned from what worked and what didn’t.
We followed the only recipe known by humanity to build global-scale systems: create a minimal, extensible and composable set of building blocks; make it easy and attractive for others to build using them; and ensure quality and volume through competition, incentives, and standards.
It’s what Nature uses for evolution. It’s what got us the Internet and the Web, and even before that, the globally distributed knowledge creation process that we call science. Rafael has argued extensively on these points, as have Robert Wright, George Dyson and others. Likewise, we think this recipe, not a massive top-down effort, is the only way we can achieve the grand objectives of OAA.
Furthermore, our approach does not tackle AI safety in isolation. Rather, it recognizes the interconnected nature of our shared world system and simultaneously supports corporate strategy and public governance. In doing so, it directly supports ongoing global efforts to redesign capitalism and governance – which are gaining traction with increasing recognition of the urgency associated with other aspects of the metacrisis, such as climate change and social cohesion. We think this general-purpose approach of "meeting users where they are" is the only pathway to achieve the wide and deep adoption required for OAA's success.
The goals of this piece are twofold:
To argue that, by following this recipe, we will converge on a system and world model that meet all the OAA requirements.
To invite researchers and developers to collaborate with us on the ongoing design, implementation and validation of this system.
Gaia in…
One sentence
An evolving repository and economy of causal models and real-world data, used by agents2 to be incrementally less wrong about the world and the consequences of their decisions.
Five bullet points
Open agency as a massive multiplayer inference game.
Adaptive, distributed learning and coordination driven by a knowledge economy.
Simultaneously solve for shared sensemaking and decision-making in collectives of arbitrary stakeholders (humans, AIs, and other synthetic entities such as corporations and governments…).
A shared metabolism/accounting system based on free energy minimisation (the ATP of agents).
Immediate real-world adoption and impact by directly plugging into and enhancing existing processes in science, public policy, and business.
Five paragraphs
The Gaia Network is like the Web, but instead of pages and links between them, we have (causal, probabilistic) models of specific aspects or subsystems of the world, and relationships of containment, abstraction and communication between the models. Stakeholders (humans, AIs, or entities) will use this model network to make inferences and predictions about the future, about parts of the world that they can’t observe, and especially about “what-if'' scenarios.
Whereas each Web page is “about” a specific topic (defined by the page’s title tag and potentially disambiguated in the page contents), each Gaia model is “about” a specific target, a system or a set of systems in the world (this “aboutness” is defined by a machine-readable semantic context, or just context3). The context specifies how the model is “wired” to the target: which input data streams or sensors are accessible/relevant to the model; which actuators are controlled by the model outputs; and how both of these are connected to the model’s internal variables (the “local ontology”). A Gaia agent is uniquely identified by its semantic context (using content-addressable identifiers) and must always include a model, even if that model is trivial. A context may be concrete (addressing a target we identify as a single system in the world, ex: “this square of land that today is used as a farm”, “that person”) or abstract (addressing a set of systems bound by some commonality, ex: “all farms”, “all people”), but this is just a fuzzy distinction to help conceptually (ex: when discussing hierarchies of models).
The Gaia Network is more like the Web as a whole than like a single knowledge base like Wikipedia. Unlike the latter, which is effectively a unitary knowledge base, knowledge in the Gaia Network is pluralistic: stakeholders can host Gaia agents containing their own models of the domains that they know best (for which they have data). Instead of forcing an agreement on a single “global” version of the truth like Wikipedia, the Gaia Network allows for a plurality of models for the same domain to coexist and compete with each other: agents choose which model makes the most accurate predictions for them, like competing websites.
The Gaia protocol is to Gaia as HTTP/S to the Web: it’s the set of operations that agents use to establish and interact within the network. These operations include publishing agents and their contents (contexts, models, plans and decisions), and querying the network for estimated outcomes of potential plans, model discovery, updating shared models, etc. While HTTP/S is completely content-agnostic, the Gaia protocol enforces just the bare minimum to help agents distinguish grounded from ungrounded models and data. Namely,
the requirement for contextuality, as discussed above;
Gaia agents can use any representation for models, as long as it’s compatible with (active) approximate Bayesian (or Infra-Bayesian4) inference, enabling agents to interpret models as Bayes-coherent claims about the world5.
Another aspect that distinguishes Gaia from the Web is that it comes with built-in support for accounting (or, in biological terms, metabolism) and economics6. As all models are Bayesian and refer to a specific context, every agent can calculate (negative) variational free energy (VFE) as a private accounting unit for knowledge about that context. Agents also calculate the related quantity called (negative) expected free energy (EFE) as a private accounting unit for the likelihood of its strategy to achieve its preferred future. These accounting principles (jointly known as the Free Energy Principle) allow agents to account for epistemic uncertainty (whether about model representations/ontologies, functional structure, and model parameters), aleatoric uncertainty, risk, realised loss – all using the same unit (free energy reduction). So agents can use free energy reduction as a shared unit of account to compare and share beliefs about observations, states of the world, models, and plans. This allows them to discover prices for knowledge, calculate incentives for delegated or coordinated actions, and so forth.7
To read the rest of the article, please proceed to the post on LessWrong.
Table of contents of the rest of the article:
Gaia Network in one diagram
This is also very aligned with Beren Millidge's vision of "Preference Aggregation as Bayesian Inference".
In the initial exposition of Gaia in the following sections, we distinguish between
- Gaia agents: computational (or cybernetic, if they are attached to actuators "in the real world", although the distinction between informational/computational actuation and "real world" actuation is very blurred) systems that attach into the Gaia Network via the particular set of standards (that we call Gaia Protocol); and
- stakeholders such as humans, other conscious beings (animals or AIs), and organisations, and collectives who can query the Gaia Network and use its data for arbitrary purposes in the economy and the society.
However, in this single-sentence description of Gaia, as well as later in the article, we intentionally blur the distinction between Gaia agents and stakeholders by using the general term agents. This is because, on the deeper inspection, there is actually no bright line between Gaia agents and stakeholders: Gaia agents could be stakeholders themselves. Consider how the Gaia Network (i.e., the Gaia agents in it) attends to the "real world" which includes itself, recursively (or self-reflectively, we should say). See the diagram in the section "Difference in the agentic structure".
This line of reasoning also leads to very important questions: how would the Gaia Network affect the future dynamics of individual vs. collective agency and consciousness? And would this impact be net good or net bad? We don't have ready answers to these questions, and we postpone their discussion to the complete paper that will be published soon. Cf. the related discussion in Beren Millidge's "BCIs and the ecosystem of modular minds".
This conforms with the formalisation of context in Fields, C., & Glazebrook, J. F. (2022). Information flow in context-dependent hierarchical Bayesian inference. Journal of Experimental & Theoretical Artificial Intelligence, 34(1), 111–142. https://doi.org/10.1080/0952813X.2020.1836034
We are open to Davidad’s arguments that Infra-Bayesianism can be critical for compositional reasoning about risk. However, we think it’s important to start building the Gaia Network with computational tools that are available today for approximate Bayesian inference and develop the computational infrastructure for Infra-Bayesian inference in parallel. When the latter is production-ready, Bayesian models could be gradually replaced with Infra-Bayesian ones in the Gaia Network. Note that if the agenda of building Infra-Bayesian infrastructure is stalled for some reason, it would still probably be better for robustness to civilisational risks to have “mere Bayesian” Gaia Network deployed in the world than not having any equivalent of such a knowledge and agent network.
Our reference implementation uses NumPyro, a Python-based probabilistic programming language, which we expect to become a de facto standard.
The accounting and economic systems are described here without any motivation and arguments for why this design should have attractive properties. This is deferred to the complete paper that will be published soon.
The step from free energy-based accounting to autonomous, free energy-based price discovery and trading requires additional work and resolution of important questions about fungibility. We have made progress on this but defer a fuller discussion to the forthcoming complete paper.