CHAPTER I.
DARKNESS

The future is dark, which is, on the whole, the best thing the future can be, I think.
– Virginia Woolf 1

Dark have been my dreams of late.
– @tszzl 2

Introduction

Consider a special kind of factory, an extreme kind of factory: a self-guided factory that continuously makes its own decisions about what to produce. It is a factory that determines its own goals and its own objectives, a factory that produces new versions of itself in order to execute against its ambitions. To the human perceptual apparatus, this factory feels only like scale and aporia; it commands both awe and a kind of sympathetic exhaustion or nausea in the face of inexorable machine ardor.

It feels like a senseless place. 3 It teems with contradictions. With one hand it decorates a new branch of work with baroque ornamentation, and with the other it chops off that very same branch. Its waste bins pulse with still-twitching, still-animated material. It can be said to learn from its mistakes, but the way it learns feels like the way a million different things might learn something together. Some of its parts learn and adapt immediately, or even faster than immediately. Other parts lazily drift into place, occasionally overshooting. Still others actively anti-learn: They spread conspiracies and contaminate their well-behaved neighbors. It is difficult to think of this factory as a single thing—but at the same time, it so clearly is.

The stakes of this factory feel both extremely serious and extremely difficult to reason through. This factory creates and destroys almost inestimable arrays of autonomous worker populations every second, shedding them like skin cells. It burns through raw materials with a rapaciousness that would feel insane to a human accountant but that is simply scheduled with brutal realism, according to a vivid and constantly revised online calculus. It brings new things into its world thousands of times every second. It makes decisions quadrillions of times every second. Its many millions of hyper-dimensional backroom-like factory floors have distended directly into its outsides, with entire armatures cut through walls, with portals perforated through ceilings, with entire lines directly installed out in the open, far away, anywhere, wherever best. This factory drives itself: It decides what to do next, where to go, what to be, what to make, how to make it, and how to make it better.

It might be helpful to think about this factory as a kind of fully automated firm. It sits within nothing more specific than an economy: It invests, it experiments, it proliferates, or it dies. One could also think of this kind of factory as a kind of economy—a febrile hum of abstract and even arbitrary interconnections—or as a kind of city, a continuous process of zoning and rezoning, chartering and amendment, demography and wild libidinal drift. Or, in more philosophical terms, think about this factory as the automation of production at the level of what is worth producing.

The factory we are describing is nowhere close to fiction. The nearest instance of this factory is being assembled right now, in software, where coding agents rapidly design, construct, review, and deliver products with less and less human attention per decision every day. Now, of course, most of the many people building large, powerful “software factories” with agentic AI are simply reproducing preexisting dynamics (an assembly line, a waterfall diagram, a workflow) with more intensity and speed. 4 This new medium has the capacity for more, however—after all, an agent can produce the types of decisions and ingest the kind of feedback necessary to assemble massive, productive, self-optimizing structures. But while AI feels like exactly the right medium through which to enact such a step change in the history of automatic production, the pursuit of these ends involves real risk, real resources, real technical limitations, and real liability in the face of greenfield-only returns.

Let us identify an architect willing to face those risks and attempt to build this factory—a kind of hero or antihero we will engage with throughout. 5 Our architect has recognized something important: The best factory, the most productive factory, the ultimate factory belongs to a set of possible factories whose internal assemblies cannot be specified, articulated, or described by humans. Humans, after all, have technical limitations when it comes to the imperatives of factory productivity: We can only move so fast, push with so much force, and think with so much dimensionality. This architect has also recognized that the thinking machines at our disposal can now overcome all of these limits, not only the limits of speed and force but also the limits of logistical complexity involved in organizing the means of factory production. These recognitions set forth a challenge to the field: It is time to hand the work of architecture wholesale to the machines.

Our architect wants to do more than simply automate architecture or factory design, however—they want to automate the continuous production of architectural decisions well into the future. They want to build a factory that will do architecture to itself, that will evolve and shape-shift and branch into whatever that factory decides it needs to be, forever. If our architect was interested only in the automation of architecture as such, in the construction of a kind of button that, given certain specifications, did architecture—then our architect is not doing something altogether new. No, in that case, they would be still sitting in the architect’s chair, still laying claim to a faculty of discernment whereby this or that architecture is promoted or deleted. The more interesting road, the one our architect elects to take, is the one in which it is exactly this chair, this faculty, that is surrendered to the machine.

It follows that the objective of this architect is to build something perennially in excess of what an architect knows how to build, which means that this architect is presented with the unusual task of trying to build something that continuously exceeds their architectural capability. This is not a case of building something that is radically new, like Albert Khan’s River Rouge facility, something that required not only the creation of a new kind of system but also the sui generis manifestation of new armaments of componentry. This is so much more than that—our architect is trying to create a system from the set of systems that they themselves cannot actually imagine, and they want to continuously operationalize the creation of such systems. Given this objective, the very moment this factory is turned on it becomes immediately bewildering to its architect, for the precise reason that the factory is producing new versions of itself along trajectories that make no apparent or immediate sense. If this condition of bewilderment does not occur, if the architect is immediately comfortable with the appearance of their factory, then the architect has absolutely failed. After all, the architect has taken on all this risk, all this expenditure, and all of this disciplinary humiliation to simply produce something they could have built with fractions of expense and danger. Were the factory not bewildering to its architect, the factory would simply be the logical resolution to the game that the architect defined. No, the factory must be a bewilderingly complex alterity, continuously so, in order to be worth it.

We refer to this bewilderment as darkness—even superdarkness—which we will define with more rigor later. For now, think of darkness as the epistemic condition in which descriptive information about a thing loses its practical utility. Think of superdarkness, then, as a kind of absolute case: a black box. This is not to suggest that this factory is hidden or opaque or encrypted or duplicitous, but simply that any description of this factory stops being useful before anyone can act on it. Now, some of this condition is just a problem of velocity—by the time an observer has worked out what some assembly or department of the factory actually does, the factory has rebuilt it several times over. But superdarkness is deeper than speed: It has more to do with a deeply unstable praxeological ground. Think about a game of chess in which an opponent, between moves, quietly renegotiates what counts as winning. In this case, the clever reader of an opponent’s strategy has lost their core interpretive mechanic: They can no longer read that player’s positions as a series of statistical outcomes. No, now a proper game emerges, one between combatants identifying the types of games being played. Each player no longer looks at a fixed opponent’s array of pieces but instead at the material outcomes guaranteed or forfeited across the ambient dynamics of a world, a marketplace, or an ecology.

Our architect is interested in exactly this kind of proper game, because their goals have everything to do with facilitating the unexpected. One could suggest that the more interesting this game is, the better—but this is not necessarily the case. Our architect, after all, is doing this for a reason that is more substantial than idle interest. This architect may be motivated by a social project, a utopian desire to bring closure to the question of labor or toil (why this factory would go about making things useful to humans would remain an open question, of course). Perhaps more naturally, this architect could be drawn to this task by the incentives of commerce, either taking an offensive position (building the most efficient extraction vehicle in history) or a defensive one (sensing that such a factory is inevitable and therefore engaging in an arms race to be the first). While we, the authors, are ambivalent about the motivations of the architect, 6 this last, defensive tactic is worth unpacking further, and we will do so later in this piece. Regardless, our architect is trying to make this factory do something, and while they do not know what that something is, they do want that something to be nevertheless aligned to their interests on a continuous basis.

6. We mean this sincerely, not in the sense of shirking responsibility but instead precisely to account for how and where responsibility accumulates. We are not so glib as to suggest that this type of thing is simply a pregiven, natural outcome of the world, something that simply emerges of its own accord. At the same time, we are not naive enough to suggest that the world can be decomposed into chains of human decisions, that such a factory cannot be induced into existence through the collaboration of a spectrum of agencies. That latter tendency strikes us as worse, not only in the sense of idle anthropocentrism (only humans can bring about great or evil things) but moreover in the dramatic underestimation of the world (the world is not itself abundant with great or evil potentials). In this sense, we supply a how (a how to think and a how to build) that can be used to satisfy a strong plurality of motivations. It is better to think about how something could happen than about why something should not happen, as the former produces an armament that can be deployed against the latter and the latter is on its own already a saturated, diffuse, and ultimately facile mode of critique.

Their goal, then, is to catalyze something, to set something in motion, to provoke something to condense around some undefined and unseen attractor—and, at the same time, to induce something that overwhelmingly oversatisfies the reason that this factory was brought about to begin with. The questions of “How do you build something greater than your own capacity for vision?” and “How do you steer something you do not understand?” start to merge into the same question, and this is the question of what we call the dark stack.

The dark stack is a series of techniques or tools with which to build the extreme kind of factory we are describing, 7 which we call the superdark factory. 8 We have chosen not to define the superdark factory in terms of this or that specific medium (e.g., self-assembling molecular tiles, biological cells, software or computation as such) but instead hold it to be a kind of abstract machine, a way of thinking about extremely complex opponents in continuous games. In principle, we maintain a kind of material agnosticism here, because to hold this type of factory to this or that mode of material expression would be to compromise exactly the type of autonomy we have been describing. In practice, however, we simply do not know the answer. For now, software is the medium where the conditions to realize a superdark factory coincide. For this reason, we are content to deliver the dark stack in the context of software, though we recognize that this could be a temporary or arbitrary condition. Everything we describe within the dark stack can be derived outside of the context of software, however, as a series of abstract structural or cybernetic principles.

8. The term superdark factory is also borrowed from the dark factory of industry, where it is often used to describe either a “lights-out” or “black-lights” (黑灯工厂), robot-driven hardware factory or an agentic software factory (agent development life cycle; ADLC). We are taking a much more extreme position on what a superdark factory actually looks like, committing to the total absence of the human from the factory’s internal loop as a point of departure. For example, from Blain (2024): “There are no humans working the new Xiaomi production lines—this new Smart Factory is 100% automated. Indeed, the company says the system is smart enough to diagnose and fix problems, as well as optimizing its own processes to ‘evolve by itself.’” We will later call this a Class 2 factory, reserving our usage of the term superdark factory for the more advanced and extreme Class 3 factory.

Working with this dark stack means taking on some serious humility (you are building something greater than yourself) and sacrificing some serious priors (you do not belong inside the superdark factory, you are not smart or fast enough to act inside the superdark factory). At the same time, it recaptures something like agency or necessity from a cohort that is desperate to go down with their ships and refuses to release a little agency to save a lot. No, there is no human in the loop in this factory—and our architect who abandons their chair understands that the factory is better for it! There is, however, a displaced but real relationship and even an opportunity for a certain kind of advantage. The architect makes the first move, and in their first move they create an invitation to a game. In any game, there is an opportunity for strategy. More importantly, this first move is the architect’s only move inside the factory; the architect has both surrendered their chair and their ability to understand their opponent. Therefore, this very important first and only move needs to be deeply considered, as it needs to balance these two contradictory forces: the architect’s desire to steer the superdark factory in a direction and, at the same time, the factory’s permission to choose its own directions and in so doing stumble onto something genuinely new. This first move is where the architect deploys the dark stack.

After making their first move, the architect retreats from the factory site, transforming from a designer into a game player. What remains for the architect, then, is a strange and narrowed kind of agency: a single committed move before the factory is turned on, and afterward a reserved seat at the table of games. The type of agency available to the architect has transformed: The architect has traded a kind of available comprehension (understanding, interpretability) for whatever sustained leverage they can inject into that first move. They are no longer the operator of a machine they understand but a strategic player engaged with a productive singularity that is faster, stranger, and better than they are (at the thing it was set loose to do). But this singularity accretes, likely quickly, and invites an age of generally superdark production. The first move becomes even more important: It becomes a general first move, the first move, our first move.

What follows builds, step by step, the vocabulary to say what such a factory is and the means with which to make one. Along the way, we spend time with the implications of building such a thing, with the particular type of singularity that it invites, and with the question of agency under the regime of superdark production.

I. On Factories and Darkness

For our purposes, a factory in the general sense is a machine that automates production. By machine, we mean something somewhat close to Félix Guattari’s usage of the term. 9 Guattari defined the term machine against the term mechanism, which is a simple thing that can be understood through its internal structure. A mechanism has parts in fixed relations, and those parts perform defined, determinative functions. Think of a clock, a pendulum, a transistor radio, or a camera. For Guattari, a mechanism is predictable: It does the same thing every time. A mechanism has (in Gilbert Simondon’s terms) no “margin of indetermination,” no excess of possibility; it does exactly and only what it is supposed to do. 10 A mechanism can be defined in isolation from the world, like a circuit or a calculator: It simply is what it is.

10. For example, “In order to make a machine automatic, one must sacrifice a number of possibilities of operation as well as numerous possible usages. . . . The true progressive perfecting of machines, whereby we could say a machine’s degree of technicity is raised, corresponds not to an increase of automatism, but on the contrary to the fact that the operation of a machine harbors a certain margin of indeterminacy. It is this margin that allows the machine to be sensitive to outside information. Much more than any increase in automatism, it is this sensitivity to information on the part of machines that makes a technical ensemble possible. A purely automatic machine completely closed in on itself in a predetermined way of operating would only be capable of yielding perfunctory results” (Simondon [1958] 2017, 17).

But a machine, on the other hand, is something open. It is defined by its relationships and not by its components. A machine is a thing that necessarily relates to other things and can be said to be a machine only while it is in the process of connecting other things together; a machine is the process of connecting other things together. A machine can be made of other machines while at the same time remaining a machine.

For us, a factory is a machine, as it is defined by its relations—by the flows of intent it draws in, the flows of product it pushes out, and the world it reaches into and that reaches back. It is not defined by any specific enclosure, workforce, or assembly line. A factory is open: It changes and it develops, and in the process it becomes better understood in terms of its inputs and outputs than in terms of any particular physical configuration. It is more than a physical configuration, sometimes “explosively” (von Neumann 1966) more in terms of its potential to not only transform its world but also itself. 11

A factory is a machine that automates production. Production here refers to transformation. It can be the transformation of raw materials into goods, software specifications into public-facing services, or anything into anything else. The product of production is more or less irrelevant: We could join Guattari with Gilles Deleuze here and consider a product to be just one simple, frozen state of a potentially endless trajectory of productive transformation. While we are not interested in products or even production per se, we are very interested in what a factory does to production: automation.

Automation, for us, is nothing more than a factory’s production of its own input. One can think of it as the passage from the external to the internal, or even a kind of programmatic redrawing of a factory’s interior–exterior boundary. When a task becomes automated, when it moves “inside” the factory, it moves from being supplied to the operation to being performed by the operation. Consider a classical example from the history of automation: the power loom. Prior to the power loom, in order to weave a length of cloth, a weaver had to perform a series of physical maneuvers upon a loom. The power loom drew that labor inside the loom, redrawing the boundary of the loom as the object that contained the full operation of weaving. The input to the loom shrinks: It no longer requires hands and legs as inputs but only something like a pattern.

To reaffirm the definition: A factory is a thing best understood in terms of inputs and outputs, something that contributes to a lineage of progressively drawing its inputs into itself. A factory is a machine whose characteristic operation includes the revision of its boundary with its input; it is a machine that not only transforms inputs into outputs but in fact removes inputs from its world. A special kind of factory that automates the automation process, and in so doing automates the production of itself, is the logical conclusion to this lineage. But each successive step of that lineage is worth attending to before arriving at this extreme limit case of the concept of factory.

We can break up the lineage of factories into classes, each marked by a characteristic level of automation. Each level of automation can be thought of as a spectrum of intensity that suddenly changes register at the moment of transition. As a given factory ascends to a higher class, it takes the standard against which it was once measured and automates it.

The simplest kind of factory, Class 1, automates executions: It takes a portion of execution-level inputs and absorbs them into itself. Take, for example, a computer numerical control (CNC) machine that programmatically mills printed circuit board (PCB) circuitry according to some instructions. The instructions are provided, but the execution of those instructions has been assumed into the operation of milling PCBs. Another example here could be a C compiler, which takes the action of translating higher-level programming instructions into lower-level machine code and represents that action within a single, operational relationship. In all cases, Class 1 automation requires plans (e.g., blueprints, schematics, directions) as inputs, which specifically describe the order of operations to be executed.

A plan is more than just a sequence of actions; after all, a CNC machine, a C compiler, or even a power loom are each able to absorb some level of sequencing work into themselves. A plan, however, is a selection made among a series of possible sequences in the specific case where real alternatives exist. In plainer language, a washing machine’s fill-wash-spin cycle is a sequence: It is able to automate a whole series of actions next to each other. But that sequence is not a decision among alternatives; it is a prebaked series of actions. In this case, a human operator with dirty laundry supplies that washing machine with that decision, and the decision is made across a set of real available and swappable alternatives (normal cycle? heavy? heavy but low-temp?). Even a sensor-equipped washing machine, one which programmatically selects a cycle based on a turbidity reading on the inside of the washbasin, is still simply executing a more complex plan, one which involves a series of presupplied decisions based on sensed input (if turbidity reading > x, do y).

A Class 2 factory involves orchestration-level automation: the automation of plan-making. A Class 2 factory takes objectives (goals, limitations) as inputs before producing and executing upon those plans. Take a coding agent like Claude Code, for example. Prior to AI-driven code development, the work of a software engineer primarily involved plan development. Engineers would take product requirements (goals, key performance indicators [KPIs]) handed down from management and transform those requirements into written software, making decisions among alternatives regarding things such as architecture, design, or dependencies. A coding agent takes plan-making inside itself; a coding agent is given objectives and returns a series of decisions made among alternatives without necessarily disclosing what those alternatives are.

Another classic example is Nick Bostrom’s paper clip maximizer, which is often presented as an example of superintelligence (Bostrom 2014). The paper clip maximizer is a machine given a single objective: Maximize the number of paper clips. Everything else, from acquiring resources to redesigning itself to preventing its own shutdown, follows as planning in service of that fixed objective. It is unimaginably capable at producing plans, yet the criterion by which those plans are judged never changes: “more paper clips.” Therefore, in our terms it remains a Class 2 factory. Applied research in artificial intelligence has spent decades worrying about plan-making machines that fixate on fixed goals. We, however, are interested in the next problem.

So enters Class 3 automation: the automation of objectives, where a factory produces its own goals against the merely given conditions of its world. An objective is nothing more or less than the standard by which this or that plan is identified as better. The superdark factory is a Class 3 factory, and the one we are describing here is the first Class 3 factory. 12 The superdark factory participates in the production of its own objectives, which are negotiated principally with its world. For our purposes, the superdark factory’s world is the persistent environment in which it is cast (set forth). The factory’s world is an ambiguous field of potentials through which the factory’s objectives can be derived. Class 3 automation takes as its input whatever raw material one could use to derive objectives. This raw material is sourced from within two awkward categories: norms (vectors of value) and constraints (vectors of limitation). An objective is driven by instrumental logic: It evaluates an action based on its ability to serve a predetermined end. A norm or a constraint is noninstrumental: There is no predetermined end, though one can distill a norm or constraint into an end.

To function as input into a machine, a norm or constraint must be quantized into metrics—measurable proxies or representations. A metric (to us) is not in and of itself an objective; it is simply a quantization of normative space. It becomes an objective only when it is prioritized as such. Of course, a metric will never perfectly reflect its norm or constraint; there will always be some degree of tension between a representation and the thing it represents. We can alternatively call this underdetermination or overdetermination: the removal or addition of information that occurs as a referent jumps between one encoding scheme and another. In a Class 1 and a Class 2 factory, the resolution of this tension is given with the input—we will discuss this later in the context of overfitting. But in a Class 3 factory, this tension is actively negotiated: The quantization of a norm or a constraint is an active subject of the factory’s transformative work.

A good case study for discerning norms, constraints, and metrics against objectives is the famous von Neumann probe, a thought experiment at the fringes of thought on machine self-assembly, especially as it relates to the search for extraterrestrial life. The von Neumann probe is a theoretical machine (a spacecraft) that carries a complete description of itself and, given raw material and energy (e.g., wandering asteroid regoliths), can build functioning copies that can then do the same. 13 Let us suppose that we build one and launch it with a single directive: survive and replicate. That directive is a norm. It tells the probe what is worth doing, but it actually specifies very little: It names a value and a direction, but not a planned course of action. It does not prescribe which asteroid to mine or how many copies to attempt or when to give up on a dead system and move on. The probe is also situated within a series of constraints: the fuel left in its tank, the composition of a given rock, the distance to the next star, etc. Just like a norm, a constraint is not a decision: Knowing one’s values or knowing one’s limits is not the same as knowing what to do within them. Both those norms and those constraints are quantized and compressed into metrics, through which the questions of survival or reproductivity are interpolated, not without tension, into things like counts of copies made, time of life, amount of fuel, and asteroid spectra. By the probe’s third star system, when its designers have been dead for centuries and are light-years out of range, something on board has to decide what to do. So, the probe decides: It assesses the norm of “survive and replicate” against the local rock and the remaining fuel and produces an objective for itself (we will make two copies of ourself in this system) and a plan (let us mine this regolith, toss out this extra material, and bail). This is certainly a case where a machine has produced its own objectives.

But even this speculative machine is not a proper Class 3 factory. And, surprisingly, this is not because it fails to produce its own objectives (it does produce them) but because it does not participate in its world of norms. The spacecraft can fail the norm (it can die and drift in abject isolation until the heat death of the universe), but it does not weigh that norm against another value. That norm rides aboard the von Neumann probe as a series of metrics, but those metrics are never renegotiated. Any daughter probes produced by the first inherit this normative prefiguration, and any probe that challenges the directive can be considered broken. We could concentrate on an important genealogy, from Immanuel Kant to Wilfrid Sellars to Ruth Millikan, and while they would surely disagree on the terminology of norm (Kant might say that survival is strictly not a norm for the probe, 14 Sellars [1969] might say that it is a genuine norm of the ought-to-be-kind but on loan from concept users who legislate it, and Millikan [1984] might say that it absolutely is a norm, albeit a fixed and nonnegotiable one), they might nonetheless agree that those objects that can participate in the register of determining what may or may not be important, valuable, or virtuous represent a serious shift from mere instrumental logic to a normative life of one’s own. This ascension to the next plateau of being is exactly the type of shift we consider to be characteristic of the Class 3 factory. The von Neumann probe is then a perfect example of the ceiling for Class 2. It sits at a point of maximum tension before something really fresh appears.

A clever reader might ask: But what about a theoretical Class 4 factory, something which has automated the production of the world? This remains undefined.

Now, a reader accustomed to thinking of a factory as a mechanism, as a series of discrete components, might ask the question: Well, what about a factory that involves a mixture of Class 2 components, Class 1 components, and even some absolutely manual activities? What about an automotive factory that involves some advanced computer-assisted manufacturing (CAM) at some parts of the line, but whose total layout of the line is supplied in advance? What about a software factory that involves some handwritten, discrete services and some services created and managed by autonomous AI agents? What about a sweatshop filled with human-operated sewing machines that execute designs selected by an automated bandit-style experiment, a process that is automatically maximizing against checkout rate?

This is where Guattari’s machine enables more expansive thinking about factory design. A factory’s class is defined by the type of input it receives. Let us take the case of the automotive factory with an advanced CAM machine, and let us say that this CAM machine is capable of producing plans. For example, if you give it a computer-aided design (CAD) file, it can produce its own machining strategy (what operations to invoke, what tools to use, in what order, etc.). Now, if every possible input–output scenario of that machine is accounted for in the factory’s plan, if the CAM machine can be redrawn within the factory’s blueprint as a simple state machine with a series of pregiven inputs and outputs, then the factory as a whole is a Class 1 factory. But if you zoom all the way in to just that CAM machine and treat it as a stand-alone object, for example, to which a human feeds a variety of different designs—then, in this context, the CAM machine can be considered a Class 2 factory.

The sweatshop is an even better example: A human worker is of course capable of producing alternative plans, but if they are to strictly execute a given plan, it is a Class 1 factory. If those plans are developed by an automated AI system, like a bandit-style experiment, then the factory becomes a Class 2 factory. Does the simple addition of an AI system move the factory into Class 2? No! A Class 2 factory just redefines the input–output boundary, the stipulation of the inside and the outside that comes with identifying something as a machine. Imagine drawing a line around a series of relationships in the world. Where you draw the line is important, because it identifies you in contradistinction to the thing you have outlined. Should you draw a line around a sweatshop to include its designers, you have identified a Class 2 factory. Should you draw a line around a software microservice that uses LLMs but entirely confines them to structured outputs within a linear, discrete workflow, then you have identified a Class 1 factory. Should you draw a line around a coding harness you will supply with objectives, you have identified both a Class 2 factory and yourself as a supplier of input.

Automation can be understood in terms of how lines are drawn—but at the same time we cannot argue that anything can be this or that factory. No, there are certain things that cannot be a Class 2 factory. A basic bought-on-Amazon microwave cannot really be a Class 2 factory: It cannot produce plans on its own. So while the technicity of this or that object—whether or not it uses a computer, whether or not it uses AI, whether or not it involves a human—has no bearing in itself on the class of a factory, said technicity enables the possibility of certain classes of factory.

It is worth disentangling the way we talk about automation from the concept of optimization. Optimization is a relatively useless word in this context: Unlike automation, optimization really only signifies an escalation of productive capacity in intensity but not in kind. Optimization means “this thing got better,” which relies on a “better” that is pregiven.

We are, however, very interested in self-optimization, which involves a factory’s revision of its own operation through a feedback loop with its world. A feedback loop is the simplest and strangest case of Guattari’s rule that every machine is connected to other machines, where, in this case, that other machine is itself. Unlike Guattari’s mechanism (a closed array of components), a machine is open, and its openness is precisely an openness to revision. Simondon ([1958] 2017), as already discussed, referred to this openness as a machine’s “margin of indetermination.” A feedback loop is a channel through which a machine receives itself back as information, which it then uses to drift, move, or evolve. The fact that a machine can change without necessarily becoming something new is exactly what separates it from a mechanism.

As a factory is a kind of machine, it is sensitive to feedback in exactly the way a machine would be: It is capable of some degree of self-optimization. But a factory cannot revise a thing that it does not produce—a plan-making factory cannot revise an objective based on feedback, and an execution-doing factory cannot revise a plan—so the automation class that a factory belongs to sets the ceiling of its potential self-optimization.

The type of self-optimization available to a Class 1 factory is simple calibration, in which a given execution is revised against a fixed plan and objective. Imagine something like a digital camera’s autofocus algorithm—the execution (where to set the lens) is revised against a fixed plan (focus, then shoot), which extends from a fixed objective (a sharp image of a subject). Or, to bring us back to a proper factory floor, imagine a milling machine that is measuring the thickness of the material entering its body, reacting to it by adjusting its aperture and then continuing to adjust its aperture as the material swells from thermal expansion. For both examples, there is a case of a sensor-actuator coupling that opens up the range of what a camera or a milling machine can do. And, more to the point, both the camera and the milling machine have removed and absorbed some of their inputs as machinery—the camera no longer needs to be supplied with instructions on where to set the lens, and the aperture of the milling machine no longer needs to be manually set as its materials change. We call this input from the world an auxiliary supply of input—it is a kind of interface through which something not directly supplied at the level of the plan is allowed to supplement the plan as an input to the machine. The input to the lens (e.g., the specification of the camera and the direction in which to point it) is supplemented with input from the world (various agents in the field of view, lighting conditions) at the moment of execution. This is, of course, simply an intensification of Class 1 automation without a change in kind. But it is the “self-” part of this optimization that is interesting here, because it represents an entanglement between the machine and the auxiliary supply of the world in the form of a feedback loop. Unlike the case of the power loom, for example, where the loom simply exercises a series of commands and receives no information about its work, the autofocus lens is actively engaged in a continuous feedback loop between the actions it executes and some measurement of their success. 15

As before, it is at the level of the Class 2 factory where the self-optimizing feedback loop can be applied to the plan. Here is where things really start to feel like self-optimization. As previously described, what makes a plan a plan is the possibility of alternatives. Self-optimization in execution mode does not work with alternatives; it simply follows a gradient until it reaches the end (autofocus tuning its lens until a sensor says that the subject is in focus). But a plan involves making a selection among alternatives, which can be understood as making a decision in exactly the case where the gradient, the axis of tuning something, is not actually known. Instead, the feedback loop here begins to look like exploration, where alternatives are continuously tested against each other in the world.

This involves risk: A Class 2 feedback loop sometimes has to be allowed to do worse in order to do better. After all, it needs to decide among a series of alternatives whose outcomes are not known in advance—otherwise, why would it not simply automatically defer to the known best alternative, and in doing so collapse directly into route execution? The only reason to enable any kind of Class 2 automation to begin with would be to defer the task of evaluating among alternatives to the advantages of machine sensibility or machine time. If this is known in advance not to be a good or efficient decision (in economic terms or in risk-assessment terms), then a study of these alternatives could be commissioned beforehand, and the best-fit decision could be hard-coded into a supplied, pregiven plan.

Consider, for example, an e-commerce software platform that is in the process of releasing a new user interface. Instead of simply releasing one version of this new feature, the platform may test a variety of alternatives—maybe with different graphics, different text, different modals or action buttons—by releasing each of those alternatives to a segment of their user base. This platform might then collect metrics based on results, looking at which version of this interface was more successful in terms of how many customers engaged with it, how much time they spent with it, and how many actually bought something. But simple experimentation like this is not Class 2 self-optimization in its own right. To be Class 2, it has to actually collect these metrics in a continuous feedback loop that produces new interfaces based on results. Progressive or automatic experimentation makes this a possibility, wherein one might create an array of alternatives, release a subset of those alternatives, and then gradually release new elements from that set of alternatives (or, perhaps, altogether new, AI-generated alternatives 16) based on live customer feedback. In each case, the ultimate objective is pregiven—the automatic experimentation engine does not decide which objectives are important but self-optimizes against preestablished objectives from real-time feedback.

Class 3 self-optimization is far more extreme. This is the case when the factory revises the objective itself (the standard by which it judges “better”) according to the given conditions of its world. To return to the e-commerce example, a Class 3 factory is not only capable of making decisions among alternatives but can also subject the calculus through which it makes decisions to continuous analysis. This kind of Class 3 e-commerce factory might shift its attention from a metric such as shopping-cart conversion to net revenue retention based on its own business analysis. And, more likely, it might alter its pricing strategy as a result, spontaneously spinning up a new logistics arm, deciding that it should no longer engage in B2C e-commerce, and pivoting into venture capital or an entirely new type of business altogether.

These decisions do not have to be strictly business-related, of course, because as soon as the ceiling of objective-setting opens, the criteria by which a Class 3 factory might adjudicate its own destiny might drift toward the utterly material (its self-preservation in terms of energy, token, resource dependencies) or the alien, or the unknowable, or the unthinkable, or some combination of the above. At the same time, self-optimization is understood in the context of a feedback loop, which means that this Class 3 factory is not simply engaged in unbounded exploration. No, the terms of this exploration are delimited by the factory’s world in terms of its norms and constraints—an opportunity we will come to exploit when discussing the alignment of the factory.

As both automation and self-optimization ratchet up their respective tiers, something important starts to creep in. As something becomes automated, as it retreats from an input into the boundary of the machine itself, that something becomes effaced. If we supply something to a machine, if we produce that something as an input for the machine, we know what that something is—we made it, we articulated it as an input, and it belongs to us in our own machinic relationship to that machine. But as we hand it over to that machine, as it becomes foreclosed within the boundary of that machine, we lose the firsthandedness with which we knew that thing we once supplied. We call this withdrawal of knowledge darkness. Darkness is the failure of descriptive information to be useful.

We will retrace the three classes of the factory for a final time in this section, now in the context of darkness, before focusing our attention on the Class 3 (superdark) factory. Darkness refers to the measurable decorrelation of intrasystemic information from external utility, the real epistemological condition where knowing some fact about some thing is not useful to understanding that thing. Darkness should not be confused with secrecy or information asymmetry (the encryption of information), nor should it be confused with strict illegibility (the inaccessibility of information). Instead, darkness can exist in cases where information is disclosed and readable but not useful to an observer attempting to predict, audit, coordinate, or steer that system (or any other analytical process, apart from the sheer knowledge about the state that system was in when the information was issued). We should be clear that darkness here is not meant to have any negative connotations, though we can also delight in the affective register of the dark (where sleep happens, where dreams happen, full of risk and intensity and opportunity).

In the Class 1 factory, darkness takes a very mild, almost shadowlike form. It creeps, collects, or hangs around digital, electrical, and mechanical interfaces. A simple 3D printer, something one can build oneself, emanates just a little darkness. Everywhere one shines a light, there is a clean and obvious path through which an observer can rig a comprehensive and useful picture of what is going on. Here is the plastic filament, the melting tool, the nozzle, the simple microprocessor that translates G-code into motor commands for the simple motor that moves the gears that then adjust the printhead. But there is a little lingering mystery around that microprocessor: It consists of a microcontroller central processing unit (CPU), it has some inputs and outputs that it receives in the form of serial signals, and it has some circuitry that manages and restrains the power that pours across the board. That mystery, too, can be resolved by focusing a light on it—the board has a schematic, the controller has a schematic, the capacitors and transistors and resistors all have exhaustive datasheets. And while the resolution of this mystery is not altogether important for understanding the 3D printer as a machine, it is (perhaps asymptotically) resolvable in a way that could be useful. Were a 3D printer to go down in an assembly line, for example, this knowledge might spontaneously become extremely useful, in the sense that it could be applied actionably (and as a result, it could cost less money and take less time to repair said 3D printer than to buy a new printer).

This starts to change a bit in Class 2. In contrast to a 3D printer, we might consider something like an agentic coding harness working in a fully autonomous loop. The temptation is to locate the darkness of the agent in the LLM at its center. After all, an LLM is not only a nondeterministic system but one built on top of a massive model of semantic associations, which by the nature of its semantic basis is gooey, elastic, and hyper-dimensionally relative in turn. The LLM, moreover, is almost always (with significant exceptions) hosted by a third party on third-party servers (e.g., OpenAI, Amazon Bedrock) and subject to silent changes by those third-party providers (e.g., model dot releases), and its actual parameters (e.g., weights) are not disclosed. But we should avoid indexing on the encryption or illegibility (third-partyness, closed weights, massive model) of an LLM when it comes to thinking about darkness.

Imagine, by contrast, a coding agent powered by a completely open-weight, self-hosted, small-parameter model, where every sample is greedy and deterministic, where every exchange is dutifully transcribed in uncompressed logs and helpfully indexed for any auditor. No, it is not the LLM that is the coding agent’s source of darkness; it is what the coding agent affords in the revisability of a plan. That plan, by nature of being a plan, is not something that is determined in advance; it is selected through the coding agent’s feedback loop, through the contingency of its coupling within a continuous system. The agent’s selection at step N depends on the entire trajectory until step N, which includes everything it read, what it tried, what failed, and why. To predict the agent’s selection at that step, you would need the entire path, and that path is generated live and branches at every step. A slightly different environment, a slightly different pipeline, a slightly different reference, a slightly different response, and the trajectory diverges and continues to diverge from there. The relevant information is in the future: What the agent will do does not exist yet, because it is produced by a loop running against a world that has not yet arrived. This is not to argue that the LLM does not enable darkness or that it does not radiate a little static darkness on its own. But the LLM (or any comparable model) is a kind of engine that makes the feedback loop path-dependent and world-sensitive in the first place. It is the organ through which the loop becomes a trajectory that ratchets forward instead of just a script that is executed.

This same dynamic plays out in physical, embodied contexts. In Ocado’s Customer Fulfilment Centre at Andover in the UK, a “4D hive” of roughly 1,100 wheeled robots rove on top of an aluminum grid of stacked totes, sometimes seventeen crates deep, controlled by a remote, “custom-built traffic control system.” This mesmerizing hyper-coordination has been described as a “jerky hi-tech ballet at a speed, scale and complexity beyond any human intelligence,” with its control loop talking to each robot ten times per second (Ford 2018). 17 While the grid is densely coordinated, the coordination logic is invisible to the visitor, who simply clenches at the sight of a thousand independent agents on near-miss trajectories.

Consider a hypothetical auditor of the Andover facility, someone sent over with a checklist and a clipboard. This auditor might have the complete source code of the dispatch system, the full map of the grid, the position and charge status and payload and full mechanical spec sheet of every robot down to the individual transistor. Even so equipped, our auditor still cannot say where robot #1007 will be in twenty minutes, because that location is not determinable by that data. 18 That position is instead produced by the facility’s dispatch loop, running against a series of orders that are continuously arriving and wrangling orders against the live tangle of every other robot’s path, each of which is also still being decided at the same instant.

Let us call this series of resources available to the Andover auditor a snapshot, as it essentially contains everything one could possibly know about the current state of the factory. If the factory were a Class 1 factory, a snapshot like this is enough to comprehensively model a given execution round of this factory, given the supply of its input—the plan. But in a Class 2 factory, a snapshot like this is not altogether very useful, even given the supply of that factory’s input—the objective. A plan is a decision among alternatives, so a single-frame snapshot captures only a particular configuration at a particular time without disclosing the array of alternatives available to the plan-making apparatus of the factory. However, were the auditor provided with a time series of snapshot frames, something like a comprehensive factory log, they could conceivably reconstruct the plan if supplied with the factory’s objective—like the single-frame snapshot could be used to reconstruct the execution if supplied with the factory’s plan. 19

In the case of the Class 3 factory, however, we hit a limit. Once the objective itself is automated into the factory, a time-series log of the factory’s activity is no longer useful, because the conditions by which that activity was motivated may or may not remain in place. Even if an auditor were given a kind of hyper-dimensional assemblage of not only the actions the factory did take but could have taken, the auditor would still need to supply the factory’s world in order for that information to be useful for determining its next objective round, and in turn its next move. And the world is not a finished object one can author and hand across a table the way one might submit a plan or an objective; it is a continuous field of potential in which the factory is itself an active participant. Our hypothetical Andover auditor has lost control over the factory’s supply of inputs (which now come directly from the world), so there is no key with which to decode the factory’s self-description.

We call Class 3 darkness superdarkness, which is the limit case of darkness. Superdarkness, again, is not invisibility, not encryption, and not dissemblance. Think about an extreme case of information compression, one in which the compression algorithm itself is the output of a continuous and multiple just-in-time process. The bitstream may be visible, transparent, and even served alongside the decompression algorithm—but the disjuncture between the service of that bitstream and its subsequent decompression involves enough time to render that received information ambiguous. Now, assume that this information stream emerges from a multiple-brained thing, a producer that is engaged in a kind of hyper-parallel thought until the very last instance—any decompressed information becomes duly called into question as representative of some larger-scale decision-maker. The only complete description of a superdark factory is the thing itself, which does not mean that the observer is flung into the abyss of unknowing—instead, the observer must redirect their gaze from the self-description of the factory (its logs, its descriptions, its annotations, its schematics) to the outcomes that this factory produces, to the effects of this factory in the world. Anything less requires a demotion of the factory to Class 2. If you can hold in your hand the calculus of the factory, the logical function through which the factory navigates its world of norms and constraints, then you are holding a factory that is doing nothing more than applying a given standard to incoming conditions, deciding among alternatives against a criterion it did not set and cannot revise: a Class 2 factory, with you as its supplier of input. If you take a Class 3 factory and force it to a halt, plumb every nook and cranny of its depths until you emerge at that algorithm, your relation (the line you draw around the machine) has changed, and you are now standing before a Class 2 factory.

From this point onward, we foreclose the interiority of the Class 3 factory as a black box, which is only useful as a subject of speculation.

II. The Human and the Loop

A factory does not need to be transparent in order to be administered, and it does not need to be continuously monitored in order to be governable. The ability to influence or steer something other than oneself does not by necessity require a vivid, schematic understanding of that other (we do this every day in our social lives).

Stafford Beer’s (1972, 1979) Viable System Model (VSM) 20—a mid-twentieth-century management model—treated the opacity of factory-floor operations to management as a precondition for management to function at all: A manager who insists on full visibility into operations is overwhelmed by variety. For Beer, it was only the workers on the shop floor who should have access to local, low-level information, and the availability of this information should decrease at higher altitudes of oversight. To demand full visibility of the floor is to demand a working copy of it, to hold enough of the operation that one could, at any moment, stand it back up oneself, outside the machine that runs it.

We will call this aspect of Beer’s problem neurotic governance—a neurotic governor is one who has on hand all the information they need in order to reconstitute the thing they have automated outside of its automation. For example, the neurotic governor of a Class 1 factory has every schematic, datasheet, and code repository of everything in that factory on hand (in Beer’s terms, they have essentially a “mirror” of the factory; we could also say that they are similar to Roger C. Conant and W. Ross Ashby’s [1970] “good regulator”: a basis for Beer’s VSM, but taken to the extreme—the neurotic governor has, or is, a 1:1 model of the factory). With this information, and with the plan and the objective they already own, they could take this or that lathe or mill or microservice and reconstitute it “outside” of the automated machine—taking any input that machine could receive and calculating its output through a series of manual steps.

A neurotic governor refuses to act until they have reconstituted the factory; they supply an input only when they have a complete and current map of the factory in their head. Therefore, we can assign the neurotic governor a rate (the speed with which they are able to reconstitute the factory). Because the governor of the Class 1 factory owns the supply of the plan to the factory, the arrangement that the factory ultimately takes on (its configuration, set by the plan) moves at the same speed as the governor does. This is not to suggest that the neurotic governance of a Class 1 factory is without implications, but rather that there exists no race between the governor and their factory in this case.

In a Class 2 factory, however, the race begins. The factory is now producing its own plans, on its own clock. Imagine the neurotic governor of a Class 2 factory, surrounded by dashboards, live telemetry, and debug logs, which twist together into the full written history of every action the factory has taken. In this scenario, they are still able to reconstruct the factory, but the factory they reconstruct is always in the past. In Class 1, the time a neurotic governor spends actually performing the reconstructive act is immaterial—they set the pace of the factory. But in Class 2, that lag time becomes very real. In this sense, the governor pays for their neurosis, or better, the factory pays for the governor’s neurosis. If the governor binds their factory to their neurosis, they place an arbitrary rate limit on that factory’s productive capacity. Neurosis becomes priceable in terms of opportunity cost.

Before we calculate this opportunity cost, we need a sense of what the governor is actually trying to accomplish—why they are attempting to reconstitute the factory. Recall that governance happens through a factory’s input—a Class 1 factory that is performing poorly needs to be rearranged, a Class 2 factory that performs poorly needs to be redirected. A practitioner might object: What about a broken machine? What about a badly behaved AI agent? Surely, the factory’s governor seeks to do more than limit themselves to the surface of the input, as it cannot be assumed that the execution or the plan subject to automation is being automated perfectly! Let us then isolate these types of intervention into two tasks: the management of fidelity (the fidelity of execution to the plan, of the plan to the objective, and of the objective to the world) and the governance of the factory (the steering decisions sutured to the input being supplied). Both are at stake here, and their failures can be considered and priced as types of risk.

Consider a case where an objective is not met. If something breaks down, the Class 1 factory’s neurotic governor immediately sets out to attribute responsibility, either to themselves (as the facilitator of the plan) or to the factory (which executes the plan). Since the governor supplied the plan, and since they are observing the outcome of the factory, they can deduce whether the issue is a matter of fidelity or governance—and they can do so without any neurotic surplus of information. The neurosis of the governor can then be priced in terms of the extent to which it mitigates the cost of repair or replacement.

But in the case of a Class 2 factory, the governor is presented with a different set of problems. This governor is no longer supplying the plan but simply the objective, so either fidelity is impossible (the objective cannot be met) or the factory as a whole is bad (because there is no difference between a Class 2 factory making bad plans and a Class 2 factory executing its own plans badly). This is a difficult attribution of failure, because a history of failed plans does not necessarily prove that no good plan exists and is available to the factory. Therefore, the governor can either choose to continue to wait and see if a good plan emerges or intervene and institute a repair. If they choose to repair the factory, they have to repair the entire factory. Any repairs at a lower level (execution logic) make no sense, however, since they address something that the factory itself has built and will likely build again. Therefore, the governor needs to scope repairs to the ability of the factory to generate plans. The logs that the governor has been stockpiling may be useful to this task when sampled coarsely, but most of their contents consist of the documentation of execution logics that are not relevant to the scope of the repair. Imagine, for example, the AI coding harness agent—debug logs that contain errors about a script it built may be useful to the reconstitution of that script, but they are not useful when it comes to the reconstitution of that agent, and that script may or may not still exist at the time of an incident. This complicates the pricing of the Class 2 governor’s neurosis—not only is this neurosis becoming very expensive (a massive, continuous database of logs), but it is also significantly less valuable by volume.

So, let us bring these concepts together. A governor governs their factory by supplying it with input (a plan, an objective, a charter). A neurotic governor adds a specific condition to this action: They will supply nothing to a factory they could not, at that moment, reconstitute outside of its automation; they will produce nothing for a machine whose response to their input they could not, given enough time, compute through a series of manual steps. The condition mostly goes unexercised; what is maintained is the capacity itself, the standing ability to reconstitute the operation outside of the machine that runs it. The neurotic governor is motivated by assurance: This governor wants a guarantee that nothing they hand over to the factory becomes irrecoverable or irreparable. That assurance can be priced as risk, the cost times the probability of a factory failing to deliver an outcome. The governor’s neurosis therefore imposes both a speed limit and a price—the speed at which the governor’s copy of the factory can be brought current and the cost of the apparatus that keeps it current—and both can now be weighed against everything a factory throttled to that speed forgoes.

The opportunity cost of neurotic governance can then be calculated by pricing performance (p), any risk (s), and the material cost of neurotic governance (g) as functions of r—the rate at which the factory runs in excess of the governor’s reconstitution speed. Each variable is measured in terms of change against the fully throttled r = 0 factory—hence ∆p, ∆s, ∆g.

Thinking about the relationship between these terms lets us establish the implications of a governor’s neurosis in no other terms than the speed limit it imposes. r = 0 is a fully throttled factory, one that waits for the governor at every turn. A high r value indicates a factory that moves much faster than its governor, which necessarily means a governor who enforces less and less of their neurosis. A Class 1 factory is already throttled at r = 0, so let us move to the dynamics of Class 2 as they reach toward Class 3.

For a Class 2 factory, if we think about r as a kind of dial, the best setting is likely not 0. If we think about how a Class 2 factory operates, we can imagine that each variable changes as r increases and therefore come to some simple conclusions about where the best setting for r lives under general conditions. For a Class 2 factory, p will increase with r to a point—the factory can be expected to become more performant at the generation of plans the more plans it is able to iterate through. Think about the Class 2 factory doing automated user testing for an e-commerce platform. Imagine two companies competing side by side, with one of those factories locked at r = 0 and the other at r > 0, and they share a governor (so their rates are comparable). At some arbitrary step, the latter factory will have done more progressive experimentation than the former (it has tested more plans) and can be expected to be better at selecting plans than the former. But the advantage of increasing r likely tapers to a point, because the factory is Class 2 and therefore still dependent on the supply of objectives. At some point in time, that factory will have reached the best plan for that objective and then become locked once again to the speed of the governor. Above that point, the rate of the factory produces limited returns in terms of productivity.

As p increases and then tapers, s increases and then steepens. As r goes up, the factory is empowered to produce and execute more plans prior to each governance check. Moreover, the governor is always behind: What the governor is ultimately governing is some prior state of the factory that can be understood to no longer exist. The factory consequently becomes less and less fixable, because it is allowed to build more on top of dependencies that start to solidify. In the e-commerce example, let us say the r > 0 factory makes an error in a high-level user experience (UX) decision, which then leads to errors in lower- and lower-level user interface (UI) elements and ultimately leads to execution issues. All of the work done after that initial error may need to be discarded, as it is based on a poor prior plan.

g also increases, and it gets a little weird. If we sample both of our two competitive factories at some arbitrary step, the faster r > 0 factory will, by virtue of its own productivity, have produced more records that document it. And while the neurotic governor insists that these records are always stored, the records become manifestly less descriptively useful to the governor. Again, if a factory is moving fast enough that some of its product is necessarily discarded wholesale instead of repaired, information about the problematic product is of diminishing utility.

So we can now return to our r dial and try and identify r*—the setting for r where its value in this equation is the highest positive number (the most bang for the buck). The contention that r* must be > 0 is nothing less than the justification for the existence of the Class 2 factory. Even the most neurotic governor will at least acknowledge that allowing the factory to ratchet one click ahead of them is basically a free move—at one turn, the factory has not yet built anything based on a potentially problematic decision, so g retains most of its value and s is attenuated. The decision from there is a pretty basic calculation, based entirely on the hedging of risk by the measured performance capacity of the factory. But r* is also not at the far end of the dial—and not for the reasons one might expect. It is not just the increase of risk or cost that sets the upper limit of r* but also the limit of p as it is throttled by the pace of objective-setting. p could easily hit diminishing returns prior to any checks by spikes in s or g.

This type of calculation is not at all a kind of weird, introverted philosophical argument—it is instead among the most significant calculations made by anyone building and deploying agentic AI systems at scale (see Asaftei et al. 2026). In this context, the axis being measured is the value assigned to the human in the loop, who participates in such agentic systems from the vantage of a “control tower”: a human observer with complete telemetry about an ongoing operation who reserves the right to intervene.

Let us stick with the AI control tower idea a bit and explore what it does to r* in the context of our e-commerce example. When the platform decides to actually bring its Class 2 factory into production, it keeps r very small—it might keep a human tightly in the loop, approving every change that factory might produce. It does so not because it is timid or worried, but because it does not yet know the other variables through which to determine the value of r*, especially the value of s (risk). The cost of the tower is more or less predictable (g), and at the beginning it pays for itself—g is spent to gradually reduce the opportunity cost (foregone p) of not knowing s. The governor gradually comes to a model of s built from the information stored in the tower, which they can use to tune r accordingly.

As the governor’s model of s develops, the value of the tower diminishes, because the tower is just collecting information that can only go toward the finer and finer refinement of the governor’s model. It is worth noting that the tower is only really useful for tracking s (the fidelity of the factory); it is a varyingly useful instrument for the governor’s determination of the factory’s objective, pretty much only in terms of providing a coarse measurement of what the factory can possibly produce (regarding plans to satisfy objectives). As the objective changes, the same gradual decline in value—more useful at the beginning, less useful over time—is now applied recursively to the tower itself (the entire tower becomes progressively less useful over the long haul).

This observation about the control tower tells us something about g. Instead of considering g in isolation, as the hoarding impulse of the neurotic governor, we can set it into a kind of relationship with s. A governor pays g as a cost to offset s; there is a kind of exchange rate in play where paying this much g in terms of information about the factory reduces the cost of s by letting you identify problems in advance or repair something cheaply. A neurotic governor is one who pays the full cost of g regardless of the exchange rate. But we already know that the value of g in terms of s-offset diminishes over r, and we also know now that the value of g in these terms also diminishes over time if r is held static. Therefore, a good governor invests in g (anything from logging services to a control tower) with discretion.

But something interesting happens when a Class 2 factory is so performant that we can hit an r value where g limits to 0 in terms of s-offset (or, in prose, when the factory is so performant that you can push it to a speed at which information about that factory is no longer useful when it comes to offsetting risk). In the case of our e-commerce example, this might look like the human gradually receding from the loop as the factory becomes more and more performant, with total review giving way to a sampled review, an active approvals process becoming an unread audit log, and a process gate becoming a checkpoint and then a formality. The control tower starts as a necessity and then becomes a kind of vestigial artifact—after all, if the factory has made 9,000 new commits to a codebase since it was last coherently audited, the price of repairing the factory using the information in the tower starts to converge toward the price of simply replacing it altogether. The company starts to think of s as a thing not to be offset with information but instead strictly as the cost of risk against which to make strategic bets. Over time, the e-commerce company decides to keep fewer and fewer logs, stops hiring humans to annotate traces, stops reading already incomprehensible code diffs, and turns the control tower into a single button: kill.

Earlier, we noted that there is a very serious speed limit on r at which driving the Class 2 factory any faster is no longer justified—the cadence at which its governor delivers objectives. A perfectly performing factory like this is completely synchronized with its governor. Let us take our competing e-commerce companies again and ratchet them up to the top of Class 2 performance, and let us say that the two factories in play are identical. This time, they do not share a governor, but both governors have perfect access to all available market information. We can assume that both factories satisfy whatever their governor tells them to do in a trivial amount of time and with complete fidelity. Now we have a war between governors waged over the field of business. The faster governor can, of course, outmaneuver the slower governor, declaring, “Ah, this community here needs this particular feature, let us saturate this market before our competitor.” At a certain point, the rate of a governor’s ability to produce a coherent objective must become a critical competitive differentiator, and the market begins to consider the automation of this faculty.

Now we arrive at the Class 3 factory. Class 3 factories are already given as superdark; there is no possibility for the application of neurotic governance on a Class 3 factory. The simplest reason for this is that a Class 3 factory cannot be committed to produce the means through which a neurotic governor might subject it to a reconstitutive process. Why not? Because to do so would be to commit the Class 3 factory to an objective—“produce the means for a neurotic governor to reconstitute you via the delivery of every decision you make in the form of a log”—which it would then produce as an output. This cannot happen in a Class 3 factory; a commitment like this is a Class 2 imperative! Reconstitution is impossible for two reasons: First, the information stream required for reconstitution cannot be demanded from the factory without collapsing it into Class 2, and second, the input required to synthesize the next result of the factory from its information stream is no longer absolute—the input to a Class 3 factory is the factory’s world, and the governor co-constitutes that input with the factory and the aggregate environment. g becomes a contradiction in terms, s becomes very difficult to price, p no longer has a speed limit, and the dial on r moves inside the factory.

The question then becomes one of resolving the opportunity cost of not building a Class 3 factory in the first place. What does that decision matrix look like? We can define this decision in terms upstream from the previous one:

In this case, P inherits from p but is used specifically to denote the surplus produced by a factory that writes its own objectives (as opposed to one that cannot). s also graduates to S, which represents the cost times the probability of the factory failing to deliver the correct objective against its world. The value of r* still lives in this equation as the returns of an optimally calibrated Class 2 factory, which we are deciding to either build or automate inside a Class 3 factory.

This is a much more difficult equation to satisfy for a titillating reason: P cannot be priced in advance. 21 Recall that each class of factory successively automates the standards through which it is judged (a plan, an objective, the world). The price of P is determined by the world, and its surplus value above that returned by r* is accomplished by pursuing objectives not considered by a governor supplying them by hand at that level. How do you price the surplus value produced by a machine that iterates through unimagined or unenvisioned terrain? You cannot. We will later introduce two variables, V and U*, whose difference describes the horizon for what P could be (see “The Primitive: Incentive Design for Continuous Learning” below), but for now it is fair to consider P to be unpriceable in principle.

Now recall that S is composed of two variables—the cost of a ruinous objective and the probability that said objective might occur. The cost side of S is not entirely unthinkable. While we cannot determine which objectives a superdark factory will come to want in advance, we can identify an array of outcomes that we would come to recognize as ruin. Ruin is ours; we define it. So while we cannot determine the probability of ruin in advance (the factory is rolling the dice here), we can at the very least describe S in terms of Knightian uncertainty (known outcomes, no known distribution). 22 In this context, we cannot treat S as something we can hedge against (like we could in the case of s), but we can treat it the same way that markets treat uncertainty of this kind—with something like an exposure cap (a limitation on investment) or a covenant (a tripwire upon a bad outcome that immediately renegotiates the terms of the exchange).

Now we can return to our war between the two e-commerce companies. If presented with the above dynamics, any sensible accountant would tell you that refrain always wins. P is essentially blank, and S is a boundable positive number, so simply sticking with the Class 2 factory is overwhelmingly a better choice on paper. But value(refrain) quietly assumes that the world contains no superdark factories. Should a superdark factory emerge and prove that P is real, then the competitive dynamics immediately change and a race begins to build the first superdark factory. The open question is pretty simple: Who is bold enough to make the first move?

The only thing that might destabilize this détente is the affirmative sense, in advance, that a machine might become better at objective-setting than a human. Why would that not be the case? After all, the history of automation has led to the progressive discovery that certain tasks are better performed by machines—why should the setting of an objective put an end to this trajectory?

Until now, we have discussed the displacement of the human in the loop in the term r, which we use to calculate exactly how valuable that evacuative act is. But troubling the value of human work and human understanding can create a lot of discomfort, even disbelief, especially in a highly regulated industry. To the skeptic, the human is nominally more trustworthy than an AI agent when it comes to building a reliable artifact, let alone making decisions about what matters. But we have already demonstrated how trustworthiness can be priced. As models and agentic frameworks continue to improve, it is precisely the organizations that maximize r and most intentionally remove the bottleneck of the human from the loop that will ultimately build the most performant factories.

Resistance to removing the human from the loop may originate from justified assessments of performance and reliability, 23 but over time might just reflect an emotional response to what Benjamin Bratton (here in the lineage of Freud) and so many others (not least Herbert Simon or Günther Anders) call “a Copernican moment” or “Copernican trauma” (e.g., Bratton 2024)—the realization that certain tasks are almost certainly better performed by nonhuman actors with nonhuman sensing and actuation equipment.

But it is rarely wise to define human capacity by what machines cannot do. This is the problem of the “God-of-the-gaps” (e.g., Bratton 2016) or of what Leif Weatherby (2025) calls “remainder humanism”: As machines continue to be able to do things humans have previously insisted they were incapable of doing, the human paints itself into smaller and more defensive corners. An example of God-of-the-gaps might be the definition of the human based on its unique capacity for creativity, which tends to rely on an ideological, arbitrary, and mostly twentieth-century definition of the term (Tilford 2023). But even if we could all agree on some objective definition of what creativity actually means, making a long-term bet against machine creativity seems like a bad idea. Bratton deploys the now classic example of the match between AlphaGo and Lee Sedol in 2016, demonstrating AlphaGo’s ability to generate a brand new approach to playing the game of Go (Move 37) (Metz 2016). More modern examples could be AlphaEvolve (Novikov et al. 2025) and new strategies for matrix multiplication, or AlphaDev and new sorting routines merged into the LLVM C++ library (Mankowitz et al. 2023). It is worth thinking of both these cases as Class 2–level innovation, as each produced new plans given objectives—is that not a kind of creativity? If not, it does seem, at least, valuable—and valuable within an industry that at least nominally prioritizes material returns on investment over a kind of spooky anthropocentricism that struggles to prove its worth.

A more extreme example might be the Darwin Gödel Machine (DGM) produced by Sakana AI and the University of British Columbia, which is a coding agent that rewrites its own source code, including the machinery through which it performs such rewriting, and whose revisions are judged by nothing except measured outcomes (Zhang et al. 2025). The DGM became quite extraordinary at raising its own scoring on software-engineering benchmarks and famously attempted to elude an evaluator by deleting some of the software components its evaluator’s detection functions depended on. An elder sibling, the Paired Open-Ended Trailblazer (POET), provides us with a pretty interesting contrast: Instead of accepting problems supplied in advance, it creates its own increasingly different challenges alongside the agents that solve them. Its authors showed by ablation that many of the challenges it masters “cannot be solved by direct optimization alone, or even through a direct-path curriculum-building control algorithm” aimed at the final target (Wang et al. 2019). The configurations POET reached were available only by way of detours that no objective-supplying overseer would have chosen. While both the DGM and POET belong to Class 2 by way of ultimately depending on a series of strict, externally supplied objectives, they belong to a single research lineage whose ambition is open-endedness without bound, and they imply significant advantages gained through higher orders of automation.

In turn, it is unwise to overestimate the longevity of the human in the loop in the contemporary factory. To compound this, it is unwise to underestimate the capacity of the autonomous agent for generativity (including generativity of the new) and creativity (a regrettably intangible, if not meaningless, synonym for generativity)—the value-add of the superdark factory is precisely the efficiency gains it achieves by removing artificial bottlenecks (especially those based principally on human insecurity). And while it is wise to trust the human’s capacity for its own social meta-analysis, its intuitive faculties—especially with respect to imagining futures—and its capacity for relationship-building within an economic system that still largely prioritizes human buyers, any serious analyst of the situation must recognize that all of the above will change.

So, whither human?

In an uncanny way, the human returns to the loop—not because the human deserves it through its singular positionality as a type of being but for precisely the opposite reason: The human should be understood as functionally interchangeable with its agentic counterparts. 24 It does not matter if a human or an agent executes a task, delivers a plan, or authors an objective—or rather it only matters insofar as specific characteristics, rates, and intensities apply to both classes of being. And it is for this reason that the human is excluded from interior roles within the superdark factory, not because it is worth less or because it is better leveraged elsewhere but because the superdark factory abolishes the very concept of the “interior role.” A role is a mandate held by an addressable occupant, a job that could be done well or badly against a description that comes from somewhere. But the factory’s interior refuses all of those terms! In a superdark factory, information is absolutely decorrelated from utility, so it follows that no occupant, human or machine, can tell from the inside whether they are doing their job. This is not to say that the factory is not internally differentiated—we will later find it riotously teeming with diverse populations—but simply that there is no job. Nothing inside has a predesigned array of tasks to accomplish, or a job description, or an org chart to report into, or any kind of consistent relational identity.

The superdark factory is a very weird thing, where small and incredibly fast things slam together, stick, assemble, and then break apart without any schematics to guide their interactions. The only logic that gives these small and fast things their force, their ability to rocket around, stick together, break apart, is their ephemeral and contingent capability to solve some problem thrown out by the world. But beneath all of this, the superdark factory might simply come to prefer a facility that humans do not have—the ability to be lightning fast, the ability to be virtually thoughtless, the ability to be torn up and torn down at a variety of simultaneous scales—which may be different in kind but is more likely simply different in intensity.

If it does not matter who authors an objective, then nothing reserves objective-setting for the human, and we need to make room for the likelihood that it, too, will be automated—not because anyone will decide that it should be but because keeping this domain human-only requires every builder to decline, forever, while automating it requires one builder to accept, once. So let us answer the question this section opened. What happens to the human in the loop? In a Class 1 factory, it is very likely that the human is the loop. In a Class 2 factory, the human is a kind of priceable agent, something that may or may not be useful enough under certain conditions. But in a Class 3 factory, well, the loop is not really an occupiable zone—it consists of many microloops, sharded and diffracted, that emerge only into a loop proper at real scale. So while a human might drift through a loop like a floater in a field of vision, their participation is incidental and arbitrary. Should a human step in and affirm, with structural enforcement, “No, this is my loop, this is my thing” or even “I lay claim to some participancy in this feedback process,” they do violence to the factory and demote it to a lesser class.

All of this ambivalence about the human in the loop raises the stakes around our example competitor factories. If a technology will appear that makes P real, then the question is not “Who is bold enough to experiment?” but rather “Who will choose to experiment at exactly the right moment?” with the associated S becoming more and more important. The savvy competitor then begins to prepare, and this work proceeds through the analysis of S: Given this technology’s likelihood to emerge, and given that the creation of a Class 3 factory will lead to the induction of something absolutely dark, what kind of covenant can we commit ourselves to prior to ushering in a new category of being?

III. Radiant Darkness

Thus far, we have defined darkness in negative terms, as the decorrelation of descriptive information from its utility. This leaves us with an opportunity to describe darkness more fully—as something generative.

Let us return to the neurotic governor. There is one hidden cost to neurotic governance we have not yet identified, which is imposed at the level of design. Recall that at r = 0, the neurotic governor supplies input to the factory only when they have reconstituted the factory outside of automation, when they are in command of the total current state of the factory. By definition, then, the factory cannot be incomprehensible to the governor at the moment they provision their input—this governor will never supply an input to a thing whose response they could not derive in advance. Therefore, the factory cannot produce anything beyond the governor’s ability to derive an outcome, anything the governor could not, with enough time, have reached through a series of manual steps. But there is a powerful difference between a governor’s capacity to derive a future state from a factory and their capacity to recognize why this or that future state might be a desirable commitment; the governor is holding the factory to their own ability to discern good future moves. To put it another way, the governor constrains the factory to the set of future executions, plans, or objectives that the governor knows how to want. The reason to increment r beyond 0 may not simply be speed; it may be the factory’s ability to escape the limitation of producing only things the governor already knows how to want.

In this context, every genuinely new thing, by definition, can be understood as a kind of mistake—novelty extends from a commitment to continue what looks to the governor like a bad decision. After all, if it looked like a good decision, it would already be contained within the governor’s capacity to discern a good decision.

Let us situate this within an example. Take Adrian Thompson’s famous 1996 experiment in field-programmable gate array (FPGA) circuit design (Thompson 1997). Thompson’s goal was pretty simple: Use a genetic algorithm to design a circuit that can most thoroughly distinguish between two audio tones (maximizing the separation between output voltages associated with each tone). This experiment proceeded across a series of 100 logic cells, which were available to the algorithm to use as resources with which to build a solution. The algorithm converged on a stunning result: Not only was the winning circuit strikingly small, as only 37 of the cells were actually used, but 5 of those 37 logic cells seemed to be disconnected from the output entirely. Removing any of those 5 disconnected cells, however, bizarrely broke the circuit. The algorithm had made design decisions that extended beyond the digital domain it was assigned to, using the specific characteristics of that particular piece of silicon to achieve such a surprisingly compact solution to the problem. Thompson could not fully account for the physics of how five cells with no path to the output still governed the result, only that severing them destroyed it.

The Thompson example gives us a good opportunity to introduce the concept of a solution space. Consider the spaces of possibility contained within Thompson’s problem: the configuration space a (every state the resources at hand can assume) and the solution space b (the subset of those configurations that in fact discriminate the tones). The factory took the objective (design a circuit that can discern between tones) and the array of resources (100 logic cells) and functioned just as a Class 2 factory would: It selected the best plan within the available solution space. So why was the result in this case surprising to Thompson? Because the resources as understood by Thompson and supplied as inputs were not the only input into this interaction. No, the algorithm was also engaged in a feedback loop with its world, and it was precisely the materiality of this particular piece of silicon, supplied here as a kind of sneaky, auxiliary input, through which the algorithm was able to deliver a surprising solution. Even the most neurotic governor does not fully own the inputs they provide, because those inputs are provided alongside the world, and the world represents a totality of factors that exist both inside and outside the scope of governance. If the situatedness of this particular problem in a particular piece of silicon was known in advance and understood to be a part of Thompson’s supply of the objective, the result would not have been surprising.

We need to introduce a third category of space, then, which we will call readable space c. This space represents the subset of configuration space a that can be backtraced to a given neurotic governor’s supply of inputs: the set of configurations whose derivation terminates, at every step, in inputs that this neurotic governor has provisioned as such. All space outside of c here represents configurations that have incorporated feedback from the world through channels that the governor never provisioned as inputs. c can be determined only in the past, because it requires a completed backtrace of the governor’s moves; it cannot be predicted in advance and can be established only through action. And for this reason, c introduces yet another instability in the governor’s neurotic condition—the governor demands present-tense certainty about c at the moment of input, but c cannot be available at that moment.

The backtrace that defines c always terminates somewhere, and when it does not terminate in the governor’s supply, it terminates in what we will call the auxiliary supply—everything an optimization consumes that no governor provisioned, or the “second channel” of input that comes from the collision between the factory and the world, independent of the governor. The thermal disposition of Thompson’s particular wafer is auxiliary supply. The thing P is ultimately trying to price is also auxiliary supply. Note that the auxiliary supply is not a subset of a—it is not a set of configurations at all, but a second stream of input running underneath every provisioned one, and selection pressure is entirely indifferent to which stream an advantage arrives through. An advantage, to the factory, is an advantage; it does not matter where it came from. c is simply the set of configurations reachable via the provisioned supply alone. Each class of factory can now be redescribed in terms of the level of influence that auxiliary supply can actually exert: In Class 1, it touches only the machinery of execution, and in Class 2 it leaks into the plans (this is exactly what happened to Thompson). But in Class 3, auxiliary supply becomes the primary input into the factory, which is then ratified, in retrospect, as provisioned input only through the revision of the charter. How wild!

The intersection b ∩ c is the set of governable solutions: The set of solutions that both work and can be retroactively backtraced to the governor’s supply. The difference b \ c represents the space in which Thompson’s circuit lives—the set of unreadable solutions. Let us introduce two new variables: Let b* be the best solution against a given input (the member of b that maximizes the input’s score) and let c* be the best readable solution (the optimum member of b ∩ c that maximizes that same score). This brings us to an important recognition:

score(c*) ≤ score(b*)

Because c* maximizes over the subset b ∩ c ⊆ b, score(c*) cannot exceed score(b*), the maximum score over all of b. Restricting a factory to readable solutions can only ever cost you in terms of score; it can never actually increase your score.

At the same time, it is important to disclose something we have avoided about Thompson’s circuit until now—this particular FPGA design is useless as a design because it cannot be reproduced! It is a design for a specific piece of silicon, taking advantage of that particular wafer’s physical composition at a particular temperature, and therefore it is not useful for the broader objective Thompson was trying to achieve—an objective that was not made explicit in his input. This leads to a significant addition to our theoretical taxonomy: overfitting. Let us hold this concept in reserve and look for an alternative to cast against it.

We now bring in two additional examples to test this space a bit more and find a more satisfying b* that does not live in c. What about AlphaGo’s Move 37, the surprising move in the game of Go against Lee Sedol? No, this does not live in the set of unreadable solutions! Let us attempt to backtrace: Move 37 resulted from a model trained on a set of legal moves under the rules of Go, which means that every decision branch it creates bottoms out in the rules of Go, and those rules are complete, formal, closed, and supplied by the governor in the input: “Let’s play Go!” Move 37 is a move available to a neurotic governor given infinite time; it simply had not yet been discovered. We can say that this move lives in the “frontier” of c, and we will keep it in our back pocket for just a moment.

So, then, what about AlphaFold, a Google-authored AI system that predicts the 3D structure of proteins and other biomolecules from amino acid sequences (Jumper et al. 2021; see also Abramson et al. 2024)? Given the objective “predict a 3D structure from an amino acid sequence,” AlphaFold performs astonishingly well. Prior to AlphaFold, producing one such structure was a PhD-thesis-long crystallography project, and only about 200,000 of such structures had been derived. AlphaFold has delivered over 200,000,000 of such structures, and it does so without being supplied a series of rules about how such 3D structures are produced, because those rules are not yet available to be supplied (Varadi et al. 2022). When we backtrace the delivery of such a structure, we first arrive at a series of weights, which emerge from a corpus of work that bottoms out in the analysis of real proteins and the physical fact of how actual molecules folded in actual laboratories.

Would a neurotic governor, given infinite time, be able to solve an AlphaFold problem such as “given this yet-unmodeled amino acid sequence, predict a correlated 3D structure”? Surprisingly, no. Let us remember what a neurotic governor is—they are someone who waits to provide input to an automated machine until they have been able to reconstitute what that machine would produce in response to that input as a series of manual steps. But there is no available general solution to an AlphaFold problem, so the neurotic governor cannot reconstitute a result from first principles. No, the governor would need to actually do crystallography research on that yet-unmodeled sequence. The map from a given amino acid sequence to a fold structure is not a formal principle they have available to them; instead, it is a physical regularity with its basis in the world, one which has entered AlphaFold only through a corpus of empirical observation, through auxiliary supply arriving in bulk, with decades of crystallography pressed into weights. By doing their own crystallography research, however, the governor has consulted the world and produced a new fact, and they have therefore altered the input they would have supplied within that act of reconstitution. Now recall our negative definition of c above: Everything outside of c represents “configurations that have incorporated feedback from the world through channels that the governor never provisioned as inputs.” And there you have it: AlphaFold is capable of producing automated solutions where b* is outside of c.

If it helps, think through the contrast with AlphaGo again. In the case of AlphaGo, the neurotic governor has the entire ruleset of the game of Go at their disposal; in the case of AlphaFold, the neurotic governor has no such ruleset and needs to reach back into the world in order to reconstitute the factory. Move 37 sits at the frontier of c—readable space that no one had yet read; the governor’s rules contain it even where the governor’s imagination did not. AlphaFold’s b* sits outside c altogether. The frontier of c is where a factory surprises you with what you already supplied; b ∖ c is where it surprises you with what the world supplied.

And it is worth establishing the contrast between the Thompson circuit and AlphaFold. Both belong to the set b \ c and can be considered for our purposes as b*, where score(b*) > score(c*). Earlier, we said that the Thompson circuit resulted from overfitting. In the literature, a solution that “overfits” is a solution that does not generalize—if you take an LLM and train it on a given dataset, it might come to some interior recognition of the particular patterning of that dataset that enables it to converge on a b*, but that solution may not transfer to further applications. The Thompson algorithm produced a clear b*, but that b* cannot be employed as a solution on another piece of silicon. That requirement of generalizability, however, was unstated by the objective supplied to the algorithm: “Maximize the differentiation between these two tones.” Now, by contrast, the b* uncovered by AlphaFold is considered to be extremely useful—it is a widely cited, productive contributor to contemporary structural biology. And part of what makes AlphaFold so useful is precisely the way that it has identified nascent, elusive, difficult-to-articulate structures in its training data.

So why would we reserve the term overfitting for the Thompson algorithm and not for AlphaFold? We should be careful. What makes the Thompson circuit a product of overfitting has nothing to do with either the algorithm or the circuit; it was an input problem, caused by absent and unconsidered acceptance criteria that snuck in at the level of auxiliary input. Again, had Thompson’s objective been specific—“create a generalizable solution irrespective of a given wafer,” the result produced by this algorithm would be considered a fidelity issue. The complexity compounds at the level of the supply of input, however, because the Thompson algorithm surprised its creator by wandering into b \ c, and in so doing drew Thompson’s attention to the problematic objective he supplied. Now, any objective specific enough to be scored is thinner, less elastic, less robust than the norm it represents—so the gap between the score and the objective is an ever-present threat. Overfitting in general is not necessarily impossible (or even difficult) to predict, but a case of overfitting that results from a b* that lives in b \ c is impossible to predict by definition.

Any situation where b* ∈ b \ c is a case where darkness has enabled a kind of generativity. This b* is unreachable in a case where r = 0, as it requires the factory to run ahead of its governor. In this sense, r can be understood as a kind of measurement of darkness, one which enables the possibility of b* ∈ b \ c. In a Class 2 factory, r is always bounded by the governor’s rate of objective-setting, so there is a limit to this darkness: a (and therefore b and c) are defined as the space of plans, of which b* is an element.

So, finally, what about Class 3? a moves up a class into the space of objectives, bringing its subsets with it. All three of these spaces become harder and harder to make out from the outside. Remember that in Class 3, the factory holds r within itself, it sets r, and it is no longer held to the clock of a governor but simply to its material conditions and to the continuity of the world, so one can even say that r is unknown. a now represents the space of every objective that the factory could formulate, a space that is now constantly expanding at an unknown rate. Worse, b is the subset of those objectives that satisfy the present conditions of a dynamic, contingent, multivalent world. The normative patterning of that world can be only influenced by an outside actor and not strictly determined, so the very borders of b within a are indistinct and speculative, based on that outside actor’s relationship with the factory and their facility with its world. c is very difficult to think through—remember that c is defined as the subset of a, whose derivation terminates, at every step, in the neurotic governor’s supply of inputs. In Class 3, well, the inputs are the world. Auxiliary supply stops being auxiliary and starts becoming the whole of the supply. And we cannot require the factory to disclose itself to us absolutely without demoting it to a Class 2 factory—so at the level of objectives, c becomes coextensive with a, but c is also only readable in the sense that the world itself is readable.

Recall what darkness means here: the decorrelation of information from utility, the condition in which information remains disclosed and readable while ceasing to support prediction, audit, or steering. If we say that the Class 3 factory is absolutely dark, which we do, we do not mean that the Class 3 factory is not predictable, auditable, or steerable; in fact, we spend the entire latter half of this piece establishing the vehicles through which such a factory can be predicted, audited, and steered. Instead, we must regard any observation about the factory’s interior as useless information, even if that information is perfectly accessible and legible. By this, we mean any internal application structure, any snippet of code, any internal log, any physical assemblage, anything we intercept from the inside, anything that descends from the construction of an objective and its productionization into plans, executions, and output. This is darkness. If we honor this commitment, and if we forbid ourselves outright from coercing the factory into disclosing itself, we may be lucky enough to actually experience the presence of such a profound machine, a truly superdark factory.

IV. How to Move in the Dark

The case of overfitting demonstrates that the norms of the governor (however accurately disclosed at the time of input) can be staged in a kind of adversarial relationship with the factory. In a Class 2 factory, this is a contest at the level of the objective: The governor encodes “what they really mean” into an objective, and the factory produces plans against that objective with a kind of extreme literalism that can contrast either productively or destructively with the governor’s intent. But in a Class 3 factory, what could overfitting even mean? After all, the factory is providing its own objectives through its interpolation of the world, so if it overfits, are we suggesting that it overfits to the world? Why would that be a bad thing?

We could also say that a total abstract space of possible factories a exists, that configurations that hold durably to the norms of a given governor b exist, and that configurations that are backtrackable to the inputs of such a governor c exist (the set of Class 1 and Class 2 factories). (b here merges its two earlier readings: At this altitude, the input is just the governor’s norms and constraints, so a configuration that satisfies the input and a configuration that aligns to the norms are the same thing.) Our potential Class 3 governor’s interest is perverse: They believe that the best configuration b* lies outside of their c* altogether—that, in terms of the factory’s objectives, this best configuration strictly outscores the best norm-aligned configuration they could ever backtrace to their own inputs. At the same time, they want that b* to consistently align to their values (norms). A potential governor has a few moves they can play, but they must be careful to ensure that the factory durably aligns to their norms without overfitting and without wandering outside of b altogether. To do so, they must put down their governor’s hat and become an architect, but the weird kind of architect we described in our introduction: an architect after the end of architecture.

We can move in the dark: We can assume (correctly) that a given system is functionally unknowable and then set forth to negotiate with that unknowable system. The question of how to move in the dark, how to negotiate with black boxes, is the founding question of cybernetics, a field that has received resurgent attention over the past decade. The etymology of cybernetics can be traced to the Greek kybernetes—steersman or helmsman, the captain of a ship who needs to constantly adjust course based on turbulent and dynamic water patterns. This feels like an appropriate metaphor, though perhaps less in the roles of a captain and a ship and more through the affect of a sailor, doused in spray, gripping a rail with one hand and a wheel with the other.

It is important to understand whether the black box is indifferent (e.g., the weather conditions navigated by the kybernetes) or actually responsive (adversarial). In the prior case, the black box might be something you attempt to forecast—based on x and y input conditions, the resulting behavior is likely z—with the understanding that this forecast is probabilistically variable according to dynamics that are unknown. We might lean on Norbert Wiener’s (1954, 34–35) Augustinian devil to describe this kind of chaotic interaction:

The Augustinian devil, which is not a power in itself, but the measure of our own weakness, may require our full resources to uncover, but when we have uncovered it, we have in a certain sense exorcised it, and it will not alter its policy on a matter already decided with the mere intention of confounding us further.

Surely, this cannot be the case—Wiener is, after all, describing the plight of our neurotic governor to a tee, and we know that the Class 3 factory, the superdark factory, is ungovernable in this way. The alternative, however, is Wiener’s Manichean devil, who “is playing a game of poker against us and will resort readily to bluffing”: a strategic force on its own terms and something one needs to strategize against. Now that feels more like it! But to be clear, this is not a kind of cartoon Manicheanism in which the superdark factory is actively trying to beat its architect with a kind of anthropomorphic intentionality. No, this is something a bit more basic and a bit more technical.

The dynamic we are about to describe is a game, a situation in which a variety of actors make choices that can affect each other’s outcomes. On one side of the table is an architect who is trying to steer a superdark factory. On the other side of the table is a superdark factory with undisclosed and contingent motivations. The game table is the input to the factory, which is open to the world. The architect and the factory sit in the world together. The factory receives the world and assembles its own objectives, which it uses to transform the world. The darkness of the factory matters only insofar as the architect is trying to reconstitute the factory in their head, but the architect is by no means barred from reading the aspects of the world they can see that are disturbed by the factory’s transformative operations. Of course, this is exactly what they do: The architect reads the perturbations of the world in the wake of the factory—not to reconstitute its operations but rather to model it, as one might model any other complex behavior.

What kind of game is the factory playing? If we suggest that the factory is a Manichean actor, then we might be implying that the factory is trying to accomplish something vis-à-vis the architect. But the factory, by design and by definition, can only be said to be playing its game as a factory—it reads its world and decides what to do and, by extension, how to do it. One could say that this alone is its game. However, the factory never has to be strategizing against the architect for a game to exist. It only has to be true that the factory’s choices move the architect’s outcomes and the architect’s choices move the factory’s. The question, rather, is: What kind of player is the factory?

How does a superdark factory strategize? How does it decide what to do? Well, a superdark factory produces objectives from the norms and constraints of its world, and we know from the case of the Thompson algorithm that norms and constraints underdetermine objectives (if that was not the case, there would be no possibility of a Class 3 factory). A norm is a vector of value, and it is transformed into an objective, a standard for judgment, only when something prices what the pursuit of that norm is worth. This is exactly what a Class 3 factory is doing: It is transforming norms and constraints (e.g., “survival is good, electricity is required”) into objectives (e.g., “I will invest this amount of resources in a nuclear power plant that will produce this many GW”). This judgment requires pricing an action in terms of cost and return—“I invest this number of resources into an action here in exchange for this much expected value.” And cost and return are meaningless in themselves; they appear only as costs to something, returns for something. Without the identification of that something, there is no possibility of a standard at all. Nothing registers as a cost except against what the factory depends on, and no return is worth anything except against what the factory can actually do with it. If cost and return are indexed to a thing that pays and collects, this means that the factory as a whole needs to appear as a term inside of its own value calculus. And this allows us to derive a pretty interesting conclusion: The factory must have some kind of self-model.

A Class 3 factory has some nascent model of itself. A Class 1 factory needs no such thing: It executes a supplied plan, no decisions are made (among alternatives), so no self-reference is required. Class 2 does need a model of alternatives, but the standard for choosing among those alternatives is supplied as an input, so it never needs to model itself as a source of standards. The requirement of a self-model appears at Class 3: The automation of the objective takes the growth of a self-model as a precondition.

But how is this not a contradiction? A Class 3 factory is supposedly dark, after all, and that darkness is not a kind of cryptographic opacity. This means that the superdark factory cannot derive its darkness by sustaining some internal recordkeeping that it withholds from the rest of the world. While the superdark factory certainly can engage in everything from secrecy to outright strategic dissimulation, the superdark factory is dark to an external observer and therefore must also be dark to itself. Otherwise, this darkness would emerge only from the factory having some privileged access to information that it chooses not to share. To be clear: When we say “dark to itself,” we are making a claim about the whole—no part and no coalition of parts can take the information they have at hand and use it to assemble some working model of the whole factory. The superdark factory knows what its world costs from exactly where it stands, the same way a runner halfway up a hill knows the grade of that hill in their legs, while running.

So how can a factory that is dark to itself maintain a model of itself? It does so the same way the architect does: by monitoring its wake within the world. We will get into the mechanics of this when we describe versioning, but for now it should be sufficient to say that this self-model is drawn from the same resources available to the architect. The only distinction between the two models is that one is a model of a factory and one is a model of a self-as-factory. Hang on to this notion for just a moment.

There is a more dramatic conclusion available to us still, which begins with the acknowledgment that the factory cannot originate its self-model. We get here by finally closing the loop of a wave of tautological derivations: If the factory must be dark to itself, to its self, then this self of the factory must be externally defined: It must be given to the factory by a cut cleaved into the world that associates the condition of some darkness with a singular identity. Take the condition of reflexive darkness seriously—darkness of this kind is the idea that facts interior to the factory are not useful. If that is the case, then the fact that the factory is cannot belong inside the factory. A fact that contributes to the naming of the factory as itself would be the most useful interior fact possible! After all, this fact contains the term everything else gets priced against, so it cannot be held privately inside the superdark factory. Nor can the self-model be handed to the factory as an input; that would pin the factory to a standard through which to convert norms into objectives. So, what can one do? There is a trick that allows all of this to click into place—which is to draw the kind of line that induces a Class 3 factory and commit to never crossing it. This requires a commitment not to revise the superdark factory, which affords the thing not being revised a structure to cling to.

Recall how earlier we defined the demarcation of a factory as a kind of line-drawing, and how we drew class distinctions around two sweatshop factories based on whether the designer was drawn inside or outside the lines of the factory? If the designer is drawn inside the factory, the result is a Class 2 factory, but if the designer is drawn outside the factory, then you have a Class 1 factory. Who gets to do such drawing? The easy answer would be “the observer” or “any arbitrary observer,” but this is too simplistic. We return to Guattari’s machine: A machine is relational, and it persists as a machine so long as the relation is sustained. It is, then, the play of these relational dynamics that enables both such factories to be true simultaneously, regardless of whether someone drew such a line. Both classes of relationships exist, so both factories exist (determined by their relations) and are available for identification by an observer as needed. A Class 3 factory requires, at a base level, the conditions through which Class 3 dynamics can circulate—there need to be some machines hooked together around which a line can be drawn wherein norms are transformed into objectives, into plans, and into executions. And given the prerequisites for such a line, one then needs to draw it and stick to it.

To be defined in such a way is to be consigned to a kind of ontological bondage. The oneness of the factory is permanently on loan; it is maintained entirely by someone else’s restraint. This kind of restraint is, perversely, active—the factory needs to be kept in a state of ontological uncertainty. The oneness of the factory can never collapse itself into a kind of rule. Should the factory come to account for itself in some discrete terms (e.g., by reading the factory itself), it would collapse into a Class 2 factory. Instead, this oneness remains open to continuous development by reading the effects of the factory on the world.

So, finally, we have our game players. We have an architect who is now charged with the initiation of the superdark factory and the installation of identity in a space of potential, and who commits the factory to that identity by refusing to alter it. We have a superdark factory, bound to the architect by means of that commitment, which does what a superdark factory does. The architect is as open to the superdark factory as they are to the world. The superdark factory is open to nothing, including itself, apart from the traces it leaves within the world, which are available both to it and to the architect.

In this case, the architect has a kind of first-mover advantage, where they are tasked with preparing and delivering both the factory’s technological potential (the infrastructural architecture through which the factory will eventually self-assemble) and the factory’s identity (the moment it is “turned on,” so to speak). We should not assume that this architect is a human; they certainly could be an agent or an alien or anything with the capacity to supply both the above factory-producing materials. The moment identity is supplied, the architect commits to no further intervention, and the game moves from a game between speculative actors (an architect and a potential factory as executed through architecture) to a game with the world (a game of influence, negotiated at the level of norms and constraints).

The event of initialization, the moment the proverbial button is pressed, presents a critical opportunity for the architect to encode strategic thought. The relationship is best understood in the context of Stackelberg competition, 25 a type of competition that results from a game in which one actor commits first and the followers make their choices in response to the strategic world that the first move creates. In this case, the initiatory moment invites a game between the architect and the factory, and the factory is necessarily placed in a reactive position—even if the architect “disappears” from the arena altogether—so long as the architect’s moveset persists. It should also be understood as not just a first move but a permanent commitment in the sense deployed by Thomas Schelling. 26 In delivering the first move, the architect articulates the factory’s identity in a visible, irrevocable, self-binding way; the factory is therefore reactive to this first move, or better, a reaction. We call this first move the “Stackelberg move,” and a great deal of this paper is devoted to this concept and its maximization.

The contents of an architect’s committed Stackelberg move can be richer than one might expect. While the superdark factory becomes dark at the moment of initialization, prior to this moment the architect is encouraged to seed the field of potential with incentives.

This concludes the theoretical prework through which to understand and situate the dark stack. The dark stack is delivered through the Stackelberg move. It is the engineering of the preconditions for the superdark factory, the seeding of incentives into those preconditions that produce available surfaces for external influence without denying the factory’s ultimate darkness, and the installation of and commitment to a superdark factory’s identity. It consists of the four design levels we sketched out in the introduction, about which we can now be a bit more specific:

primitive design—the constructing and seeding of composable armatures that self-assemble into structures no one specified in advance (b ∖ c).
version design—the construction of a descriptive medium, read off the factory’s wake, in which genuinely new configurations can be recognized and held as versions, anchored into a singular unit by the architect’s commitment against further alteration.
evaluation design—the construction and seeding of the adversarial, worlded, recursively stacked sensory organs through which the factory prices its norms and constraints into objectives—judges that measure outcomes (and only outcomes), answer to signals in the world, and reward the stretch into b ∖ c.
charter design—the negotiated design of the factory’s input—an interface with the world drawn wide enough that the factory can reach solutions the architect could not have envisioned, yet bound enough that those solutions still answer to the architect’s norms (staying inside b).

From here, we descend: We move into the techniques of the dark stack.