As author Ted Chiang has noticed, “Most fears about A.I. are finest understood as fears about capitalism.” Once we think about rogue AI programs optimizing the world to dying, we’re really describing what many companies already do. If a company’s ethos is misaligned, any know-how platform it creates will amplify that misalignment.
Take a second to mirror on the group you maintain most expensive: Does it have an ethos, or is it drifting with out one? Does it combat for human flourishing or in opposition to it? Is it creating worth or extracting it? Are you shaping its ethos, or is it shaping yours? Regardless of your formal title, you’ve gotten affect over this group. But for those who can’t reply these questions with confidence, you aren’t in management. You’re a passenger in a car you suppose you’re driving.
But when no person is consciously driving, the place is the group going? If the conduct of superorganisms had been random, we might see as many evolving towards flourishing as towards corruption. As a substitute, the overwhelming majority wind up in the identical place, with indistinguishable values.
I just lately had a dialog with the CEO of a multibillion‑greenback firm who’d come to see me for assist with a baffling drawback. He had made a daring new wager on the promise of AI to revolutionize his enterprise. He’d personally overseen the event of a brand new AI product and examined it with clients. They liked it. Glad that he had created one thing superb, he turned it over to the remainder of his workforce to commercialize and scale.
Six months later, this new product was going nowhere. The gross sales workforce wasn’t promoting it. Builders had been loath to work on it. Marketing wouldn’t put it up for sale. When confronted, the executives would insist that the product was a part of their quarterly priorities. Every of them was telling their groups to deal with it. But, one way or the other, line workers and center managers had been persistently avoiding the AI product. It was a sample all through the group.
When the CEO personally intervened, he might simply get particular person clients to undertake the brand new product. However he couldn’t get his workforce to do the identical, even after investing extensively in coaching, efficiency bonuses, and prime‑down directives. He couldn’t establish anyone individual or group actively blocking the AI initiative. He even changed a number of key executives with new ones explicitly chosen for his or her enthusiasm for this new route. And but, inside months, the uneasy establishment prevailed.
He shook his head. “ what’s driving me loopy? I based this firm fifteen years in the past. Once I first proposed the core thought, everybody thought I used to be nuts. Traders, potential workers, even my household.
“Now that we’re profitable, after I counsel we have to pivot, most of the identical individuals, together with my very own management workforce, resist as a result of ‘we will’t danger what we’ve constructed.’ ”
He threw up his fingers. “They’re defending one thing that exists solely as a result of I took the ‘loopy’ danger all of them thought was unattainable. Now they suppose I’m reckless and might’t appear to understand what number of instances they’ve benefited from my pushing by way of the unattainable to get us to new success.”
He might really feel the stress his insistence was placing on his workers, he instructed me, as in the event that they had been being torn between two authorities. However who, precisely, was this different authority? Like so many founders, he had assumed that being in cost meant the group would mirror his values. How might or not it’s that he had all of the management on paper however so little in day by day follow? And if he wasn’t in management, who was?
John Steinbeck noticed this predicament clearly in The Grapes of Wrath. Tenant farmers are being evicted from their land in the course of the Mud Bowl, however once they confront the lads finishing up the evictions, they’ll’t get a transparent reply as to who’s making the choice to devastate them. The lads despatched to do the soiled work reply to the financial institution supervisor, who solutions to distant homeowners, and people homeowners reply to shareholders. Every of them has a task within the group, however one way or the other none of them is in cost.
This results in the farmers’ bewildered query: Who’s the financial institution?
Steinbeck portrays it as a faceless system by which no particular person could be held accountable, regardless that actual individuals are dropping their properties and livelihoods. He calls the financial institution “the monster. Males made it, however they’ll’t management it.” My pal the CEO, grappling with resistance he couldn’t find, skilled the identical bewilderment. If the individual on the prime isn’t in cost, who’s?
The CEO I used to be counseling wasn’t preventing his personal individuals on the AI initiative. He was wrestling with a unique sort of residing being—one which had developed its personal character, its personal preferences, and its personal will. His private ethos of innovation was at battle with the group’s emergent character, which craved security, certainty, and predictability. The truth that he had created it mattered little or no.
Life is outlined not by flesh and blood however by particular properties. Dwelling issues preserve boundaries between themselves and their atmosphere. They metabolize sources into power. They develop, adapt, and reproduce. They exhibit behaviors that emerge from their elements however can’t be predicted by finding out these elements in isolation. Most significantly, they show a will to outlive that shapes each motion they take.
Organizations exhibit each one among these properties. They preserve authorized and cultural boundaries that outline who’s inside and who’s out. They metabolize capital, expertise, and uncooked supplies into services and products. They develop by way of hiring, adapt to market pressures, and spawn subsidiaries or merge with others. An organization’s “tradition” emerges from hundreds of day by day interactions however can’t be present in any worker handbook or org chart. And when threatened, organizations exhibit a tenacity that may override the preferences of any particular person inside them—even their nominal chief.
Every one is a superorganism unto itself, working in response to its personal guidelines of existence.
Now we will reply Steinbeck’s query, Who’s the financial institution? An emergent intelligence, each bit as alive as you or me. A superorganism.
That is who the financial institution is.
Once we inform horror tales about AI—programs that optimize ruthlessly towards a slender objective, blind to human prices—we’re, in some ways, describing the financial institution. Males made it, however they’ll’t management it. Our fears about AI are, as Chiang suggests, usually fears that we are going to pour unprecedented technical energy into establishments whose ethos is already misaligned with human flourishing.
If we wish completely different outcomes, we can’t simply ask what our machines are optimizing for. Now we have to ask the identical query of our organizations.
Excerpted from Incorruptible by Eric Ries, revealed by Authors Fairness. Copyright © 2026 by Eric Ries. Reprinted by permission of Authors Fairness.
