Strolling into World Labs—the San Francisco startup cofounded by Fei-Fei Li, recognized in tech circles because the “godmother of AI”—guests are met with a wall tiled in rainbow hues. A grand piano beckons on the finish of the deep, sunlit foyer. Chair-size iridescent spheres relaxation towards the partitions, as if a large forgot to select up his toys. However across the nook, the partitions and flooring look scrambled, like a cubist portray not noted within the rain. Fortunately, this “world” is only a simulation of the World Labs foyer I’ve constructed utilizing Marble, the corporate’s generative AI app.
My actual tour had taken place per week earlier on the firm’s precise places of work, the place actuality—together with the rainbow wall, spheres, and piano—stayed mercifully intact. “Marble is the primary a part of [our] journey—it’s not a totally matured mannequin,” CEO Li had advised me, carrying a dishevelled maroon World Labs hoodie and settling her 5-foot-3 body right into a chair in a comfortable wood-lined convention room.
A world-renowned AI scientist recognized for her self-effacing demeanor, Li is being characteristically modest about Marble’s capabilities. The app, launched final November, makes use of a brand new kind of AI referred to as a “world mannequin” to generate interactive 3D replicas of any area—actual or imaginary—from visible or written prompts. These “worlds” might be something, from floating castles and sci-fi vistas to, effectively, a sure startup’s entryway.
My outcomes have been a bit off, however Marble nonetheless conjured the entire thing in a matter of minutes from a couple of photographs I haphazardly snapped with my iPhone. In case you truly know what you’re doing, Marble’s world mannequin can produce 3D environments wealthy and expansive sufficient to make use of in digital movie manufacturing, architectural design, and robotics coaching—all domains through which World Labs already has paying prospects.
Li gained’t touch upon present income or engagement metrics; one of many firm’s cofounders, Ben Mildenhall, says that Marble remains to be in a “postlaunch discovery section” after coming out of beta final November. World Labs provides tiered subscriptions to people, however Li says she envisions its main enterprise mannequin as “closely B2B,” with corporations paying to pipe Marble’s capabilities into their very own merchandise.
But it’s the world mannequin beneath that actually excites Li—and World Labs’ traders, which embrace Andreessen Horowitz, AMD, Nvidia, and Autodesk.
The marketplace for world fashions
World Labs emerged from stealth in September 2024 with no product, $230 million in funding, and a billion-dollar valuation. Final February, 4 months after asserting Marble, which Li nonetheless refers to as “a proto-product,” it raised a billion {dollars} extra—$200 million from Autodesk alone.
Why? As a result of AI, only a few years after LLMs reminiscent of ChatGPT and Claude started remaking the worldwide financial system, is already looking for its subsequent act. Li believes it should middle on what she calls “spatial intelligence”: AI that may transcend next-word prediction and find out how the true world works. Contemplate all of the data you leverage simply to traverse a crowded sidewalk: distance and motion, time and area, trigger and impact. It’s stuff a 3-year-old can grasp however nonetheless stumps your common trillion-parameter chatbot.
World fashions (so the considering goes) will use “multimodal” coaching knowledge drawn from movies, photos, industrial sensors, and digital simulations to unlock this extra expansive intelligence, enabling the whole lot from totally self-driving automobiles and autonomous robots to AI-powered factories and laboratories. Rev Lebaredian, Nvidia’s vp of Omniverse and simulation know-how, advised the Monetary Occasions final 12 months that the potential marketplace for world fashions may very well be $100 trillion, or roughly the scale of the whole international financial system.
If that sounds just like the equal of claiming “infinity {dollars},” PwC issued a barely much less galaxy-brained estimate in March, pegging the full marketplace for “bodily AI” (its synonym for “spatial intelligence”) at $503 billion by 2030, a big portion of which may very well be addressed by world fashions.
“Spatial intelligence, to me, is the North Star,” Li says. “And a world mannequin is a method [to get there].”
Competitors within the spatial intelligence subject
She’s hardly alone in her quest. The identical week I visited World Labs, Nvidia held its annual GTC convention in San Jose, the place CEO Jensen Huang proclaimed that finally, through its personal newest world fashions—Cosmos 3 and GR00T N2, introduced in March—“each industrial firm will develop into a robotics firm.” Google DeepMind has developed a video-based world mannequin, too, referred to as Genie 3, which it makes use of as a coaching instrument for Waymo’s self-driving automobiles and sees as “a stepping stone” towards AGI, or synthetic basic intelligence. Elon Musk claims that world fashions are “important” to coaching Tesla’s Optimus humanoid robots; xAI poached world-model specialists from Nvidia final summer season.
In the meantime, smaller gamers from disparate industries are elbowing in. Runway, an AI video-generation startup that has a partnership with Hollywood studio Lionsgate, hit a $5 billion valuation in 2026 by pivoting to world fashions. Normal Instinct, which raised $134 million final 12 months from traders reminiscent of Khosla Ventures (one among OpenAI’s early backers), is utilizing billions of online game spotlight reels to construct a world mannequin for AI brokers.
And Yann LeCun—one among three different so-called AI “godparents” (along with Nobel Prize winner Geoffrey Hinton and Yoshua Bengio, a College of Montreal pc scientist)—give up his publish as Meta’s chief AI scientist final 12 months to launch his personal world-model startup, AMI Labs. He, too, has raised a billion {dollars} in funding.
In a panorama this scorching and crowded, what units World Labs aside, apart from having an precise product, as Li dryly factors out, is the corporate’s dedication to what she calls “human-centered AI.” Whereas different world-model corporations like Normal Instinct and Tesla seem to chase automation über alles—“autonomous management of each spatial workload,” as described to me by Nicole Fraenkel, a associate at VC agency Khosla, which invests in Normal Instinct—World Labs needs its AI to be a copilot, not an autopilot.
“A few years in the past, I stored listening to this phrase ‘infinite productivity,’” Li says, referring to the ever-inflating expectations round AI. “I don’t know tips on how to react to [that]. We’re making an attempt to construct a rational enterprise. We consider Marble can reply a ache level.”
That may be the persistent excessive value—to not point out tediousness—of making 3D simulations, significantly in leisure, media, and robotics. However moderately than having its world mannequin swallow the duty entire, World Labs has made a Photoshop-like instrument aimed toward creatives, builders, and researchers: professionals who need to do their jobs higher with AI, however nonetheless, you realize, do them.
Perhaps that sounds quixotic as a technique, however Li’s most outstanding backers are shopping for in. “It’s our job to create AI that could be a associate—this is a vital a part of our ethos. Fei-Fei understands this intimately,” says Autodesk CEO Andrew Anagnost. Even Martin Casado, a basic associate at Andreessen Horowitz (the enterprise agency whose cofounder Marc Andreessen blithely claimed final 12 months that one of many solely jobs protected from AI automation was his personal), agrees. “The imaginative and prescient is so clear. We perceive the way you construct a enterprise on prime of this,” he says.
Li acknowledges the second that World Labs finds itself in. Three years in the past, LLMs redefined what AI might do; now world fashions look poised to do it once more. However whose model—and imaginative and prescient—of them?
“I’m keenly conscious of a number of clocks,” Li says, in her gentle however steely voice. “They’re all ticking.”
World Labs’ consistency concept
LLMs, as everyone knows by now, are “ebook good” at finest—they ingest and manipulate phrases, hallucinating as they go. World fashions, as defined to me by Bengio (sure, he’s creating a world mannequin of his personal), goal to seize one thing each bigger and extra strong: “coherence with the true world.”
For a self-driving automobile, this may imply understanding {that a} cease signal nonetheless means “cease” even when it’s hooked up to the aspect of a college bus. (Waymo has trouble with this.) For robots, it would imply reasoning about trigger and impact earlier than appearing. Stepping over a ditch, for instance, normally makes extra sense than entering into it. A world mannequin means not having to search out out the arduous method.
At World Labs, a coherent world mannequin begins with one thing easier, however arguably extra necessary for AI to get proper: the 3D consistency of the world itself. Iridescent spheres that keep their dimension and form as you progress round them. Pianos that keep put. This fundamental spatial consistency “is so basic—it’s the linchpin that connects notion with motion,” Li says. It’s the proper basis, in different phrases, for World Labs to start out constructing an AI that goes past ebook smarts.
Li had already been an AI-research veteran for practically 20 years—with stints at Princeton, Stanford, and Google—when she bought the concept to discovered a world-model firm in the summertime of 2023.
ChatGPT was barely six months outdated, and Silicon Valley was swooning for LLMs. Marc Andreessen was totally satisfied that language-based AI would “save the world” (as he put it in his 7,000-word manifesto on the topic). At a lunch he had convened for “AI luminaries” at his dwelling—the place, as Li recollects, “everybody was speaking about LLMs”—she discovered herself in a nook whispering to Casado, a network-security knowledgeable who oversees Andreessen Horowitz’s infrastructure follow.
“On the time, it was a really, very contrarian place to consider that language wasn’t all you wanted,” Casado recollects. “Fei-Fei leans over to me and says, ‘You already know what someone must work on? A world mannequin.’ I had come to the identical conclusion, however not practically [with] the depth that she [did].”
Li already had in thoughts her former scholar Justin Johnson, who she says was “getting headwind” on spatial AI as a researcher at Meta, as a cofounder. Casado launched her to Mildenhall and Christoph Lassner, each consultants in utilizing AI for 3D graphics. In November 2023, World Labs was born.
For Li, the corporate is the results of “a career-long conviction” that imaginative and prescient lies on the coronary heart of synthetic intelligence. Again in 2012, early in her tenure as a researcher at what’s now generally known as Stanford’s Imaginative and prescient and Studying Lab, her ImageNet dataset helped neural networks efficiently study to acknowledge tens of millions of images of objects, kicking off the “deep studying revolution” (as AI was referred to as earlier than ChatGPT got here alongside). However Li additionally believes that seeing is vital to intelligence, interval.
“Simply shut your eyes,” she says. “Take into consideration the typical day and the way a lot you depend on your spatial intelligence”—the truth that your visible actuality stays intact, as an alternative of dissolving into hallucinations. We take it with no consideration, however this cohesion is precisely what Marble’s world mannequin is skilled to provide.
And it’s more durable than it sounds. DeepMind’s Genie 3, as an illustration, can’t keep spatial consistency for longer than a few minutes. However my humble duplicate of the World Labs foyer will grasp collectively so long as the corporate’s servers do.
Going for “Gaussian splats”
Each Marble and Genie depend on the identical underlying know-how as LLMs—generally known as the “transformer structure.” (Marble additionally makes use of different strategies that Li declined to debate intimately.) However they take vastly completely different approaches.
Genie and different video-based world fashions use AI to provide a stream of recent frames based mostly on those that got here earlier than, they usually shortly run out of steam due to value; creating these photos is the computational equal of setting cash on fireplace. Sora, Open AI’s transient foray into world modeling, was reportedly burning by $15 million a day earlier than the corporate killed off the app in March.
Marble’s main output, nevertheless, isn’t even footage. It’s “Gaussian splats” (sure, you learn that proper): overlapping blobs of math that encode how a scene seems to be from any angle. The simulation I made in Marble accommodates about 300,000 splats. As soon as the world mannequin squirts all of them into place, no extra should be generated, no matter what number of instances the view adjustments. They keep put—similar to that piano.
“All the pieces Marble does is as a result of we’ve [this] permanence,” says Li. For now, it’s a bonus that video-based world fashions—i.e., most of them—can’t match.
Online game engines and digital twins have provided high-fidelity, persistent 3D simulations for years. However they should be painstakingly arrange by hand, at vital expense—sometimes between $30,000 and $40,000 for a single scene, based on Casado. Marble can create a 3D scene for lower than a greenback, in minutes moderately than days. “It was straightforward to point out that particular use case as having worth,” he says.
World Labs’ prospects agree. Marble “genuinely accelerates part of our course of that was once a lot slower or purely summary,” says Hugues Bruyère, cofounder and chief technologist of Dpt., a Montreal-based inventive studio that designs interactive and immersive experiences for purchasers like Bentley and Radisson Resorts. “That’s an actual shift.”
Bruyère at the moment subscribes to Marble’s highest pricing tier: $95 per thirty days. World Labs gained’t specify how a lot it expenses for personalized enterprise subscriptions, however on condition that high-end enterprise plans from Anthropic and Salesforce can value upwards of $400 per thirty days, World Labs might be able to cost enterprise prospects a whole bunch of {dollars} above its off-the-rack subscription costs. And Li’s imaginative and prescient for World Labs as an enterprise operation is already underway.
OpenArt, a generative-art platform launched by ex-Googlers in 2022, started providing 3D world-generation (through World Labs’ API) to its neighborhood of 8 million month-to-month lively customers earlier this 12 months. Preview.io, a Sequoia Capital–backed startup that gives AI-powered movie manufacturing instruments to Fortune 100 corporations, makes use of Marble’s spatial management to assist producers meet demanding briefs from purchasers within the automotive business. And Lightwheel, which designs digital “rooms” to coach AI-powered robots, credit World Labs for altering its workflow in a single day.
“We have been utilizing the identical kitchen, the identical lounge, the identical warehouse time and again,” says John Stephens, Lightwheel’s chief evangelist. With World Labs’ API, “we will generate a thousand rooms a day.”
For a frontier analysis lab, World Labs appears effectively on its solution to product-market match. Nevertheless, Li can’t assist however point out another excuse she selected “pixel lovers”—her identify for the sort of expertise she hires and sells to—as Marble’s main clientele.
“The inventive neighborhood,” she says, “is extra tolerant [of] the mannequin not being nice but.” That’s nothing towards artists or designers, “and it’s not for lack of ambition” at World Labs, she provides. Robotics, healthcare, and even scientific discovery are all on her highway map. However within the content-creation enterprise, “no one will get killed when your mannequin is just not ok.”
The “messy center”
Li describes herself as a “techno-optimist.” Nevertheless, she avoids utilizing the time period “AGI” (Why? “As a result of I’m a scientist,” she says flatly), and he or she worries about folks: Her school-age children. Her employees. Her educational colleagues. It’s not a matter of what AI might “do” to them, however what they could or could not be capable of do with AI—due to an more and more polarized international dialog that relentlessly facilities on know-how as an alternative of the people it’s supposedly for.
She’s not even keen on the “godmother” moniker she earned from the pioneering computer-vision analysis she began at Princeton in 2006 and continued at Stanford in 2009.
Raised in Chengdu, the capital of China’s Sichuan province, Li was inspired by her unconventional mother and father to chase butterflies and pore over English literature, as she wrote in her 2023 memoir, The Worlds I See. At 16 she immigrated to Parsippany, New Jersey, the place she entered public highschool realizing little English—and left with a scholarship to Princeton, which she attended whereas working her mother and father’ dry-cleaning enterprise on weekends.
After ImageNet later cemented her repute as a researcher, Li took a 21-month sabbatical to steer AI at Google Cloud earlier than returning to Stanford in 2019 to cofound its Institute for Human-Centered AI (HAI), which she says she continues to be “strategically concerned in.”
It was there that she and her collaborators invented the time period “basis mannequin” to explain the already-looming social penalties of generative AI. Below her steerage, HAI additionally labored with Congress to go the Nationwide AI Initiative Act in 2021, which led to a pilot program (supported by each the Biden and Trump administrations) that deployed $100 million in AI computing assets throughout 14 federal companies.
These days, Li could discover herself closing billion-dollar funding offers, however her views on AI haven’t wavered. Final 12 months, Anthropic CEO Dario Amodei made headlines by issuing doomerish warnings about AI wiping out data work, whereas Elon Musk assured credulous traders that AI and robots would “eradicate poverty.” Li, in distinction, used her keynote deal with on the AI Motion Summit in Paris to urge the worldwide policymakers in attendance (together with Emmanuel Macron, JD Vance, and India’s Narendra Modi) to steer their choices utilizing “science, not science fiction.” It made far fewer headlines.
“She says it’s about being within the messy center” of the is-AI-good-or-bad debate, says HAI government director Russell Wald. “The center isn’t essentially enjoyable, as a result of it’s simpler to be on these polarizing sides.”
It appeals to traders, although. Li’s ideas are an enormous a part of what attracted Autodesk’s $200 million funding in World Labs. “She has this model round caring deeply in regards to the human-centricity of AI,” Autodesk CEO Anagnost says. “There’s a lot of hyperscalers on the market which might be world fashions. You need different forces within the ecosystem. You need somebody passionate in regards to the know-how and its implications. These issues collectively led me to maintain chasing, chasing, chasing her.”
(Li acknowledges that her personal stance on AI doesn’t essentially lengthen to everybody who could use World Labs’ know-how, or world fashions usually. “Humanity is messy,” she says. “We have now to stay with hope in addition to our darkish underbelly, and attempt to do the most effective [we can].” When requested if World Labs would ever probably reject a buyer or associate on moral grounds, she declined to reply instantly, stating solely that “human-centered AI is a framework, not a method to an finish.”)
Anagnost seems to be ahead to the day when spatially clever AI can remedy the “clean slate drawback” very similar to ChatGPT does, apart from designing buildings as an alternative of writing time period papers. He can envision a “hyperautomated manufacturing unit” the place a next-gen world mannequin might let “you perceive full utilization of the area, reconfigure it quickly, and monitor it over time.” Within the meantime, Autodesk plans for “interoperability” between Marble and Stream Studio, its 3D modeling suite for leisure and visible results.
Fei-Fei Li’s benefit
That is precisely the sort of discuss that different world-model traders don’t have any endurance for. Khosla Ventures’ Fraenkel is enthusiastic in regards to the hyperautomated half, however skeptical bordering on dismissive about Gaussian splats being a solution to get there. In her view, merchandise like Marble are “simply an costly digicam” and “in the end economically ineffective”—a distraction from world fashions’ true endgame.
“Brokers and robots, they don’t want a prettier image of the world,” Fraenkel asserts. “They should perceive what occurs in it after they act. That is the prize.” On this planet-model race, corporations that go “straight for the jugular, straight for autopilot”—just like the Khosla-backed Normal Instinct, or Tesla—could have the benefit, she says.
Even these enjoying on World Labs’ crew acknowledge that its method has dangers. The corporate’s world mannequin wants extra coaching to maneuver past fundamental spatial coherence—compute and knowledge are a part of what the billion {dollars} is earmarked for—however “we don’t even know if the structure is correct,” says Casado, referring to the underlying technical design of the world mannequin. “Actually, no one’s constructed one earlier than.”
Li gained’t disclose precisely what combine of knowledge World Labs intends to make use of, or the place it’ll come from, however she acknowledges that world-model coaching knowledge isn’t precisely straightforward to return by: “There’s no web of textual content [for this]. There’s a considerable amount of knowledge we’ve to obtain.” As a small startup, World Labs might additionally battle to draw “good expertise” to its idiosyncratic mission, she says. She even concedes that rivals may merely find yourself shifting quicker.
Nonetheless, Li has confronted lengthy odds earlier than. Her ImageNet venture is lauded because the cradle of contemporary AI now, however when she began, it was seen as so audacious it might have jeopardized her pathway to tenure, based on HAI’s Wald.
Li cites one other supply of what she considers her “entrepreneurial shamelessness”: “I’m an immigrant,” she says. “My mother and father and I constructed a life right here. I’m used to being humbled. I’m very snug on floor zero.” If each tech startup wants an “unfair benefit”— Silicon Valley’s go-to phrase for a novel aggressive asset—World Labs’ may simply be its personal diminutive, decided CEO.
After all, Li would by no means say so herself. “All people tells me I’m too quiet,” she says. “My buddies, my traders, my colleagues.” At first she didn’t even need to give an interview for this story, however her crew twisted her arm. “I don’t stroll round considering, ‘I’m Fei-Fei Li,’” she says. “I at all times construct for a mission.” Her imaginative and prescient for World Labs—and its human-centered future—is each constant and protracted. It’s like a simulation in her world mannequin. As soon as in place, it stays put.
