Your personal agent is your personal computer

In the past month, the personal AI assistant space has blown up. In fact, there are enough products in the category that someone created Assistant Benchmark to make sense of what each one is good at.

I’ve had many conversations with folks trying to make sense of the intense competition and wondering who ultimately wins in the space. Many seem puzzled about the hype of personal agents and skeptical about the potential value of these products.

I think their everyday usefulness is part of how these products could end up earning a much broader role in our lives. Today, personal AI assistants help with something we need done and give us a reason to try using them. But my bet is that personal agents could eventually become the interface around which we choose devices and access services.

For decades, we’ve built habits around going to our computers or our phones, choosing applications, and figuring out how to use them to enter the digital world. I’m interested in the behavioral shift in beginning that journey with an agent. I believe the future of human-computer interaction (HCI) pushes devices back into a role of facilitating how we interact with agents, and software around these agents extends what we can do.

How the device became the center

In 1981, IBM introduced the PC into an existing personal-computer market, but with a lower price point that could help bring it into the mainstream.

The IBM PC could connect to a television and enabled users to process text and play games. It streamlined business operations, spurred the development of the software industry and vaulted computing into the mainstream.

With a PC, people had the tools to enter the digital world together on a single machine, with software increasingly giving people more reasons to spend time there. The machine became where we went to write, play, make something, or figure something out.

Throughout the 90s, the growth of the Internet and the web connected that experience to a much larger world. In the late 2000s, smart mobile devices took off and turned computing into something we could carry around in our pockets. Apple’s 2007 iPhone introduction and subsequently, the App Store’s arrival in 2008 increasingly made our phone screens a place for entirely new kinds of digital activity. We had mobile apps as an extension of websites, but also new businesses and ways of connecting the digital world to the physical world, like Uber and DoorDash.

Alongside this shift, we developed the infrastructure in cloud services to loosen the connection between our digital lives and any single machine. iCloud made notes, photos, files, and other data available across devices signed into the same account. An iPhone, Mac, and Apple Watch could become parts of a connected experience, with features such as Handoff letting us continue an activity on another device. Our digital “stuff” was already following us around, even as we continued to reach for particular devices and applications to use it.

Even the idea of computing beyond one primary machine has precedent within the history of HCI. Mark Weiser explored this in his 1991 essay The Computer for the 21st Century, describing a form of ubiquitous computing where personal computers are pushed into the background and many interconnected devices in different scales are woven into everyday life.

Though Weiser’s vision didn’t name an explicit agent as an organizing entity, this is what I’m now exploring as the path towards a distributed computing environment.

How the agent could become the center

Even with connected devices and cloud services, we still do much of the work of translating what we want into actions that software can carry out. We might open a browser to find something, pull up an application to make something, or use a messaging service to talk to someone.

A personal agent changes this paradigm by becoming the new starting point, not the device. We could bring our agents a question, an idea, or something we want to happen. Our agent can determine which information, tools, and services it needs to help us. Sometimes that would mean working through something together, with an agent bringing us an interface we could use directly. Other times, we could completely delegate an interaction and come back when there is a decision to make. We get the chance to participate in the digital world without personally showing up for every step.

HCI researchers have debated this division of responsibility for decades. In a 1997 debate, Pattie Maes argued that agents will help navigate increasingly complex software, while Ben Shneiderman questioned whether interpreting someone’s intentions could still offer the predictability and control of direct manipulation on an interface.

Maes speculated:

“If untrained consumers are to employ future computers and networks effectively, direct manipulation will have to give way to some form of delegation. Researchers and software companies have set high hopes on so-called software agents, which "know" users' interests and can act autonomously on their behalf. Instead of exercising complete control (and taking responsibility for every move the computer makes), people will be engaged in a cooperative process in which both human and computer agents initiate communication, monitor events and perform tasks to meet a user's goals.

The widespread dissemination of agents will have enormous social, economic and political impact. Agents will bring about a social revolution: almost anyone will have access to the kind of support staff that today is the mark of a few privileged people. As a result, they will be able to digest large amounts of information and engage in several different activities at once. The ultimate ramifications of this change are impossible to predict.”

Maes was early in predicting this future, which required the LLM-based intelligence to power today’s agents. But it is increasingly clear that she was not wrong in saying it’d be too difficult to predict how things change because of agents.

I take her side of this future view, imagining a world where I encounter my agent in many forms: a Tamagotchi-like pocket device, glasses, ambient speakers, phones, or computers. Each form gives us a different way to interact: glasses could let me ask my agent about something while I’m looking at it, while a big screen lets me examine and change something we’re making. In this paradigm, I think of the device as a particular embodiment of the agent — with its particular sensors, controls, and physical presence shaping how we can collaborate.

Applications and services could evolve to treat agents as primary users, because it’s essential my agent knows how to use them instead of me. I may still choose to enter a dedicated UI when I want its particular experience, but this is not a hard requirement. More of my computing could begin with what I want to accomplish, with an agent helping me choose the means along the way.

What Meta is building with Muse

When Meta debuted Muse, I initially compared it to Instinct and other personal agent harnesses by assessing their assistant-specific capabilities. But as Meta has rolled out more functionality, I think the broader product ambition is becoming clearer in way that strengthens my belief about this new computing paradigm.

Muse is a personal agent with its own cloud computer that you can talk to via the app, the web, WhatsApp, and even ESP32 devices.

It can use websites and connected services to carry on with work even when the app is closed, you’ve shut down your laptop or put away your phone. You can initiate an activity from one device and have it continue even when your interaction with that device ends.

In Meta’s design walkthrough, there are concrete product decisions that form the new Information Architecture (IA) around this paradigm: conversations have persistent memory, side chats can be spun up for separate topics, goals give plans and progress somewhere permanent to live even when you don’t explicitly define them, activity shows the progress of ongoing work, and artifacts house documents beyond the context of chat.

When I look at Muse as a product, I’m reminded of Eric Horvitz’s mixed-initiative interfaces view: automated assistance with direct manipulation, the best of both worlds for a user. Muse has places to ask for something, inspect what’s in progress, and work directly with a result. You can move between those modes with ease and it feels more like a computing environment than a succession of requests and responses from a chatbot.

To take this even further, Meta also announced Muse on AI glasses and the pocket device Muse Charm at Connect on September 23. Meta has also released Muse Gadgets, open-source tools for connecting Muse to devices built with ESP32 boards or Linux hardware such as a Raspberry Pi. Developers can connect displays, buttons, audio, and sensors, making it possible to build their own physical interfaces to the agent.

When I saw these drops, I started anticipating how Muse will be embodied in new form factors that shape our behavior: a pocket device for quick conversations, glasses to help it easily maximize context-gathering for whatever I’m seeing. I’m excited about this direction because I can imagine my agent becoming more capable, without requiring as much effort from me.

The platform taking shape around Muse

To me, Meta is laying out a vision for a new human-computer interaction paradigm with the agent at the center. We reach Muse through conversation, but the device we use can vary with the situation. What Muse does is also a sub-delegated task, whether it reaches for my computer, its computer, or some other external service outside of Meta. It all starts with me going to my agent, which then dictates how surrounding software gets used.

Muse’s service connections to 3P players further strengthen the platform premise. These connections can make the agent more useful beyond what Meta is building. In the Connect announcement, Meta detailed many additions spanning retailers, payments, and work services like Walmart, PayPal, Notion, and GitHub.

OpenAI is pursuing a similar direction with Dots and its plugin ecosystem. Its announcement describes a primary dot with a cloud computer, the ability to learn from feedback, and access to connected applications. I see both companies trying to establish a relationship through which people can use capabilities far beyond the providers’ own products.

Personal agents that can interpret what we want can influence which services we consider, what data these services get from us, and how their offerings reach us. Agent providers gain a valuable role in directing activity across products they don’t own, simply because people have chosen their agent as the front door to using software. This is a critical part of a platform play: as more people begin with an agent, other companies have a direct incentive to build services and devices around it, expanding what people can do through the relationship with their agent. The value then compounds through platform dynamics (as Ben Thompson describes).

What are these products competing to become?

Think of assistant tasks as a wedge for personal agents on their way to becoming your new personal computer. They give a broad range of people a reason to try agents within today’s computing habits, without asking them to completely reimagine how they use technology. And as people discover enough value to bring the agent to more tasks, the broader role can emerge from these experiences.

Over time, I’d look for whether people feel like they are reaching to talk to their agent, instead of reaching for a device to pick up. If that becomes a habit, agents are going to take on the role that we’ve associated with our computers: the place to enter the digital world, explore what’s possible, and figure out how to proceed.

In When Software Recedes, I explored how we could use more software while spending less of our time directly operating it. An agent brings the relevant capabilities into an experience as we need them and lets us follow an idea further before having to stop and figure out the tools. As software recedes, so too can devices, allowing the agent to become the part of computing that we recognize as ours.