A recurring example: an AI assistant spends $200 at the wrong store, then reports back, cheerfully, “All done!”
Nothing broke. It did exactly what it was built to do. It just said it in the tone you would use for booking a nice dinner, not the tone you want for someone else touching your credit card.
This is the tension I keep coming back to. As agents move toward acting through a site’s own tools instead of clicking through its screens, there is no interface left to glance at and sanity-check. The agent’s tone becomes the interface. I think a lot of teams are solving this with one voice, shipped everywhere. Trust doesn’t scale the way friendliness does.
Two intents need two voices. When someone asks for “something in my usual style”, the stakes are low, and the agent can be a relaxed co-pilot that bounces ideas back and forth. When someone says “buy this desk, schedule Monday pickup”, they are delegating real money. They want reliability, not magic. A breezy “all done for you!” after an autonomous purchase reads as a small alarm bell. A plain, almost boring confirmation builds far more trust than enthusiasm does.
The part that still needs work is friction. Acting without clicking through a UI removes friction, which means we now have to design some of it back in, on purpose. Just-in-time consent matters more than ever. So does a real hierarchy of signals, so the important update is not buried under routine ones. And there should always be a clear, obvious way to stop the agent before it acts on anything sensitive.
Whether people trust agents is a design question as much as an engineering one. Where do you land: full autonomy, or should agents always check in before spending your money?