A thousand agents walk into a conference…
Last week I joined data, AI, and IT leaders at the Agentic AI Exchange in New York. Attendees described various states of maturity and adoption in their organizations. All were there in search of feedback and best practices from their peers. Nearly everyone cited the tremendous pace of change in AI. Unanimously, we all wondered how the world would be different six months hence, and we marveled at how far things have come in a relatively short time.
Five things stood out to me from the conversations we had:
1) Agents are table stakes, but few have figured out how to orchestrate them
Everyone in the room had deployed agents in their organization. Some were in production; others were still evaluating and proving the value. But if we rewind to the beginning of the year, just a small fraction of those same leaders were even experimenting with agents. While there is certainly some selection bias in terms of who attended the event given the theme, the consensus is that organizations are increasingly trusting agents with workflows of varying complexity.
That said, leaders were comparing notes on how others orchestrate agents, what protocols they have established for how agents communicate with one another. And that’s just internally. What about agentic consumers and customers? Andreesen Horowitz recently published a worthwhile read about “B2A” commerce, in which companies that have long tried to deter bots from their sites now vie for their business. Which brings me to…
2) “Customer experience” is shifting toward “agent experience”
One presenter explored how to balance customer experience and agent experience. His internet service provider (ISP) had frequent outages, and he wanted a credit on his bill for the interrupted service. So, he built a few agents to look for outages and to call up his ISP’s customer service to argue for a credit on his behalf.
It’s an interesting use case. It illustrates how customer service and customer success teams need to build for machine interactions. That said, the presenter’s ISP should also focus on the fundamentals. They should prepare for more bots to call on behalf of their human users and design a superior agent experience. But they also shouldn’t forget that the best agent experience is one that never needs to happen in the first place. They can both make customer service more accessible to AI agents while addressing the underlying problems, anticipating customer needs, and resolving issues before customers call in. Most ISP consumers are not creating agents, but many are frustrated with outages and arrive in customer service queues with an axe to grind. Is your data prepared both to handle agents calling in, as well as improving operations and fundamentals?
3) Leaders are demanding ROI of AI
If you had “tokenomics” on your buzzword bingo card during this event, you almost certainly would have won. AI ROI is on everyone’s lips.
It’s not that leaders or boards want data and AI teams to stop experimenting. Rather, their tolerance of “shiny new toy” exploration of AI is giving way to more disciplined fiscal approaches. They’re done marveling at what it can do and are instead waiting to see that magic impact their top and bottom lines.
Interestingly, one presenter even noted a trend in which companies were shifting toward localizing models and moving out of the cloud and on- prem, where appropriate. AI adoption does not look like people asking Claude whether it will rain today. That’s a task for the Weather app, not a reason to burn tokens. Harness engineering and matching the right prompt to the right model based on complexity will be a big part of prudent agentic adoption and scaling going forward to manage token spend responsibly. Products that do this prompt-model matching automatically will need to provide transparency and auditability. Products that leave customers to do this will face requests for flexibility and rationalization.
4) Anxiety around AI-related job loss impacts adoption
Attendees noted they are in different stages of AI adoption. We visited the familiar topics of training, enablement, tooling, and so on. But one topic that really resonated was how AI-related job loss can affect whether and how much teams embrace the power of AI and agentic workflows.
Team members have plenty of ideas about how agents can help with the tedious parts of their jobs. They have fewer answers about what they do instead when agents are actually taking action on their behalf. Not to mention questions about who is accountable for an agent’s misbehavior or success. As a result, the leaders responsible have to approach the adoption of agents not just as change management but also from the lens of retraining, replacement, and support.
5) Context can be a garden or an untended field
I had the privilege of facilitating a panel that asked participants to explore whether context should be inherited or inferred. That is, should we tend and manage context like a garden, or should we let it grow wild like an open field? There might be reasons to do both. Frontier models are getting good at inferring context and avoiding overreach. And in some cases, context engineering can be time-intensive, overly process-heavy, and too slow to respond to change.
On the other hand, the risks of agentic overreach or hallucination can be significant, especially in regulated industries. DataOS allows the teams producing data to include minimally viable context when engineering data products to get the best of both worlds: eliminate hallucination while also requiring low levels of effort. In DataOS, context isn’t something you specify separately; it’s an integrated part of how data products are built from the ground up to create trust and value.
The path forward
There is obviously a lot of optimism about the workloads agents can handle. I understand why. But I couldn’t help but scratch my head, to return to my second takeaway, at the idea of a bot sitting on hold on behalf of a human user, waiting to take its turn to talk to a customer service bot. Is that really the kind of agent-to-agent communication that will make the world a better place?
There was another quote from one presentation that stuck with me: “If your customers could leave you in an hour, would they?” That is, if AI and agents made switching costs so low that your customers could either transfer their data or build a replica in an hour, would they bother remaining a customer of yours?
Put another way, companies adopting agents should consider two unfair advantages:
- Customer data
- Customer experience
Regarding customer data, customers increasingly want to own their first-party data. They will increasingly judge harshly any organization that prevents them from taking their data with them. And they have every right to. I expect more legislation or regulation in the future that give people more control and portability over their data. Can your data platform offer them that?
Secondly, if data flexibility and time-to-value are increasingly in demand, that shifts the premium from the data companies have about customers to the amount of delightful and meaningful experiences companies can create for their customers, with or without agents.