The Challenge of ‘Jagged Intelligence’ in AI Agents

A persistent obstacle to the widespread adoption of autonomous AI agents lies in a phenomenon Salesforce terms “jaggedness” or “jagged intelligence.” This refers to the perplexing tendency of AI systems to excel at complex tasks while inexplicably faltering on simpler ones that humans routinely solve with ease. Savarese notes that this “jaggedness” often manifests in scenarios demanding “common sense” reasoning.

The solution, according to Salesforce, isn’t simply building larger and more complex models. “It’s not about training bigger and bigger models,” Savarese stated. “Large Language Models (LLMs) are currently trained on massive datasets, but we’re reaching a point of diminishing returns. Improvement is becoming marginal.” Instead, the focus must shift to enabling AI to learn from experience, incorporating feedback from both the environment and user interactions.

However, allowing AI agents to learn “on the fly” during live deployment carries inherent risks. “We want to perform learning training *before* deployment,” Savarese emphasizes, “by constructing simulation environments where agents can be thoroughly tested, evaluated, and refined until they reach the desired performance level.”

How eVerse Creates Realistic AI Training Environments

eVerse addresses this need by simulating realistic enterprise environments populated with synthetic data and metadata that closely mirrors real-world customer data. Within these virtual environments, teams can rigorously test AI agents, measuring both failure rates and successful outcomes to iteratively improve performance before deployment. This approach allows for controlled experimentation and refinement without the potential disruptions of live testing.

Salesforce successfully utilized eVerse in the development of Agentforce Voice, announced in October, a platform designed to empower organizations to build voice-enabled agents capable of handling complex conversations in real-time. Prior to its launch, Agentforce Voice underwent thousands of simulated conversations within eVerse, exposing it to a wide range of real-world complexities such as background noise, varying accents, and crosstalk.

“Consider the challenges of a poor cell phone connection, distracting background noise, or even strong accents – like my own Italian accent,” Savarese illustrates. “We need AI agents that can navigate these conditions with fluency, naturalness, and consistency. eVerse allows us to create these realistic voice interactions through simulation and accurately assess agent behavior.”

UCSF Health Pioneers eVerse in Healthcare Billing

Salesforce customer UCSF Health is currently piloting eVerse to refine AI agents for healthcare billing processes. Sara Murray, MD, VP and Chief Health AI Officer at UCSF Health, reports that eVerse has already begun to streamline one of the most complex aspects of healthcare administration.

UCSF Health manages approximately 2.5 million outpatient visits annually, generating around 9,000 billing inquiries each month. These inquiries consume thousands of hours of human agent time.

“Healthcare billing can be incredibly overwhelming for patients,” Murray explains. “Our human agents spend significant time simply answering questions. Initially, an AI agent could handle about 70% of these inquiries, but 30% still required human intervention.”

Through the Healthcare Learning Engine powered by eVerse, UCSF’s team has iteratively trained the AI agent, increasing its ability to resolve billing inquiries to approximately 80%. This represents a substantial improvement in efficiency and patient experience.

What impact will advancements in AI-powered simulation have on other complex industries beyond healthcare? And how will these tools shape the future of work for human agents?

Frequently Asked Questions About Salesforce eVerse