The Rise of the Small Language Model

AT&T’s experience highlights a growing recognition that bigger isn’t always better when it comes to language models. Markus believes the future of agentic AI lies in the proliferation of SLMs. “We find small language models to be just about as accurate, if not as accurate, as a large language model on a given domain area,” he stated. This accuracy, coupled with significantly lower computational costs, makes SLMs an attractive alternative for many enterprise applications.

Building with LangChain and Microsoft Azure

The company’s internal “Ask AT&T” personal assistant and the recently launched “Ask AT&T Workflows” are prime examples of this strategy in action. Workflows, built in collaboration with Microsoft Azure, provides employees with a no-code/low-code graphical interface to automate tasks. These agents tap into AT&T’s proprietary tools for document processing, natural language-to-SQL conversion, and image analysis, ensuring decisions are driven by the company’s own data.

However, AT&T isn’t solely relying on homegrown solutions. The company embraces a pragmatic approach, prioritizing “interchangeable and selectable” models and avoiding unnecessary reinvention. They actively evaluate off-the-shelf options, deprecating internal tools as industry standards mature. Their “Ask Data with Relational Knowledge Graph,” for example, currently leads the Spider 2.0 text-to-SQL accuracy leaderboard, demonstrating their commitment to cutting-edge performance. Spider 2.0 Leaderboard

The Importance of Human Oversight

Despite the increasing autonomy of these agentic systems, human oversight remains crucial. All agent actions are meticulously logged, data is isolated, and role-based access controls are enforced. This ensures accountability and prevents unintended consequences. “Things do happen autonomously, but the human in the loop still provides a check and balance of the entire process,” Markus emphasized.

AI-Fueled Coding: A New Paradigm for Software Development

AT&T’s innovative approach extends beyond task automation and into the realm of software development itself. “AI-fueled coding” leverages agile methodologies and function-specific build archetypes to generate production-grade code with remarkable efficiency. This process, likened to retrieval-augmented generation (RAG), significantly reduces the iterative back-and-forth typically associated with traditional coding. The company has even built internal data products in as little as 20 minutes – a process that previously took six weeks.

This shift isn’t limited to professional developers. Non-technical teams can now utilize plain language prompts to prototype software, democratizing the development process and accelerating innovation. But what are the long-term implications of this trend for the role of the software engineer?

Frequently Asked Questions About AT&T’s AI Strategy

• What are small language models (SLMs) and why are they important?

  Small language models are AI models with fewer parameters than large language models. AT&T has found they can be just as accurate as LLMs for specific tasks, while being significantly more cost-effective and faster.

• How does AT&T’s multi-agent system work?

  AT&T’s system uses “super agents” powered by large language models to direct smaller “worker” agents, each focused on a specific task. This division of labor improves efficiency and reduces costs.

• What is “Ask AT&T Workflows” and who is it for?

  “Ask AT&T Workflows” is a no-code/low-code platform that allows AT&T employees to automate tasks using AI agents. It’s designed for both technical and non-technical users.

• How is human oversight incorporated into AT&T’s AI systems?

  All agent actions are logged, data is isolated, and role-based access controls are enforced. A human reviews the process to ensure accuracy and prevent unintended consequences.

• What is “AI-fueled coding” and how is it changing software development at AT&T?

  “AI-fueled coding” uses AI to generate production-grade code quickly and efficiently, significantly shortening development timelines and increasing output.