Apple Evaluates PrismML Technology to Run Compressed AI Models on iPhone

The Technology Behind the Compression

PrismML, a spinout from the California Institute of Technology (Caltech), focuses on shrinking complex AI models that typically require server-grade hardware to function. The company claims to achieve this by simplifying how internal model information is stored. By reducing values from 16 bits to one or three possible values—a process described as an advancement beyond the chip industry’s move from eight-bit to four-bit computing—the startup can drastically reduce memory requirements. To demonstrate this capability, the company publicly released a compressed version of Alibaba’s open-source Qwen model. PrismML stated that it reduced the model from approximately 54 GB to less than 4 GB, allowing all 27 billion of its parameters to run on an iPhone 15 or newer. According to PrismML, their compressed models offer significant efficiency gains compared to conventional versions:

• Memory usage: 10 to 15 times less memory required.
• Speed: Generates responses six to eight times faster.
• Energy efficiency: Consumes three to six times less energy.

Hassibi acknowledged that there is a performance trade-off, noting that these models typically lose a few percentage points of overall accuracy. Factual recall tends to weaken before core skills like math, coding, and reasoning.

Strategic Implications for Apple’s AI Strategy

The evaluation comes as Apple works to enhance Siri and integrate more artificial intelligence features into its ecosystem. Apple recently opened the public beta of iOS 27, which includes a significant overhaul of its voice assistant. Industry analysts suggest that the ability to run larger, more capable models directly on a device addresses a fundamental constraint in Apple’s strategy. Currently, the most sophisticated AI models require processing power and memory that exceed the capacity of smartphones. By shifting more tasks to the device, Apple could achieve several objectives: * Reduced Latency: Eliminating the need to send data to a remote server for processing. * Enhanced Privacy: Keeping sensitive personal information, such as health and medication data, local to the device. * Operational Costs: Lowering cloud-computing expenses and reducing reliance on external data centers. * Offline Functionality: Allowing AI-driven features to remain operational without an active internet connection. Carolina Milanesi, president and principal analyst at Creative Strategies, noted that on-device processing is particularly beneficial for features involving computational photography, video generation, and health tools that rely on private user data.

The Future of Localized Intelligence

The technology developed by PrismML originated from Hassibi’s research group at Caltech, with the university retaining the underlying patents and granting an exclusive license to the startup. In March, PrismML raised $16.25 million in a seed round backed by investors including Khosla Ventures. Looking ahead, the company plans to compress additional models, including Google’s open-source Gemma. Beyond smartphones, the startup envisions its technology being applied to robotics, autonomous systems, and other hardware that requires rapid, localized decision-making. Horace Dediu, founder of Asymco, noted that Apple’s unique position as both a chip and software designer gives it a potential advantage in implementing this technology. By controlling both the hardware and the software, the company can more effectively optimize how AI models function within the physical limits of an iPhone. While Apple currently uses a combination of on-device processing and cloud infrastructure, the goal is to determine how large and “clever” a model can be fitted onto a handset to support more demanding tasks.

Find more reporting in our Technology section.