Apple’s groundbreaking AI research could transform iPhones, ahead of Google and Microsoft

However, things are about to change. In a new research paper, Apple has demonstrated a breakthrough technology that could help power artificial intelligence on iPhones. This technology involves simplifying massive LLM programs using flash storage optimization. When Apple integrates advanced AI into the iPhone, it will be another important turn of events. The Cupertino-based tech giant announced significant advances in artificial intelligence with two new research papers it presented this month. The paper unveils new techniques for 3D avatars and efficient language model inference.

This new research “LLM in a Flash: Efficient Large Language Model Inference with Limited Memory” published on December 12 has the potential to transform the iPhone experience because it can offer a more immersive visual experience and users will be able to access complex AI systems on iPhone and iPad devices. The paper mainly focuses on running large language models efficiently on devices with limited DRAM capacity. DRAM is dynamic random access memory used in personal computers and is known for its high speed, high density, affordability, and low power consumption.

Here are some quick takeaways from the research that will put Apple ahead of its peers

The paper addresses the challenge of running LLMs that already exceed available DRAM, storing model parameters in flash memory, and running them in DRAM on demand. It talks about the heuristics cost model developed to optimize data transfers from flash memory, taking into account the characteristics of flash and DRAM.

The techniques discussed in the paper are Windowing which reduces data transfer by reusing previously activated neurons, and Row-Column Bundling which increases the sizes of data chunks to read flash memory efficiently.

The paper also highlights the Sparsity exploit that uses sparsity in the layers of the FeedForward Network (FFN) to selectively load parameters to enhance efficiency. Another key aspect is memory management which proposes strategies to efficiently manage data loaded in DRAM to reduce overhead.

The researchers used models such as OPT 6.7B and Falcon 7B to illustrate their approach. According to the paper, the results showed that the models achieved a speedup of 4-5x and 20-25x on CPU and GPU respectively compared to traditional methods.

When it comes to the practical application of research, both models have shown significant improvements in resource-constrained settings.

New Apple research shows an innovative approach to efficiently run LLM programs in hardware-constrained environments. It paves a new direction for future research in on-device and next-generation user experience.

What does it mean for iPhone users?

From the user’s perspective, the results on efficient LLM inference with limited memory can greatly benefit Apple and iPhone users. With powerful LLM software that runs efficiently on devices with limited DRAM like iPhones and iPads, users will be able to experience enhanced AI capabilities at their fingertips. These capabilities include improved language processing, more advanced voice assistants, improved privacy, potentially reducing internet bandwidth usage, and most importantly making advanced AI available and responsive to all iPhone users.

Regardless of the future capabilities that demonstrate how Apple is working to dominate AI research and applications, experts appear to be in a cautious mode. Some experts seem to suggest that the tech giant will need to exercise a great deal of caution and responsibility while incorporating the research findings into real-world use cases. Some even highlighted the need to consider privacy protections, and ways to mitigate potential misuse and public impact.

© IE Online Media Services Pvt Ltd

Bijan Jose is a technology journalist and currently works as Assistant Editor of Indian Express Online in New Delhi. Passionate about the transformative power of technology, he delves into its multifaceted impact on our lives and the wider world. His interest in the vast potential of technology began in his youth, paving the way for his career. His writing journey began at Maharaja Sayajirao University, Vadodara, where he earned a BA in English. At the same time, he took up the role of citizen journalist at The Times of India, where he highlighted civic challenges in Vadodara. After completing his MA in English Literature, he moved into the role of senior reporter at TOI, focusing on crime reporting for print while crafting features on the city’s culture, art and heritage. Embrace the digital world in 2016 with India Today Digital. Starting as a desk clerk, he quickly rose to lead the online news team. His versatility shone when he took on a mixed media role in The Times of India’s New Delhi bureau, coordinating interviews with diplomats and defense personnel and covering pivotal events for the editorial segment of the response. A film enthusiast, he has written film reviews and critical features, and has contributed to The Economic Times and DailyO. In 2021, he shifted his focus to technology journalism with the Economic Times. By 2023, he had found his niche at The Indian Express, where he continues to produce insightful features on the ever-evolving world of technology. His reports and stories are distinguished by his talent for simplifying complex concepts and enlightening readers with the latest developments. …Read more