Ruoming Pang’s Post

🚀 Introducing the Apple Foundation Model. I’m thrilled to participate the journey of working on the Apple Foundation Model from the very beginning, which powers numerous features in Apple Intelligence this year. 🌱 Genesis: Wrote Decoder in AXLearn Training Framework. 🔥 Ignition: Collaborated with Tom G. to build and serve of our pre-training model. 🚀 Momentum: Partnered with Zirui Wang to make the post-training model accessible to all Apple engineers for product innovation. ✨ Realization: Worked with Chong Wang and Jianyu Wang to apply optimization for integration. 🌟 Vision: Initiated the development of agentic action capability for a new experience between human and machine. It’s been an amazing experience collaborating with Qibin Chen, Feng Nan, Dong Yin, Aonan Zhang, Floris Weers, Jiarui Lu, Tao Lei, Nan Du, Bowen Zhang and many other cross-team engineers on this project. Blog: https://lnkd.in/gesvt7ZF https://lnkd.in/gFkEUu6Q #Apple #WWDC

Some shared details about the: 3B on-device and the larger server-based language models introduced at #WWDC24

An innovative perspective about AI on security and privacy. What on your device stays on your device🔒 #wwdc #machinelearning #appleintelligence #devices #security #privacy

Excited about the launch of Apple Intelligence! Thrilled to be part of this amazing journey, filled with continuous learning and collaboration with many talented teams. Proud of the hard work everyone has put in. Check out the latest blog post detailing the training process and insights into Apple's On-Device and Server Foundation Models #AppleIntelligence #MachineLearning #AI #FoundationModels

What an incredible #WWDC24! Learn more about Apple Intelligence here: https://lnkd.in/geC-K9Ug Learn about Apple Intelligence in 5 minutes: https://lnkd.in/gqW6T95P

Just read Apple's blog post on its Foundation Models. Fascinating stuff that illustrates the level of expertise and innovation required to adapt vanilla GenAI to specialized environment, namely Apple software and devices. We are thinking of these as Expert Models and Edge Models. This is just the beginning. It will be interesting to see if the "late to the dance" but much more thoughtful approach that Apple took with much success to the smartphone will replay itself here. Or will it be too little, too late, and too closed? https://lnkd.in/gk7GvuDQ

I finally got a chance to read Apples summary of their Intelligence strategy announced this week. (Link in comment) It’s a distilled read with architectural details on what I would call “how to make AI consumer-device friendly”. credit to Apple! My highlights: 🛠️ a lot of optimizations (quantization,…) for their devices "on iPhone 15 Pro we are able to reach time-to-first-token latency of about 0.6 millisecond per prompt token, and a generation rate of 30 tokens per second." 🛠️ Model Adaptation: fine-tuned submodels which can be loaded (and offloaded) in memory for specific tasks (summarization, tone-change, …) 📊 Lots of impressive results shared in a visually beautiful way. It seems like they are only second to the state-of-art (GPT 4). They didn't include GPT 4o in their comparisons. Happy reading. And I still want to have the option to turn it off when released on my Mac/iOS devices :)

Yesterday, Apple unveiled their new Apple Intelligence. What's fascinating is how they make on-device inference work. On-device inference needs to be quick and efficient without draining the iPhone battery. Apple's integrated silicon strategy uniquely positions them for this, but they also use innovative techniques: They employ a base model with 3 billion parameters(which is super small in LLM terms) and add a so-called LoRa adapter for specific tasks like summarization or proofreading. This approach allows them to transform a “vanilla” model into a fine-tuned expert in any field while maintaining only one "big model" on the device. Further that allows them to add more AI functionality to iOS by simply adding more LoRa Adapter without requiring any changes to the base model. I expect to see much more LLM applications rely on specialized task LoRa adapters in the future, similar how they are the bread and butter in the Image Generation scene. Check out more details here: https://lnkd.in/eXAm-fJ4

Apple announced yesterday the introduction of Apple intelligence, which will introduce a number of new AI based new features to our iPhones, iPad and Macs. Two things that I find really interesting and noteworthy from the announcement ( beyond the new capabilities that will come to our devices in the fall): - It uses small specialized models to run a lot of the smarts on device securely and quickly. This shows how small models working in tandem can be both effective and energy efficient. - For tasks that require more power, the request will be sent to Apple new Private Cloud Compute where data will only be used for fulfilling the user’s request and then deleted. No storing of user data, no mining or selling of user information. In this day and age where our private lives seem to be the currency of choice for so many companies, it is refreshing to see that significant technological progress can still be made without sacrificing privacy. More information about Apple intelligence, the foundation models their using and Private Cloud Compute can be found https://lnkd.in/eXVRnn4p , https://lnkd.in/eiP79cEE and https://lnkd.in/eDHm4VAu