A look at Apple's technical approach to AI, including core model performance, alignment strategies, and adapter and on-device strategy
Apple Intelligence makes a lot of sense when you get out of the AI bubble. Plus, the cool technical details Apple shared about their language models “thinking different.”
Interconnects Nathan Lambert
Related Coverage
- No Matter How You Package It, Apple Intelligence Is AI Wired · Steven Levy
- View article Newcomer
- Introducing Apple's On-Device and Server Foundation Models Windows Copilot News · Mpesce
- We now know who holds the most power in Apple's deal with OpenAI Business Insider · Hasan Chowdhury
- The key difference between Apple Intelligence and Microsoft Copilot is _privacy_. No one using Copilot - even a private instance - feels entirely sure that Microsoft isn't seeing that traffic. — Apple hopes to be able to prove to any interested parties that their system is completely secure and does not retain any data. … @mpesce@arvr.social · Mark Pesce
- A look at Apple's technical approach to AI including core model performance etc. Hacker News
- Apple Details Its AI Foundation Models and Applebot Web Scraping MacStories · Federico Viticci
- Excluding your website from Apple's AI crawler Six Colors · Dan Moren
- Opting Out of AI Model Training MacStories · John Voorhees
Discussion
-
@marcslove
Marc Love
on threads
“Apple's presentation rang very different than most AI keynotes we've seen in the last few years. While OpenAI and Google are trying to prove that they are the best at AI, Apple leaned into a narrative of what else we can do with AI. ” …
-
@maxwinebach
Max Weinbach
on x
This is from Apple's State of the Union The local model is a 3B parameter SLM that uses adapters trained for each specific feature. Diffusion model does the same thing, adapter for each style. Anything running locally or Apple's Secure Cloud is an Apple model, not OpenAI. [video]
-
@natolambert
Nathan Lambert
on x
lot's of people wrote about Apple Intelligence, hopefully mine was the most in the weeds about the model details: - did Apple train a GPT4 level model? - why did Apple use another old RL algorithm MDPO for RLHF? - what it means for the open vs closed debates?
-
@rm_rafailov
Rafael Rafailov
on x
@natolambert I am somewhat doubtful they used the original MDPO algorithm in a token-level space. There are ways to frame online *PO as mirror descent, I would assume they uses some version of that in combination with the LOO approach from Cohere.
-
@drexmcarthur
Rex McArthur
on x
@natolambert So. Many. Tricks. Seriously this industry is just “1,000,000 sort of secrets that make things great”.
-
@natolambert
Nathan Lambert
on x
seems that Apple trained a “original GPT4"ish level performance model for servers. * Beating DBRX Instruct + Mistral 8x22b by a lot isn't super easy, but also not that hard with $$$ * Loses to current GPT4 Turbo, many models do The on-device model makes sense that they would... […
-
@andrew_n_carr
Andrew Carr
on x
Easily one of the best Nato posts recently. A must read.
-
@luciascarlet
† Lucia Scarlet
on x
according to Apple's data, Apple's server-side foundational LLM appears to be insane, matching or beating GPT-4.5 in industry-standard benchmarks which really raises the question of why they even worked with OpenAI to begin with - server capacity? https://machinelearning.apple.co…
-
@natolambert
Nathan Lambert
on x
lol at Apple sneaking this RLHF gem into their “Apple foundation models” blog post and no one talked about it “We have developed two novel algorithms in post-training: (1) a rejection sampling fine-tuning algorithm with teacher committee, and (2) a reinforcement learning from...
-
r/apple
r
on reddit
AI for the rest of us
-
@viticci@mastodon.macstories.net
@viticci@mastodon.macstories.net
on mastodon
I was grossed out when Apple got to image generation in their WWDC keynote, and I was grossed out when I read that Applebot scraped “the open web” to train their AI model, with publishers only being able to opt out after the fact. Disappointing to say the least. — https://www.…
-
@craiggrannell@mastodon.social
Craig Grannell
on mastodon
MacStories suggests Apple is no better than others when it comes to AI training. I'd argue it's worse: after all, the creatives whose work is now being exploited are the people who kept Apple alive for so many years. — Disappointing. But also unsurprising. https://www.macstor…