Apple unveils Apple Intelligence for iOS, iPadOS, and macOS, with text and image generation across apps, cross-app task orchestration, App Intents, and more

Apple soon will be taking a huge bite … Aman Gupta / Livemint : Is Apple Intelligence coming to your iPhone, iPad or Mac? Check full list of compatible devices Ryan Christoffel / 9to5Mac : There's an emoji for that: meet Genmoji, Apple's AI-powered emoji generation feature in iOS 18 Charlie Sorrel / Lifewire : Apple Intelligence Just Made All Other AI Look Pointless Derek Saul / Forbes : Apple Stock Hits All-Time High On AI iPhone Optimism—Crosses $200 Per Share Will Knight / Wired : Apple Intelligence Will Infuse the iPhone With Generative AI Markus Kasanmascheff / WinBuzzer : Apple Introduces Apple Intelligence With AI Across iOS, macOS, VisionOS Viraj Gawde / Gizmochina : macOS 15 Sequoia And iPadOS 18: All New Features And List Of Compatible Devices Daniel John / Creative Bloq : Apple Intelligence could be bad news for AI gadgets Chris Barylick / O'Grady's PowerPage : WWDC: Apple releases short recap video of WWDC keynote speech announcements Marques Brownlee / Marques Brownlee on YouTube : WWDC 2024 Recap: Is Apple Intelligence Legit? Vallari Sanzgiri / MEDIANAMA : Apple Integrates ChatGPT into iOS, iPadOS, and macOS Matt Mullenweg : Apple Intelligence  —  It was so cool to see WordPress highlighted (although with a lowercase P … Threads: Benedict Evans / @benedictevans : Why would anyone have thought that all of Apple's new LLM features would be powered by Open AI (or Gemini) when Apple has been publishing papers about its own models, to run on its own devices and silicon, for most of the this year, if not longer? @panzer : Apple Intelligence castling “AI” is going to go down as one of the most diabolically slick marketing moves of all time. Eric Young / @eric.young310 : I think the best way I'd describe what Apple announced today was that Apple brought AI to the GUI world Like they did with the Mac from command line days AI before Apple, was a command prompt that users would enter an infinite and arbitrary request... and the AI would perform Apple Intelligence gives AI a simple graphical user interface for the rest of us M.G. Siegler / @mgsiegler : Interesting that it doesn't sound like Apple is going to have a way to upsell to the premium version of ChatGPT.  You can use it, but it's bring your own?  WWDC24 Parker Thompson / @parkert : Makes sense.  They can push on-device “free” features, OpenAI can pay for lead gen in return for making iOS a bit better, and power users can pay for 3rd party subscriptions.  Then later they can just turn down/off that traffic as they want to own more of that experience (maybe when all the bugs get worked out) and the tech press isn't yelling at people who ship this stuff. Mastodon: Dave Rahardja / @drahardja@sfba.social : This cloud architecture is fascinating.  —  1. Secure the hardware supply chain.  —  2. Rely on secure boot and the trust cache to assure loaded software.  —  3. Remove code that provide access in and out of the runtime.  —  4. Rely on per-boot on-disk encryption keys to secure data on Flash. … Jeffrey P. Bigham / @jbigham@hci.social : really appreciate Apple security systems people — how do you provably provide private server-side LLMs? check out the blog post,  —  https://security.apple.com/... Meg / @megmac@social.treehouse.systems : https://security.apple.com/...  This is, honestly, a pretty interesting and bold set of promises from Apple about how this system works, but I don't really understand why it should be taken as trusted that the attestations by the PCC servers about their running images (that clients use as a precondition for sending data to it) are correct? … X: Sam Altman / @sama : very happy to be partnering with apple to integrate chatgpt into their devices later this year! think you will really like it. Greg Brockman / @gdb : We're partnering with Apple to integrate ChatGPT right into iOS, iPadOS, and macOS. Will be available later this year, very excited! @elder_plinius : HOLY MOLY @APPLE IS MAKING IT RAIN ATTACK VECTORS 💦 I DON'T KNOW WHAT TO DO WITH MY HANDS 🫨 Bindu Reddy / @bindureddy : Honestly, I don't understand why Apple has to send anything to ChatGPT? It's just bizarre. Why not just get Meta to give them a license and host the 400B Llama? Starting with 70B is fine... They are so hung up on privacy and security and even alleged that you shouldn't trust [image] Simon Willison / @simonw : Some of the backlash I'm seeing against Apple AI is reinforcing my opinion: The hardest problem in computer science continues to be convincing an AI skeptic that you're not going to steal their data and train an AI model on it @mingchikuo : There's a lot to discuss about Apple Intelligence, and let's start with a quick overview. Apple Intelligence fully showcases Apple's strengths in ecosystem integration and interface design, mainly emphasizing convenience (through in-app & between-app integration) and enhancing Mike Butcher / @mikebutcher : The upshot of Apple's announcements from a security expert: part of your phone now lives 2,000 miles away in a data center. [Edit: Apple is having to jump through a LOT of hoops so as not to turn your private queries of AI services into training data for OpenAI.] @johnwilson : Apple Intelligence will also put pressure on automakers to keep supporting CarPlay. Some have started supporting ChatGPT directly, but I don't think they will be the dominant path. @sullyomarr : The apple event was great. They focused on the important things that consumers care about (privacy, easy to use etc) If anyone else had launched an on device, OS level LLM this place would of lost their mind Were the emojis cringe? Sure. Everything else was exciting Mark Gurman / @markgurman : Apple needs to put 25% of the vigor into iPadOS that it just put into Apple Intelligence because this is getting ridiculous. The iPad Pro gets incredible new hardware and an M4 chip and then iPadOS essentially gets nothing of substance. Simon Willison / @simonw : Their on-device language model was apparently trained on “licensed data, including data selected to enhance specific features, as well as publicly available data collected by our web-crawler, AppleBot” https://machinelearning.apple.com/ ... Ethan Mollick / @emollick : I wonder how much Apple AI is going to cause people to underestimate LLMs A 3B quantized model running on local silicon will be good for some stuff but not agentic reasoning. The cloud model doesn't seem to maintain state or history & isn't frontier. Is GPT-4 just for questions? Simon Willison / @simonw : Have Apple made any statements specifically about the training data they used for their new on-device image generation model? They licensed a ton of images a few months ago: Apple Signs Deal For AI Training Data from Image Service Shutterstock https://www.inc.com/... Simon Willison / @simonw : I think the product design instincts Apple displayed here are extraordinary Every single person involved appears to have had a deep understanding of AI ethics and privacy, and a hyper-realistic understanding of what these models are capable of and what responsible AI looks like @snazzylabs : I want to know the percentage of Apple employees that are—quietly—deeply disgusted by the image and text generation features shown off today. Diego Aguirre / @diegousdz : @elder_plinius @Apple Hey what is your stake of Apple Intelligence how difficult do you think it is to bypass his security and reverse engineer the tech to tell you every single detail from user? Simon Willison / @simonw : ... though I'll reserve full judgement until I see how well they have designed against the threat of prompt injection! Ian Zelbo / @ianzelbo : RIP Grammarly @elder_plinius : @DiegoUSDZ @Apple time will tell! bold move integrating to this extent, given the current state of llm security Simon Willison / @simonw : Was Apple's emphasis today that they were running their Private Cloud Compute AI accelerators on Apple Silicon a veiled dig at NVIDIA? Dylan / @dylanmcd8 : I must say, Apple Intelligence is another example of Apple doing what they do: being a little late to the game, but them absolutely blowing everyone else out of the water. Apple Intelligence is incredible and so exciting. [image] @snazzylabs : @elonmusk @MKBHD Apple doesn't hand over any data; you do, as the user, explicitly. Every time. But that would require you to spend 35 seconds researching before going on a temper-tantrum tirade. Steven Sinofsky / @stevesi : The contrast between what Apple is showing and what Google and Microsoft have shown is a stark as ever. This is really important. Apple brings their point of view to the newest technologies, again. Kelsey Hightower / @kelseyhightower : Apple Private Cloud! I always thought Apple would do something like this. I thought the use case would be serverless compute to support mobile applications but these ML use cases make more sense. Jessica Lessin / @jessicalessin : The legacy of today's Apple news will be that AI is a feature not a product. Sam Biddle / @samfbiddle : Pretty bleak to me that Apple envisions a Journalism and Mass Communication major, of all people, using an iOS LLM to write their emails for them. [image] Ben Bajarin / @benbajarin : Apple making some points about Apple Secure Cloud being 100% Apple's own data centers (not hosted at a hyper scaler), powered by Apple Silicon, 100% powered by renewable energy. Ben Bajarin / @benbajarin : In our post WWDC QA, Apple confirming they built a host of their own foundational models and used a host of licensed content to train and fine tune their models. Essentially 99% of Apple Intelligence is powered by Apple's own proprietary FMs. Alex Volkov / @altryne : is this a reaction to Apple not going with Grok? Because that's not what apple announced today 🤔 They said that they will support other models when on device is not cutting it, and announced a partnership with OpenAI Kelsey Hightower / @kelseyhightower : Apple Intelligence might have me switching back to Apple's native apps such as Mail, Calendar, and Pages. @javi : Apple's ability to tell a clear story about how they will help normal people like you get stuff done is night and day compared to Google. Dare Obasanjo / @carnage4life : Apple is a company that believes very deeply in creating a narrative about how technology makes people's lives richer and then builds products that match that narrative. Google is good at shipping technology while Microsoft is good at creating such narratives for CIOs & CTOs. Alex Kantrowitz / @kantrowitz : This is why we need AI [Image: Play the podcast that my wife sent the other day] @autismcapital : Imagine texting your mom an AI photo of her looking terrified. [image] Andrej Karpathy / @karpathy : Actually, really liked the Apple Intelligence announcement.  It must be a very exciting time at Apple as they layer AI on top of the entire OS.  A few of the major themes.  Step 1 Multimodal I/O. Enable text/audio/image/video capability, both read and write.  These are the native human APIs... Hamza Shaban / @hshaban : Another interpretation of Apple Intelligence = more elaborate forms of green bubble shaming Steven Sinofsky / @stevesi : Excellent. Today was super high on “vision” for Apple with tons of future tense. At the same time their strong point of view is abundantly clear. This is not just privacy and on device, but how they see integration at the platform level. The idea for example of building on top of Aaron Zollo / @zollotech : Apple Intelligence Compatible devices. [image] @deliprao : coming out of my self-imposed social media break to observe a moment of silence to mourn for thousands of consumer gen AI builders and their investors who did not listen to me. sorry to be that guy but the beatings will continue until the morale is improved. Peter Sterne / @petersterne : It is impossible to overstate how much this will improve journalists' lives. https://www.theverge.com/... [image] @snazzylabs : yyyyyyyyyyyyyyyyyyyikes don't like that [Mom in cape genmoji] Gene Munster / @munster_gene : Apple Intelligence's critical advantage is the personalization. They can use your personal data without giving it to the cloud. The personalization is done on device. Something only Apple can delivery. From a feature perspective you can use generative AI in Mail, Messages, Michael Gartenberg / @gartenberg : Apple Intelligence is free... next word is but Max Weinbach / @maxwinebach : Language, Images, Action, Personal Context The LLMs in Apple Intelligence will now prioritize notifications based on what's important New writing tools system wide to rewrite, proofread, and summarize. Automatically available everywhere Julia Alexander / @loudmouthjulia : Apple Intelligence: most of these features will likely be less good than individual AI applications and LLMs from competitors, but the fact that it's in one device and touches all parts of the ecosystem on widely available phones is a major AI experimentation and adoption moment. Emad / @emostaque : Hey @google, some free branding for you Awesome Google Intelligence Get that AGI inside Gene Munster / @munster_gene : They're making it clear that Apple Intelligence is personalized, which means it is trained on your data. Because of Apple's position on privacy, I believe consumers will be ok with the model training. Rich DeMuro / @richontech : Did Apple really just say you'll only be able to get Apple Intelligence on the iPhone 15 Pro and later? @autismcapital : The logical conclusion of this Apple Intelligence is to just create an artificial instance of ourselves that lives a whole life automated and independent of us. Wait — but then what would be the point of us existing? Uh oh! M.G. Siegler / @mgsiegler : “AI for the rest of us” is the first time they've said “AI” I think?! #wwdc24 Alex Styl / @alexstyl : Did Apple just invent Android's Intents API? Aaron Zollo / @zollotech : Apple has a new webpage just for Apple Intelligence. https://www.apple.com/... @panzer : Apple Intelligence castling “AI” is going to go down as one of the most diabolically slick marketing moves of all time. M.G. Siegler / @mgsiegler : Undoubtedly hard to read into it THIS closely, but Wall Street still digesting. Seemed to hate it at first. Then started to warm up with iOS. Then really not like the obvious AI announcements until the OpenAI partnership was announced. But even that not enough so far... #WWDC24 [image] Eric Seufert / @eric_seufert : It seems that Apple discarded the “Web Eraser” tool visible in the iOS 18 developer version of Safari that potentially could have been used as an ad blocker. Steven Sinofsky / @stevesi : “AI for the rest of us” This was awesome. It is a lot of vision. So much of this will be available later. It isn't worth making a list of the specifics in this presentation to track to this summer beta. Instead what matters is the direction and overall framework/pillars. Dariusz Dziuk / @dariuszdziuk : This is exactly the integration everyone thought Apple will do, nicely executed. If it works well they will own a big LLM powered AI marketshare given their distribution. But that ChatGPT button is anything but “seamless.” #wwdc2024 Julia Alexander / @loudmouthjulia : Which isn't to say Apple wins or that genAI is actually where the hype thinks it is. The overall presentation was meh — but the opportunity Apple has from a central hardware and ecosystem perspective, and access to top developers, will be a momentous next step for potential usage Max Weinbach / @maxwinebach : “AI for the rest of us” Steven Sinofsky / @stevesi : This is why it is so important to use the native capabilities of an OS. Use text view and just pick up all these features. Cross platform means you fall behind right away. [image] Ben Bajarin / @benbajarin : Apple frames Apple Intelligence as AI for the rest of us. Very Apple way to say it. Austen Allred / @austen : Apple Intelligence but for connecting my Airpods to the right device Matt Binder / @mattbinder : Apple is announcing its AI product “Apple Intelligence” and gave an example of how you can wish you friend Happy Birthday with an AI generated image of them then they threw this up on the screen why would i do my friend dirty like this on her birthday [image] Adam Singer / @adamsinger : Apple intelligence is going to help you figure out how to get to the airport on time. Because you're a complete moron and couldn't do that before @ijustine : For more AI processing power that can't be done on device Apple created specific servers with privacy at top of mind called Private Cloud Compute. [video] Ben Bajarin / @benbajarin : Federighi, just prior to showing the ChatGPT integration said “we want you to access external models” which makes it sound like most of Apple Intelligence is powered by Apple's own proprietary models. Seems GPT is just for web searches and composing fresh writing. Aaron Zollo / @zollotech : Apple Intelligence - Capabilities - Create images in 3 styles when sending in messages and other apps - Understands what you are doing to find what you're looking for Architecture - On device - Private Cloud Compute [image] @snazzylabs : Apple is promising the same crap Humane and Rabbit have promised, except, I think Apple Intelligence will actually mostly work. Jon Prosser / @jon_prosser : APPLE INTELLIGENCE IS ON-DEVICE PROCESSING?! WTF BROOOOO WHAT HOLY SHITT Scott Stein / @jetscott : Apple AI-generated memory albums... [video] @panzer : hooooooo boy [image] Aaron Zollo / @zollotech : More about Apple Intelligence - Rewrite to get suggestions - Proofread with suggested edits - Summarize key points - New Smart Reply Suggestions in Mail - Email Summaries based on email content - Priority Notifications on Lock Screen - New Reduce Interruptions Focus [image] Vadim Yuryev / @vadimyuryev : Apple Intelligence in iOS 18 can rewrite, proofread, and summarize text system-wide. RIP Grammarly! #wwdc24 [image] Trung Phan / @trungtphan : The AI industry competitive landscape after Apple launches Apple Intelligence [image] @autismcapital : So Apple Intelligence basically reads every single aspect of your life and sends it to Apple. Got it. Vadim Yuryev / @vadimyuryev : Honestly, Apple Intelligence is WAY better than CoPilot+ #wwdc24 [image] Vadim Yuryev / @vadimyuryev : HOLY SMOKES! You can rewrite your email in different tones with Apple Intelligence!!! And it works system-wide, even third party apps!! This is by far the BEST feature in iOS 18 and all the other softwares! #wwdc24 [image] Ray Wong / @raywongy : Apple really demoing how Apple Intelligence can help write an email for someone from a “Dept of Journalism and Mass Communication 2024” Bruhhhh #wwdc [image] @ijustine : Personal context is key to Apple Intelligence with on device processing keeping your data private. I love this example of cross referencing all these different things on your device to get the info you're looking for. #WWDC [video] @ijustine : Apple Intelligence using image generation and your personal context. Haha I can't wait to make images of all my friends. No one is safe! #WWDC [video] @snazzylabs : It's weird how awesome and terrible this is at the same time. Some features (like hyper-specific image search) are AMAZING! And others are... unsettling and depressing. Julia Alexander / @loudmouthjulia : I don't know man, I feel like all these AI-prompted images is just reiterating how important it is that we financially support real artists so we can have art that doesn't look like the same image in different shades defining what we look at every day, like, 20 years from now. Peter Sterne / @petersterne : Apple's implementation of AI seems really good. Summarizing notifications, proofreading emails and creating custom emoji all seem more useful and easy to use than just something like ChatGPT. Keshav / @keshav_027 : Apple has avoided the usage of “AI” but obliterated so many “AI” startups. Of course Apple incorporates AI across their ecosystem! Max Woolf / @minimaxir : “No need to engineer the prompt.” PROMPT ENGINEERING IS NOW MAINSTREAM Max Weinbach / @maxwinebach : Apple was never behind on AI, were they? This kills any other consumer AI product right now Dr. Manu Kumar / @manukumar : The economic value of AI will accrue mainly to the incumbents. Miguel de Icaza / @migueldeicaza : I am surprise that Google and Gmail did not do any of the features Apple just showed for Email. Steven Sinofsky / @stevesi : This integration is so exactly right. It shows that a “chat app” was a short term “demo” UI. Combine that with all the context on the OS and the privacy and this is all how AI can really come to life the right way. Eric Seufert / @eric_seufert : Writing Tools for App Intelligence address a core Grammarly use case. Chris Saad / @chrissaad : Answer wizard in email replies - sweet! #WWDC Matthew Lynley / @mattlynley : semantic index sounds a whole lot like a vector database tbh Steven Sinofsky / @stevesi : “over the next year” will get mocked (Apple “doesn't do that") but the architecture is right from the start and they won't need to back in. Much of this is clearly aspirational or early plans. M.G. Siegler / @mgsiegler : “Over the next year” Apple is finally fully shipping in iteration. #WWDC24 Eric Seufert / @eric_seufert : Apple confirms that it will use servers to process certain AI tasks. The device determines whether the device is capable of processing a task and sends data to its “Private Cloud Compute” servers if it can't. Kelsey Hightower / @kelseyhightower : Apple is reading the room on AI/ML. Personal context is how I want to think about my data. The last thing I want to do is send my data anywhere. Ben Bajarin / @benbajarin : Apple device experience deepens in value the more Apple devices you use. Apple Intelligence will only deepen this and make the strongest case ever for customers to invest in more of the Apple ecosystem. Average devices per customers is just shy of 1.5 so a lot of upside here. Ray Wong / @raywongy : Craig is dunking on “AI clouds” and saying how Apple Intelligence uses on-device processing for privacy #wwdc [image] Anshel Sag / @anshelsag : This is a total no-brainer feature, customer support and user manuals are already saving big companies TONS of money. Just helping people use their iPhones and solve problems. #WWDC [image] Alexei Oreskovic / @lexnfx : the “original intelligent assistant” Steven Sinofsky / @stevesi : Exactly right.  This is how to add a semantic index and LLMs to a platform.  [Slide from presentation: “  • Your data is never stored  • Used only for your requests  • Verifiable privacy promise “] Denver Sean / @dnvrsn : CAVEAT! sometimes a request might require a larger generative model to achieve something (a cloud) — so for those tasks they've created Private Cloud Compute. [image] Gene Munster / @munster_gene : Apple is hitting Privacy hard. New Private Cloud Compute allows you do more advanced personalized AI with privacy in the cloud. Privacy and personalization is Apple's differentiator. Hope King / @hopeking : The moment Tim Cook uttered ‘artificial intelligence’ on the #WWDC YouTube livestream [image] Ben Bajarin / @benbajarin : Entirely as expected, Apple is leaning heavily into their privacy efforts/standards as it relates to how they implement Apple Intelligence. M.G. Siegler / @mgsiegler : Um, 3 whole styles for images. That will get old, fast. But I suppose limit IP/copying concerns? #WWDC24 Ben Bajarin / @benbajarin : Apple Intelligence focusing on actions, assumes an underlying contextual action model. LinkedIn: Bryant Courtney : With Private Cloud Compute, Apple Intelligence can flex and scale its computational capacity and draw on larger, server-based models for more complex requests. …

TechCrunch 2024-06-11 Brian Heater