How SLMs like Microsoft's Phi-3, which can run locally on phones or PCs without major compromises, open up new AI use cases by being more responsive or private
literally. Recent advancements by Microsoft have made it possible … Ian Mullane : This may have passed you by, but think about the potential. We have moved from only being able to run LLMs in the cloud (think ChatGPT, Gemini … Martijn Rasser : “When ChatGPT was released in November 2023, it could only be accessed through the cloud because the model behind it was downright enormous. … Craig Scroggie : Research from Microsoft shows that AI models can now be made small enough to run on personal devices like phones and laptops without significant loss of functionality. …
Context & Ripple Effects
Microsoft had already positioned its small-model line for constrained hardware with Phi-2’s phone-sized deployment claim, then made Phi-3 generally available, including an on-device variant for Copilot+ PCs. This coverage focuses on the practical implication: local execution makes responsiveness and privacy product features rather than solely cloud-service attributes.
The story matters because it recasts small language models as a complement to cloud AI. For tasks that fit their capability envelope, a device can handle inference locally instead of requiring every interaction to traverse a remote model service.
First-order effects
- Microsoft and developers using Phi-3 can design phone and PC features that respond without a cloud round trip, while keeping the relevant processing on the device.
- Users and organizations with privacy-sensitive or connectivity-constrained workflows gain a more viable local-AI option; the trade-off is that applications must select tasks suited to a smaller model.
Second-order effects
- AI application teams are pushed toward workload routing: use local small models for immediate or private interactions, and reserve cloud models for requests that need greater capability.
- PC and phone vendors gain a clearer reason to emphasize hardware that supports on-device inference, while cloud AI providers face more pressure to show where remote-scale models add value.
Third-order effects
- If small models continue to deliver useful performance on personal devices, AI products are likely to settle into hybrid architectures rather than a cloud-only model, with inference location becoming a core product-design choice.
- The competitive boundary may shift from access to a single large model toward how well platforms coordinate device hardware, local models, and cloud escalation—though the division will depend on task quality and device constraints.
The trend: This is one data point in the shift toward hybrid AI, where inference is distributed between personal devices and the cloud according to latency, privacy, and capability needs.