How SLMs like Microsoft's Phi-3, which can run locally on phones or PCs without big compromises, open up new AI use cases by being more responsive and private
literally. Recent advancements by Microsoft have made it possible … Ian Mullane : This may have passed you by, but think about the potential. We have moved from only being able to run LLMs in the cloud (think ChatGPT, Gemini … Martijn Rasser : “When ChatGPT was released in November 2023, it could only be accessed through the cloud because the model behind it was downright enormous. … Craig Scroggie : Research from Microsoft shows that AI models can now be made small enough to run on personal devices like phones and laptops without significant loss of functionality. …
Context & Ripple Effects
Microsoft’s Phi line had already established the small-model direction with Phi-2 positioned as capable enough to run on a phone. The Phi-3 release made that direction more concrete, including a 3.3B-parameter variant intended for Copilot+ PCs in Microsoft’s broader Phi-3 rollout.
The significance is less a replacement for cloud models than a deployment shift: tasks that benefit from low latency or local handling can be designed around the device rather than a remote endpoint.
First-order effects
- Developers and device users can deploy Phi-3-class language capabilities locally on phones and PCs, reducing the need to send every request to a cloud-hosted model.
- Microsoft gains a route to make AI features more responsive and privacy-oriented on compatible personal devices, alongside its cloud AI offerings.
Second-order effects
- AI application teams must choose which requests run locally and which require larger cloud models, accelerating hybrid product designs rather than a single cloud-only path.
- Device AI features become more viable for use cases where responsiveness or local data handling is central, raising the value of software integration with on-device models.
Third-order effects
- If small models continue to retain useful capability at lower resource demands, inference will increasingly be distributed across devices and cloud infrastructure rather than concentrated solely in data centers.
- The competitive boundary may shift toward model efficiency and deployment tooling: providers that can route workloads effectively across local and cloud systems could have an advantage.
The trend: Small language models are pushing AI from cloud-first services toward hybrid, device-native inference for latency- and privacy-sensitive work.