Privacy startup Duality says it has developed a private LLM inference framework that uses fully homomorphic encryption to let LLMs answer encrypted prompts
Rina Diane Caballar / IEEE Spectrum :
Context & Ripple Effects
Duality previously positioned homomorphic encryption as a way to deliver privacy-preserving data analysis, backed by a $16M round led by Intel Capital. Its new framework extends that privacy focus from analytics to LLM inference.
The surrounding coverage shows that fully homomorphic encryption's commercial relevance depends on execution speed: the Intel-Microsoft-DARPA DPRIVE effort targeted custom silicon to make the technique practical. That makes encrypted-prompt inference a meaningful application test rather than a standalone cryptographic claim.
First-order effects
- Duality can market a private-inference framework in which prompts remain encrypted while an LLM produces an answer, targeting deployments where exposing prompts to the inference provider is unacceptable.
- The framework puts performance and operational viability at the center of Duality's product case, because fully homomorphic encryption adds computation to each inference request.
Second-order effects
- LLM providers and enterprise AI teams evaluating sensitive workloads gain a potential alternative to architectures that require plaintext prompts at the serving layer; they will need to compare privacy protection against latency and cost.
- Demand for faster homomorphic-encryption execution could strengthen the case for specialized compute initiatives such as Intel's Heracles FHE accelerator, while conventional inference stacks face pressure to add privacy-preserving options.
Third-order effects
- If encrypted inference becomes operationally viable, privacy constraints could become an architectural choice in AI deployment rather than a reason to exclude certain data from LLM workflows.
- The durable shift would be toward policy-aware inference systems that separate access to user inputs from model execution, though adoption will depend on whether encryption overhead fits production service requirements.
The trend: This is one data point in the move to make AI inference usable for sensitive workloads by embedding privacy controls directly into the serving stack.