Google HEIR Compiles PyTorch Models for Fully Homomorphic Encryption
In this article
Google has released HEIR (Homomorphic Encryption Intermediate Representation) as an open-source compiler toolchain that compiles pre-trained AI models—built for conventional plaintext inputs—so they can execute inference directly on encrypted data. A Python-annotated model can, in principle, move from standard unencrypted inference to fully homomorphic encryption (FHE) execution without a ground-up rewrite. For engineers building privacy-sensitive AI pipelines, this is the most concrete infrastructure-level development in the FHE space to assess, alongside broader shifts where pipeline architecture rather than model improvements is driving significant AI deployment gains.
What HEIR Does Architecturally
HEIR's core mechanism is an intermediate representation layer that abstracts the translation between standard ML computation graphs and HE-compatible execution. Developers annotate Python source to designate which data types should be encrypted, then invoke HEIR to compile the annotated code for homomorphic execution. The toolchain uses torch_mlir as a first-stage export step, converting a PyTorch model into MLIR before HEIR handles HE-specific lowering. Google describes the IR as capable of representing and scaling complex models across diverse dialects, keeping the abstraction backend-agnostic rather than tied to a single HE scheme.
The compilation pipeline is not yet fully automated; it currently involves discrete manual steps beginning with the torch_mlir export. The gap between "annotate and compile" as a description and the actual multi-step process is a real friction point for adoption teams to assess.
Demonstrated Use Cases
Google has applied HEIR internally across four domains: private content recommendation without exposing user behavioral data, credit card fraud detection while keeping sensitive financial records encrypted throughout processing, network intrusion detection without revealing packet contents to the detection service, and hotword recognition in audio streams without disclosing recordings to the service provider. Each maps to deployments where a third-party service must operate on data it cannot be permitted to see in plaintext—the category that end-to-end encryption alone cannot address because it requires decryption before computation.
Performance Overhead
FHE's computational overhead is the central engineering tradeoff. Community discussion surfaced concrete figures: a 64-bit equality check under FHE takes approximately 80 ms; addition and subtraction run at roughly 100 ms each; division reaches as much as 8 seconds. The commonly cited overhead ratio is on the order of 10³ relative to plaintext computation.
| Operation | FHE Latency | FHE Suitability |
|---|---|---|
| 64-bit equality check | ~80 ms | Poor — high cost for a primitive op |
| Addition / Subtraction | ~100 ms | Moderate — LLMs rely heavily on these |
| Division | ~8 s | Very poor — avoid in critical paths |
| Branching / conditionals | Not quantified | Poor — FHE handles branching least efficiently |
The architectural composition of LLMs creates a partial mitigation. Transformer-based inference is dominated by matrix multiplications—which decompose into additions and multiplications—and does not depend heavily on branching, placing LLM workloads in a more favorable position in the FHE performance landscape than general-purpose programs. A 1,000× overhead converts a 1 ms image classification task to roughly 1 second, which remains viable for latency-tolerant applications. Google has not published specific throughput or latency benchmarks for HEIR applied to LLM inference, despite including benchmarking infrastructure in the repository.
Availability and the Local vs. Cloud Privacy Tradeoff
HEIR is available now as an open-source project. The current requirement to manually invoke torch_mlir before HEIR's HE lowering means teams should treat this release as an early-adopter platform rather than a production-ready one-click tool, despite Google's framing.
The deeper architectural debate the toolchain surfaces is whether FHE-based cloud inference is the right privacy primitive at all, versus local on-device deployment. Critics argue that running a model on local hardware eliminates the need to encrypt queries entirely; proponents counter that for data worth protecting, the additional compute cost of FHE is an acceptable tradeoff compared to exposing sensitive inputs to a remote service. This tension maps directly to questions of where AI security boundaries should be enforced in production systems.
If Google closes the automation gap and publishes LLM inference benchmarks, HEIR transforms from a research-grade toolchain into a deployable privacy primitive for regulated industries. The pace of that maturation—not the current release state—is what engineers evaluating FHE for healthcare, finance, or enterprise data pipelines should be tracking.