Google DeepMind SL2T Brings Sign Language Translation to Pixel 11

Google DeepMind has released Sign Language to Text (SL2T), a multilingual model that translates continuous sign language into written text, shipping it inside Gboard and Live Transcribe on the Pixel 11 — the first time a production-grade sign language translation system has reached a consumer device. The initial deployment covers American Sign Language (ASL) to English, with additional sign languages and devices announced as forthcoming, at no additional cost to users.

The technical gap SL2T closes is wider than it first appears. Spoken-language transcription is a sequential mapping within a single language; sign language translation is cross-lingual machine translation over a spatiotemporal visual signal. Meaning is encoded simultaneously across hands, arms, torso, head, and face, demanding high-frame-rate whole-body pose estimation before any language modelling can occur. Earlier hardware-centric approaches — sign language gloves being the canonical example — were structurally unable to capture this, because sign languages are not a manual encoding of English syntax.

Architecture and Training Data

SL2T ingests a sequence of geometric pose landmark coordinates rather than raw video frames. An on-device MediaPipe Holistic model tracks body keypoints locally; only those coordinate sequences leave the device for server-side translation, allowing the source video to be discarded immediately. This satisfies a privacy constraint without sacrificing translation fidelity, since the geometric representation retains the spatial relationships needed for language modelling while discarding pixel-level identity information.

The translation backbone bypasses gloss annotations entirely. Glosses — intermediate written tokens approximating individual signs — cannot encode non-manual markers (facial grammar, mouthing) or spatial constructions in which location in the signing space itself carries grammatical meaning. By training directly from landmark sequences to target-language text, SL2T removes the gloss vocabulary ceiling and allows quality to scale with data volume.

That data volume is substantial: the model was trained on over 100,000 hours of signing across more than 50 sign languages, with roughly one quarter of that corpus in ASL. Joint multilingual training — across languages, dialects, and proficiency levels — caused the model to outperform single-language baselines in internal experiments, consistent with cross-lingual transfer findings in spoken-language NLP. The approach mirrors the philosophy behind compact but generalist model design, where shared structure across tasks improves per-task performance.

Benchmark Results

SL2T achieves a zero-shot BLEURT score of 70 on the FLEURS-ASL sd-test benchmark for ASL-to-English translation, which DeepMind describes as significantly higher than any previously reported score on that evaluation.

DeepMind identified several practical failure modes and addressed them during development: streaming latency minimisation, hallucination suppression on non-signing inputs, fairness for left-handed signers (approximately 10% of the signing population), and accuracy for one-handed signing used when the other hand holds a phone. Qualitative examples from FLEURS-ASL show strong translation of complex sentences, with residual errors in rapid fingerspelling (the source cites "prey" rendered as "grey"), passive constructions, classifier depictions, and tense resolution without surrounding context.

Community Governance and Deployment

The project originated with Sam Sepah, a Deaf Googler, and Deaf partners were involved in data collection, user studies, and impact assessment throughout. DeepMind established the AI Sign Language Advisory Committee (AISLAC), comprising global Deaf organisations and subject-matter experts. The committee co-authored a joint impact report for the SL2T 1.0 release — a transparency mechanism DeepMind says it intends to continue for all major sign language releases.

SL2T is available now in Gboard and Live Transcribe on Pixel 11. No API, SDK, or open-weights release is described in the source material; developer access beyond those consumer integrations has not been announced.

Where recent coverage of real-time translation infrastructure has concentrated on spoken-language audio models, SL2T demonstrates that pose estimation can serve as a viable bridge modality — privacy-preserving by construction and capable of exceeding gloss-based academic baselines at scale. With more than 50 sign languages in the training corpus and consumer hardware as the deployment surface, the architectural pattern is already positioned to extend beyond ASL without fundamental redesign.