Google DeepMind SL2T Brings Sign Language Dictation to Pixel 11

Google DeepMind published details on August 12, 2026 of SL2T, a massively multilingual sign-language-to-text model now shipping inside Gboard and Live Transcribe on the Pixel 11 at no additional cost. Trained on over 100,000 hours of data spanning more than 50 sign languages — with roughly a quarter of that corpus in American Sign Language — SL2T is the first sign language AI to move from lab research into a consumer product at this scale. For engineers working in multimodal and accessibility pipelines, the architectural decisions offer a direct case study in privacy-preserving input design, gloss-free sequence translation, and multilingual transfer for a modality that mainstream NLP tooling has largely ignored.

An estimated 70 million Deaf and hard of hearing people use one of the world's more than 200 sign languages as a primary language. Until now, those users had no dictation equivalent — the frictionless input that hearing users take for granted in voice-to-text workflows. SL2T closes that gap: signers can compose messages, run web searches, and query Gemini anywhere text input is ordinarily expected, including one-handed while holding the phone.

Architecture: Pose Landmarks Over Raw Video

The central privacy constraint shaped the entire inference pipeline. Rather than streaming raw camera frames to a server, SL2T ingests only geometric coordinates produced on-device by MediaPipe Holistic, which tracks landmark positions across the hands, arms, torso, head, and face. The original video is discarded immediately after landmark extraction, so the server-side model never sees pixel data — only a temporal sequence of body-point coordinates — sidestepping both privacy risk and the bandwidth cost of transmitting high-frame-rate video.

That coordinate sequence is translated directly into text. SL2T skips the intermediate gloss layer that prior work typically inserts between visual input and textual output. Glosses fail to represent non-manual markers (facial expressions, mouth morphemes, head tilts) and spatial grammatical constructions, and they impose a hard vocabulary ceiling. Removing glosses lets translation quality scale with data volume rather than with the completeness of a hand-crafted gloss lexicon. This approach shares conceptual ground with end-to-end strategies in other sequence transduction tasks, such as OCR systems that bypass intermediate character segmentation.

Joint training across more than 50 sign languages was a deliberate architectural choice. DeepMind reports that multilingual joint training causes the model to learn shared underlying structure, outperforming single-language models in internal experiments — a cross-lingual transfer strategy now validated for a visuo-spatial modality.

Benchmark Performance and Real-World Tradeoffs

On the FLEURS-ASL sd-test benchmark, which evaluates ASL-to-English translation quality, SL2T achieves a zero-shot BLEURT score of 70 — significantly higher than any previously reported score on that benchmark.

The team explicitly flags that benchmark optimisation alone does not guarantee usable real-world performance. Engineering work addressed streaming latency, hallucination suppression on non-signing inputs, left-handed signer fairness (approximately 10% of the signing population), and one-handed signing support. Published translation examples from the FLEURS-ASL benchmark show where errors currently cluster: rapid fingerspelling (the source word "prey" surfaces as "grey" in output), passive voice constructions, classifier depictions that encode instrument or manner, and tense resolution without prior context.

Governance and Deployment

DeepMind established the AI Sign Language Advisory Committee (AISLAC) to provide participatory oversight throughout development, drawing on multiple global Deaf organisations and subject-matter experts. The committee co-authored a joint impact report — released alongside SL2T 1.0 — that explicitly documents both capabilities and current limitations. The project was conceptualised by Sam Sepah, a Deaf Googler, and Deaf partners were involved in data collection and user studies. This structure, where affected communities hold direct influence over development priorities rather than serving only as test subjects, distinguishes SL2T's release process from typical model launches.

The feature is live in Gboard and Live Transcribe on Pixel 11, starting with ASL-to-English, with additional devices and sign languages on the roadmap. As multimodal AI moves toward real-time modality translation across diverse input types, SL2T demonstrates that privacy-preserving geometric abstraction combined with gloss-free end-to-end training and large-scale multilingual data can close gaps that annotation-heavy pipelines failed to bridge for decades.