Google announced its latest accessibility update Monday — real-time phone call captions for Google Pixel users who cannot or prefer not to speak on the phone.
The Live Caption feature for phones will let users initiate two-way text-to-speech tools within a call, providing live captions of what the other person says in real time and letting users type a response that’s read out loud by Google’s text-to-speech voice during the call. Once initiated, the phone call converts into an easily navigable text chat exchange on the user’s side. It also alerts the caller on the other line that the person is using these services, making it easier for everyone involved.
When receiving a phone call, users can enable Live Caption directly on screen.
Credit: Google
When enabled, Live Caption automatically transcribes the speaker’s voice.
Credit: Google
Pixel users can type their responses, which are then read out loud to the caller.
Credit: Google
Live Caption is the product of Google’s Live Relay research, an initiative that began in 2019 to devise better captioning and other live accessibility features for callers who are deaf. Live Caption tools were originally introduced to Google apps and devices to help caption audio played through computers and phones, as well as transcribe environmental sounds and external noises. Initial versions of live captions for calls were introduced to select devices in 2020, and then expanded in 2021. The newest iteration is an updated tool available across Pixel phones. It’s also the latest in a series of accessibility features from the company, like Live Transcribe & Notification, which provides two-way conversation transcription and can alert users to ambient sounds around them, as well as Google Meet closed captioning and improved captions for YouTube content.
Sam Sepah, accessibility research program manager and a member of the Deaf community himself, consulted on the research and worked with Google’s design team to ensure the company’s accessibility features are living up to the day-to-day needs of users who are deaf. “As an individual, as an adult, I want to have a life without relying on too many people or too many services,” Sepah said, explaining his own needs. “I want to be able to make independent decisions about how I communicate with the world. Making personal phone calls is a hugely powerful experience.”
Important to the feature, and for many users, Sepah said, is the fact that it’s integrated into the device, without the need for third-party apps, other tech, or traditional relay services, which provide a variety of accessibility tools and even video interpreters that help individuals who are deaf to make phone calls. He explained that these services have sometimes proven difficult for him and others to use on the daily, especially for simple day-to-day tasks like quickly calling a friend while in line at the store.
“A lot of people who do not have a disability or hearing loss might not realize that Live Caption for calls is, really, a transformational experience for our community, especially the Deaf and hard-of-hearing community,” Sepah said. “Live captions are really exciting for me — I thought that I would see that sort of technology when I was an old man.”
To use Live Caption’s text-to-speech responses, all Pixel 6 users need is their phones. Access Live Captions in the “Accessibility” section of your phone’s settings menu. When receiving or placing a call, wait for the Live Caption pop-up window or select the small captions logo at the bottom right side of the calling screen. You can then tap the keyboard icon under the captions to send a response.
Live Caption for transcribing the voices of callers is currently available in English for Pixel 2 or newer phones. The option to respond in calls with text-to-speech technology will be available in English, French, Italian, German, and Spanish, for Pixel 6 and Pixel 6 Pro this month.