When you think of tools infused with artificial intelligence (AI) these days, it’s natural for ChatGPT and Bing Chat to spring to mind. But Facebook owner Meta wants to change that with SeamlessM4T, an AI-powered “universal language translator” that could instantly convert any language in the world into whatever output you want.
Meta describes SeamlessM4T as “the first all-in-one multilingual multimodal AI translation and transcription model.” That’s quite a mouthful, but in simple terms, it means it can convert languages in a range of different ways, such as taking speech audio and switching it into text in a different tongue.
According to Meta, the tool’s speech recognition and translation features can work in a few different ways:
• Speech recognition for nearly 100 languages
• Speech-to-text translation for nearly 100 input and output languages
• Speech-to-speech translation for nearly 100 input languages and 36 output languages (including English)
• Text-to-text translation for nearly 100 languages
• Text-to-speech translation for nearly 100 input languages and 35 output languages (including English)
Meta says this will “allow people to communicate effortlessly through speech and text across different languages.”
Coming soon to Facebook?
SeamlessM4T is being released under a research license, and Meta states it’s doing this to “allow researchers and developers to build on this work.” As well as that, the metadata of the dataset that was used to train the translation model, called SeamlessAlign, is also being publicly released. This consists of “270,000 hours of mined speech and text alignments,” Meta claims.
However, Meta did not make it clear where these 270,000 hours of “mined speech” have been sourced. Concerns have been raised over the privacy implications of Meta’s work on AI chatbots, while other AI tools have already been caught stealing protected work. There will no doubt be fears that Meta could have done something similar when it trained SeamlessM4T.
There are already other translation tools like Google Translate that can convert text to text and speech to text, but Meta says its own efforts are superior. SeamlessM4T, Meta argues, “reduces errors and delays, increasing the efficiency and quality of the translation process.”
Meta has not said whether the new tool will be integrated into its apps like Facebook and Instagram, but the company did reveal that it aimed to “explore how this foundational model can enable new communication capabilities” in the future. We’ll have to see what that entails.
