Broadcom just announced an AI Chipset that translates audio in real time directly to the device
Broadcom and a company called camb.ai are teaming up to bring the device’s audio rendering to the chipset. This will allow SOC-based devices to perform translation, decoding and audio interpretation functions without immersing themselves in the cloud. In other words, it can greatly improve consumer accessibility.
Companies promise ultra-low latency and improved privacy, that all processing is kept local to the user’s device. Wireless bandwidth should also be significantly reduced.
As for the audio description part, there is a demo video of the tool used in the episode in the film Ratatouille. The AI can be heard explaining the scene in different languages, in addition to the written version that appears on the screen. This seems to be very useful, especially for those with vision problems.
There is a big caveat. This is a tightly controlled piece with a lot of editing. We have no idea how this technology will work in the real world. Also, we don’t know how bad the details will be. It features a voice model already used by organizations such as NASSAR, ComCast and Eurovision.
The companies boast that this will enable “device translation in over 150 languages.” We don’t know when the chips will start showing up in TVs and other gadgets. The tech is in the testing phase now, so it will take time. Broadcom also recently teamed up with Opelai to help the latter company make its own chips.
