Mark Zuckerberg owned Meta on Tuesday unveiled a new AI model 'ImageBind' that combines different senses just like people do.
In a Facebook post, Zuckerberg shared the video explaining how ImageBind works. "It understands images, video, audio, depth, thermal and spatial movement," he said.
Introducing ImageBind by Meta AI: the first AI model capable of binding data from six modalities at once. This breakthrough brings machines one step closer to the human ability to bind together information from many different senses.
More on this new open source work ⬇️May 9, 2023AdvertisementIn a statement, Meta AI said that the model learns a single embedding, or shared representation space, not just for text, image/video, and audio, but also for sensors that record depth (3D), thermal (infrared radiation), and inertial measurement units (IMU), which calculate motion and position.
ImageBind equips machines with a holistic understanding that connects objects in a photo with how they will sound, their 3D shape, how warm or cold they are, and how they move.
AdvertisementThe company said that ImageBind can outperform prior specialist models trained individually for one particular modality, as described in their research paper. But most important, it helps advance AI by enabling machines to better analyze many different forms of information together.
"For example, using ImageBind, Meta's Make-A-Scene could create images from audio, such as creating an image based on the sounds of a rain forest or a bustling market," the company added.
"ImageBind is part of Meta's efforts to create multimodal AI systems that learn from all possible types of data around them. As the number of modalities increases, ImageBind opens the floodgates for researchers to try to develop new, holistic systems, such as combining 3D and IMU sensors to design or experience immersive, virtual worlds," the artificial intelligence company said.
Essential Business Intelligence, Continuous LIVE TV, Sharp Market Insights, Practical Personal Finance Advice and Latest Stories — On NDTV Profit.