IBM Research and MIT collaborate to advance frontiers of artificial intelligence in real-world audio visual comprehension technologies

YORKTOWN HEIGHTS, N.Y. - 20 Sep 2016: IBM Research (NYSE: IBM) today announced a multi-year collaboration with the Department of Brain & Cognitive Sciences at MIT to advance the scientific field of machine vision, a core aspect of artificial intelligence. The new IBM-MIT Laboratory for Brain-inspired Multimedia Machine Comprehension’s (BM3C) goal will be to develop cognitive computing systems that emulate the human ability to understand and integrate inputs from multiple sources of audio and visual information into a detailed computer representation of the world that can be used in a variety of computer applications in industries such as healthcare, education, and entertainment.

Guru Banavar, Chief Science Officer, Cognitive Computing and VP, IBM Research & Professor James DiCarlo, head of the Department of Brain & Cognitive Sciences (BCS) at the Massachusetts Institute Technology (MIT) (Credit: IBM Research)

The BM3C will address technical challenges around both pattern recognition and prediction methods in the field of machine vision that are currently impossible for machines alone to accomplish. For instance, humans watching a short video of a real-world event can easily recognize and produce a verbal description of what happened in the clip as well as assess and predict the likelihood of a variety of subsequent events, but for a machine, this ability is currently impossible.

Beginning in September 2016 in Cambridge, the BMC3 collaboration will bring together leading brain, cognitive, and computer scientists to conduct research in the field of unsupervised machine understanding of audio-visual streams of data, using insights from next-generation models of the brain to inform advances in machine vision. The vision is that this integrated cross-disciplinary research will lead to advances that are likely to change both our personal and professional lives - from helping clinicians improve elderly and disabled care to helping organizations maintain and repair complex machinery as well as a host of cross-industry applications.

“In a world where humans and machines are working together in increasingly collaborative relationships, breakthroughs in the field of machine vision will potentially help us live healthier more productive lives,” said Guru Banavar, Chief Scientist, Cognitive Computing and VP at IBM Research. “By bringing together brain researchers and computer scientists to solve this complex technical challenge, we will advance the state-of-the-art in AI with our collaborators at MIT.”

The BM3C will be led by Professor James DiCarlo, head of the Department of Brain & Cognitive Sciences (BCS) at MIT, who will be supported by a team of faculty members, researchers, and graduate students from both the Brain & Cognitive Sciences department and the MIT Computer Science and Artificial Intelligence Lab (CSAIL). MIT researchers will collaborate with IBM scientists and engineers who will provide technology expertise and advances from the IBM Watson platform.

“Our brain and cognitive scientists are excited to team up with cognitive computing scientists and engineers from IBM to achieve next-generation cognitive computing advances as exposed by next-generation models of the mind,” said Jim DiCarlo. "We believe that our fields are poised to make key advances in the very challenging domain of unassisted real-world audio-visual understanding and we are looking forward to this new collaboration."

Cognitive computing systems collaborate with human experts in natural ways, learn through this interaction, and help enable individuals and teams to make more informed decisions by making sense of massive amounts of unstructured data. Key to the success of these systems is ongoing research in machine learning and reasoning, machine vision, decision techniques, domain-specific knowledge understanding, data assurance and trust, and radically efficient computing infrastructures.

To this end, IBM Research has built a network of university research collaborations that put together IBM researchers with academic researchers in fields including computer science, engineering, brain and cognitive science, to help make fundamental advances in AI. The goal is to apply the advances to domains such as health, education, law, and business decisions. The network addresses the entire spectrum of cognitive computing capabilities from curating data and knowledge and developing new algorithms, to building the new computing infrastructures needed to optimize the new data-intensive workloads of a truly digital world.

IBM currently works with more than 250 universities to help teach courses in various cognitive computing disciplines. These courses offer students real-world case study learning experiences and access to Watson technology via the cloud. IBM also hosts university competitions, challenging students to identify industry-specific challenges and providing support to commercialize their ideas.