M23.1 CONNECT THE MECHANISM
Connect information expressed in different forms
A phone hears a kettle whistle and sees steam. Where should the two clues meet? Pick the wrong place and some questions become impossible to answer.
LESSON OVERVIEW15 min lesson
Lesson overview
A phone hears a kettle whistle and sees steam. Where should the two clues meet? Pick the wrong place and some questions become impossible to answer.
What you’ll explore
- Multimodal systems align information across modalities and combine it through defined interfaces; paired data, timing, and missing-modality behavior determine what can be learned.
GO TO THE SOURCE
Original explanations, connected to the research.
Multimodal Machine Learning: A Survey and Taxonomy (Baltrušaitis, Ahuja & Morency, 2017)Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering (Goyal et al., 2017)Flamingo: a Visual Language Model for Few-Shot Learning (Alayrac et al., 2022)Learning Transferable Visual Models From Natural Language Supervision (CLIP; Radford et al., 2021)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.