Searching for general models that learn compositionally

Abstract. For all their impressive results, current machine learning algorithms are only able to solve one problem at a time, and they can’t generalize even across closely related tasks: A system trained on chess will have to learn to play checkers from scratch. One reasonable hypothesis for the difference between algorithms and flexible intelligent beings, who do not have this limitation, is that only the latter are capable of compositional learning. That is, when faced with a new challenge, an intelligent being will 1) try to combine skills it already possesses in novel ways to solve the new task; 2) if new skills are called for, it will store these newly acquired skills in its cognitive toolkit, to recycle them in the future. In this talk, I will describe our efforts to develop a very general algorithm that displays the same compositional learning capabilities. The ideas we are exploring include: allowing the system to progressively freeze some of its weights to store previously acquired skills; separating input/output steps from computation steps, to allow the system to perform multi-step “reasoning” on its acquired knowledge; learning objectives that encourage generalization rather than rote learning.
Joint work with: Tomas Mikolov, Germán Kruszewski, Adam Liska, Rahma Chaabouni, Allan Jabri

Short Bio. Marco Baroni received a PhD in Linguistics from the University of California, Los Angeles, in the year 2000. After several experiences in research and industry, he joined the Center for Mind/Brain Sciences of the University of Trento, where he’s been associate professor since 2013. In 2016, Marco joined the Facebook Artificial Intelligence Research team. Marco’s work in the areas of multimodal and compositional distributed semantics has received widespread recognition, including a Google Research Award, an ERC Starting Grant and and the ICAI-JAIR best paper prize. Marco’s current research focuses on developing computational systems that can flexibly adapt to new situations just like living beings do.