Paper Image

Syntactic structure boosts compositional generalization

Published on:

7 November 2023

Primary Category:

Computation and Language

Paper Authors:

Danial Kamali,

Parisa Kordjamshidi

Bullets

Key Details

Syntactic parsing provides structural information about command components

Attention masking forces models to respect syntax when grounding commands

Weight sharing reduces model complexity and improves generalization

State-of-the-art results on compositional benchmarks like ReaSCAN

Especially effective on test cases requiring deeper reasoning

AI generated summary

Syntactic structure boosts compositional generalization

This paper shows that using the syntactic structure of language commands, obtained through dependency parsing, improves compositional generalization in multimodal models that ground language in vision. Weight sharing in the transformer encoder also enhances the model. Results on compositional benchmarks like ReaSCAN demonstrate state-of-the-art performance, especially on test cases requiring deeper reasoning.

Answers from this paper

Comments

No comments yet, be the first to start the conversation...

Sign up to comment on this paper

Sign Up