Paper Image

Multi-modal reasoning with model selection

Published on:

12 October 2023

Primary Category:

Machine Learning

Paper Authors:

Xiangyan Liu,

Rongxue Li,

Wei Ji,

Tao Lin

Bullets

Key Details

Proposes model selection framework for multi-modal reasoning agents

Represents reasoning process as graph with subtask dependencies

Learns to predict execution status for input and model choices

Creates new benchmark dataset for model selection research

Enables dynamic model selection and improves reasoning robustness

AI generated summary

Multi-modal reasoning with model selection

This paper proposes a framework to select optimal AI models for each step in a multi-modal reasoning process. An agent decomposes complex tasks into subtasks and sequences AI models collaboratively. Model selection is critical but overlooked. This method represents reasoning as a graph, jointly modeling input, selected models, and subtask dependencies to predict execution status. Experiments on a new benchmark dataset demonstrate the approach enhances reasoning robustness by dynamic model selection.

Answers from this paper

Comments

No comments yet, be the first to start the conversation...

Sign up to comment on this paper

Sign Up