Paper Image

Selectively updating foundation models to expand knowledge

Published on:

23 August 2023

Primary Category:

Computer Vision and Pattern Recognition

Paper Authors:

Wenxuan Zhang,

Paul Janson,

Rahaf Aljundi,

Mohamed Elhoseiny

Bullets

Key Details

Proposes method to expand foundation models' knowledge when learning new concepts

Identifies model layers & parameters most relevant to new data

Updates only small subset of model parameters

Demonstrates knowledge expansion on new datasets

Preserves original model capabilities with minimal loss

AI generated summary

Selectively updating foundation models to expand knowledge

This paper proposes a method to expand the knowledge of large foundation models like CLIP when learning new concepts, while preserving their original capabilities. It identifies model layers and parameters most relevant to new data, and updates only those sparsely. Evaluated on classification tasks, it expanded knowledge on new datasets, with minimal loss of original model strengths.

Answers from this paper

Comments

No comments yet, be the first to start the conversation...

Sign up to comment on this paper

Sign Up