23 August 2023
Computer Vision and Pattern Recognition
Proposes method to expand foundation models' knowledge when learning new concepts
Identifies model layers & parameters most relevant to new data
Updates only small subset of model parameters
Demonstrates knowledge expansion on new datasets
Preserves original model capabilities with minimal loss
Selectively updating foundation models to expand knowledge
This paper proposes a method to expand the knowledge of large foundation models like CLIP when learning new concepts, while preserving their original capabilities. It identifies model layers and parameters most relevant to new data, and updates only those sparsely. Evaluated on classification tasks, it expanded knowledge on new datasets, with minimal loss of original model strengths.
No comments yet, be the first to start the conversation...
Sign up to comment on this paper