Paper Image

Aligning multilingual AI through cultural reasoning

Published on:

9 September 2023

Primary Category:

Computation and Language

Paper Authors:

Bin Wang,

Zhengyuan Liu,

Xin Huang,

Fangkai Jiao,

Yang Ding,

Ai Ti Aw,

Nancy F. Chen

Bullets

Key Details

SeaEval benchmark evaluates multilingual AI across language, reasoning, culture, and cross-lingual knowledge sharing

Includes new datasets for cultural reasoning and cross-lingual consistency assessments

Models struggle with instruction sensitivity, label bias, inconsistent factual knowledge between languages, and imbalanced multilingual proficiency

Cultural reasoning and cross-lingual alignment are key for advancing multilingual AI

AI generated summary

Aligning multilingual AI through cultural reasoning

This paper introduces SeaEval, a benchmark to evaluate how well large AI models can understand and reason across languages and cultures. Key aspects assessed are language tasks, complex reasoning, cultural knowledge, and cross-lingual knowledge sharing. New datasets focus on cultural reasoning and cross-lingual consistency. Experiments reveal models still struggle with inconsistent responses to rephrased instructions, label bias, sharing factual knowledge between languages, and balanced multilingual proficiency.

Answers from this paper

Comments

No comments yet, be the first to start the conversation...

Sign up to comment on this paper

Sign Up