arXiv：cs.AI· Yue Xu, Wenjie Wang·· 16 小时前
Genres：从社会关系视角评测多模态大语言模型中的性别偏见
From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationship
AI 导读
研究者提出 Genres，一个从叙事生成中的社会关系视角评估多模态大语言模型（MLLMs）性别偏见的基准，基于 Fiske 关系模型理论构建了 1,440 组多模态叙事诱导配对，覆盖四种基本关系类型、多样社会场景和三个年龄组。对六款近期 MLLMs 的实验显示，性别差异持续存在且高度依赖关系类型与模型，部分模式在人际互动情境下更为明显。团队还评估了两种互补的生成阶段缓解策略。
正文
Abstract:Ensuring the trustworthiness of Multimodal Large Language Models (MLLMs) requires rigorous evaluation of gender bias, particularly in socially sensitive domains. Existing benchmarks primarily assess bias in isolated contexts and are less suited to capturing biases that emerge through interpersonal interactions. We introduce Genres, a benchmark for evaluating gender bias in MLLMs through the lens of social relationships in narrative generation. Grounded in Fiske's relational models theory, Genres comprises 1,440 multimodal narrative elicitation pairs spanning four fundamental relationship types, diverse social scenarios, and three age groups. Its dual-character generation tasks elicit socially situated interpersonal narratives and enable fine-grained assessment across multiple dimensions of gender bias. Experiments on six recent MLLMs reveal persistent yet strongly relationship- and model-dependent gender disparities, with some patterns becoming more pronounced in interpersonal settings. We further evaluate two complementary generation-stage mitigation strategies. Our findings demonstrate the value of relationship-aware evaluation for uncovering subtle, interaction-driven gender bias in MLLMs.
| Comments: | Accepted by COLM2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY) |
| Cite as: | arXiv:2506.23101 [cs.CL] |
| (or arXiv:2506.23101v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2506.23101 arXiv-issued DOI via DataCite |
Submission history
From: Yue Xu [view email]
[v1]
Sun, 29 Jun 2025 06:03:21 UTC (1,290 KB)
[v2]
Tue, 13 Oct 2026 03:18:15 UTC (5,208 KB)
来源：arXiv：cs.AI · arxiv.org