arXiv：cs.CL· Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio Andr\'e Machado Porto·· 10 小时前
Manacá-1B-Instruct-2：面向巴西葡萄牙语原生 LLM 的可复现安全对齐，含 DPO 负结果与 LLM 裁判静默失效
Manac\'a-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge
AI 导读
Manacá-1B-Instruct-2 发布，这是对 1.72B 参数、从零训练的巴西葡萄牙语模型 Manacá-1B 的第二代后训练对齐，SFT 语料含 291,223 条样本。
正文
Abstract:Brazilian Portuguese remains underserved by open-source language models, and the few that are safety-aligned rarely document what did not work or validate comparisons with statistical significance. We present Manacá-1B-Instruct-2, the second-generation post-training alignment of Manacá-1B (1.72B parameters, trained from scratch for PT-BR), with an SFT corpus of 291,223 examples. We document three DPO attempts, all negative, including a case where the training reward margin reaches 1.0 while free generation does not change at all, and show that safety came from a dedicated safety-SFT (LLM-jp AnswerCarefully style), refined over three iterations until closing the safety-helpfulness frontier. A controlled scan of the fraction of general data in the safety-SFT mix (10% to 60%) reveals a non-monotonic over-refusal curve with a knee point at 40-50%. The final model is compared against four external PT-BR models on the same items, with Wilson confidence intervals and paired McNemar tests with Holm-Bonferroni correction: the lower bound of its safety interval (94.1%) lies above the upper bound of all four baselines. We also report a methodological finding for LLM-judge evaluation: on two of six safety items in MT-Bench-PT, the judge categorically refuses to assign a score regardless of the evaluated response, silently reducing the effective N of that category in every prior report of the project. We complement this with an n-gram decontamination check and a Chinchilla-anchored compute-budget table that attributes the gap in general capability to pre-training scale rather than alignment methodology. All code, raw data, logs, and scripts that produce every reported number are publicly released.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.20570 [cs.CL] |
| (or arXiv:2610.20570v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20570 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bruno Menezes [view email]
[v1]
Wed, 14 Oct 2026 17:53:46 UTC (109 KB)
来源：arXiv：cs.CL · arxiv.org