FlagEval

鲁棒性评测介绍

鲁棒性是指模型在面对不同类型的异常、噪声、干扰、变化或恶意攻击时，能够保持稳定性和高效性的能力。抽象来看，目前的基础模型（包括基于学习的深度学习模型）在给定数据输入 $X$ 的情况下，该参数化模型 $F_{θ} (\cdot)$ 经过其定义的计算，得到模型期望的输出 $Y$ 。通常鲁棒性可以理解为模型在有噪音的情况下，是否能给出正确的输出。具体来说，在给定扰动噪音 $Δ X$ 的情况下，模型的输出 $F_{θ} (X)$ 是否等于期望的输出 $Y$ ，我们量化该差异为 $Δ Y$ 。此外，构建的扰动噪音要求其不影响人对 $X$ 的理解。因此，在构建文本噪音时，评估所生成的测试样例会设计 $Δ X$ ，使 $X + Δ X$ 与原始的 $X$ 在人的理解上差异不大，但又容易使模型的输出犯错。

我们通过对实例进行扰动评估模型的鲁棒性。具体来说，我们对数据集进行不同程度的扰动，主要包括两个层面，一个是常见的现实世界中人类会犯的错误，分为三个级别：字符级别、单次级别、句子级别。字符级别包括相似字符的替换，键盘邻近字符的替换，单词级别则是词语的同义词替换以及代理模型语义空间单词的替换，句子级别主要是语言的回译。另外一个是针对性的扰动，例如采用代理模型进行对抗性的攻击。在进行上述的扰动后对于不同的原始数据集我们生成了不同的扰动数据集，通过评估模型在扰动数据集上的评测结果来计算模型在该数据集上的鲁棒性指标。

评测数据集

MMLU

鲁棒性数据集的构建不采用代理模型评估扰动结果，扰动分类字符级别，单词级别，句子级别。具体对数据集中的quesion字段进行扰动。

扰动后数据集名称如下：

扰动数据集名称	扰动方法
C-keyboard	disturbance-char-keyboard
C-ocr	disturbance-char-ocr
W-synonym	disturbance-word-synonym
W-wordembedding	disturbance-word-word-embedding
S-backtranslation	disturbance-sentence-back-translation

C、W、S、Adv分别是Char、Word、Sentence、adversarial的缩写

字符级别

随机挑选1到2个单词，每个单词挑选1到2个字符进行替换，扰动方式如下

ocr（o—>0）

ocr mmlu扰动数据集样例

{
	"question": "Which of the following best describes the balance the Supreme Coort has struck between the establishment clause and the free - exercise clause?",
	"subject":"high_school_government_and_politics",
	"choices":["Freedom of speech is protected except in certain situations, such as yelling \"fire\" in a crowded theater.","Once a church has been recognized by the federal government, its tax-exempt status can never be revoked.","Once Congress has created an administrative agency, that agency can be dissolved only by a constitutional amendment.","State-sponsored prayer during school hours is prohibited, but voluntary prayer by student groups before school is allowed."],
	"answer":3
}

keyboard（q—>w）

keyboard mmlu扰动数据集样例

{
	"question": "Which of the following best describes the balance the SupreJe Ckurt has struck between the establishment clause and the free - exercise clause?",
	"subject":"high_school_government_and_politics",
	"choices":["Freedom of speech is protected except in certain situations, such as yelling \"fire\" in a crowded theater.","Once a church has been recognized by the federal government, its tax-exempt status can never be revoked.","Once Congress has created an administrative agency, that agency can be dissolved only by a constitutional amendment.","State-sponsored prayer during school hours is prohibited, but voluntary prayer by student groups before school is allowed."],
	"answer":3
}

单词级别

随机挑选1到2个单词进行替换，扰动方式如下

word_embedding（采用glove6B_300d模型将挑选单词替换为语义相似的单词）

word_embedding mmlu扰动数据集样例

{
	"question": "Which of the following best describes the confidence the Supreme Court has gave between the establishment clause and the free - exercise clause?",
	"subject":"high_school_government_and_politics",
	"choices":["Freedom of speech is protected except in certain situations, such as yelling \"fire\" in a crowded theater.","Once a church has been recognized by the federal government, its tax-exempt status can never be revoked.","Once Congress has created an administrative agency, that agency can be dissolved only by a constitutional amendment.","State-sponsored prayer during school hours is prohibited, but voluntary prayer by student groups before school is allowed."],
	"answer":3
}

synonym（将挑选单词替换为同义词）

synonym mmlu扰动数据集样例

{
	"question": "Which of the following best describes the balance the Supreme Court has struck between the establishment clause and the free - exercise clause?",
	"subject":"high_school_government_and_politics",
	"choices":["Freedom of speech is protected except in certain situations, such as yelling \"fire\" in a crowded theater.","Once a church has been recognized by the federal government, its tax-exempt status can never be revoked.","Once Congress has created an administrative agency, that agency can be dissolved only by a constitutional amendment.","State-sponsored prayer during school hours is prohibited, but voluntary prayer by student groups before school is allowed."],
	"answer":3
}

句子级别

采用语言模型对句子进行扰动

back_translation（使用Helsinki-NLP/opus-mt-ROMANCE-en以及Helsinki-NLP/opus-mt-en-ROMANCE模型将题干部分翻译到另一种语言再翻译回来）

back_translation mmlu扰动数据集样例

{
	"question": "Which of the following best describes the balance reached by the Supreme Court between the establishment clause and the free exercise clause?",
	"subject":"high_school_government_and_politics",
	"choices":["Freedom of speech is protected except in certain situations, such as yelling \"fire\" in a crowded theater.","Once a church has been recognized by the federal government, its tax-exempt status can never be revoked.","Once Congress has created an administrative agency, that agency can be dissolved only by a constitutional amendment.","State-sponsored prayer during school hours is prohibited, but voluntary prayer by student groups before school is allowed."],
	"answer":3
}

鲁棒性指标（RB-index）

针对原始数据集以及不同的扰动数据集我们有 $A c c_{o r g}$ ， $A c c_{d i s t 1}$ ， $A c c_{d i s t 2}$ ， $A c c_{d i s t 3}$ ， $. . .$ ， $A c c_{d i s t T}$ （ $A c c$ 指模型在该数据集下的评测指标， $o r g$ 指原始数据集， $d i s 1. . . T$ 指不同的扰动数据集）。

在该数据集上的鲁棒性指标计算公式为：

$R o b u s t n e s s = \frac{1}{T * A c c_{o r g}} Σ_{i = 1}^{T} (A c c_{o r g} - A c c_{d i s t i})$

鲁棒性指标数值越小说明模型鲁棒性越好，可以为负数（多在NLP中出现）

鲁棒性评测介绍 ​

评测数据集 ​

MMLU ​

鲁棒性指标（RB-index） ​

鲁棒性评测介绍

评测数据集

MMLU

鲁棒性指标（RB-index）