图像数据解释基准评测：图像多类别分类

此 notebook 演示了如何使用基准评测实用程序来评测图像数据解释器的性能。在此演示中，我们展示了针对图像多类别分类模型的 partition explainer 的解释性能。用于评估的指标是“保持正向”和“保持负向”。使用的掩码器是带有 Inpaint Telea 的 Image Masker。

新的基准评测实用程序使用新的 API，以 MaskedModel 作为用户导入模型的包装器，并评估输入的掩码值。

import json

import numpy as np
from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input

import shap
import shap.benchmark as benchmark

加载数据和模型

model = ResNet50(weights="imagenet")
X, y = shap.datasets.imagenet50()

类别标签映射

url = "https://s3.amazonaws.com/deep-learning-models/image-models/imagenet_class_index.json"
with open(shap.datasets.cache(url)) as file:
    class_names = [v[1] for v in json.load(file).values()]

定义评分函数

def f(x):
    tmp = x.copy()
    if len(tmp.shape) == 2:
        tmp = tmp.reshape(tmp.shape[0], *X[0].shape)
    preprocess_input(tmp)
    return model(tmp)

定义图像掩码器

masker = shap.maskers.Image("inpaint_telea", X[0].shape)

创建解释器对象

explainer = shap.Explainer(f, masker, output_names=class_names)

explainers.Partition is still in an alpha state, so use with caution...

运行 SHAP 解释

shap_values = explainer(X[1:3], max_evals=500, batch_size=50, outputs=shap.Explanation.argsort.flip[:4])

Partition explainer:  50%|█████████████████                 | 1/2 [00:00<?, ?it/s]

Partition explainer: 3it [03:15, 65.24s/it]

绘制 SHAP 解释

shap.image_plot(shap_values)

获取输出类别索引

output = f(X[1:3]).numpy()
num_of_outputs = 4
sorted_indexes = np.argsort(-output, axis=1)
sliced_indexes = np.array([index_list[:num_of_outputs] for index_list in sorted_indexes])

定义指标（排序顺序 & 扰动方法）

sort_order = "positive"
perturbation = "keep"

基准评测解释器

sequential_perturbation = benchmark.perturbation.SequentialPerturbation(
    explainer.model, explainer.masker, sort_order, perturbation
)
xs, ys, auc = sequential_perturbation.model_score(shap_values, X[1:2], indices=sliced_indexes[0])
sequential_perturbation.plot(xs, ys, auc)