V1

2022/12/28阅读：31主题：科技蓝

利用mAP评估目标检测模型

1. 从预测分数到类别标签

``y_true = ["positive", "negative", "negative", "positive", "positive", "positive", "negative", "positive", "negative", "positive"]``

``pred_scores = [0.7, 0.3, 0.5, 0.6, 0.55, 0.9, 0.4, 0.2, 0.4, 0.3]``

``import numpypred_scores = [0.7, 0.3, 0.5, 0.6, 0.55, 0.9, 0.4, 0.2, 0.4, 0.3]y_true = ["positive", "negative", "negative", "positive", "positive", "positive", "negative", "positive", "negative", "positive"]threshold = 0.5y_pred = ["positive" if score >= threshold else "negative" for score in pred_scores]print(y_pred)``
• `y_pred`
``['positive', 'negative', 'positive', 'positive', 'positive', 'positive', 'negative', 'negative', 'negative', 'negative']``

``r = numpy.flip(sklearn.metrics.confusion_matrix(y_true, y_pred))print(r)precision = sklearn.metrics.precision_score(y_true=y_true, y_pred=y_pred, pos_label="positive")print(precision)recall = sklea``
• 结果
``# Confusion Matrix (From Left to Right & Top to Bottom: True Positive, False Negative, False Positive, True Negative)[[4 2] [1 3]]# Precision = 4/(4+1)0.8# Recall = 4/(4+2)0.6666666666666666``

2. PR 曲线

1. 真实标签。
2. 样本的预测分数。
3. 将预测分数转换为类别标签的一些阈值。

``import numpyy_true = ["positive", "negative", "negative", "positive", "positive", "positive", "negative", "positive", "negative", "positive", "positive", "positive", "positive", "negative", "negative", "negative"]pred_scores = [0.7, 0.3, 0.5, 0.6, 0.55, 0.9, 0.4, 0.2, 0.4, 0.3, 0.7, 0.5, 0.8, 0.2, 0.3, 0.35]``

``[0.2,  0.25,  0.3,  0.35,  0.4,  0.45,  0.5,  0.55,  0.6,  0.65]``

``import sklearn.metricsdef precision_recall_curve(y_true, pred_scores, thresholds):    precisions = []    recalls = []        for threshold in thresholds:        y_pred = ["positive" if score >= threshold else "negative" for score in pred_scores]        precision = sklearn.metrics.precision_score(y_true=y_true, y_pred=y_pred, pos_label="positive")        recall = sklearn.metrics.recall_score(y_true=y_true, y_pred=y_pred, pos_label="positive")                precisions.append(precision)        recalls.append(recall)    return precisions, recalls``

``precisions, recalls = precision_recall_curve(y_true=y_true,                                              pred_scores=pred_scores,                                             thresholds=thresholds)``
• 以下是精度列表中的返回值。
``[0.5625, 0.5714285714285714, 0.5714285714285714, 0.6363636363636364, 0.7, 0.875, 0.875, 1.0, 1.0, 1.0]``
• 这是召回列表中的值列表。
``[1.0, 0.8888888888888888, 0.8888888888888888, 0.7777777777777778, 0.7777777777777778, 0.7777777777777778, 0.7777777777777778, 0.6666666666666666, 0.5555555555555556, 0.4444444444444444]``

``matplotlib.pyplot.plot(recalls, precisions, linewidth=4, color="red")matplotlib.pyplot.xlabel("Recall", fontsize=12, fontweight='bold')matplotlib.pyplot.ylabel("Precision", fontsize=12, fontweight='bold')matplotlib.pyplot.title("Precision-Recall Curve", fontsize=15, fontweight="bold")matplotlib.pyplot.show()``

f1 指标衡量准确率和召回率之间的平衡。当 f1 的值很高时，这意味着精度和召回率都很高。较低的 f1 分数意味着精确度和召回率之间的失衡更大。

``f1 = 2 * ((numpy.array(precisions) * numpy.array(recalls)) / (numpy.array(precisions) + numpy.array(recalls)))``

``matplotlib.pyplot.plot(recalls, precisions, linewidth=4, color="red", zorder=0)matplotlib.pyplot.scatter(recalls[5], precisions[5], zorder=1, linewidth=6)matplotlib.pyplot.xlabel("Recall", fontsize=12, fontweight='bold')matplotlib.pyplot.ylabel("Precision", fontsize=12, fontweight='bold')matplotlib.pyplot.title("Precision-Recall Curve", fontsize=15, fontweight="bold")matplotlib.pyplot.show()``

3. AP

``AP = numpy.sum((recalls[:-1] - recalls[1:]) * precisions[:-1])``
• 这是计算 `AP` 的完整代码。
``import numpyimport sklearn.metricsdef precision_recall_curve(y_true, pred_scores, thresholds):    precisions = []    recalls = []        for threshold in thresholds:        y_pred = ["positive" if score >= threshold else "negative" for score in pred_scores]        precision = sklearn.metrics.precision_score(y_true=y_true, y_pred=y_pred, pos_label="positive")        recall = sklearn.metrics.recall_score(y_true=y_true, y_pred=y_pred, pos_label="positive")                precisions.append(precision)        recalls.append(recall)    return precisions, recallsy_true = ["positive", "negative", "negative", "positive", "positive", "positive", "negative", "positive", "negative", "positive", "positive", "positive", "positive", "negative", "negative", "negative"]pred_scores = [0.7, 0.3, 0.5, 0.6, 0.55, 0.9, 0.4, 0.2, 0.4, 0.3, 0.7, 0.5, 0.8, 0.2, 0.3, 0.35]thresholds=numpy.arange(start=0.2, stop=0.7, step=0.05)precisions, recalls = precision_recall_curve(y_true=y_true,                                              pred_scores=pred_scores,                                              thresholds=thresholds)precisions.append(1)recalls.append(0)precisions = numpy.array(precisions)recalls = numpy.array(recalls)AP = numpy.sum((recalls[:-1] - recalls[1:]) * precisions[:-1])print(AP)``
• 这都是关于平均精度的。以下是计算 `AP` 的步骤摘要：
1. 使用模型生成预测分数。
2. 将预测分数转换为类别标签。
3. 计算混淆矩阵。
4. 计算精度和召回率指标。
5. 创建精确召回曲线。
6. 测量平均精度。

4. IoU

1. 图片
2. 图像检测结果的真实框

`IoU` 是根据下面等式计算的，通过将 2 个框之间的交叉区域除以它们的联合区域。 `IoU` 越高，预测越好。

1. `gt_box`：真实边界框。
2. `pred_box`：预测边界框。

``def intersection_over_union(gt_box, pred_box):    inter_box_top_left = [max(gt_box[0], pred_box[0]), max(gt_box[1], pred_box[1])]    inter_box_bottom_right = [min(gt_box[0]+gt_box[2], pred_box[0]+pred_box[2]), min(gt_box[1]+gt_box[3], pred_box[1]+pred_box[3])]    inter_box_w = inter_box_bottom_right[0] - inter_box_top_left[0]    inter_box_h = inter_box_bottom_right[1] - inter_box_top_left[1]    intersection = inter_box_w * inter_box_h    union = gt_box[2] * gt_box[3] + pred_box[2] * pred_box[3] - intersection        iou = intersection / union    return iou, intersection, union``

1. 左上角的 x 轴。
2. 左上角的 y 轴。

``gt_box = [320, 220, 680, 900]pred_box = [500, 320, 550, 700]``

``import imageioimport matplotlib.pyplotimport matplotlib.patchesdef intersection_over_union(gt_box, pred_box):    inter_box_top_left = [max(gt_box[0], pred_box[0]), max(gt_box[1], pred_box[1])]    inter_box_bottom_right = [min(gt_box[0]+gt_box[2], pred_box[0]+pred_box[2]), min(gt_box[1]+gt_box[3], pred_box[1]+pred_box[3])]    inter_box_w = inter_box_bottom_right[0] - inter_box_top_left[0]    inter_box_h = inter_box_bottom_right[1] - inter_box_top_left[1]    intersection = inter_box_w * inter_box_h    union = gt_box[2] * gt_box[3] + pred_box[2] * pred_box[3] - intersection        iou = intersection / union    return iou, intersection, unionim = imageio.imread("cat.jpg")gt_box = [320, 220, 680, 900]pred_box = [500, 320, 550, 700]fig, ax = matplotlib.pyplot.subplots(1)ax.imshow(im)gt_rect = matplotlib.patches.Rectangle((gt_box[0], gt_box[1]),                                       gt_box[2],                                       gt_box[3],                                       linewidth=5,                                       edgecolor='r',                                       facecolor='none')pred_rect = matplotlib.patches.Rectangle((pred_box[0], pred_box[1]),                                         pred_box[2],                                         pred_box[3],                                         linewidth=5,                                         edgecolor=(1, 1, 0),                                         facecolor='none')ax.add_patch(gt_rect)ax.add_patch(pred_rect)ax.axes.get_xaxis().set_ticks([])ax.axes.get_yaxis().set_ticks([])``

``iou, intersect, union = intersection_over_union(gt_box, pred_box)print(iou, intersect, union)``
• 结果

IoU 分数 0.54 意味着真实边界框和预测边界框之间有 54% 的重叠。看着方框，有人可能在视觉上觉得它足以得出模型检测到猫对象的结论。其他人可能会觉得模型还不准确，因为预测框与真实框不太吻合。

``import matplotlib.pyplotimport matplotlib.patchesimport imageiodef intersection_over_union(gt_box, pred_box):    inter_box_top_left = [max(gt_box[0], pred_box[0]), max(gt_box[1], pred_box[1])]    inter_box_bottom_right = [min(gt_box[0]+gt_box[2], pred_box[0]+pred_box[2]), min(gt_box[1]+gt_box[3], pred_box[1]+pred_box[3])]    inter_box_w = inter_box_bottom_right[0] - inter_box_top_left[0]    inter_box_h = inter_box_bottom_right[1] - inter_box_top_left[1]    intersection = inter_box_w * inter_box_h    union = gt_box[2] * gt_box[3] + pred_box[2] * pred_box[3] - intersection        iou = intersection / union    return iou, intersection, union, im = imageio.imread("pets.jpg")gt_box = [10, 130, 370, 350]pred_box = [30, 100, 370, 350]iou, intersect, union = intersection_over_union(gt_box, pred_box)print(iou, intersect, union)fig, ax = matplotlib.pyplot.subplots(1)ax.imshow(im)gt_rect = matplotlib.patches.Rectangle((gt_box[0], gt_box[1]),                                       gt_box[2],                                       gt_box[3],                                       linewidth=5,                                       edgecolor='r',                                       facecolor='none')pred_rect = matplotlib.patches.Rectangle((pred_box[0], pred_box[1]),                                         pred_box[2],                                         pred_box[3],                                         linewidth=5,                                         edgecolor=(1, 1, 0),                                         facecolor='none')ax.add_patch(gt_rect)ax.add_patch(pred_rect)gt_box = [645, 130, 310, 320]pred_box = [500, 60, 310, 320]iou, intersect, union = intersection_over_union(gt_box, pred_box)print(iou, intersect, union)gt_rect = matplotlib.patches.Rectangle((gt_box[0], gt_box[1]),                                       gt_box[2],                                       gt_box[3],                                       linewidth=5,                                       edgecolor='r',                                       facecolor='none')pred_rect = matplotlib.patches.Rectangle((pred_box[0], pred_box[1]),                                         pred_box[2],                                         pred_box[3],                                         linewidth=5,                                         edgecolor=(1, 1, 0),                                         facecolor='none')ax.add_patch(gt_rect)ax.add_patch(pred_rect)ax.axes.get_xaxis().set_ticks([])ax.axes.get_yaxis().set_ticks([])``

5. mAP

``y_true = ["positive", "negative", "positive", "negative", "positive", "positive", "positive", "negative", "positive", "negative"]pred_scores = [0.7, 0.3, 0.5, 0.6, 0.55, 0.9, 0.75, 0.2, 0.8, 0.3]``

``y_true = ["negative", "positive", "positive", "negative", "negative", "positive", "positive", "positive", "negative", "positive"]pred_scores = [0.32, 0.9, 0.5, 0.1, 0.25, 0.9, 0.55, 0.3, 0.35, 0.85]``

IoU 阈值列表从 0.2 到 0.9，步长为 0.25。

``thresholds = numpy.arange(start=0.2, stop=0.9, step=0.05)``

``precisions, recalls = precision_recall_curve(y_true=y_true,                                              pred_scores=pred_scores,                                              thresholds=thresholds)matplotlib.pyplot.plot(recalls, precisions, linewidth=4, color="red", zorder=0)matplotlib.pyplot.xlabel("Recall", fontsize=12, fontweight='bold')matplotlib.pyplot.ylabel("Precision", fontsize=12, fontweight='bold')matplotlib.pyplot.title("Precision-Recall Curve", fontsize=15, fontweight="bold")matplotlib.pyplot.show()precisions.append(1)recalls.append(0)precisions = numpy.array(precisions)recalls = numpy.array(recalls)AP = numpy.sum((recalls[:-1] - recalls[1:]) * precisions[:-1])print(AP)``

``mAP = (0.949 + 0.958)/2 = 0.9535``

总结

参考资料

[1]

Source: https://www.kdnuggets.com/2021/03/evaluating-object-detection-models-using-mean-average-precision.html

V1