pointpillars代码阅读----检测指标篇
Brief
kitti衡量指标。
代码⽂件在kitti_datat.py中（SECOND）
代码 1
五虎断门枪detections += net(example)
。。。
result_dict = eval_datat.datat.evaluation(detections,
str(result_path_step))
（1）detections 经过了⽹络的前向传播得到了预测的box_pred,cls_pred和dir_pred
（2）后续的评价⽹络是研究的重点。
评价函数----evaluation
该函数定义为：
def evaluation(lf, detections, output_dir):
对应的gt如下，这是最原始的
ground_truth_annotations format:
{
bbox: [N, 4], if you fill fake data, MUST HAVE >25 HEIGHT
alpha: [N], you can u -10 to ignore it.
occluded: [N], you can u zero.
truncated: [N], you can u zero.
name: [N]
location: [N, 3] center of 3d box.
dimensions: [N, 3] dim of 3d box.
rotation_y: [N] angle.
蜡开头的四字成语}
输⼊的第⼀个参数是预测的内容，第⼆个参数是想要输出的⽂件夹。gt是⼀个⾃带的属性
1 convert_detection_to_kitti_annos
gt_annos = [info["annos"] for info in lf._kitti_infos]
dt_annos = lf.convert_detection_to_kitti_annos(detections)
（1）将预测出来的detection转化为kitti_annos。和对应的gt_nos对应.经过转化后，和gt⼀样含有5个维度[bbox, location, dimensions, rotation_y, score]解释：
bbox：【xmin，ymin，xmax，ymax】⼆维边界框
location：3D中⼼
dimensions：w,h,l
rotation_y：yaw旋转⾓度
score：预测的置信度
offical_eval
⽂件在eval.py中。
def get_official_eval_result(gt_annos,
dt_annos,
current_class,
difficultys=[0, 1, 2],
z_axis=1,
z_center=1.0):
（1）作者代码中写了8个类别的eval，如下图，stack之后min_overlaps 变成再经过训练类别选取得到：min_overlaps 为do_eval_v3
采⽤这个函数进⾏计算对应的准确率。
def do_eval_v3(gt_annos,
dt_annos,
current_class,
min_overlaps,
compute_aos=Fal,
difficultys=(0, 1, 2), z_axis=1,
z_center=1.0):
（1）gt_annos和dt_annos⼀样含有上⾯的5个元素的dict。min_overlaps在这⾥是。
如下如：这⾥的的含义：
（2）但是在输出界⾯上会有⼀个第四⾏，作者后续计算了⼀个mAPaos 的精度。应该是朝向预测？
给闺蜜的话
[2,3,8]
[2,3,4]
[2,3,4][2,3,5]
if compute_aos:
mAPaos = get_mAP(metrics["bbox"]["orientation"][j, :, i])
detail[class_name][f"aos"] = list()
mAPaos = ", ".join(f"{v:.2f}" for v in mAPaos)
result += print_str(f"aos AP:{mAPaos}")
coco eval
还有⼀种评价指标:coco_eval，对应的含义和官⽅检测是⼀样的。魔方的玩法
result_coco = get_coco_eval_result(
gt_annos,
dt_annos,
lf._class_names,
z_axis=z_axis,
z_center=z_center)
最后检测效果
在kitti上进⾏了多分类，效果如下：
采⽤官⽅的计算⽅法：
Evaluation official
Car AP(Average Precision)@0.70, 0.70, 0.70:
bbox AP:90.76, 89.45, 88.17
bev AP:89.83, 86.53, 79.62
3d AP:88.02, 77.50, 75.73
aos AP:0.23, 1.26, 2.32
Car AP(Average Precision)@0.70, 0.50, 0.50:心肺复苏的目的
bbox AP:90.76, 89.45, 88.17
bev AP:90.79, 90.06, 89.46
3d AP:90.79, 89.98, 89.27
aos AP:0.23, 1.26, 2.32
Cyclist AP(Average Precision)@0.50, 0.50, 0.50:
bbox AP:82.30, 70.63, 69.67
bev AP:79.81, 66.11, 60.95
3d AP:77.80, 60.30, 58.31
aos AP:0.90, 2.06, 1.94
Cyclist AP(Average Precision)@0.50, 0.25, 0.25:
bbox AP:82.30, 70.63, 69.67
bev AP:85.80, 68.40, 66.92
3d AP:85.79, 68.40, 66.91
aos AP:0.90, 2.06, 1.94
Pedestrian AP(Average Precision)@0.50, 0.50, 0.50:
bbox AP:62.27, 60.92, 54.80
bev AP:66.87, 59.97, 54.21
3d AP:60.86, 53.67, 51.13
aos AP:6.38, 6.48, 6.01
Pedestrian AP(Average Precision)@0.50, 0.25, 0.25:
bbox AP:62.27, 60.92, 54.80
bev AP:79.75, 73.70, 71.25
3d AP:79.75, 73.68, 71.25
aos AP:6.38, 6.48, 6.01
Van AP(Average Precision)@0.70, 0.70, 0.70:
bbox AP:45.80, 39.60, 34.79
bev AP:45.12, 39.24, 34.52
3d AP:39.13, 34.98, 30.87
aos AP:0.46, 0.24, 0.62
Van AP(Average Precision)@0.70, 0.50, 0.50:
bbox AP:45.80, 39.60, 34.79
bev AP:45.57, 40.04, 39.10
3d AP:45.56, 39.87, 39.02
aos AP:0.46, 0.24, 0.62
疑惑，为什么每⼀个类别含有两个得分，分别对应mod和easy两种
coco⽅式
Evaluation coco
Car coco AP@0.50:0.05:0.95:
bbox AP:72.62, 68.94, 67.36
供应链案例bev AP:70.40, 66.24, 64.21
3d AP:61.57, 56.84, 54.95
aos AP:0.17, 0.93, 1.76
Cyclist coco AP@0.25:0.05:0.70:
bbox AP:79.73, 67.53, 65.18
bev AP:73.43, 58.34, 56.04
3d AP:70.07, 54.83, 52.43
aos AP:0.87, 1.93, 1.79
Pedestrian coco AP@0.25:0.05:0.70:
bbox AP:54.61, 53.78, 50.98
bev AP:59.19, 54.69, 50.82
3d AP:54.40, 50.16, 47.19
aos AP:5.73, 6.01, 5.94
Van coco AP@0.50:0.05:0.95:
bbox AP:33.89, 29.62, 27.60
bev AP:35.59, 30.77, 28.82
3d AP:28.08, 24.45, 22.28
aos AP:0.36, 0.16, 0.54
函数debug
1. 在train.py中，对所⽤的eval数据⽣成了dict，再经过nms得到了最后的detections(dict,shape是[num_batch])，函数调⽤代码如下，
result_dict = eval_datat.datat.evaluation(detections,
str(result_path_step))
2. 该函数的实现在kitti_datat.py⽂件中的class KittiDatat的evaluation函数。上⽂也已经说过了。
3. get_official_eval_result函数定义了官⽅的测试函数。该函数主要调⽤函数eval_class_v3，eval_class_v3定义如下：
def eval_class_v3(gt_annos,
dt_annos,
current_class,
difficultys,
metric,
min_overlaps,
compute_aos=Fal,
z_axis=1,
z_center=1.0,
num_parts=50):
（1）输⼊参数如下表：
输⼊参数shape含义
gt_annos list,len=eval_num
list中的每⼀个元素分别表⽰每⼀个点云的gt内容。每⼀个gt内容包括有{’name’,truncated,occluded,alpha,bbox,dimensions,location, rotation_y,score,index,group_ids,difficulty,num_points_in_gt}这些元素的含义和⼀⼀对应。bbox表⽰了⼆维bbox，,
dim，location,rotation_y共同表⽰三维bbox
dt_annos list,len=eval_num内容同上，不过是预测出来的
current_class list,len=num_list(=4)只检测{car,cyclist,pedestrain,van}四类difficultys list,len=3内容是{0,1，2}应该是指的困难级别
min_overlaps[2,3,4]其中2表⽰{easy,modrate（重要待补充）}, 3表⽰{box,bev,3d},4表⽰4类
metrics int eval type. 0: bbox, 1: bev, 2: 3d
输⼊参数shape含义国家公派留学
（2）分成75个块（3769//50=75），分别进⾏iou计算，作者表⽰这样分块可以加速，应该是并⾏计算吧。每⼀块50个.
。。。。内容太多，先放弃了
重点
本篇博客主要是帮助博主⾃⼰理解kitti evalute的各个含义，
对于官⽅的预测⽅式，对车的如下，每⼀个class都有两种，具体含义如最后的图：
Car AP(Average Precision)@0.70, 0.70, 0.70:
bbox AP:90.76, 89.45, 88.17
bev AP:89.83, 86.53, 79.62
3d AP:88.02, 77.50, 75.73
aos AP:0.23, 1.26, 2.32
Car AP(Average Precision)@0.70, 0.50, 0.50:
车载音乐dj歌曲大全
bbox AP:90.76, 89.45, 88.17
bev AP:90.79, 90.06, 89.46
3d AP:90.79, 89.98, 89.27
aos AP:0.23, 1.26, 2.32
其实博主之前就是这么理解的，但是很不解的是同⼀个class上下两次⼀样的easy的最⼩IOU（=0.7）的时候却有着不⼀样的AP是什么意思?。最后的aos的含义根据作者的回复是可以忽略的。下⾯是作者的回复：
Car@0.70, 0.70, 0.70 means evaluate car performance in easy, moderate, hard and u 0.7 (easy), 0.7 (mod), 0.7 (hard) as overlap threshold.
bbox means calculate overlap (interction of union area) by bbox overlap, bev means bev overlap, 3d means 3d overlap. the aos… you can ignore this.
