机器学习：Loss损失函数
我们分配了许多prior bboxes，我们要想让其预测类别和⽬标框信息，我们先要知道每个prior bbox和哪个⽬标对应，从⽽才能判断预测的是否准确，从⽽将训练进⾏下去。
不同⽅法 ground truth boxes 与 prior bboxes 的匹配策略⼤致都是类似的，但是细节会有所不同。这⾥我们采⽤SSD中的匹配策略，具体如下：
第⼀个原则：从ground truth box出发，寻找与每⼀个ground truth box有最⼤的jaccard overlap的prior bbox，这样就能保证每⼀个groundtruth box⼀定与⼀个prior bbox对应起来(jaccard overlap就是IOU，如图3所⽰，前⾯介绍过)。 反之，若⼀个prior bbox没有与任何ground truth进⾏匹配，那么该prior bbox只能与背景匹配，就是负样本。
⼀个图⽚中ground truth是⾮常少的，⽽prior bbox却很多，如果仅按第⼀个原则匹配，很多prior bbox会是负样本，正负样本极其不平衡，所以需要第⼆个原则。
第⼆个原则：从prior bbox出发，对剩余的还没有配对的prior bbox与任意⼀个ground truth box尝试配对，只要两者之间的jaccard overlap⼤于阈值（⼀般是0.5），那么该prior bbox也与这个ground truth进⾏匹配。这意味着某个ground truth可能与多个Prior box 匹配，这是可以的。但是反过来却不可以，因为⼀个prior bbox只能匹配⼀个ground truth，如果多个ground truth与某个prior bbox的IOU ⼤于阈值，那么prior bbox只与IOU最⼤的那个ground truth进⾏匹配。
注意：第⼆个原则⼀定在第⼀个原则之后进⾏，仔细考虑⼀下这种情况，如果某个ground truth所对应最⼤IOU的prior bbox⼩于阈值，并且所匹配的prior bbox却与另外⼀个ground truth的IOU⼤于阈值，那么该prior bbox应该匹配谁，答案应该是前者，⾸先要确保每个ground truth⼀定有⼀个prior bbox与之匹配。
⽤⼀个⽰例来说明上述的匹配原则：
黑茶保质期有多久图像中有7个红⾊的框代表先验框，黄⾊的是ground truths，在这幅图像中有三个真实的⽬标。按照前⾯列出的步骤将⽣成以下匹配项：
下⾯来介绍如何设计损失函数。
将总体的⽬标损失函数定义为 定位损失（loc）和置信度损失（conf）的加权和：
L(x,c,l,g) = \frac{1}{N}(L_{conf}(x,c)+\alpha L_{loc} (x,l,g)) (1)L(x,c,l,g)=N1(Lconf(x,c)+αLloc(x,l,g))(1)
其中N是匹配到GT（Ground Truth）的prior bbox数量，如果N=0，则将损失设为0；⽽ α 参数⽤于调整confidence loss和location loss之间的⽐例，默认 α=1。
confidence loss是在多类别置信度(c)上的softmax loss
其中i指代搜索框序号，j指代真实框序号，p指代类别序号，p=0表⽰背景。其中x^{p}_{ij}=\left\{1,0\right\}xijp={1,0} 中取1表⽰第i个prior bbox匹配到第 j 个GT box，⽽这个GT box的类别为 p 。C^{p}_{i}Cip表⽰第i个搜索框对应类别p的预测概率。此处有⼀点需要关注，公式前半部分是正样本（Pos）的损失，即分类为某个类别的损失（不包括背景），后半部分是负样本（Neg）的损失，也就是类别为背景的损失。
⽽location loss（位置回归）是典型的smooth L1 loss
其中，l为预测框，g为ground truth。(cx,xy)为补偿(regress to offts)后的默认框d的中⼼,(w,h)为默认框的宽和⾼。更详细的解释看-看下图：
值得注意的是，⼀般情况下negative prior bboxes数量 >> positive prior bboxes数量，直接训练会导致⽹络过于重视负样本，预测效果很差。为了保证正负样本尽量平衡，我们这⾥使⽤SSD使⽤的在线难例挖掘策略(hard negative mining)，即依据confidience loss对属于负样本的prior bbox进⾏排序，只挑选其中confidience loss⾼的bbox进⾏训练，将正负样本的⽐例控制在positive：negative=1:3。其核⼼作⽤就是只选择负样本中容易被分错类的困难负样本来进⾏⽹络训练，来保证正负样本的平衡和训练的有效性。
举个例⼦：假设在这 441 个 prior bbox ⾥，经过匹配后得到正样本先验框P个，负样本先验框 441−P
个。将负样本prior bbox按照prediction loss从⼤到⼩顺序排列后选择最⾼的M个prior bbox。这个M需要根据我们设定的正负样本的⽐例确定，⽐如我们约定正负样本⽐例为1:3时。我们就取M=3P，这M个loss最⼤的负样本难例将会被作为真正参与计算loss的prior bboxes，其余的负样本将不会参与分类损失的loss计算。
本⼩节介绍的内容围绕如何进⾏训练展开，主要是3块：
英吉沙刀先验框与GT框的匹配策略
损失函数计算
难例挖掘
这3部分是需要结合在⼀起理解，我们再整个梳理下计算loss的步骤
1）先验框与GT框的匹配
按照我们介绍的⽅案，为每个先验框都分配好类别，确定是正样本还是负样本。
计算loss
按照我们定义的损失函数计算 分类loss 和 ⽬标框回归loss
负样本不计算⽬标框的回归loss
难例挖掘
上⾯计算的loss中分类loss的部分还不是最终的loss
因为负样本先验框过多，我们要按⼀定的预设⽐例，⼀般是1:3，将loss最⾼的那部分负样本先验框拿出来，其余的负样本忽略，重新计算分类loss
完整loss计算过程的代码见model.py中的 MultiBoxLoss 类。
class MultiBoxLoss(nn.Module):
"""
The loss function for object detection.
对于Loss的计算，完全遵循SSD的定义，即 MultiBox Loss
This is a combination of:
(1) a localization loss for the predicted locations of the boxes.
(2) a confidence loss for the predicted class scores.
"""
def __init__(lf, priors_cxcy, threshold=0.5, neg_pos_ratio=3, alpha=1.):
super(MultiBoxLoss, lf).__init__()
lf.priors_cxcy = priors_cxcy
lf.priors_xy = cxcy_to_xy(priors_cxcy)
lf.threshold = threshold
<_pos_ratio = neg_pos_ratio
lf.alpha = alpha
lf.smooth_l1 = nn.L1Loss()
def forward(lf, predicted_locs, predicted_scores, boxes, labels):
"""
Forward propagation.
Forward propagation.
:param predicted_locs: predicted locations/ the 441 prior boxes, a tensor of dimensions (N, 441, 4)
:param predicted_scores: class scores for each of the encoded locations/boxes, a tensor of dimensions (N, 441, n_class) :param boxes: true object bounding boxes in boundary coordinates, a list of N tensors
:param labels: true object labels, a list of N tensors
:return: multibox loss, a scalar
"""
800字检讨batch_size = predicted_locs.size(0)
n_priors = lf.priors_cxcy.size(0)
n_class = predicted_scores.size(2)
asrt n_priors == predicted_locs.size(1) == predicted_scores.size(1)
true_locs = s((batch_size, n_priors, 4), dtype=torch.float).to(device) # (N, 441, 4)
芭提雅
true_class = s((batch_size, n_priors), dtype=torch.long).to(device) # (N, 441)
# For each image
for i in range(batch_size):
n_objects = boxes[i].size(0)
overlap = find_jaccard_overlap(boxes[i], lf.priors_xy) # (n_objects, 441)
# For each prior, find the object that has the maximum overlap
overlap_for_each_prior, object_for_each_prior = overlap.max(dim=0) # (441)
# We don't want a situation where an object is not reprented in our positive (non-background) priors -
# 1. An object might not be the best object for all priors, and is therefore not in object_for_each_prior.
# 2. All priors with the object may be assigned as background bad on the threshold (0.5).
# To remedy this -
# First, find the prior that has the maximum overlap for each object.
_, prior_for_each_object = overlap.max(dim=1) # (N_o)
# Then, assign each object to the corresponding maximum-overlap-prior. (This fixes 1.)
object_for_each_prior[prior_for_each_object] = torch.LongTensor(range(n_objects)).to(device)
# To ensure the priors qualify, artificially give them an overlap of greater than 0.5. (This fixes 2.)饭店英文
电脑突然自动关机
overlap_for_each_prior[prior_for_each_object] = 1.
# Labels for each prior
label_for_each_prior = labels[i][object_for_each_prior] # (441)
# Set priors who overlaps with objects are less than the threshold to be background (no object)
label_for_each_prior[overlap_for_each_prior < lf.threshold] = 0 # (441)
# Store
true_class[i] = label_for_each_prior
# Encode center-size object coordinates into the form we regresd predicted boxes to
true_locs[i] = cxcy_to_gcxgcy(xy_to_cxcy(boxes[i][object_for_each_prior]), lf.priors_cxcy) # (441, 4)
# Identify priors that are positive (object/non-background)
positive_priors = true_class != 0 # (N, 441)
# LOCALIZATION LOSS
# Localization loss is computed only over positive (non-background) priors
loc_loss = lf.smooth_l1(predicted_locs[positive_priors], true_locs[positive_priors]) # (), scalar
# Note: indexing with a torch.uint8 (byte) tensor flattens the tensor when indexing is across multiple dimensions (N & 441) # So, if predicted_locs has the shape (N, 441, 4), predicted_locs[positive_priors] will have (total positives, 4)
# CONFIDENCE LOSS
华清池温泉# Confidence loss is computed over positive priors and the most difficult (hardest) negative priors in each image
# That is, FOR EACH IMAGE,
# we will take the hardest (neg_pos_ratio * n_positives) negative priors, i.e where there is maximum loss
# This is called Hard Negative Mining - it concentrates on hardest negatives in each image, and also minimizes pos/neg imbalance
# Number of positive and hard-negative priors per image
n_positives = positive_priors.sum(dim=1) # (N)
n_hard_negatives = lf.neg_pos_ratio * n_positives # (N)
# First, find the loss for all priors
conf_loss_all = lf.cross_entropy(predicted_scores.view(-1, n_class), true_class.view(-1)) # (N * 441)
conf_loss_all = conf_loss_all.view(batch_size, n_priors) # (N, 441)
# We already know which priors are positive
conf_loss_pos = conf_loss_all[positive_priors] # (sum(n_positives))
# Next, find which priors are hard-negative
# To do this, sort ONLY negative priors in each image in order of decreasing loss and take top n_hard_negatives
conf_loss_neg = conf_loss_all.clone() # (N, 441)
conf_loss_neg[positive_priors] = 0. # (N, 441), positive priors are ignored (never in top n_hard_negatives)
conf_loss_neg, _ = conf_loss_neg.sort(dim=1, descending=True) # (N, 441), sorted by decreasing hardness
hardness_ranks = torch.LongTensor(range(n_priors)).unsqueeze(0).expand_as(conf_loss_neg).to(device) # (N, 441)
hard_negatives = hardness_ranks < n_hard_negatives.unsqueeze(1) # (N, 441)
conf_loss_hard_neg = conf_loss_neg[hard_negatives] # (sum(n_hard_negatives))
# As in the paper, averaged over positive priors only, although computed over both positive and hard-negative priors
conf_loss = (conf_loss_hard_neg.sum() + conf_loss_pos.sum()) / n_positives.sum().float() # (), scalar
# return TOTAL LOSS
友谊岁月return conf_loss + lf.alpha * loc_loss
