由6,14以及68点⼈脸关键点计算头部姿态
前⾔
关于头部姿态估计理论部分的内容，⽹络上包括我所列的参考⽂献中都有⼤量概述，我不再重复。这⾥直⼊主题，如何通过图像中2D⼈脸关键点计算出头部姿态⾓，具体就是计算出俯仰⾓(pitch)，偏航⾓(yaw)和翻滚⾓(roll)。
计算头部姿态需要的若⼲数据：
1、⾸先，你需要拿到2D⼈脸关键点坐标，通过dlib的可以很容易的计算出⼈脸68个关键点的位置。但是在具体计算头部姿态的时候可以选择性的使⽤这68个关键点。我看⽹上⼤量的⽂章都是摘取的其中6个关键点(如下图)。我分别试验了6点、14点以及68点这三种情形。
2、从上⼀步计算出的⼈脸关键点中选出N(例如:6)个，但因为这N个点只是2D坐标，你还要想办法计算出他们对应的3D坐标。你或许会想，这⼀步是不是需要图像中的⼈脸的3D模型?只能说理论上是这样的，⽽实际应⽤过程中，⼀个通⽤的3D模型就可以满⾜了，更近⼀步，你只需要知道通⽤模型中关键点对应的3D坐标位置就可以⼲活了，例如：
⿐尖: ( 0.0, 0.0, 0.0)
下额 : ( 0.0, -330.0, -65.0)精忠报国简谱
左眼⾓ : (-225.0f, 170.0f, -135.0)
右眼⾓：( 225.0, 170.0, -135.0)
左嘴⾓：-150.0, -150.0, -125.0)
右嘴⾓：(150.0, -150.0, -125.0)
3、摄像机的内部参数。
实测
1、⼈脸关键点
关键点的获取由dlib来实现，其中需要⽤到官⽅预训练好的模型，地址如下:
具体实现可以参考以下代码：
#!/usr/bin/env python
# coding: utf-8
import dlib
import cv2
import matplotlib.pyplot as plt
import numpy as np
detector = _frontal_face_detector() #加载dlib⾃带的⼈脸检测器
pic_path = "1033.jpg"
孩子识字img = cv2.imread(pic_path)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) #opencv读到的是BGR的矩阵
faces = detector(img, 1) #检测⼈脸，返回检出的⼈脸框，可能有多张
r = faces[0] #只取第⼀张脸
x0,y0,x1,y1 = r.left(),r.top(),r.right(),r.bottom()
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat') #加载关键点检测模型ldmk = predictor(img, face) #对指定的⼈脸进⾏特征点检测
points_68 = np.matrix([[p.x, p.y] for p in ldmk.parts()])
for _, p in enumerate(points_68):
pos = (p[0,0], p[0,1])
cv2.circle(img, pos, 2, (0,255,255), -1, 8)
plt.imshow(img)
2、头部姿态计算
下⾯我分别尝试了6点、14点以及68点做头部姿态的计算。
怎么提高智商
what?⼀个完全正确的都没有。就68点的情形靠谱点，⾄少在Y,X⽅向上基本是对的，但Z⽅向⽐较诡异。我这个出的是由欧拉⾓计算出来的实际⾓度。我把代码附在后⾯，欢迎⼤家⼀起来改进！
【参考⽂献】
【源代码】
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import print_function
import os
import cv2
import sys
import numpy as np
import math
class PoEstimator:
"""Estimate head po according to the facial landmarks"""
def __init__(lf, img_size=(480, 640)):
lf.size = img_size
# 3D model points.
(0.0, 0.0, 0.0), # No tip
(0.0, -330.0, -65.0), # Chin
(-225.0, 170.0, -135.0), # Left eye left corner
(225.0, 170.0, -135.0), # Right eye right corne
利润分成合作协议(-150.0, -150.0, -125.0), # Left Mouth corner
(150.0, -150.0, -125.0) # Right mouth corner
], dtype=float) / 4.5
(6.825897, 6.760612, 4.402142),
(1.330353, 7.122144, 6.903745),
(-1.330353, 7.122144, 6.903745),
(-6.825897, 6.760612, 4.402142),
爱和陪伴
(5.311432, 5.485328, 3.987654),
(1.789930, 5.393625, 4.413414),
(-1.789930, 5.393625, 4.413414),
(-5.311432, 5.485328, 3.987654),
(2.005628, 1.409845, 6.165652),
(-2.005628, 1.409845, 6.165652),
(2.774015, -2.080775, 5.048531),
(-2.774015, -2.080775, 5.048531),
(0.000000, -3.116408, 6.097667),
(0.000000, -7.415691, 4.070434)], dtype=float)
[-73.393523, -29.801432, -47.667532], [-72.775014, -10.949766, -45.909403], [-70.533638, 7.929818, -44.84258 ], [-66.850058, 26.07428 , -43.141114], [-59.790187, 42.56439 , -38.635298], [-48.368973, 56.48108 , -30.750622], [-34.121101, 67.246992, -18.456453], [-17.875411, 75.056892, -3.609035], [ 0.098749, 77.061286, 0.881698], [ 17.477031, 74.758448, -5.181201], [ 32.648966, 66.929021, -19.176563], [ 46.372358, 56.311389, -30.77057 ], [ 57.34348 , 42.419126, -37.628629], [ 64.388482, 25.45588 , -40.886309], [ 68.212038, 6.990805, -42.281449], [ 70.486405, -11.666193, -44.142567], [ 71.375822, -30.365191, -47.140426], [-61.119406, -49.361602, -14.254422], [-51.287588, -58.769795, -7.268147], [-37.8048 , -61.996155, -0.442051], [-24.022754, -61.033399, 6.606501], [-11.635713, -56.686759, 11.967398], [ 12.056636, -57.391033, 12.051204], [ 25.106256, -61.902186, 7.315098], [ 38.338588, -62.777713, 1.022953], [ 51.191007, -59.302347, -5.349435], [ 60.053851, -50.190255, -11.615746], [ 0.65394 , -42.19379 , 13.380835], [ 0.804809, -30.993721, 21.150853], [ 0.992204, -19.944596, 29.284036], [ 1.226783, -8.41454
1, 36.94806 ], [-14.772472, 2.598255, 20.132003], [ -7.180239, 4.751589, 23.536684], [ 0.55592 , 6.5629 , 25.944448],
[ 8.272499, 4.661005, 23.695741], [ 15.214351, 2.643046, 20.858157], [-46.04729 , -37.471411, -7.037989], [-37.674688, -42.73051 , -3.021217], [-27.883856, -42.711517, -1.353629], [-19.648268, -36.754742, 0.111088], [-28.272965, -35.134493, 0.147273], [-38.082418, -34.919043, -1.476612], [ 19.265868, -37.032306, 0.665746], [ 27.894191, -43.342445, -0.24766 ], [ 37.437529, -43.110822, -1.696435], [ 45.170805, -38.086515, -4.894163], [ 38.196454, -35.532024, -0.282961], [ 28.764989, -35.484289, 1.172675], [-28.916267, 28.612716, 2.24031 ], [-17.533194, 22.172187, 15.934335], [ -6.68459 , 19.029051, 22.611355], [ 0.381001, 20.721118, 23.748437], [ 8.375443, 19.03546 , 22.721995], [ 18.876618, 22.394109, 15.610679], [ 28.794412, 28.079924, 3.217393], [ 19.057574, 36.298248, 14.987997], [ 8.956375, 39.634575, 22.554245], [ 0.381549, 40.395647, 23.591626], [ -7.428895, 39.836405, 22.406106], [-18.160634, 36.677899, 15.121907], [-24.37749 , 28.677771, 4.785684], [ -6.897633, 25.475976, 20.893742], [ 0.340663, 26.014269, 22.220479], [ 8.444722, 25.326198, 21.02552 ],
[ 24.474473, 28.323008, 5.712776],
[ 8.449166, 30.596216, 20.671489],
[ 0.205322, 31.408738, 21.90367 ],
[ -7.198266, 30.844876, 20.328022]])
lf.focal_length = lf.size[1]
lf.camera_center = (lf.size[1] / 2, lf.size[0] / 2)
lf.camera_matrix = np.array(
[[lf.focal_length, 0, lf.camera_center[0]],
慰问简报[0, lf.focal_length, lf.camera_center[1]],
[0, 0, 1]], dtype="double")
# Assuming no lens distortion
校园时光lf.dist_coeefs = np.zeros((4, 1))
# Rotation vector and translation vector
lf.r_vec = np.array([[0.01891013], [0.08560084], [-3.14392813]])
lf.t_vec = np.array([[-14.97821226], [-10.62040383], [-2053.03596872]])
def get_euler_angle(lf, rotation_vector):
# calc rotation angles
theta = (rotation_vector, cv2.NORM_L2)
# transform to quaterniond
w = s(theta / 2)
x = math.sin(theta / 2)*rotation_vector[0][0] / theta
y = math.sin(theta / 2)*rotation_vector[1][0] / theta
z = math.sin(theta / 2)*rotation_vector[2][0] / theta
# pitch (x-axis rotation)
t0 = 2.0 * (w*x + y*z)
t1 = 1.0 - 2.0*(x**2 + y**2)
pitch = math.atan2(t0, t1)
# yaw (y-axis rotation)
t2 = 2.0 * (w*y - z*x)
if t2 > 1.0:
t2 = 1.0
if t2 < -1.0:
t2 = -1.0
yaw = math.asin(t2)
# roll (z-axis rotation)
t3 = 2.0 * (w*z + x*y)
t4 = 1.0 - 2.0*(y**2 + z**2)
roll = math.atan2(t3, t4)
return pitch, yaw, roll
熏蒸托盘def solve_po_by_6_points(lf, image_points):
"""
Solve po from image points
Return (rotation_vector, translation_vector) as po.
"""
points_6 = np.float32([
image_points[30], image_points[36], image_points[45],
image_points[48], image_points[54], image_points[8]])
_, rotation_vector, translation_vector = cv2.solvePnP(
points_6,
lf.camera_matrix,
lf.dist_coeefs,
rvec=lf.r_vec,
