search
HomeDatabaseMysql TutorialOpenCV 脸部跟踪(2)

【原文:http://www.cnblogs.com/mikewolf2002/p/3474188.html】 前面一篇文章中提到,我们在一副脸部图像上选取76个特征点来描述脸部形状特征,本文中我们会把这些特征点映射到一个标准形状模型。 通常,脸部形状特征点能够参数化分解为两个变量,一个是全

【原文:http://www.cnblogs.com/mikewolf2002/p/3474188.html】

前面一篇文章中提到,我们在一副脸部图像上选取76个特征点来描述脸部形状特征,本文中我们会把这些特征点映射到一个标准形状模型。

      通常,脸部形状特征点能够参数化分解为两个变量,一个是全局的刚体变化,一个是局部的变形。全局的刚体变化主要是指脸部能够在图像中移动,旋转,缩放,局部的变形则是指脸部的表情变化,不同人脸的特征等等。

 

     形状模型类主要成员如下:

class shape_model 
    {                         //2d linear shape model 
    public: 
        Mat p;                                  //parameter vector (kx1) CV_32F,参数向量 
        Mat V;                                   //shape basis (2nxk) CV_32F, line subspace,线性子空间 
        Mat e;                                   //parameter variance (kx1) CV_32F 参数方差 
        Mat C;                                   //connectivity (cx2) CV_32S 连通性

//把一个点集投影到一个可信的脸部形状空间 
void     calc_params(const vector &pts, //points to compute parameters from 
    const Mat weight = Mat(),    //weight of each point (nx1) CV_32F 点集的权重 
    const float c_factor = 3.0); //clamping factor

//该函数用人脸模型V和e,把向量p转化为点集 
vector   calc_shape();                   //shape described by parameters @p
 

...

void    train(const vector > &p, //N-example shapes 
    const vector &con = vector(), //point-connectivity 
    const float frac = 0.95,           //fraction of variation to retain 
    const int kmax = 10);              //maximum number of modes to retain 
...

}

本文中,我们通过Procrustes analysis来处理特征点,移去全局刚性变化Procrustes analysis算法可以参考:http://en.wikipedia.org/wiki/Procrustes_analysis

      在数学上,Procruster analysis就是寻找一个标准形状,然后把所有其它特征点数据都和标准形状对齐,对齐的时候采用最小平方距离,用迭代的方法不断逼近。下面通过代码来了解如何实现Procrustes analysis。

//Procrustes分析的基本思想是最小化所有形状到平均形状的距离和 
Mat shape_model::procrustes(const Mat &X, 
    const int itol,  //最大迭代次数 
    const float ftol //精度

     ) 
    {

X矩阵就是多副样本图像76个特征点组成的矩阵,共152行,列数为图像的个数,每列表示一个样本图像特征点的x,y坐标。 
    int N = X.cols,n = X.rows/2;

    //remove centre of mass 
    //所有的形状向量(特征)对齐到原点,即每个向量的分量减去其平均值,每列是一个形状向量。
 
    Mat P = X.clone(); 
    for(int i = 0; i         { 
        Mat p = P.col(i); //第i个向量 
        float mx = 0,my = 0; 
        for(int j = 0; j //x,y分别计算得到平均值。
 
            { 
            mx += p.fl(2*j); 
            my += p.fl(2*j+1); 
            } 
        mx /= n; my /= n; 
        for(int j = 0; j             { 
            p.fl(2*j) -= mx; 
            p.fl(2*j+1) -= my; 
            } 
        }


    //optimise scale and rotation 
    Mat C_old; 
    for(int iter = 0; iter         {   

      注意下边的一行代码,会把每个图像对齐到原点特征点x,y分别加起来,求平均值,得到一个152*1的矩阵,然后对该矩阵进行归一化处理。 
        Mat C = P*Mat::ones(N,1,CV_32F)/N;  //计算形状变换后的平均值 
        normalize(C,C); //canonical shape (对x-进行标准化处理) 
        if(iter > 0) //converged?//收敛?当两个标准形状或者标准形状的误差小于某一值这里是ftol则,停止迭代。  
            {

norm函数默认是矩阵各元素平方和的根范式OpenCV 脸部跟踪(2) 
            if(norm(C,C_old)                 break; 
            } 
        C_old = C.clone(); //remember current estimate//记下当前的矩阵,和下一次进行比较 
        for(int i = 0; i             {

rot_scale_align函数求每副图像的特征点向量和平均向量满足最小乘法时候的旋转矩阵。即求得a,b值组成的旋转矩阵。

OpenCV 脸部跟踪(2)

OpenCV 脸部跟踪(2)

该函数的代码:

Mat    shape_model::rot_scale_align(<span>const</span> Mat &src,    <span>const</span> Mat &dst)
    {
    <span>//construct linear system</span>
    <span>int</span> n = src.rows/2;
    <span>float</span> a=0,b=0,d=0;
    <span>for</span>(<span>int</span> i = 0; i //x*x+y*y
        a += src.fl(2*i) * dst.fl(2*i  ) + src.fl(2*i+1) * dst.fl(2*i+1);
        b += src.fl(2*i) * dst.fl(2*i+1) - src.fl(2*i+1) * dst.fl(2*i  );
        }
    a /= d; 
    b /= d;
    <span>//solved linear system</span>
    <span>return</span> (Mat_float>(2,2) 
<br>

<br>
              Mat R = this->rot_scale_align(P.col(i),C); <span>//求两个形状之间的误差满足最小二乘时的旋转矩阵。即相当于两个形状"最靠近"时,需要的旋转的仿射矩阵 <br>
</span>            for(int j = 0; j 
                { <span>//apply similarity transform//应用相似变换,这对形状中的每一个点,应用仿射矩阵。  变化后,该特征点向量会靠近平均特征向量。之后经过反复迭代,直到平均向量和上次比较变化很小时,退出迭代。 <br>
</span>                float x = P.fl(2*j,i),y = P.fl(2*j+1,i); <br>
                P.fl(2*j  ,i) = R.fl(0,0)*x + R.fl(0,1)*y; <br>
                P.fl(2*j+1,i) = R.fl(1,0)*x + R.fl(1,1)*y; <br>
                } <br>
            } <br>
        } <br>
    return P; <br>
    } <br>

<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
<span>      通过<span>Procrustes analysis对齐的特征向量,我们要用一个统一的矩阵把平移和旋转统一起来表示<span>(成为线性表示),</span>然后把该矩阵追加到局部变形空间,注意对该矩阵表示,我们最后进行了史密斯正交处理。</span></span></p>
<p>
<img  src="/static/imghwm/default1.png" data-src="/inc/test.jsp?url=http%3A%2F%2Fimages.cnitblog.com%2Fblog%2F361409%2F201312%2F14113411-9fdaefb021b14f52a72d2e3dae833c22.png&refer=http%3A%2F%2Fblog.csdn.net%2Fzhazhiqiang%2Farticle%2Fdetails%2F20158453" class="lazy" alt="OpenCV 脸部跟踪(2)" ></p>
<p>
<span>我们通过函数 calc_rigid_basis得到该矩阵表示:</span></p>
<p>
<span>Mat  shape_model::calc_rigid_basis(const Mat &X) <br>
    { <br>
    <span>//compute mean shape</span> <br>
    int N = X.cols,n = X.rows/2; <br>
    Mat mean = X*Mat::ones(N,1,CV_32F)/N;</span></p>
<p>
<span>    <span>//construct basis for similarity transform <br>
</span>    Mat R(2*n,4,CV_32F); <br>
    for(int i = 0; i 
        { <br>
        R.fl(2*i,0) =  mean.fl(2*i  ); <br>
        R.fl(2*i+1,0) =  mean.fl(2*i+1); <br>
        R.fl(2*i,1) = -mean.fl(2*i+1); <br>
        R.fl(2*i+1,1) =  mean.fl(2*i  ); <br>
        R.fl(2*i,2) =  1.0;   <br>
        R.fl(2*i+1,2) =  0.0; <br>
        R.fl(2*i,3) =  0.0;  <br>
        R.fl(2*i+1,3) =  1.0; <br>
        } <br>
    <span>//Gram-Schmidt orthonormalization <br>
</span>    for(int i = 0; i 
        { <br>
        Mat r = R.col(i); <br>
        for(int j = 0; j 
            { <br>
            Mat b = R.col(j); <br>
            r -= b*(b.t()*r); <br>
            } <br>
        normalize(r,r); <br>
        } <br>
    return R; <br>
    }</span></p>
<p>
</p>
<p>
<span>下面我们看看train函数的实现:</span></p>
<p>
<span>两篇参考的翻译:http://blog.csdn.net/raby_gyl/article/details/13148193</span></p>
<p>
<span>http://blog.csdn.net/raby_gyl/article/details/13024867</span></p>
<p>
</p>
<p>
<span>      该函数的输入为n个样本图像的采样特征点,该点集会被首先转化为行152,列为样本数量的矩阵表示,另外还有连通性点集索引,以及方差的置信区间以及保留模型的最大数量。</span></p>
<p>
<span>void    train(const vector<vector> > &p, <span>//N-example shapes</span> <br>
    const vector<vec2i> &con = vector<vec2i>(), <span>//point-connectivity <br>
</span>    const float frac = 0.95,           <span>//fraction of variation to retain <br>
</span>    const int kmax = 10)              <span>//maximum number of modes to retain <br>
</span>    { <br>
    <span>//vectorize points</span> <br>
    Mat X = this->pts2mat(points);</vec2i></vec2i></vector></span></p>
<p>
<span><span>N是样本的数目,n是76,表示76个特征点。 <br>
</span>    int N = X.cols,n = X.rows/2;</span></p>
<p>
<span>    <span>//align shapes <br>
</span>    Mat Y = this->procrustes(X);</span></p>
<p>
<span>    <span>//compute rigid transformation</span> <span>计算得到刚体变化矩阵R <br>
</span>    Mat R = this->calc_rigid_basis(Y);</span></p>
<p>
<span>      脸部局部变形我们用一个线性模型表示,主要的思想如下图所示:一个有N个面部特征组成面部形状,被建模成一个2N维空间的点。我们要尽量找到一个低维的超平面,在这个平面内,所有的面部形状都在里面,由于这个超平面只是2N维空间的子集,占用刚少的空间,处理起来更快。可以想得到,如果这个子空间来自于一个人,则子空间的点,表现这个人的各种表情变化。前面的教程中,我们知道PCA算法能够找到低维子空间,但PCA算法需要指定子空间的维数,在启发式算法中有时候这个值很难选择。在本程序中,我们通过SVD算法来模拟PCA算法。</span></p>
<p>
 </p>
<p>
<img  src="/static/imghwm/default1.png" data-src="/inc/test.jsp?url=http%3A%2F%2Fimages.cnitblog.com%2Fblog%2F361409%2F201312%2F15192025-1229008c4629424d8379873e5a6e517b.png&refer=http%3A%2F%2Fblog.csdn.net%2Fzhazhiqiang%2Farticle%2Fdetails%2F20158453" class="lazy" alt="OpenCV 脸部跟踪(2)" ></p>
<p>
 </p>
<p>
<span>    <span>//compute non-rigid transformation</span> </span></p>
<p>
<span><span>Y是152*1294的矩阵,它是procrustes分析的结果,R是刚体变化矩阵152*4,它的转置就是4*152 <br>
</span>    Mat P = R.t()*Y; <span>//原始的位置 <br>
</span>    <span>Mat dY = Y - R*P; </span><span>//</span><span>dy变量的每一列表示减去均值的Procrustes对齐形状,<span>投影刚体运动</span></span></span></p>
<p>
<span>      奇异值分解SVD有效的应用到形状数据的<span>协方差矩阵</span>(即,dY.t()*dY),OpenCV的SVD类的w成员存储着数据变化性的主要方向的变量,从最大到最小排序。一个选择子空间维数的普通方法是选择保存数据总能量分数frac的方向最小集(即占总能量的比例为frac),这是通过svd.w记录表示的,因为这些记录是从最大的到最小的排序的,它充分地用来评估子空间,通过用变化性方向的最大值k来评估能量。他们自己的方向存储在SVD类的u成员内。svd.w和svd.u成分一般分别被成为特征值和特征矢量。</span><span><span> <br>
</span><span>    SVD svd(dY*dY.t()); <br>
    int m = min(min(kmax,N-1),n-1); <br>
    float vsum = 0; <br>
    for(int i = 0; i 
        vsum += svd.w.fl(i); <br>
    float v = 0; <br>
    int k = 0;</span></span></p>
<p>
<span><span><span>达到了95%的主成分量,退出,frac=0.95</span> <br>
    for(k = 0; k 
        { <br>
        v += svd.w.fl(k); <br>
        if(v/vsum >= frac){k++; break;} <br>
        } <br>
    if(k > m)  k = m;</span></span></p>
<p>
<span><span><span>取前k个特征向量 <br>
</span>    <span>Mat D = svd.u(Rect(0,0,k,2*n));</span></span></span></p>
<p>
<span><span>把全局刚体运动和局部变形运动结合起来,注意V的第一列是缩放,第三、四列分别是x,y偏移。</span></span></p>
<p>
<span>//combine bases <br>
    V.create(2*n,4+k,CV_32F); <br>
    Mat Vr = V(Rect(0,0,4,2*n));   <span>//刚体子空间 <br>
</span>    R.copyTo(Vr);     <span>//非刚体子空间</span> <br>
    Mat Vd = V(Rect(4,0,k,2*n)); <br>
    D.copyTo(Vd);</span></p>
<p>
<span>      最后我们要注意的一点是如何约束子空间坐标,以使得子空间内的面部形状都是有效的。在下面的图中,我们可以看到,对于子空间内的图像,如果在某个方向改变坐标值,当坐标值小时候,它仍是一个脸的形状,但是变化值大时候,就不知道是什么玩意了。防止出现这种情况的最简单方法,就是把变化的值clamp在一个范围内,通常是现在± 3 的范围,这样可以cover到99.7%的脸部变化。clamping的值通过下面的代码计算:</span></p>
<p>
 </p>
<p>
<img  src="/static/imghwm/default1.png" data-src="/inc/test.jsp?url=http%3A%2F%2Fimages.cnitblog.com%2Fblog%2F361409%2F201312%2F15192029-199dc6314d9e4c14bbf7fe4b35f098c0.png&refer=http%3A%2F%2Fblog.csdn.net%2Fzhazhiqiang%2Farticle%2Fdetails%2F20158453" class="lazy" alt="OpenCV 脸部跟踪(2)" ></p>
<p>
<span> <span>   //compute variance (normalized wrt scale)</span> </span></p>
<p>
<span><br>
   <span>Mat Q = V.t()*X; </span><span>//把数据投影到子空间 <br>
</span>    for(int i = 0; i //normalize coordinates w.r.t scale <br>
</span>        {  <span>//用第一个坐标缩放,防止太大的缩放值影响脸部识别 <br>
</span>        float v = Q.fl(0,i); <br>
        Mat q = Q.col(i); <br>
        q /= v; <br>
        } <br>
    e.create(4+k,1,CV_32F); <br>
    pow(Q,2,Q); <br>
    for(int i = 0; i 
        { <br>
        if(i 
            e.fl(i) = -1; <span>//no clamping for rigid coefficients <br>
</span>        else <br>
            e.fl(i) = Q.row(i).dot(Mat::ones(1,N,CV_32F))/(N-1); <br>
        } <br>
    <span>//store connectivity <br>
</span>    if(con.size() > 0) <br>
        { //default connectivity <br>
        int m = con.size(); <br>
        C.create(m,2,CV_32F); <br>
        for(int i = 0; i 
            { <br>
            C.at<int>(i,0) = con[i][0]; <br>
            C.at<int>(i,1) = con[i][1]; <br>
            } <br>
        } <br>
    else <br>
        {              //user-specified connectivity <br>
            C.create(n,2,CV_32S); <br>
            for(int i = 0; i 
                { <br>
                C.at<int>(i,0) = i; C.at<int>(i,1) = i+1; <br>
                }    <br>
            C.at<int>(n-1,0) = n-1; C.at<int>(n-1,1) = 0; <br>
        } <br>
    }</int></int></int></int></int></int></p>
<p>
</p>
<p>
<span>工程文件:FirstOpenCV40,</span></p>
<p>
<span>程序的运行参数为:annotations.yaml shapemodle.yaml</span></p>
<p>
<span>程序执行后,可以看到我们只保留了14个模型。</span></p>
<p>
<img  src="/static/imghwm/default1.png" data-src="/inc/test.jsp?url=http%3A%2F%2Fimages.cnitblog.com%2Fblog%2F361409%2F201312%2F14113419-908c110a67d3464abcd452edcc027668.png&refer=http%3A%2F%2Fblog.csdn.net%2Fzhazhiqiang%2Farticle%2Fdetails%2F20158453" class="lazy" alt="OpenCV 脸部跟踪(2)" ></p>
<p>
<span>我们也可以使用下面的运行参数:annotations.yaml shapemodle.yaml –mirror</span></p>
<p>
<span>这时候,每副图像的特征点,会生成一个y轴对称的镜像特征点集,这时训练的采样数目翻倍,为5828。</span></p>
<p>
<span>在工程文件FirstOpenCV41中,我们可视化了生成的模型,会连续显示14个模型的不同姿态:</span></p>
<p>
<img  src="/static/imghwm/default1.png" data-src="/inc/test.jsp?url=http%3A%2F%2Fimages.cnitblog.com%2Fblog%2F361409%2F201312%2F14113424-ac033ab541ca4a349466840ce5b41628.png&refer=http%3A%2F%2Fblog.csdn.net%2Fzhazhiqiang%2Farticle%2Fdetails%2F20158453" class="lazy" alt="OpenCV 脸部跟踪(2)" ></p>


Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
How do you handle database upgrades in MySQL?How do you handle database upgrades in MySQL?Apr 30, 2025 am 12:28 AM

The steps for upgrading MySQL database include: 1. Backup the database, 2. Stop the current MySQL service, 3. Install the new version of MySQL, 4. Start the new version of MySQL service, 5. Recover the database. Compatibility issues are required during the upgrade process, and advanced tools such as PerconaToolkit can be used for testing and optimization.

What are the different backup strategies you can use for MySQL?What are the different backup strategies you can use for MySQL?Apr 30, 2025 am 12:28 AM

MySQL backup policies include logical backup, physical backup, incremental backup, replication-based backup, and cloud backup. 1. Logical backup uses mysqldump to export database structure and data, which is suitable for small databases and version migrations. 2. Physical backups are fast and comprehensive by copying data files, but require database consistency. 3. Incremental backup uses binary logging to record changes, which is suitable for large databases. 4. Replication-based backup reduces the impact on the production system by backing up from the server. 5. Cloud backups such as AmazonRDS provide automation solutions, but costs and control need to be considered. When selecting a policy, database size, downtime tolerance, recovery time, and recovery point goals should be considered.

What is MySQL clustering?What is MySQL clustering?Apr 30, 2025 am 12:28 AM

MySQLclusteringenhancesdatabaserobustnessandscalabilitybydistributingdataacrossmultiplenodes.ItusestheNDBenginefordatareplicationandfaulttolerance,ensuringhighavailability.Setupinvolvesconfiguringmanagement,data,andSQLnodes,withcarefulmonitoringandpe

How do you optimize database schema design for performance in MySQL?How do you optimize database schema design for performance in MySQL?Apr 30, 2025 am 12:27 AM

Optimizing database schema design in MySQL can improve performance through the following steps: 1. Index optimization: Create indexes on common query columns, balancing the overhead of query and inserting updates. 2. Table structure optimization: Reduce data redundancy through normalization or anti-normalization and improve access efficiency. 3. Data type selection: Use appropriate data types, such as INT instead of VARCHAR, to reduce storage space. 4. Partitioning and sub-table: For large data volumes, use partitioning and sub-table to disperse data to improve query and maintenance efficiency.

How can you optimize MySQL performance?How can you optimize MySQL performance?Apr 30, 2025 am 12:26 AM

TooptimizeMySQLperformance,followthesesteps:1)Implementproperindexingtospeedupqueries,2)UseEXPLAINtoanalyzeandoptimizequeryperformance,3)Adjustserverconfigurationsettingslikeinnodb_buffer_pool_sizeandmax_connections,4)Usepartitioningforlargetablestoi

How to use MySQL functions for data processing and calculationHow to use MySQL functions for data processing and calculationApr 29, 2025 pm 04:21 PM

MySQL functions can be used for data processing and calculation. 1. Basic usage includes string processing, date calculation and mathematical operations. 2. Advanced usage involves combining multiple functions to implement complex operations. 3. Performance optimization requires avoiding the use of functions in the WHERE clause and using GROUPBY and temporary tables.

An efficient way to batch insert data in MySQLAn efficient way to batch insert data in MySQLApr 29, 2025 pm 04:18 PM

Efficient methods for batch inserting data in MySQL include: 1. Using INSERTINTO...VALUES syntax, 2. Using LOADDATAINFILE command, 3. Using transaction processing, 4. Adjust batch size, 5. Disable indexing, 6. Using INSERTIGNORE or INSERT...ONDUPLICATEKEYUPDATE, these methods can significantly improve database operation efficiency.

Steps to add and delete fields to MySQL tablesSteps to add and delete fields to MySQL tablesApr 29, 2025 pm 04:15 PM

In MySQL, add fields using ALTERTABLEtable_nameADDCOLUMNnew_columnVARCHAR(255)AFTERexisting_column, delete fields using ALTERTABLEtable_nameDROPCOLUMNcolumn_to_drop. When adding fields, you need to specify a location to optimize query performance and data structure; before deleting fields, you need to confirm that the operation is irreversible; modifying table structure using online DDL, backup data, test environment, and low-load time periods is performance optimization and best practice.

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

PhpStorm Mac version

PhpStorm Mac version

The latest (2018.2.1) professional PHP integrated development tool

ZendStudio 13.5.1 Mac

ZendStudio 13.5.1 Mac

Powerful PHP integrated development environment

mPDF

mPDF

mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),

EditPlus Chinese cracked version

EditPlus Chinese cracked version

Small size, syntax highlighting, does not support code prompt function