前言
人脸检测标准库FDDB详细介绍了数据库和使用方法。对于训练的模型,如何评估模型的效果呢,本文对此进行介绍。说实话,参考了很多博客,但是感觉都不是很明白(当然本文也会有瑕疵),故在此记录!
测试环境
1.安装Perl;
2.安装Gnuplot;
操作步骤
1.根据训练好的模型测试数据库的人脸检测结果,并将结果输出,输出格式与要求一致即可,即out-fold-**.txt和results.txt;
检测结果格式如下:
...
<image name i>
<number of faces in this image =im>
<face i1>
<face i2>
...
<face im>
...
shapes format:
4 a. Rectangular regions
Each face region is represented as:
<left_x top_y width height detection_score>
4 b. Elliptical regions
Each face region is represented as:
<major_axis_radius minor_axis_radius angle center_x center_y detection_score>.
这里需要得到detection_score这个参数,如何得到这个参数是一个好问题,可以使用opencv自带的函数获取,也可以使用其他方法获取(fddb_faq);
cascade.detectMultiScale(img, objs, reject_levels, level_weights, scale_factor, min_neighbors, 0, cv::Size(), cv::Size(), true);
fddb_faq:
Q. How do you compute the detection score that is to be included in the face detection output file?
The score included in the output file of a face detection system should be obtained by the system itself. The eval(-infinity to infinity). In other words, the scores are used to order the detections, and their absolute values do
本文使用的是opencv自带函数和IOU判断结合来获取的。
//re:javascript:void(0)
/************************************************************************
* File: genResult.cpp
* Coder: AMY
* Email:happyamyhope@163.com
* Date: 2018/10/15
* ChLog: score max.
* Re: http://haoxiang.org/2013/11/opencv-detectmultiscale-output-detection-score/
************************************************************************/
#include "opencv2/objdetect/objdetect.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <cctype>
#include <iostream>
#include <fstream>
#include <iterator>
#include <stdio.h>
std::vector<cv::Rect> detectAndScore(cv::Mat& img, cv::CascadeClassifier& cascade, double scale, double* score);
std::vector<cv::Rect> detectAndScoreMax(cv::Mat& img, cv::CascadeClassifier& cascade, double scale, double* score);
std::string cascadeName = "..//src//haar_roboman_ff_alt2.xml";
//compute iou.
float compute_iou(cv::Rect boxA, cv::Rect boxB)
{
int xA = std::max(boxA.x, boxB.x);
int yA = std::max(boxA.y, boxB.y);
int xB = std::min(boxA.x+boxA.width, boxB.x+boxB.width);
int yB = std::min(boxA.y+boxA.height, boxB.y+boxB.height);
float inter_area = std::max(0, xB-xA+1) * std::max(0, yB-yA+1);
float boxA_area = boxA.width * boxA.height;
float boxB_area = boxB.width * boxB.height;
float iou = inter_area / (boxA_area + boxB_area - inter_area);
return iou;
}
int main(int argc, const char** argv)
{
cv::Mat frame, frameCopy, image;
std::string inputName;
std::string dir;
cv::CascadeClassifier cascade;
double scale = 1;
cascade.load(cascadeName);
std::ofstream out_all_txt("result_all.txt");
for (unsigned int file_num=1; file_num<11;file_num++)
{
std::string str = std::to_string(file_num);
if (str.size() < 2) str = "0" + str;
std::string out_name = "out-fold-" + str + ".txt";
std::cout << "start file " << out_name << std::endl;
std::ifstream in_txt("..//FDDB-folds//FDDB-fold-" + str + ".txt");
std::ofstream out_txt(out_name);
std::string dir1 = "..//FDDB-originalPics//";
while (!in_txt.eof())
{
getline(in_txt, inputName);
if (in_txt.eof()) break;
dir = dir1 + inputName;
dir += ".jpg";
//dir = "..//FDDB-originalPics//2002//08//25//big//img_674.jpg";
//std::cout << dir << std::endl;
image = cv::imread(dir, CV_LOAD_IMAGE_COLOR);
if (!image.empty())
{
std::ofstream out_txt(out_name, std::ios::app);
double scoreBuffer[50];
std::vector<cv::Rect> faces = detectAndScoreMax(image, cascade, scale, scoreBuffer);
out_txt << inputName << std::endl << faces.size() << std::endl;
out_all_txt << inputName << std::endl << faces.size() << std::endl;
//std::cout << faces.size() << std::endl;
for (unsigned int i = 0; i<faces.size(); i++)
{
cv::rectangle(image, faces[i], cv::Scalar(0, 0, 255), 1, 1, 0);
out_txt << faces[i].x << " " << faces[i].y << " " << faces[i].width
<< " " << faces[i].height << " " << scoreBuffer[i] << std::endl;
out_all_txt << faces[i].x << " " << faces[i].y << " " << faces[i].width
<< " " << faces[i].height << " " << scoreBuffer[i] << std::endl;
}
faces.clear();
}
//cv::imshow("src", image);
//cv::waitKey(100);
}
//if (in_txt.eof()) std::cout << "[EOF reached]" << std::endl;
//else std::cout << "[EOF reading]" << std::endl;
in_txt.close();
out_txt.close();
}
out_all_txt.close();
cv::waitKey(1);
return 0;
}
std::vector<cv::Rect> detectAndScoreMax(cv::Mat& color, cv::CascadeClassifier& cascade, double scale, double* scoreBuffer)
{
cv::Mat gray;
cv::Mat img(cvRound(color.rows / scale), cvRound(color.cols / scale), CV_8UC1);
cv::cvtColor(color, gray, CV_BGR2GRAY);
cv::resize(gray, img, img.size(), 0, 0, CV_INTER_LINEAR);
cv::equalizeHist(img, img);
const float scale_factor(1.2f);
const int min_neighbors(3);
std::vector<cv::Rect> faces;
std::vector<int> reject_levels;
std::vector<double> level_weights;
cascade.detectMultiScale(img, faces, reject_levels, level_weights, scale_factor, min_neighbors, 0, cv::Size(), cv::Size(), true);
//std::cout << "faces.size(): " << faces.size() << "---level_weights.size(): " << level_weights.size() << std::endl;
for (unsigned int n = 0; n < faces.size(); n++)
{
scoreBuffer[n] = level_weights[n];
//std::cout << "level_weight: " << level_weights[n] << std::endl;
}
return faces;
}
std::vector<cv::Rect> detectAndScore(cv::Mat& color, cv::CascadeClassifier& cascade, double scale, double* scoreBuffer)
{
cv::Mat gray;
cv::Mat img(cvRound(color.rows / scale), cvRound(color.cols / scale), CV_8UC1);
cv::cvtColor(color, gray, CV_BGR2GRAY);
cv::resize(gray, img, img.size(), 0, 0, CV_INTER_LINEAR);
cv::equalizeHist(img, img);
const float scale_factor(1.2f);
const int min_neighbors(3);
//long t0 = cv::getTickCount();
std::vector<cv::Rect> faces;
cascade.detectMultiScale(img, faces, scale_factor, min_neighbors, 0, cv::Size(), cv::Size());
//long t1 = cv::getTickCount();
//double secs = (t1 - t0)/cv::getTickFrequency();
//std::cout << "Detections takes " << secs << " seconds " << std::endl;
std::vector<cv::Rect> objs;
std::vector<int> reject_levels;
std::vector<double> level_weights;
cascade.detectMultiScale(img, objs, reject_levels, level_weights, scale_factor, min_neighbors, 0, cv::Size(), cv::Size(), true);
//std::cout << "faces.size(): " << faces.size() << "---objs.size(): " << objs.size() << std::endl;
for (unsigned int n = 0; n < faces.size(); n++)
{
int iou_max_idx = 0;
float iou_max = 0.0;
for (unsigned int k=0; k < objs.size(); k++)
{
float iou = compute_iou(faces[n], objs[k]);
if ( (iou>0.5) && (reject_levels[k]>=15) && (iou>iou_max) )
{
iou_max = iou;
iou_max_idx = k;
//std::cout << "iou: " << iou << "---reject_levels[k]: " << reject_levels[k] << std::endl;
}
}
scoreBuffer[n] = level_weights[iou_max_idx];
}
return faces;
}
View Code
2.准备好图片数据库、数据库的groundtruth文件(ellipseList.txt、imList.txt)及其对应的输出文件(results.txt),根据下载的eval(runEvaluate.pl),运行该程序即可得到检测器的效果;
eval(232, 232, 232); background: rgb(249, 249, 249);">
#ifdef _WIN32
string baseDir = "..//..//FDDB-originalPics//";
string listFile = "..//..//imList.txt";
string detFile = "..//..//results.txt";
string annotFile = "..//..//ellipseList.txt";
#else
string baseDir = "..//FDDB-originalPics//";
string listFile = "..//imList.txt";
string detFile = "..//results.txt";
string annotFile = "..//ellipseList.txt";
#endif
runEvaluate.pl
#!/usr/bin/perl -w
use strict;
#### VARIABLES TO EDIT ####
# where gnuplot is
my $GNUPLOT = "/usr/bin/gnuplot";
# where the binary is
my $eval(GF, ">$gnuplotFile") or die "Can not open $gnuplotFile for writing\n";
#print GF "$GNUPLOT\n";
print GF "set term png\n";
print GF "set size .75,1\n";
print GF "set output \"$pngFile\"\n";
#print GF "set xtics 500\n";
#print GF "set logscale x\n";
print GF "set ytics .1\n";
print GF "set grid\n";
#print GF "set size ratio -1\n";
print GF "set ylabel \"True positive rate\"\n";
print GF "set xlabel \"False positives\"\n";
#print GF "set xr [0:50000]\n";
print GF "set yr [0:1]\n";
print GF "set key right bottom\n";
print GF "plot \"$rocFile\" using 2:1 with linespoints title \"$title\"\n";
close(GF);
}
my $annotFile = "ellipseList.txt";
my $listFile = "imList.txt";
my $gpFile = "createROC.p";
# read all the folds into a single file for eval(-e $detFile){
system("rm", $detFile);
}
if(-e $listFile){
system("rm", $listFile);
}
if(-e $annotFile){
system("rm", $annotFile);
}
foreach my $fi (1..10){
my $foldFile = sprintf("%s/out-fold-%02d.txt", $detDir, $fi);
system("cat $foldFile >> $detFile");
$foldFile = sprintf("%s/FDDB-fold-%02d.txt", $fddbDir, $fi);
system("cat $foldFile >> $listFile");
$foldFile = sprintf("%s/FDDB-fold-%02d-ellipseList.txt", $fddbDir, $fi);
system("cat $foldFile >> $annotFile");
}
#die;
# run the actual eval($eval($eval($eval($detDir."ContROC.txt", $gpFile, $detDir, $detDir."ContROC.png");
system("echo \"load '$gpFile'\" | $GNUPLOT");
makeGNUplotFile($detDir."DiscROC.txt", $gpFile, $detDir, $detDir."DiscROC.png");
system("echo \"load '$gpFile'\" | $GNUPLOT");
# remove intermediate files
system("rm", $annotFile, $listFile, $gpFile, $detFile);
View Code
注意不同文件目录的相对路径一定要正确。
得到的文件:ContROC.txt和DiscROC.txt、ContROC.png和DiscROC.png;
3.和其他检测器的算法结果进行比较;
将生成的两个*.txt文件放在compareROC目录,在ContROC.p和DiscROC.p(也可以是ContROC_unpub.p和DiscROC_unpub.p)文件分别对应地添加一行语句(注意对应格式一致)即可运行;
#"rocCurves/filename_DiscROC.txt" using 2:1 title 'filename' with lines lw 2
运行命令
gnuplot contROC.p
或者
gnuplot discROC.p
即可生成对应的多个算法检测结果的比较;
问题
Q1:
Incompatible annotation and detection files. See output specifications
注意直接将上面生成的txt文件复制到ubuntu16下会报错Incompatible annotation and detection files. See output specifications
,由于windows下文件和ubuntu下不同导致的。只需要在ubuntu下面创建一个txt文件,然后将内容复制进去即可。当然也有可能是生成txt文件的代码有一些小问题,需要再仔细检查一下。
Q2:
为什么同样的评估程序,对于ContROC.txt和DiscROC.txt以及对应的ROC结果图片,有时候可以得到正常的曲线,有时候却得到只是直线?有大神看到的话麻烦解答一下下啦~~~
注意
1.编写生成检测器结果文件的程序,注意文件内容的格式,可参考FDDB;
2.在FDDB网站下载评估程序;
3.注意评估程序的目录结构;
4.注意根据具体情况改写*.p和*.pl文件的内容;
总结评估的步骤:准备好检测器、以txt的形式按要求输出检测器的检测结果、修改eval(0)">fddb评估;
2.windows下fddb评估;
3.人脸检测的评价方式;
4.windows-fddb-eval(0)">windows-努力奔跑的小白博客;
10. fddb;
完