Kernel PCA feature extraction and the SVM classification algorithm for multiple-status, through-wall, human being detection

Abstract

Ultra-wideband (UWB) radar with strong anti-jamming performance and high-range resolution can be used to separate multiple human targets in a complex environment. In recent years, through-wall human being detection with UWB radar has become relatively sophisticated. In this paper, the method of kernel principal component analysis (KPCA) feature extraction and the support vector machine (SVM) classification algorithm are applied to identify and classify the multiple statuses of through-wall human being detection. This method makes full use of the KPCA of powerful, nonlinear feature extraction and SVMs, which can solve the problem of multiple-status detection and nonlinear pattern recognition. The experimental data that come from KPCA feature extraction are used as input to the SVM classification algorithm, some of which are used to train the model and the others to test the model. Experimental results showed that KPCA feature extraction and the SVM classification algorithm effectively distinguished four statuses of through-wall human being detection and achieved the desired results.

1 Introduction

Ultra-wideband (UWB) radar can emit pulses of very short duration to penetrate walls, bulkheads, and other obstacles. The UWB radar pulse has the advantages of high accuracy, strong penetrating ability, high resolving power, good anti-stealth ability, and low power consumption. Hence, it has great potential in radar detection, imaging, precise positioning and target recognition. In the fight against terrorism, disaster relief, public security during rioting and urban street fighting and other fields, it has a significant application. It operates mainly through the acquisition and analysis of the echo signal to carry information to detect objects hidden in penetrating media, such as clay walls, dielectric plates, concrete, etc. The UWB signal can realize ranging accuracy in the order of centimeters, can identify and distinguish different target types, can overcome the absorbing effect of narrowband radar, can suppress clutter echo effects in complex backgrounds, and can provide non-intrusive detection. Therefore, the technology has broad and important application prospects as the best choice for all-round human being detection. The focus of this paper is on using P410 UWB radar for multiple-status, through-wall, human being detection [1].

UWB radar has good penetration ability and is not susceptible to weather, temperature, humidity or other physical factors. Therefore, in recent years, its application regarding through-wall detection has increasingly attracted the interest of scholars. In reference [2], this paper mainly introduces the normalized difference square matrix method, and the reference moving average method with the Discrete Fourier Transform (DFT) as the detection techniques for the periodic respiratory motion of the human target. The results of experiments to detect objects behind gypsum walls and concrete walls have been separately proved for human target detection. In reference [3], this paper mainly introduces the through-wall human being detection model based on UWB radar, deducing the wavelet packet transform of the target criterion, and designing the procedure for through-wall human being detection with statistical process control. The experimental data are collected for both stationary and moving status of human beings behind a brick wall. In reference [4], this paper mainly introduces an efficient method of TOA (time of arrival) estimation using UWB through-wall radar to detect and track moving targets behind a wall based on the TWRI (through-wall radar imaging) algorithm. The processed result of the experimental data obtained from the UWB through-wall radar shows its detection and tracking effects on moving targets. In reference [5], this paper mainly introduces residual subspace analysis, addressing the anomaly detection problem in large-scale data-mining applications, and suggests a framework using compressed sensing (CS) theory. The experimental results were based on the benchmark PETS 2007 and 83 GB of real footage from three public train stations. The results of this paper proposed a method that is scalable and, importantly, its performance is comparable to conventional methods for anomaly detection when the complete data are available. Reference [6] processes fuzzy pattern recognition and the genetic algorithm to identify the multiple-status human being behind a brick wall, and recognizes the principle of maximum degree of membership function to establish target prediction function by fuzzy pattern recognition. The results show fuzzy pattern recognition performance for the multi-status human being behind the wall. Reference [7], mainly describes a complete UWB signaling tomography system for high-contrast or large-object recognition, applied to breast cancer screening. This article focuses on the implementation of a two-degree, free-imaging set to deal with asymmetric objects with lifelike breast “ghosts.” In reference [8], a radar sensor network (RSN) was used to arrive at a unified analytical framework that takes all factors into account and allows the uncertainty derivation of probability detection and location. The experimental results allow the system designer to have a clear understanding of the impact of each system parameter and the trade-off between performance and complexity. In reference [9], Yarovoy has examined the detection and positioning of human beings in complex environments with UWB radar and it has been shown that the range of a person varies by up to 0.6 cm because of their breathing movements, and novel motion or breathing detection has been presented on the basis of the measurements from radar return spatial variations. The experimental results of human being radar return have been analyzed in the frequency band of 1 to 2 GHz. In reference [10], this paper describes a complex process based on the M-sequence UWB radar estimation method for behind-wall moving-target tracking, and introduces the phase-task, solution signal processing method. Experimental results were based on the scenario of tracking a single moving object through a concrete wall, and the UWB radar signal is used to deal with the performance of the demonstration trajectory estimation method.

In recent years, multiple-status, through-wall human detection being has attracted wide attention and been applied to many fields such as national economy, space technology, and national defense. The kernel principal component analysis (KPCA) method can deal with the nonlinear relationship between variables as a multi-variable statistical process monitoring effective algorithm using this method to establish a multiple-status fault detection model. The experimental results are based on three case studies: (1) a two-dimensional toy example, (2) a realistic simulation usually used as a benchmark example, known as the Tennessee Eastman Process, and (3) real data from a methanol plant [11]. Reference [12] shows that reducing the impact of channels and handsets on system performance is one of the major issues that improve the accuracy of the most state-of-the-art speaker recognition algorithms. By adapting the model between the different channel conditions to explore the SVM framework-specific techniques to obtain completely nonlinear channel compensation, the observed changes in the particular type of marker channel are less sensitive to the system. In [13], Solomonoff proposed the SVM method to adjust the model to improve the accuracy of the speaker recognition algorithm and reduce the impact of the channel and the mobile phone on the system performance, and discussed SVM framework-specific technology to obtain complete nonlinear channel compensation. The experimental results are based on systems that are less sensitive to the particular type of marker channel changes observed during training. In [14], this paper mainly discusses the road detection algorithm of the front-view single camera using the road probability distribution model (RPDM) and the online learning method. The combination of the dynamic RPDM and the fuzzy support vector machine (FSVM) makes the algorithm self-supervising and optimizes learning from the previous results.

In this paper, we proposed an algorithm for through-wall human being detection based on KPCA feature extraction and the SVM classification algorithm. We also used linear PCA feature extraction and the SVM classification algorithm compared with the method of this paper. The remainder of the paper is organized as follows: in Section 1, we introduce the merit of UWB radar in the field of target recognition and elaborated the main research contents of this paper. In Section 2, we introduce the theory of KPCA and SVMs. The algorithm of construction and implementation is in Section 3. Experimental results for multiple-status human being detection are shown in Section 4. In Section 5 are the Discussion and our Conclusions.

2 Theory

2.1 KPCA nonlinear feature extraction theory [15, 16]

Principal component analysis (PCA) is a linear dimensionality reduction and feature extraction method for high-dimensional data. It maps the input data from the original high-dimensional space to the characteristic subspace, extracts the main feature vector of the input data, and achieves the purpose of analyzing the original data with the main component. In general, PCA can only be effectively performed on a set of observations that are best described by second-order correlations and vary linearly, or the observations are generated by a Gaussian distribution. It is well known, however, that the variations of the actual data are nonlinear and highly non-Gaussian, and that the majority of the data cannot be described by second-order correlations. Therefore, when PCA is employed, the performance is very poor. In this paper, KPCA is a nonlinear PCA method based on kernel functions which intrinsically constructs a nonlinear mapping from the input space to the feature space F by a nonlinear transformation Φ, and performs linear PCA in F. Between two input examples, x and y, in the original space, one can avoid performing the nonlinear mappings and computing both the dot products in the feature space by using a kernel function of form:

$$ k\left(x,y\right)=\varPhi (x)\cdot \varPhi (y)\cdot $$

The conceptual framework of the KPCA method is shown schematically in Fig. 1.

Fig. 1

The conceptual framework of the kernel principal component analysis (KPCA) method

There are many forms of kernel functions. According to Mercer’s theorem of functional analysis, if the kernel function is a continuous kernel of a positive integral operator, there exists a map, Φ, into a dot product space, F, such that the formula holds. As long as the requirement on the kernel function satisfies Mercer’s theorem and selects the fitting kernel function, it can achieve a good dimensionality reduction effect. Below are some typical kernel functions, such as:

$$ \mathrm{Polynomial}\ \mathrm{kernel}\ k\left(x,y\right)={\left\langle x,y\right\rangle}^d, $$

$$ \mathrm{Sigmoid}\ \mathrm{kernel}\ k\left(x,y\right)=\mathit{\tanh}\left({\beta}_0\left\langle x,y\right\rangle +{\beta}_1\right), $$

$$ \mathrm{Radial}\ \mathrm{basis}\ \mathrm{kernel}\ k\left({x}^{,}y\right)=\exp \left(\frac{{\left\Vert x-\left.y\right\Vert \right.}^2}{c}\right), $$

where d, β ₀, β ₁, and c are specified a priori by the user. The polynominal kernel and radial basis kernel always satisfy Mercer’s theorem, whereas the sigmoid kernel satisfies it only for certain values of β ₀ and β ₁. Due to the good performance of the radial basis function, in practical application the radial basis function is generally chosen as the kernel function of KPCA; so, in this paper, we use the radial basis kernel as the KPCA kernel function.

Giving a set of input data (with zero mean) X (x ₁ , ⋯ , x _N ∈ R ^m that N is the number of samples, m is the dimension of the measurement variables) through PCA and KPCA algorithm calculating their covariance matrix, the covariance of PCA can be expressed by formula (1) and the covariance of KPCA can be expressed by formula (2) in a linear feature space F instead of the nonlinear input space:

$$ C=\frac{1}{N}{\mathrm{x}}_i{\mathrm{x}}_i^T=\frac{1}{N}{XX}^T\mathrm{and} $$

(1)

$$ {C}^F=\frac{1}{N}\sum_{j=1}^N{\varPhi}_j(x){\varPhi}_j{(x)}^T, $$

(2)

where it is assumed that ${\sum}_{k=1}^N\varPhi \left({x}_k\right)=0$, and the Φ(·) is a nonlinear mapping function that projects the input vectors from the input space to F. Note that the dimensionality of the feature space can be arbitrarily large or possibly infinite. To calculate the covariance matrix, one has to solve the eigenvalue problem in the feature space:

$$ \lambda v={C}^Fv, $$

(3)

where eigenvalues λ ≥ 0 and eigenvector v ∈ F, the eigenvector v for any λ ≠ 0 can be linearly represented by Φ(x _i ):

$$ v=\sum_{i=1}^Na(i)\varPhi \left({x}_i\right) $$

(4)

Eq. (2) can be converted into the kernel eigenvalue problem:

$$ N\lambda a= Ka, $$

(5)

where an N ∗ N matrix K is a kernel matrix, K = k _ij = (Φ(x _i ) · Φ(x _j )) = k(x _i , x _j ), and α is the feature vector of the kernel matrix. When reconstruct input data from feature space, we use the fellow Eq. (5):

$$ {y}_k=\left\langle {v}_k,\varPhi (x)\right\rangle =\sum_{i=1}^N{a}_i^k\left\langle \varPhi \left({x}_i\right),\varPhi (x)\right\rangle $$

(6)

Before applying the KPCA algorithm, we assume that the data have been standardized, but in the actual application, without knowing the specific form of Φ, the data are not standardized. This can be done by substituting the kernel matrix K with:

$$ \widehat{K}=K-{1}_NK-K{1}_N+{1}_NK{1}_N, $$

(7)

where ${1}_N={\left(\frac{1}{N}\right)}_{N\times N}$

2.2 SVM classification algorithm theory [17,18,19]

Support vector machines (SVMs) are a supervised, machine learning method for classification, pattern recognition, regression analysis, and other learning tasks based on statistical learning theory (SLT). In this method, support vectors are used to represent decision boundaries; then, one maps the linearly indivisible data of the low-dimensional input space into a high-dimensional feature space to make it linearly separable; this it is based on the structural risk minimization theory and then constructs an optimal separating hyperplane in this space. The basic idea for SVMs is show in Fig. 2.

Fig. 2

Basic idea for support vector machines (SVMs)

Given a training set of N data points (y _j , x _j ) , j = 1 , 2 , 3 , … , N where x _j ∈ R ⁿ is the input pattern and y _j ∈ {1, −1} is the output pattern. The SVM method approach aims at constructing a classifier of the form:

$$ y(x)=\mathit{\operatorname{sign}}\left[\sum_{j=1}^N{a}_j{y}_j\varPsi \left(x,{x}_j\right)+b\right], $$

(8)

where sign (·) is a symbolic function used to classify, α _j represents positive real constants and b is a real constant. For Ψ(·, ·) one typically has the following choices: $\varPsi \left(x,{x}_k\right)={x}_k^Tx$ (linear SVM) and Ψ(x, x _k ) is a kernel function similar to KPCA. Here, we also select the radial basis kernel as the SVM kernel function and use the nonlinear SVM in this paper.

The classifier is constructed as follows. One assumes that:

$$ {\omega}^T\varphi \left({x}_i\right)+b\ge 1,\mathrm{if}\;{\mathrm{y}}_i=+1, $$

(9)

$$ {\omega}^T\varphi \left({x}_i\right)+b\le -1,\mathrm{if}\;{\mathrm{y}}_i=-1, $$

(10)

which is equivalent to:

$$ {y}_j\left[{\omega}^T\varphi \left({x}_j\right)+b\right]\ge 1,j=1,2,3,\dots, N, $$

(11)

where φ(·) is a nonlinear function which maps the input space into a higher-dimensional feature space. But, in order to have the possibility to violate formula (10), the function is not clearly constructed. We introduce the variables ξ _j to solve the following primal optimization problem and to obtain the separating hyperplane in the higher-dimensional space:

$$ \underset{\omega, \xi j}{\mathit{\min}}\frac{1}{2}{\omega}^T\omega +C\sum_{j=i}^N{\xi}_j, $$

(12)

subject to

$$ {y}_j\left({\omega}^T\varphi \left({x}_i\right)+b\right)\ge 1-{\xi}_j $$

(13)

$$ \xi \ge 0,j=1,2,\dots, N $$

3 Algorithm and implementation

In order to better understand the feature extraction algorithm of this paper, we give the training steps of the network as follows:

In this paper, we used the ROC (receiver operating characteristic) curve to estimate the performance of the classification algorithm more comprehensively. The ROC curve has the advantages of high credibility, accurate description of the target, and of not being affected by the data environment. It is based on a series of two different classification methods (cutoff or decision threshold). In the ROC curve, the FPR (false positive rate) is used as the abscissa and the TPR (true positive rate) is used as the ordinate. Compared with the traditional evaluation methods, the performance of the ROC curve analysis method is better and there are no excessive restrictions. Therefore, the ROC curve analysis method is applicable to a wider range. The evaluation index of the ROC is shown in the following [20] (Table 1)

Table 1 The evaluation index of receiver operating characteristic (ROC)

:

TP is the positive sample which is predicted to be true by the model, FN is the positive sample which is predicted to be false by the model, FP is the false sample which is predicted to be true by the model, and TN is the false sample which is predicted to be false by the model. The calculation formulas of FPR and TPR are as follows:

$$ FPR=\frac{FP}{TN+ FP}, TPR=\frac{TP}{TP+ FN}. $$

(14)

4 Experimental process and results

In this experimental, the UWB radar module is the P410 from the Time Domain Company. The P410 equipment development board is composed of a FPGA, a DSP, a network port, a UWB transceiver, a fan and many other components and is connected to the computer by net mouth using the UDP protocol to set the signal pulse length, frequency, and other parameters. The center frequency of the P410 UWB radar is 4.3 GHz. The wall material is a 23.5-cm thick brick. The P410 UWB radar is placed at a distance of 20 cm from the brick wall and the distance between the UWB radar and the ground is half of the brick wall. In this experimental environment, there are four statuses for through-wall human being detection. The first is nobody behind the brick wall, the second is two persons walking 1 m away from the brick wall, the third is three persons breathing normally 2 m away from the brick wall, and the last is two persons breathing rapidly 1 m away from the brick wall. The data in the four statuses consist of 500 groups of pulses and pulse sampling points set at 1000.

Based on the computer of P42.5GHZ,512M RAM and matlab R2009a simulation experiment, we selected the 300 groups of data from the four kinds of states through the feature extraction as the training set, the rest for the testing set. In order to compare the effects of feature extraction, the feature extraction method of this paper is compared with the PCA feature extraction method regarding the classification result. Additionally, the experiments were performed on the same training and testing data using the two feature extraction algorithms combined with SVMs. Through the theoretical knowledge in Section 2, we know that the KPCA method has the same mathematical and statistical properties as the linear PCA in the feature space, such as lack of correlation of the principal components; the principal component can represent the maximum variance of the sample data; the sample is reconstructed with the least squares error; in addition, it can extract more sample information than the linear PCA. Under the premise of the same classification performance, KPCA requires fewer principal elements than PCA, and it does not need to solve the nonlinear optimization problem, only the eigenvalue decomposition of the matrix, compared with other nonlinear feature extraction methods.

In the simulation results, the eigenvalues by descending order of PCA and KPCA are shown in the following: Fig. 3 shows the matrix eigenvalues of PCA and Fig. 4 the matrix eigenvalues of KPCA. In this picture, the horizontal axis represents the number of sample eigenvalues, and the vertical axis represents the eigenvalues. Compared with the two graphs, we can see that the KPCA tends to zero at the 300th eigenvalue, but the PCA starts to tend to zero at the 500th eigenvalue. If the variance contribution rate that the sum of the selected eigenvalues divided by the sum of all the eigenvalues reaches 90%, the PCA needs the first 100 eigenvalue representations, that is the 1000-dimensional column vector can be compressed into a vector of only 100-dimension. However, the KPCA only needs the first 70 eigenvalues to make the contribution rate of 90%. In other words, the 1000-dimensional column vector can be compressed into a vector of only 70-dimension. Under the same classification performance, the KPCA requires fewer principal elements and lower mean square error, and the effect of reducing the dimension is more obvious.

Fig. 3

Matrix eigenvalues of principal component analysis (PCA)

Fig. 4

Matrix eigenvalues of kernel principal component analysis (KPCA)

In this paper, the data by feature extraction of the training set is input into the SVMs for the training model. After the model training is finished, the test set data are input to the trained model for testing, and the experimental results are showed in Table 2. In Table 2, nine indicators are listed, and the feature extraction performance of the two algorithms is compared by each indicator. The accuracy of the KPCA algorithm is significantly higher than that of the PCA algorithm. The kappa statistics are an important indicator of the degree of consistency in the evaluation. If the kappa value is greater than 0.75, the consistency is good, indicating consistency between 0.75 and 0.4; less than 0.4 indicates poor consistency. From the table, we can see that the kappa value of the KPCA algorithm is greater than 0.75, the consistency is better, and the kappa value of PCA algorithm is less than 0.75 consistency. The mean absolute error and root mean squared error are used to measure the difference between the predicted value of the classifier and the actual result and the smaller the value the better the classification performance; so, it is apparent that the KPCA is superior to PCA. When the absolute error cannot reflect the error of the true size, the relative absolute error and root relative squared error can reflect the size of the error by the proportion of the error to the true value. Additionally, the smaller the relative absolute error is, the better the classification performance of the feature extraction algorithm is. To sum up, the KPCA and SVM combination algorithms are better than those of PCA and SVMs in comparison with each index of the two algorithms.

Table 2 Experimental results compared with PCA + SVM, KPCA + SVM

This paper also used the ROC curve to analyze the experimental results. The ROC curve is closer to the upper left corner, it represents the algorithm having higher accuracy. As shown in Fig. 5, the ROC curve of through-wall human being detection by the two algorithms, using the KPCA algorithm, is significantly better than that of the PCA algorithm. The ROC curve of through-wall human being detection is shown as follows:

Fig. 5

Receiver operating characteristic (ROC) of through-wall human being detection

5 Conclusion

We have presented a framework for through-wall human being detection under four statuses with P410 UWB radar using the SVM classification algorithm. In the aforementioned classification algorithm, we needed to extract the feature of the data to reduce the dimension and the time complexity. Compared to other feature extraction methods, KPCA has the following main advantages: (1) nonlinear data is effectively dealt with, (2) no nonlinear optimization is involved, (3) the calculations in KPCA are as simple as in standard PCA, and (4) the number of PCs need not be specified prior to modeling. So, in this paper, the combination of KPCA and SVMs with this method combines the advantages of KPCA component analysis and SVMs in the application of pattern recognition, so the practical application can be better than the performance of the two methods alone. Experimental results show that the algorithm can effectively distinguish between there being no person behind the brick wall, two persons walking 1 m away from the brick wall, three persons breathing normally 2 m away from the brick wall, and two persons breathing rapidly 1 m away from the brick wall which has important theoretical significance and practical application value. Compared with the traditional PCA method, the simulation results show that the proposed method has better stability and reliability, can improve the recognition rate effectively, and can effectively optimize the extraction of the radar target’s principal feature, accelerate the identification of its speed, improve the recognition performance of the target, and it also has good promoting ability. Further research work is being focused on a small sample of multiple-status, human being detection and aims to improve recognition rate with different kernel functions.