Pattern Recoctnition Pergamon Press 1977. Vol. 9, pp. 16%170. Printed in Great Britain ON A MODIFIED FORM OF PARZEN ESTIMATOR FOR NONPARAMETRIC PATTERN RECOGNITION*t K. S. SHANMUGAM Electrical Engineering Department, Wichita State University, Wichita, KS 67208, U.S.A. (Received 8 December 1976; in revised form 19 April 1977) Abstract--The use of Bayesian Strategy in pattern recognition problems involves the estimation of probability density function of each of the categories of patterns. If the functional forms of the density functions are not known, the estimation has to be done nonparametrically. A commonly used non- parametric density estimator makes use of the weighting function technique, developed for the one dimensional case by Parzen ~ and later extended to the multi-dimensional case by Murthy32~ Several authors (Shanmugam, ~3~ Koontz and Fukunaga, 141 and Duin ~s~) have suggested modified forms of Parzen estimator for a variety of applications. In this paper, we derive a form of Parzen estimator which uses a data dependent smoothing matrix and a Gaussian weighting function. We show that this form of Parzen estimator has some desirable parametric properties. We also establish that the estimator is asymptotically consistent. Pattern recognition Nonparametric estimators Probability density functions Parzen estimator INTRODUCTION The estimation of the probability density function f(x) of a p dimensional random variable X from a set of randomly selected samples X~,X2,...,X, is an important problem in the field of pattern recognition (Duda and Hart, ~6~ Patrick~7~). If the functional form of the probability density function f(x) is not known a priori, then the estimation of f(x) has to be done nonparametrically. A commonly used nonparametric estimator f,(x) off(x) has the form ./~(x) = 1 " K~ xl z X~sl XP- XsP~ (1) " where X1 x -- x2 and Xp ( jl) Xj = X j2 . JP The kernal or the weighting function K in (1) is an arbitrarily bounded probability density function and the smoothing factors {hi(n)} are sequences of positive numbers. It has been shown by Parzen ~11 and Murthy 121 that the estimator given in (1) is asymptotically * This work was partially supported by NSF Grant No. Soc. 76-12358 AO1 and Eng. 74-18981. ~ An earlier version of this paper was presented at the 3rd International Pattern Recognition Conference, San Diego, 8-11 November, 1976. ~; An estimator is said to be asymptotically unbiased and consistent if its bias and variance approach zero as the sample size approaches infinity. unbiased:~ if and that the estimator is asymptotically consistent if l{ t] Lt n Iq h,(n) -* ;c. (31 -~ i=1 Further, the estimator has been shown to be uniformly consistent if Lt n hi(n --~.-/. (4) n*z i A variety of weighting functions and smoothing factors satisfy the constraints outlined above and the selection of a particular form of K and {hi(n)) is often guided by subjective reasoning. The most commonly used weighting function has the form of a multivariate normal probability density function. The smoothing factors {hi(n)}, i= 1,2 ..... p, used in previous applica- tions fall into one of the following three classes: (1) {hi(n)} depend on n alone (SprechtlS~), (2) l hi(n)} = 1/p (trace I~) n-'/v, 0 < 7 < 0.5, where is the estimated covariance matrix, i.e. I hi(n)} are data dependent (Koontz and Fukunaga1%, (3) { hi(n)} are chosen to maximize the product of the estimated value of the probability density function at the sample points ~,Xi}; [hi(n)} are again data depen- dent (DuintS)): The selection of smoothing factors according to (2) and (3) pose the following problems. The data depen- dent method of selecting the smoothing factors pro- posed by Duin ~s~ requires excessive amounts of computation, and the asymptotic properties of the resulting estimator have not been established. In the estimator proposed by Koontz and Fukunaga 14~ the 167
