Stepwise Procedures In Discriminant Analysis

1. CHAPTER TWO
   LITERATURE REVIEW

2.0 DISCRIMINANT ANALYSIS

Discriminant analysis as an important multivariate technique offers wide range of uses. The ideas associated with discriminant analysis can be traced back to the 1920s and work completed by the English Statistician Karl Pearson. In the 1930s, R. A. Fisher translated multivariates inter-group distance into a linear combination of variables to aid in inter-group discrimination.

Huberty (1989) stated that discriminant analysis includes a set of response variables and a set of one or more grouping variables. Klecka (1980:8) showed the basic pre-requisites for conducting a discriminant analysis, i.e., “two or more groups exist which we presume differ on several variables and that those variables can be measured at the interval or ratio level. Discriminant analysis helps us analyze the differences between the groups and/or provide us with a means to classify or assign any case into the groups which it most closely resembles.

Discriminant analysis shares similarities with multiple regression (MR) analysis. Kachigan (1986) viewed discriminant analysis as an adaptation of regression analysis techniques. With a sound knowledge on the basic goals and techniques of multiple regressions, it is very easy for one to understand the association between multiple regression and discriminant analysis. Some researchers, Cohen (1968) and Knapp (1978) have derived from the same linear model which includes the use of least square weights.

According to Knapp (1978), virtually all the commonly encountered tests of significance can be treated as special cases of Canonical Correlation analysis, which appears to be the general procedure for showing differences between two sets of variables.

Thompson (1988) indicated that every parametric procedure deals with the creation of a synthetic score(s) for each individual on discriminant function. Pedhazur (1982) defined the synthetic scores as the discriminant scores created with the discriminant function coefficient.

Cooley and Lohnes (1962) viewed discriminant analysis as a technique for description and testing of between group differences. The tests associated with this are identical to those of multivariate analysis of variance (MANOVA). Separation of distinct sets of objects as well as allocating new objects to previously defined groups is a problem of discriminant analysis.

Onyeagu (2003) viewed discriminant analysis as concerned with the problem of classification, while Anderson (1958) says that the problem of classification is the problem of “Statistical decision functions”.

According to Lachenbruch (1975), the problem of discriminant analysis is that of assigning an unknown observation to a group with a low error rate.

2.1 STEPWISE DISCRIMINANT ANALYSIS

In order to ensure that important variables are not excluded during the variables selection process in a multivariate Discriminant Analysis, it is important that the researcher will carefully select the useful variables and discard the redundant ones.

Stepwise discriminant Analysis is used to ensure that only important variables are selected. It assists in the determination of relative importance of the set of variables even if no samples are to be discarded.

Stepwise Discriminant Analysis is said to have been first advanced by Efroymson (1960).

Thompson (1989) described stepwise methods in discriminant analysis as one of the most popular research practices employed in Statistical research. It is mainly employed whenever large number of independent variables are available for an experiment, and the experimenter wishes to discard those variables that are less important in the presence of other

variables for separating the groups.

Huberty (1994) noted that the stepwise methodologies which have enjoyed popular usage, especially in educational and psychological research settings, can be done in any of the following ways;

1. Forward selection;
2. Backward elimination;
3. Forward stepwise and
4. Backward stepwise analysis

However, the default settings usually result in a forward stepwise analysis. Stepwise discriminant analysis can be done with the following popular computer software packages;

1)BMDP;

2)SAS and

3)SPSS

Rencher (1995) revealed that the importance of a variable in the presence of other variables can be determined by simply examining the absolute standardized discriminant function coefficients of the variables. The variable with the highest absolute standardized discriminant function coefficient is the most important variable, while the variable with the smallest absolute standardized discriminant function coefficient is the least

important variable in the presence of other variables. Forward selection will

select the variable with the highest absolute standardized discriminant function coefficient and discard the variable with the lowest absolute standardized discriminant coefficient. The backward elimination is the reverse of the forward selection method.

LINEAR DISCRIMINANT FUNCTION

According to Ogum (2002), the variates Xi, ... ,XP, are assumed to follow a multivariate normal distribution with the variance, Vii of Xi and the covariance, Vy of Xi and Xj assumed to be equal in the populations. Then, the linear discriminant function, _/v’LiXi, is defined as the linear function of Xi that gives the smallest probability of misclassification.

Where Lis are the discriminant function coefficients.

Rencher (1995) outlined that discriminant function is the linear combination of p-variables that maximizes the distances between two groups mean vectors, given that there are y11, y12, ., y1n1, samples from population one (1) and y21, y22,...,y2n2 samples from population (2), consisting of measurements on p-variables. In fact, linear Discriminant functions are linear combinations of the p-variables that best separate

groups.

In Discriminant Analysis, our interest is centered on separation of distinct sets of objects and allocation of new objects to already defined groups with the aid of a discriminant function.

Johnson and Wichern (1992) explained that a function that separates groups may sometimes serve as a function that allocates new individuals to already defined groups and conversely, an allocatory rule (a function that assigns a new individual to an already defined group based on multivariate measurements on the individual), may suggest a discriminatory procedure. This leads to the frequent overlapping between discrimination and classification, thus making the distinction between separation and allocation so blurred.