37610-Hair6_im

Table of ContentsINTRODUCTION..........................................2 CHAPTER ONE...........................................8 CHAPTER THREE........................................37 ADVANCED DIAGNOSTICS FOR MULTIPLE REGRESSION ANALYSIS ................................................68 CHAPTER FIVE.........................................71 CHAPTER SIX..........................................91 CHAPTER SEVEN.......................................102 CHAPTER EIGHT.......................................118 CHAPTER NINE........................................129 CHAPTER TEN.........................................144 CHAPTER ELEVEN......................................154 CHAPTER TWELVE......................................169 SAMPLE MULTIPLE CHOICE QUESTIONS....................177 1 INTRODUCTION This manual has been designed to provide teachers using Multivariate Data Analysis, 6th edition, with supplementary teaching aids. The course suggestions made here are the result of years of experience teaching the basic content of this text in several universities. Obviously, the contents may be modified to suit the level of the students and the length of the term. Multivariate data analysis is an interesting and challenging subject to teach. As an instructor, your objective is to direct your students' energies and interests so that they can learn the concepts and principles underlying the various techniques. You will also want to help your students learn to apply the techniques. Through years of teaching multivariate analysis, we have learned that the most effective approach to teaching the techniques is to provide the students with real-world data and have them manipulate the variables using several different programs and techniques. The text is designed to facilitate this approach, making available several data sets for analysis. Moreover, accompanying sample output and control cards are provided to supplement the analyses discussed in the text. WHAT’S NEW AND WHAT’S CHANGED The sixth edition has many substantial changes from prior editions that we feel will markedly improve the text for both faculty member and student. Three notable additions were made to the text: • The most obvious change in the sixth edition is the new database— HBAT. The emphasis on improved measurement, particularly multiitem constructs, led us to develop HBAT. After substantial testing we believe it provides an expanded teaching tool with various techniques that are comparable to the HATCO database, which will still be available on the book’s Web site. A second major addition is “Rules of Thumb” for the application interpretation of the various techniques. The rules of thumb highlighted throughout the chapters to facilitate their use. We confident these guidelines will facilitate your utilization of techniques. and are are the • • A third major change to the text is a substantial expansion in coverage of structural equations modeling. We now have three chapters on this increasingly important technique. Chapter 10 provides an overview of structural equation modeling, Chapter 11 focuses on confirmatory factor analysis, and Chapter 12 covers issues in estimating structural 2 models. These three chapters provide a comprehensive introduction to this technique. Each chapter has been revised to incorporate advances in technology, and several chapters have undergone more extensive change: • Chapter 2 “Examining the Data” has an expanded section on missing data assessment, including a flowchart depicting a series of decisions that are involved in identifying and then accommodating missing data. • Chapter 5, “Multiple Discriminant Analysis and Logistic Regression,” provides complete coverage of analysis of categorical dependent variables, including both discriminant analysis and logistic regression. An expanded discussion of logistic regression includes an illustrative example using the HBAT database. • Chapter 7, “Conjoint Analysis,” has a revised examination of issues of research design that focuses on the development of the conjoint stimuli in a concise and straightforward manner. An important development is the continuation of the Web site “Great Ideas in Teaching Multivariate Statistics” at www.mvstats.com, which can also be accessed as the Companion Web site at www.prenhall.com/hair. This Web site acts as a resource center for the textbook as well as everyone interested in multivariate analysis, providing links to resources for each technique as well as a forum for identifying new topics or statistical methods. In this way we can provide more timely feedback to researchers than if they were to wait for a new edition of the book. We also plan for the Web site to be a clearinghouse for materials on teaching multivariate statistics—providing exercises, datasets, and project ideas. ORGANIZATION OF THE CHAPTERS IN THE TEXT The text is designed to help make your teaching as enjoyable and as simple as possible. Each chapter begins with a "Chapter Preview" so that students will understand the major concepts they are expected to learn. To facilitate understanding of the chapter material and as a ready reference for clarification, definitions of key terms are presented at the front of each chapter. The text is designed for those individuals who want to obtain a conceptual understanding of multivariate methods—what they can do, when they should be used, and how the results should be interpreted. Following this design, each chapter is structured in a step-by-step manner, including six steps. The end of each chapter includes an illustration of how to apply and interpret each technique. Basically, the approach is for the "data analyst," therefore, the math formulae and symbols are minimized. We believe it is the most practical, readable guide available to understanding and applying 3 these otherwise complex statistical tools. At the end of each chapter, we review the Learning Objectives to provide the student with an overview of what has been covered in the chapter in relation to those major concepts or issues defined in the Learning Objectives. Finally, a series of questions is provided to stimulate the student to evaluate what has been read and to translate this material into a workable knowledge base for use in future applications. ORGANIZATION OF THE CHAPTER MATERIALS This instructor's manual is designed to facilitate the preparation and conduct of classes, exams, and seminars. Materials included in the manual are organized in two sections for each chapter. (1) Chapter summaries: to refresh the instructor's memory without the necessity of re-reading the entire chapter prior to class. Each chapter summary is organized around four major sections. The objective of these sections is to identify particular issues that may be useful in organizing class discussion. The four sections are: a. What – an overview, or brief description, of the technique. b. Why – a description of the basic objectives of the technique. c. When – identification of the types of problems the technique may be used to address. d. How – description of the assumptions applicable to the technique, the data requirements for its use, the major points which are essential to the successful implementation of the research plan and the key points contained in the computer output needed for a complete and accurate interpretation of the results. (2) Answers to the end-of-chapter questions: suggested answers to the questions that can form the basis for further elaboration if desired. Sample exam questions: while essay or short answer questions are probably preferable for examinations, many times multiple choice questions can be used as a method for assessing specific knowledge about the subject. To ease the burden of writing exam questions, multiple choice questions are provided for each chapter. All of the sample exam questions have been placed in a separate section at the end of the Instructor’s Manual. 4 THE COMPANION WEB SITE – MVSTATS.COM The authors have established a Web site entitled “Great Ideas in Teaching Multivariate Statistics” with the objective of providing a clearinghouse for instructional materials and a forum for discussions about pedagogical issues. Accessed directly at www.mvstats.com or through the Companion Web site link at the Prentice-Hall Web site (www.prenhall.com/hair), the Web site will offer all of the supplementary materials for the text (datasets, control card files and output files) as well as links to additional datasets for use in class and Web-based materials for each technique. A complete set of datasets and related materials are available not only for the sixth edition, but the fifth edition (HATCO dataset) as well. We sponsor a mailing list MVSTAT-L that is open to the interested participants for asking questions related to either the teaching or application of multivariate statistics. An important adjunct is a “faculty-only” section of the Web site where additional pedagogical materials will be made available to all adopters of the textbook. The permission-based section will allow for providing the text-related materials (e.g., PowerPoint slides and image files of all figures in the text) as well as acting as a forum for faculty interested in teaching multivariate statistics. We encourage any faculty member to contribute to the mailing list or even contribute class-related materials which we will disseminate among all those interested in the subject area. We envision this to be an evolutionary project, with its growth and focus guided primarily by its users and contributors. We hope that we can provide an easy and readily available forum for discussion and collaboration among interested faculty members. The understanding and interpretation of multivariate analysis is enhanced immeasurably by the ability of students to actually analyze data (e.g., derive research questions and execute the analysis) and/or examine actual computer output to ascertain what is the form and content of the results. To avoid any unnecessary duplication of effort on the part of the instructor, the Web site contains a number of computer files related to the analyses conducted throughout the text: Datasets There are a number of datasets provided to students and faculty to perform all of the multivariate analyses described in the textbook. While some techniques require specialized datasets (e.g., multidimensional scaling, conjoint and structural equation modeling), many of the techniques can be performed using conventional survey data. To this end, a common dataset has been developed for use with many of the techniques to allow students to see the interrelationships 5 among techniques as well as the techniques themselves. The HBAT dataset has three forms utilized throughout the text: • HBAT – the primary database described in the text which has multiple metric and nonmetric variables allowing for use in most of the multivariate techniques. HBAT200 – an expanded dataset, comparable to HBAT except for 200 rather than 100 respondents, that allows for multiple independent variables in MANOVA while still maintaining adequate sample size in the individual cells. HBAT_MISSING – a reduced dataset with 70 respondents and missing data among the variables. It is utilized to illustrate the techniques for diagnosis and remedy of missing data described in Chapter 2. • • In addition to these datasets, there are several others used with specific techniques, including conjoint analysis, multidimensional scaling and structural equation modeling. These datasets include: • • HBAT_CPLAN and HBAT_CONJOINT – the datafiles needed to perform the “full profile” conjoint analysis available in SPSS. HBAT_MDS and HBAT_CORRESP – the datafiles used in performing the multidimensional scaling and correspondence analyses described in the text. HBAT_SEM – the original data responses from 400 individuals which are the basis for the structural equation analyses of Chapters 10, 11 and 12. HBAT_COV is the covariance matrix derived from HBAT_SEM that is used as input to structural equation programs such as LISREL, EQS or AMOS. • Finally, two additional datasets are provided to allow students access to data other than the HBAT datafiles described in the textbook: • HATCO – this dataset has been utilized in past versions of the textbook and provides a simplified set of variables amenable to all of the basic multivariate techniques. SALES – this dataset concerns sales training and is comprised of 80 respondents, representing a portion of data that was collected by academic researchers. Also, a copy of the sales training questionnaire is provided. • 6 Given the widespread interchangeability of data formats among statistical programs, all of the datasets are provided in two formats. First is the .SAV format used in SPSS, which allows for documentation of variable descriptions, etc., in a standard format. Also, all of the datasets are contained in an EXCEL worksheet, amenable to input to any statistical program. Between these two formats the student or faculty member should be able to employ the data with any available statistical software program. Control Card Files: To assist the instructor in performing the analyses illustrated in the text, control card files containing program syntax commands are provided for the statistical software programs SPSS (Statistical Package for the Social Sciences, SPSS Inc.) and LISREL (Scientific Software, Inc.). Computer Outputs If computer access is not available for a particular technique, files of the actual outputs (from SPSS for Chapters 2 – 9 and LISREL for Chapters 10-12) for each analysis are also provided. This enables faculty and students with the complete computer outputs even without actually running the programs. Acknowledgements The authors wish to express their thanks to the following colleagues for their assistance in preparation of the current and previous versions of the Instructor's Manual and other course supplements: Rick Andrews, Scott Roach, Barbara Ross, Sujay Dutta, Bruce Alford, Neil Stern, Laura Williams, Jill Attaway, Randy Russ, Alan Bush, Sandeep Bhowmick and Ron Roullier. 7 CHAPTER ONE INTRODUCTION TO MULTIVARIATE ANALYSIS This presentation will approach the general idea of multivariate data analysis by attempting to answer the basic questions of "What?," "Why?," "When?," and "How?". What is multivariate analysis? 1. Multivariate analysis, in this text, includes both multivariable techniques and multivariate techniques. The term "multivariate analysis" really stands for a family of techniques, and it is important that you realize this fact from the onset. The common characteristic is that multiple variables are used for the dependent and/or independent variables in the analysis. Each technique has its own special powers and unique instances of applicability. These will be revealed to you in detail in subsequent chapters in the text. 2. For now, it is only necessary that you realize in general terms the special powers represented by this family of analyses: • • Description – Some of the techniques have rather amazing abilities to describe phenomena. They can find patterns of relationships where the human eye and even univariate or bivariate statistics fail to do so. Explanation – Other techniques have special capabilities to explain or to help explain relationships. For instance, they can isolate the impact of one variable on another; show relative differences between the magnitude of impact for two or more variables; or even reveal how one set of variables impinges on another set. Prediction – Certainly every decision maker desires the ability to predict, and multivariate techniques afford that promise. Certain multivariate analyses are explicitly designed to yield predictive modes with specific levels of accuracy. Control – Finally, certain multivariate techniques can be used for control. Of course, the techniques themselves will not institute control, but their application can help a decision maker develop cause-andeffect relationships and models which will ultimately enhance their ability to control events to some degree. • • 8  Nonmetric data are categorical variables that describe differences in type or kind by indicating the presence or absence of a characteristic or property. o Definition: a linear combination of variables with empirically determined weights. There are primary and secondary uses of each of the multivariate techniques as well as cases in which they are not applicable. o Variables are specified by the researcher. the extension to multiple variables or variates introduces a number of issues which must be understood before examining each technique in detail. "P" indicates a primary ability of the technique. can be metric or nonmetric. in the form of measurement scales. o Metric vs nonmetric: Data. The researcher must make sure that he or she has the appropriate type of data to employ the chosen multivariate technique. This single value. the variate produces a single value that represents the entire set of variables.3. Figure 1 at the end of this section is a reference in this respect. perhaps it would be helpful to show the "fit" of each to the four abilities just enumerated. For this reason. • The Role of the Variate – Multivariate techniques differ from univariate techniques in that they employ a variate. The "NA" means that the technique is not applicable. While multivariate analyses are descended from univariate techniques. o Single value: When summed. the type of data held by the researcher is instrumental in the selection of the appropriate multivariate technique. is important in all multivariate techniques. Given that you will be introduced to each one in the text chapters. 4. while "S" represents a secondary ability. and the weights are assigned by the technique. in addition to the specific contribution of each variable to the variate. In the Figure.  9 . • Specification of measurement scales – Each multivariate technique assumes the use of a certain type of data. Metric data are continuous measures that reflect differences in amount or degree in a relative quantity or distance. Alpha and beta are inversely related. 10 .  Validity is the degree to which a measure accurately represents what it is supposed to.  Specifying type II error (beta) also specifies the power of the statistical inference test. Interpretation of statistical inferences requires the specification of acceptable levels of error. which adds residual error to the observed variables. o Specifying statistical error levels. • Statistical significance and statistical power.  • • • Power is determined by 3 factors: its inverse relationship with alpha. o Need for assessment of both the validity and the reliability of measures. the effect size. o Power: the probability (1-beta) of correctly rejecting the null hypothesis when it should be rejected. including data entry errors and the use of inappropriate gradations in a measurement scale.• Identification of measurement error o Definition: the degree to which the observed values in the sample are not representative of the true values in the population.  Type I error (alpha) is the probability of rejecting the null hypothesis when it is actually true. o All variables in a multivariate analysis have some degree of measurement error.  Reliability is the degree to which the observed variable measures the true value and is error-free. o Sources: several sources are possible. Almost all of the multivariate techniques discussed in this text are based on statistical inference of a population's values from a randomly drawn sample of the population.  Type II error (beta) is the probability of failing to reject the null hypothesis when it is actually false. and the sample size. the variables interact with each other as well. previous experiences with the product. conversations with neighbors. interest rates. In addition to the main impact of these variables. it is most commonly the case that you have several variables observed a great many times. a manager is faced with a complete system of complicated relationships in any single decision-problem. When planning research. and a good deal more. explanations. multivariate analysis is appropriate whenever you have two or more variables observed a number of times. is related to taxes. When do you use multivariate analysis? Essentially. and season of the year. To managers or persons responsible for making decisions within an organization. In short. Inflation. strive to make some sense out of their environment. You could even choose to ignore the impacts of some variables. and • then select the sample size and the alpha level to achieve the desired power level. They seek some order to the chaos of interrelationships. the money supply. or simplification. for instance. They are the means of achieving parsimonious descriptions. credibility of the actor used in the commercial. competitors. Buyers' reactions to an advertisement are related to the price of the item. the actual power should be calculated in order to properly interpret the results. You can use intuition. There are many ways of attempting to achieve order.  Upon completion of the analyses. From these two situations—the myriad of interrelated variables and the need for an accepted framework—springs the need for multivariate analysis techniques. the business cycle. Why is there multivariate data analysis? The need for multivariate analysis comes from the simple fact of life that just about everything is in some way interrelated with other things. each decision is affected by many "variables. oil prices. the researcher should: • estimate the expected effect size. Humans." These variables are identifiable entities which have some impact or relationship to the focal topic. In fact. foreign wars. also known as a "data 11 . and predictions of reality. logical systems such as set theory. But most intelligent managers prefer to rely on some framework which is more accepted. by their very nature. warranty terms. factor analysis might result in ten factors. scaling of variables. MANOVA might reveal 12 cases of significant differences.000 cells of data. It should be evident that parsimony can be achieved by using multivariate techniques when analyzing most data sets. each of whom answers 100 questions. when completing univariate tests. dependence or interdependence." It is traditional to envision a data set as being comprised of rows and columns. as well as the empirical. The following summary provides an overview of each of the six steps. Depending on the type of question. Sample sizes. the researcher should define the research question(s). such as each person or each completed questionnaire in a large survey. The rows pertain to each observation. Hence the data set would be 1. Based on an underlying theoretical model. Multivariate techniques aid the researcher in obtaining parsimony by reducing the number of computations necessary to complete statistical tests.000 by 100. issues for each technique are discussed.000 separate values computed. there would be close to 5. multivariate analysis would require many less computations. and Multivariate Technique To Be Used. Obviously. if an average were computed for each variable. The conceptual. such as a response to a question or an observed characteristic for each person. Multiple regression could identify six significant predictor variables. Objectives. Cluster analysis might yield five clusters. Data sets can be immense. For example. Step 2: Develop the Analysis Plan Each multivariate technique has an associated set of issues which are relevant to the design of an analysis plan. 100 means would result. Step 1: Define the Research Problem. pertain to each variable. the researcher should also identify the type of multivariate technique.000 or more respondents. 12 .set. The six steps defined in the following chapters should be viewed as guidelines for better understanding the process of applying each technique. How do you use multivariate data analysis techniques? This text follows a structured approach to multivariate model building. In sharp contrast. on the other hand. a single study may have a sample size of 1. and if a correlation were computed between each variable and every other variable. and estimation methods are common issues which must be evaluated by the researcher prior to data collection. For example. Multiple discriminant analysis perhaps would find seven significant variables. The columns. the need for parsimony is very evident. which is most appropriate. or 100. Step 6: Apply the Diagnostics to the Results Finally. the overall model fit is compared to specified criteria. After the model is estimated. or practical. you will find that these issues are identified several times across a number of multivariate techniques. Coefficients of the variables are examined. the researcher must evaluate the underlying assumptions of the chosen multivariate technique. The objective is to identify findings which can be generalized to the population. Interpretation may lead to model respecification.Step 3: Evaluate the Assumptions Underlying the Multivariate Technique Once the data are collected. • Establish Practical Significance as Well as Statistical Significance Not only must the researcher assess whether or not the results of a multivariate analysis are statistically significant. Step 5: Interpret the Variate Once an acceptable model is achieved. Sample Size Affects All Results The size of the sample will impact whether or not the results achieve statistical significance. these guidelines serve as a "general philosophy" for completing multivariate analyses. The model may be respecified for better fit if necessary. As you review each chapter. implications for action. 13 • . Researchers should always assess the analysis results in light of the sample size. All techniques have conceptual and statistical assumptions which must be met before the analysis can proceed. six guidelines which are applicable to all multivariate analyses are given below. Some General Guidelines for Multivariate Analysis Further. For this reason. Step 4: Estimate the Multivariate Model and Assess Overall Model Fit Following a testing of the assumptions. These two actions provide the researcher with support for the research findings. the researcher must assess whether the results are unduly affected by a single or a small set of observations and determine the degree of generalizability of the results by validation methods. he or she must also determine whether or not the results have managerial. a multivariate model is estimated with the objective of meeting specific characteristics. the nature of the multivariate relationship is investigated. Too little or too much power can be the result of sample sizes. the analyst must first determine the number of dependent variables. the analyst must address two issues: objective of the analysis and type of variables used. Strive for Model Parsimony The researcher should evaluate those variables chosen for inclusion in the analysis. First. If the relationship is dependence based. but are representative and generalizable to the population. the analyst must evaluate the theoretical nature of the problem and determine if the objective is to assess a dependence relationship (predictive) or is an interdependence approach (structure seeking) needed.2 in the text provides a general decision process with which to select the appropriate multivariate technique. In selecting among the interdependence techniques. Validation procedures ensure the analyst that the results are not merely specific to the sample. Diagnostic measures are available to evaluate the nature of sets of multivariate variables. Second. the first model estimation does not provide the best model fit. the analyst must define whether structure among variables. the researcher should analyze the prediction errors and determine potential changes to the model. Specification error (omission of relevant variables) and high multicollinearity (inclusion of irrelevant variables) can substantially impact analysis results. 14 . Errors serve as diagnostics for achieving better model fit. Validate Your Results The researcher should always validate the results. Thus. respondents or objects is desired. The objective is to create a parsimonious model which includes all relevant variables and excludes all irrelevant variables. the analyst must determine how the dependent variables will be measured. Look at Your Errors Often.• Know Your Data Multivariate analyses require rigorous examination of data. • • • Selecting a Multivariate Technique Figure 1. In doing so. FIGURE 1 MULTIVARIATE TECHNIQUES AND THEIR ABILITIES Ability Technique Control Multiple Regression NA Multiple Discriminant NA MANOVA P Canonical Correlation NA Factor Analysis NA Cluster Analysis NA Multidimensional Scaling NA Conjoint Analysis NA Structural Equation Modeling NA Legend: P: S: NA: Primary ability Secondary ability Not applicable Describe Explain Predict S S NA P P P S S S S S S S S S P P P P P S S NA NA S S P 15 . not solely in the number of variables or observations. DEFINE MULTIVARIATE ANALYSIS. b. c. Answer a. The development of several fairly sophisticated "canned" computer programs for carrying out multivariate techniques.ANSWERS TO END-OF-CHAPTER QUESTIONS 2222222 (1) IN YOUR OWN WORDS. (2) NAME SEVERAL FACTORS THAT HAVE CONTRIBUTED TO THE INCREASED APPLICATION OF TECHNIQUES FOR MULTIVARIATE DATA ANALYSIS IN RECENT YEARS. Advances in computer technology which make it feasible to attempt to analyze large quantities of complex data. The authors adopt a fairly inclusive description of the term. The ability to conceptualize data analysis has increased through the study of statistical methods. and more sophisticated techniques for data analysis are needed. all the variables must be random variables which are interrelated in such ways that their different effects cannot easily be studied separately. 16 .usually referring to techniques used in the simultaneous analysis of more than two variables. d. they avoid becoming bogged down in the nuances of "multivariable" and "multivariate. Multivariate ." The distinction between these terms is made as follows: Multi-variable . Answer a. The multivariate character lies in the multiple combinations of variables. In so doing.to be considered truly multivariate. and The research questions being asked are becoming more and more complex. (2) (3) (4) (5) (6) 17 .simultaneously analyzes several dependence relationships (e. (1) Multiple Regression (MR) .used to transform nonmetric scale responses into metric form. Multivariate Analysis of Variance (MANOVA) simultaneously analyzes the relationship of 2 or more metric dependent variables and several nonmetric independent variables.variables are divided into dependent and independent. where there is only one metric dependent variable.(3) LIST AND DESCRIBE THE MULTIVARIATE DATA ANALYSIS TECHNIQUES DESCRIBED IN THIS CHAPTER. Multiple Discriminant Analysis (MDA) . Canonical Correlation Analysis (CCA) simultaneously correlates several metric dependent variables and several metric independent variables..the objective of MR is to predict changes in a single metric dependent variable in response to changes in several metric independent variables. It is concerned with the joint effect of two or more nonmetric independent variables on the ordering of a single dependent variable. several regression equations) while also having the ability to account for measurement error in the process of estimating coefficients for each independent variable. Note that this procedure can be considered an extension of MR. A related procedure is multivariate analysis of covariance (MANCOVA) which can be used to control factors other than the included independent variables. A related technique is multiple correlation. Structural Equation Modeling .g. Conjoint Analysis . CITE EXAMPLES FOR WHICH EACH TECHNIQUE IS APPROPRIATE. Answer a. DEPENDENCE TECHNIQUES .the objective of MDA is to predict group membership for a single nonmetric dependent variable using several metric independent variables. (5) WHY IS KNOWLEDGE OF MEASUREMENT SCALES IMPORTANT TO AN UNDERSTANDING OF MULTIVARIATE DATA ANALYSIS? Answer Knowledge and understanding of measurement scales is a must before the proper multivariate technique can be chosen. Answer The multivariate techniques can be viewed as a "family" of techniques in that they are all based upon constructing composite linear relationships among variables or sets of variables. INTERDEPENDENCE TECHNIQUES . Inadequate understanding of the type of data to be used can cause the selection of an improper technique. Multidimensional Scaling (MDS) . The two major approaches are component analysis and common factor analysis. (2) (3) (4) EXPLAIN WHY AND HOW THE VARIOUS MULTIVARIATE METHODS CAN BE VIEWED AS A FAMILY OF TECHNIQUES.used to analyze the interrelationships among a large number of variables and then explain these variables in terms of their common. 18 .used to classify a sample into several mutually exclusive groups based on similarities and differences among the sample components. The family members complement one another by accommodating unique combinations of input and output requirements so that an exhaustive array of capabilities can be brought to bear on complex problems.a technique used to transform similarity scaling into distances in a multidimensional space. (1) Factor Analysis (FA) .b.all variables are analyzed simultaneously. underlying dimensions. Measurement scales must be understood so that questionnaires can be properly designed and data adequately analyzed. with none being designated as either dependent or independent. Cluster Analysis . which makes any results invalid. 19 . and Multivariate Technique to Be Used The starting point for any analysis is to define the research problem and objectives in conceptual terms before specifying any variables or measures. Stage Three: Evaluate the Assumptions Underlying the Multivariate Technique All techniques have underlying assumptions.(6) WHAT ARE THE DIFFERENCES BETWEEN STATISTICAL AND PRACTICAL SIGNIFICANCE? IS ONE A PREREQUISITE FOR THE OTHER? Answer Statistical significance is a means of assessing whether the results are due to change. Each technique must be considered individually for meeting these and other assumptions. Statistical significance would be a prerequisite of practical significance. (7) WHAT ARE THE IMPLICATIONS OF LOW STATISTICAL POWER? HOW CAN THE POWER BE IMPROVED IF IT IS DEEMED TOO LOW? Answer The implication of low power is that the researcher may fail to find significance when it actually exists. Objectives. that impact their ability to represent multivariate assumptions. These issues include: (1) sample size. both conceptual and empirical. linearity. and (3) special characteristics of the technique. Answer Stage One: Define the Research Problem. needed to achieve the desired objectives. dependence or interdependence. nonmetric. (2) type of variables (metric vs. Stage Two: Develop the Analysis Plan A plan must be developed that addresses the particular needs of the chosen multivariate technique. FOCUSING ON THE MAJOR ISSUES AT EACH STEP. Power may be improved through decreasing the alpha level or increasing the sample size. independence of error terms. and equality of variances. Techniques based on statistical inference must meet the assumptions of multivariate normality. (8) DETAIL THE MODEL-BUILDING APPROACH TO MULTIVARIATE ANALYSIS. This will lead to an understanding of the appropriate type of technique. Practical significance assess whether the result is useful or substantial enough to warrant action. Then based on the nature of the variables involved a specific technique may be chosen. a model is estimated considering the specific characteristics of the data. At this stage the influence of outlier observations is also assessed. Stage Five: Interpret the Variate With acceptable model fit.Stage Four: Estimate the Multivariate Model and Assess Overall Model Fit With assumptions met. interpretation of the model reveals the nature of the multivariate relationship. and achieves practical significance. Stage Six: Validate the Multivariate Model The attempts to validate the model are directed toward demonstrating the generalizability of the results. the overall model fit is evaluated to determine whether it achieves acceptable levels of statistical criteria. 20 . Each technique has its own ways of validating the model. After the model is estimated. identifies proposed relationships. and independence of error terms. this presentation will address the basic questions of "Why?. or data examination." "When?. Data examination enables the researcher to analyze the multivariate assumptions of normality." and "How?" as applied to examining your data prior to the application of a multivariate technique. To ensure that the statistical and theoretical underpinnings of the chosen multivariate technique are upheld. In addition. this approach is time consuming and does not enable the researcher to get the "big picture.CHAPTER TWO EXAMINING YOUR DATA Similar to the style of Chapter One. Examining a compilation of cases reduces individual observations to easily interpretable summaries. The two approaches are: • Case-by-case evaluation — although necessary in the examination of response bias." "What?. or relationships. provide the analyst with a means to present data descriptors in a manageable form." • Compilation of cases — this preferred method provides a more meaningful interpretation of the cases. Why examine data? Data examination is needed for several reasons: 1. including information about the relationships among the variables. To gain a basic understanding of the data set. 2. homoscedasticity. 21 . Each multivariate technique has underlying assumptions which will be highlighted in the following chapters. variable associations. linearity. Descriptive statistics. can be calculated from the raw data and represented simply in reduced form. 3. identifying outliers. however. some errors will occur. the process is tedious and is not recommended given the computing power available. • What is involved in examining data? Data examination techniques vary from a simple visual examination of graphical representations to complex statistical analyses which address missing data problems and the assumptions underlying the multivariate technique. but it also gives the analyst a better understanding of the nature of the data set. In addition to computer packages. such as the input of data. To analyze the impacts of uncertainties inherent in data collection. errors may result from incorrect coding or the misinterpretation of codes. In fact. • Controllable factors — controlled by the researcher or analyst. which will call attention to any impossible or improbable values which require further attention. an analyst should examine every new data set and should reexamine any dataset being used for a new multivariate application. In addition. evaluating missing data. Not only does examination provide the analyst with a test of the underlying assumptions of the multivariate technique. are designated in data examination techniques. For example. 22 . including controllable and uncontrollable factors which may influence the data set. When do you examine your data? Essentially. data examination may also be computed by hand. Many techniques are available for examining data sets. cases with a large number of missing values may be identified. and assessing assumptions. This chapter provides a detailed description of data examination in four phases: • • • • graphical representation analysis. Uncontrollable factors — characteristic of the respondent or the data collection instrument. Data examination provides the analyst an overview of the data. Most statistical software packages offer techniques for the evaluation of data. No matter how carefully the data is input. or extreme cases. For example. may also be detected via data examination. outliers. data examination is a necessary first step in any multivariate application. Many packages refer to data examination as descriptive statistics. evaluation of the possible causes and remedies for missing data in the variables in the analysis. and residuals.. linear. • Scatterplot — the most common form of graphical display for examining the bivariate relationships among variables. • Histogram — the most common form of graphical representation of the data. curvilinear. The stem is the root value to which each leaf is added to derive the actual data value. • 23 . but also includes the actual data values.e. including actual values. Relationships between two or more variables may be examined by graphical plots. The nature of the variable can be evaluated by examining the shape of the distribution. Stem and leaf diagram — similar to histograms. etc.). graphically displays the data distribution by frequencies and data categories. The scatterplot is a graph of data points. there are four phases of data examination: • • • • graphical examination of the variables in the analysis. The variable observations can be many values. expected values. Histograms can be used to examine any type of variable. and assessment of the ability of the data to meet the statistical assumptions specific to the selected multivariate technique. It displays the frequency of occurrences of the data values (X axis) with the data categories (Y axis). where the horizontal axis is one variable and the vertical axis is another variable. Scatterplot matrices — scatterplots computed for all combinations of variables. The diagonal of the matrix contains the histograms for each variable.How do you examine your data? As outlined in the chapter. identification of outliers. The patterns of the data points represent the relationship between the two variables (i.. Phase 1: Graphical Examination of the Data 1. • 2. such that the box contains the middle 50% of the values. • Glyphs or Metroglyphs: some form of circle with radii that correspond to a data value or a multivariate profile which portrays a bar-like profile for each observation. with each variable representing a different feature. o Box length is the distance between the 25% percentile and the 75% percentile. Testing for group differences requires examination of 1) how the values are distributed for each group. Iconic representation: pictorially represents each variable as a component of a whole picture. 4. Mathematical transformation: transformation of the original data into a mathematical relationship which can be displayed graphically. The asterisk inside the box identifies the median. o Outliers (marked O) are observations which range between 1. one of three types of multivariate profiles is appropriate.3.0 and 1. 2) if outliers are present in the groups. with the upper and lower boundaries of the box marking the upper and lower quartiles of the data distribution.5 quartiles away from the end of the box. and 3) whether or not the groups are different from one another. • Box plot — a pictorial representation of the data distribution of each group. Each group is represented by a box. • • 24 . o Extreme values are marked E and represent those observations which are greater than 1. When the analyst wishes to graphically examine more than two variables. o Lines or whiskers extending from each box represent the distance to the smallest and the largest observations that are less than one quartile range from the box (also marked by an X).5 quartiles away from the box. This most common form is a face. o Ignorable missing data operate at random. 25 . the researcher evaluates the pattern of the missing data and determines the potential for remedy. observed and missing. 3. they are often unidentifiable and cannot be accommodated in the research design. The analyst must identify the missing data process (reasons underlying missing data) before he or she can select a remedy. Missing data can produce hidden biases in the analysis results and can also adversely affect the sample size available for analysis.  • Unidentifiable — When the missing data are due to an action of the respondent. • Exclusion of too many variables due to missing data can substantially affect the sample size. Specific remedies are not needed since the allowances are inherent in the technique used. 2. the observed values are a random sample of the total set of values. any observation with missing data on any of the variables will be excluded from the analysis. • Ignorable missing data — When the missing data process is known and is external to the respondent. and it can be accommodated in the research plan. • Without remedy. o Examples of ignorable missing data:  observations in a population which are not included in the sample.Phase 2: Evaluating Missing Data 1. In this case. or censored data (observations which are not complete because of their stage in the missing data process). We know that sample size will impact the power of any statistical tests and affect whether or not the results achieve statistical significance. or appropriate course of action. A missing data process may be of two kinds: a systematic event external to the respondent (ignorable) or any action on the part of the respondent which leads to missing values (unidentifiable). o Overall test of randomness: Analyze the pattern of missing data on all variables and compare it to the pattern expected for a random missing data process. the missing data can be classified as MCAR. The correlations indicate the degree of association between the missing data on each variable pair. but not on Y. o Dichotomized correlations for a pair of variables: For each of the two variables. then compute correlations for the missing values of each variable. If significant differences are found. meaning that the missing data should be classified as MAR. 26 . If no significant differences are found. replace each valid value with a value of one and each missing value with a value of zero. and test for significant differences between the two groups on any other variables of interest. This occurs when X biases the randomness of the observed Y values. Assessing the degree of randomness will identify one of two types: missing at random (MAR) and missing completely at random (MCAR). one group being those observations with missing data and another group being those observations with valid values. a nonrandom missing data process is present. • Approaches for diagnosing the randomness of the missing data process o Significance tests for a single variable: Form two groups.4. such that the observed Y values do not represent a true random sample of all actual Y values in the population. • Missing completely at random (MCAR): When the observed values of Y are truly a random sample of all Y values. Low correlations denote randomness in the pair of variables. the missing data can be classified as MCAR. If all variable pairs have low correlations. • Missing at random (MAR): When the missing values of Y depend on X. Imputation methods should be used only if the data are MCAR. Cold deck imputation: missing values are replaced with a 27  . but is an alternative which can be used if the data are MAR or MCAR. o Most effective for data which are not missing at random. maximizing pairwise information. o Used only if the missing data are missing completely at random (MCAR). Imputation methods replace missing values with estimates based on the valid values of other variables and / or cases in the sample.  Mean substitution: missing values for a single variable are replaced with the means value of that variable based on all responses. • Imputation methods. • Delete case(s) and/or variable(s). when used with data which are missing at random (MAR). • Use of only observations with complete data. o Five imputation methods are available:  Case substitution: observations with missing data are replaced by choosing another nonsampled observation. only data from  The all-available approach uses all available valid observations to estimate missing data. the results are not generalizable to the population.  The complete case approach uses observations that have no missing data. o Selecting values or observations to be used in the imputation process. Approaches are available for dealing with missing data that are selected based on the randomness of the missing data process.5. the researcher would include only those observations with complete data. The researcher would delete the case(s) and/or variable(s) which exceed a specified level from the analysis. o Default in many statistical programs. When conducting analysis. • Beneficial outliers — when they are indicative of characteristics in the population that would not be discovered in the normal course of analysis. • Outliers arising from a procedural error. Estimated values are based on their relationship with other variables in the data set. These outliers cannot be explained. Often. Outliers can be classified into four categories. constant value derived from external sources or previous research.  • Model-based procedures. these observations are deleted from the data set. Phase 3: Identification of Outliers 1. • 2. Multiple imputation: a combination of several methods. but instead must be viewed within the context of the analysis and should be evaluated by the types of information that they may provide regarding the phenomenon under study. • Outliers resulting from an extraordinary event with an explanation These outliers can be explained. 28 . Outliers cannot be categorically characterized as either beneficial or problematic. either through a process specifically designed for missing data estimation or as an integral portion of the standard multivariate analysis. If found to be representative of the population. Regression imputation: missing values are replaced with predicted estimates from a regression analysis. • Outliers resulting from an extraordinary event with no explanation. Model-based procedures incorporate missing data into the analysis. two or more methods of imputation are used to derive a composite estimate for the missing value. These outliers result from data entry errors or mistakes in coding. Problematic outliers — when they are not representative of the population and are counter to the objectives of the analysis. They should be identified and eliminated during data cleaning. they should be kept in the data set. ) • Bivariate detection: examine scatterplots of variable pairs and select as outliers those values which fall markedly outside the range of the other observations. (For large sample sizes.5 may be potential outliers. 3. the value may increase to 3 or 4.e. Influence plot. While these values cannot be distinguished individually. • Univariate detection: examine the distribution of observations for a variable and select as outliers those values which fall at the outer ranges of the distribution. • Multivariate detection: assess each observation across a set of variables and select as outliers those values which fall outside a specified range specific to the statistical test employed. where the point varies in size in proportion to its influence on the relationship and the largest points are potential outliers. Identification can be made from any of three perspectives: univariate. o Conservative values (i. o When standardized. In these cases. . If possible.• Ordinary values which become unique when combined with other variables. or multivariate.001) for the statistical tests should be set for identification of potential outliers. o Mahalanobis D2 is commonly used in multivariate analyses to identify outliers. o 29 . It is a measure of the distance in multidimensional space of each observation from the mean center of the observations. data values which are greater than 2. Those values falling outside the range are potential outliers. multiple perspectives should be utilized to triangulate the identification of outliers. bivariate. they become very noticeable when combined with other values across other variables. the observations should be retained unless specific evidence to the contrary is found. o Ellipse representing a confidence interval may be drawn around the expected range of observations on the scatterplot. Multivariate techniques may also be employed to trace the underlying causes of outliers. Multivariate analyses require that the assumptions underlying the statistical techniques be tested twice: once for the individual variables and once for the multivariate model. • Unnecessary deletion of outliers will limit the generalizability of the analysis.the data points should closely follow the diagonal line. Stage 4: Testing the Assumptions of Multivariate Analysis 1. 2. • • • Most fundamental assumption in multivariate analyses.the distribution should approximate a bell-shaped curve.4. Only observations which are truly unique from the population should be designated as outliers. The researcher should be careful of identifying too many observations as outliers. o Normal probability plot -. Visual checks: The simplest way to evaluate the normality of a variable is to visually check a histogram or a normal probability plot. Normality: Each variable in the analysis must be normally distributed. o Histogram -. Sufficient non-normality invalidates statistical tests which use the F and t statistics. The assumptions for the variate for each technique will be discussed in the appropriate chapter. The following discussion relates only to assumptions underlying the individual variables. • Profiles on each outlier should be generated and the data should be examined for the variable(s) responsible for generating the outlier. The researcher should be able to classify the outlier in one of the four categories discussed above. 30 . Outliers should be deleted from the analysis only if they are proven to be not representative of the population. • Statistical tests: The two most common are the Shapiro-Wilks and Kolmogorov-Smirnov tests. a new variable which represents the nonlinear relationship can be created. Homoscedasticity: dependent variables should exhibit equal levels of variance across the range of predictor variables. The most common test is the Levene test. o Statistical tests for equal variance dispersion relate to the variances within groups formed by nonmetric variables. Identification: graphical versus statistical.• • Transformations: When a distribution is found to be non-normal. o Graphical plot of residuals will reveal violations of this assumption. data transformations should be computed. • • 4. Impact: Violation of this assumption will cause hypothesis tests to be either too conservative or too sensitive. Linearity: variables should be linearly related. • 31 . • Common sources: Most problems with unequal variances stem from either the type of variables included in the model or from a skewed distribution. • Remedies: Heteroscedastic variables can be remedied through data transformations. Other than a transformation. • Identification: Scatterplots of variable pairs are most commonly used to identify departures from linearity. 3. which is used to assess if the variances of a single metric variable are equal across any number of groups.58 are indicative of a nonnormal distribution. Examination of the residuals in a simple regression analysis may also be used as a diagnostic method. Skewness: Skewness values exceeding +2. Other statistical tests are available in specific statistical software programs. the most direct approach is to transform one or both of the variables. When more than one variable is being tested. the Box's M test should be used. Nonlinearity: If a nonlinear relationship is detected. • Residual plots should not contain any recognizable pattern. o Transformations should be applied to the independent variables except in the case of heteroscedasticity. • Patterns in the error terms reflect an underlying systematic bias in the relationship. • • Basis: Transformations can be based on theoretical or empirical reasons.0. o Cone-shaped distribution which opens to the right should be transformed using an inverse. Prediction errors should not be correlated. and linearity and to improve the relationships between variables. Data transformations enable the researcher to modify variables to correct violations of the assumptions of normality. o Positively skewed distributions transformed by taking logarithms o Negatively skewed distributions transformed by taking the square root. • Violations of this assumption often result from problems in the data collection process. o Select the variable with the smallest ratio from item 1. A cone shaped distribution which opens to the left should be transformed by taking the square root. 6. o Ratio of a variable's mean divided by its standard deviation should be less than 4. • General guidelines for performing data transformations.5. 32 . homoscedasticity. including squaring the variable and adding additional variables termed polynomials. o Flat distribution is the most common transformation is the inverse. Distribution shape: The shape of the distribution provides the basis for selecting the appropriate transformation. o Nonlinear transformations can take many forms. o Transformations may change the interpretation of the variables. Effects coding assign a value of -1 to the comparison group while still using 1 to designate the other group. • • Indicator coding assigns a value of 1 to one group. and zero to the comparison group (males). There are two general methods of accomplishing this task. Incorporating Nonmetric Data with Dummy Variables When faced with nonmetric variables in the data the researcher may wish to represent these categorical variables as metric through the use of dummy variables. Any nonmetric variable with k groups may be represented as k .o Heteroscedasticity can only be remedied by transformation of the dependent variable in a dependence relationship.1 dummy variables. 33 . for instance females. If a heteroscedasticity relationship is also nonlinear. the dependent and perhaps the independent variables must be transformed. Respondent: 1) Misunderstanding of the question 2) Response bias. These extreme responses must be evaluated as to the type of influence exerted and dealt with accordingly. Problematic outliers may be the result of data input errors. a respondent's misunderstanding of the question.ANSWERS TO END-OF-CHAPTER QUESTIONS (1) LIST POTENTIAL UNDERLYING CAUSES OF OUTLIERS. if only one respondent from a lower income group is included in the sample and that respondent expresses an attitude atypical to the remainder of the sample. An extraordinary observation with no explanation. d. Answer a. 34 . or response bias. An ordinary value which is unique when combined with other variables. Beneficial outliers are indicative of some characteristic of the population which would not have been otherwise known. Researcher: 1) Data entry errors 2) Data coding mistakes c. (2) DISCUSS WHY OUTLIERS MIGHT BE CLASSIFIED AS BENEFICIAL AND AS PROBLEMATIC. b. BE SURE TO INCLUDE ATTRIBUTIONS TO BOTH THE RESPONDENT AND THE RESEARCHER. this respondent would be considered beneficial. such as yea-saying 3) Extraordinary experience b. Answer a. For example. Problematic outliers are not indicative of the population and distort multivariate analyses. the analyst may wish to consider deletion of cases with a great degree of missing data. the missing data are at random. the analyst should only use a modeling-based approach which accounts for the underlying processes of the missing data. Cases with missing data are good candidates for deletion if they represent a small subset of the sample and if their absence does not otherwise distort the data set. the choice of case deletion versus imputation method should be based on theoretical and empirical considerations. If the data are missing completely at random. When the missing data are missing at random (MAR). This occurs when X biases the randomness of the observed Y values. EXPLAIN HOW EACH TYPE WILL IMPACT THE ANALYSIS OF MISSING DATA. 35 . If the sample size is sufficiently large. Answer The researcher must first evaluate the randomness of the missing data process. c. Data which are missing at random cannot employ an imputation method. Answer a. such that the observed Y values do not represent a true random sample of all actual Y values in the population. b. or employing an imputation method. such as using only observations with complete data. as it would introduce bias into the results. Missing at Random (MAR): If the missing values of Y depend on X. the analyst may use any of the suggested approaches for dealing with missing data. When the missing data are missing completely at random (MCAR). (4) DESCRIBE THE CONDITIONS UNDER WHICH A RESEARCHER WOULD DELETE A CASE WITH MISSING DATA VERSUS THE CONDITIONS UNDER WHICH A RESEARCHER WOULD USE AN IMPUTATION METHOD. deleting case(s) or variable(s). Missing Completely at Random (MCAR): When the observed values of Y are truly a random sample of all Y values. deleting a case is the only acceptable alternative of the two. but not on Y. If the data are missing at random. Only cases with data which are missing completely at random would utilize an imputation method.(3) DISTINGUISH BETWEEN DATA WHICH ARE MISSING AT RANDOM (MAR) AND MISSING COMPLETELY AT RANDOM (MCAR). HOMOSCEDASTICITY. The degree of missing data will influence the researcher's choice of information used in the imputation (i. whereas multiple discriminant analysis is primarily sensitive to violations of multivariate normality. case substitution. or multiple imputation). mean substitution. it is not the only consideration. however. (6) DISCUSS THE FOLLOWING STATEMENT. each multivariate technique has a set of underlying assumptions which must be met. 36 . For example." Answer As will be shown in each of the following chapter outlines. complete case vs. (5) EVALUATE THE FOLLOWING STATEMENT. AS LONG AS THE SAMPLE SIZE IS ADEQUATE. cases with missing dependent variable values are often deleted. consider the amount of missing data when selecting this option.For instance. Although sample size is an important consideration in multivariate analyses. LINEARITY. regression imputation. AND INDEPENDENCE. If the sample size is small. IT IS NOT NECESSARY TO MEET ALL OF THE ASSUMPTIONS OF NORMALITY. The analyst should." Answer False.e.e. the analyst may wish to use an imputation method to fill in missing data. all-available approaches) and the researcher's choice of imputation method (i. multiple regression analysis is sensitive to violations of all four of the assumptions. cold deck imputation. "IN ORDER TO RUN MOST MULTIVARIATE ANALYSES. "MULTIVARIATE ANALYSES CAN BE RUN ON ANY DATA SET. Analysts must also consider the degree of missing data present in the data set and examine the variables for violations of the assumptions of the intended techniques. The degree to which a violation of any of the four above assumptions will distort data analyses is dependent on the specific multivariate technique. • When do you use factor analysis? Factor analysis is used when: • the analyst wishes to examine the underlying structure of a data matrix. but an interdependence technique in which all variables are considered simultaneously. thereby reducing the size of the data set to equal the number of factors extracted. Factor analysis can meet the objectives of: • Data reduction: factor analysis calculates scores. Why do we use factor analysis? 1." "Why?. number of the original variables. What is factor analysis? 1. but representative." and "How?" as applied to Factor Analysis. . which represent a variable's interrelationship with the other variables in the data set.CHAPTER THREE FACTOR ANALYSIS The following presentation will address the basic questions of "What?. Summarization: factor analysis derives the underlying dimensionality or structure of a data set and allows the researcher to describe the data with a fewer. Factor analysis is a summarization and data reduction technique that does not have independent and dependent variables. factor analysis is the appropriate analysis technique. Factor analysis examines the interrelationships among a large number of variables and then attempts to explain them in terms of their common underlying dimensions. and substitutes these scores for the original variables. or • when analysts wish to understand the structure of the interrelationships among the variables in a data set. 2. 3." "When?. These common underlying dimensions are referred to as factors. • • • Identification of the structure of relationships among either variables or respondents. The following are important points for each step of the process. five or more).How do you use factor analysis? Factor analysis follows the six-stage model-building process introduced in Chapter 1. but include a sufficient number of variables to represent each proposed factor (i. 3. Identification of representative variables from a much larger set of variables for use in subsequent multivariate analyses. There are two approaches to calculate the correlation matrix that determine the type of factor analysis performed: • R-type factor analysis: input data matrix is computed from correlations between variables. Stage 1: Objectives of Factor Analysis 1.e. 2. . which are much smaller in number in order to partially or completely replace the original set of variables for inclusion in subsequent multivariate techniques. The researcher should minimize the number of variables included in the analysis. • Q-type factor analysis: input data matrix is computed from correlations between individual respondents. Creation of an entirely new set of variables. Stage 2: Designing a Factor Analysis 1. Variables in factor analysis are generally metric. Factor Analysis has three primary objectives. Dummy variables may be used in special circumstances. Thus. . Stage 3: Assumptions in Factor Analysis 1. although dummy variables may be used (coded 0-1). The most basic assumption is that the set of variables analyzed are related. Sample size is an important consideration in factor analyses. o Example: if you had 20 unrelated variables. • The ratio of observations to variables should be at least 5 to 1 in order to provide the most stable results. • Factor analysis does not require multivariate normality. you would have 20 different factors. Factor analysis assumes the use of metric data. • Variables must be interrelated in some way since factor analysis seeks the underlying common dimensions among the variables. • The sample size should be 100 or larger. If the variables are not related. 2. factor analysis has no common dimensions with which to create factors. then each variable will be its own factor.4. • Metric variables are assumed. • Multivariate normality is necessary if the researcher wishes to apply statistical tests for significance of factors. some underlying structure or relationship among the variables must exist. Sample sizes between 50 and 100 may be analyzed but with extreme caution. • The sample should be homogenous with respect to some underlying factor structure. When the variables are unrelated. • Principal components factor analysis inserts 1's on the diagonal of the correlation matrix. The data matrix must have sufficient correlations to justify the use of factor analysis. the Bartlett test of sphericity. Determining the number of factors to extract should consider several criteria: • Latent Root criterion: specifies a threshold value for evaluating the eigenvalues of the derived factors. o only factors with an eigenvalue greater than 1 are significant and will be extracted.3. • • Rule of thumb: a substantial number of correlations greater than . o Most appropriate when there is a desire to reveal latent dimensions of the original variables and the researcher does not know about the nature of specific and error variance. • Common factor analysis only uses the common variance and places communality estimates on the diagonal of the correlation matrix. so an extracted factor should account for the variance of at least a single variable.50). .30 are needed. and the measure of sampling adequacy (MSA greater than . o Most appropriate when the concern is with deriving a minimum number of factors to explain a maximum portion of variance in the original variables. Stage 4: Deriving Factors and Assessing Overall Fit 1. thus considering all of the available variance. This is due to the notion that an individual variable contributes a value of 1 to the total eigenvalue. 2. and the researcher knows the specific and error variances are small. Two extraction methods are: principal components and common factor analysis. Tests of appropriateness: anti-image correlation matrix of partial correlations. • Scree Test criterion A scree test is derived by plotting the eigenvalues of each factor relative to the number of factors in order of extraction. o At some point. the cumulative percent of variance explained is used to determine the number of factors. . the first factors extracted are those which are more homogeneous across the entire sample. The researcher may desire to specify a necessary percentage of variance explained by the solution. o Earlier extracted factors have the most common variance. but once specific variance becomes too large the plot line will become horizontal. o While the factors contain mostly common variances the plot line will continue to decline sharply. • Percentage of Variance criterion As factor analysis extracts each factor. the amount of specific variance begins to overtake the common variance in the factors. o Point where the line becomes horizontal is the appropriate number of factors. causing rapid decline in amount of variance explained as additional factors extracted. A scree plot reveals this by a rapid flattening of the plot line. This criterion is useful if the researcher is testing previous research or specific hypotheses. o Scree test almost always suggests more factors than the latent root criterion.• A Priori criterion The researcher may know how many factors should be extracted. • Heterogeneity of the Respondents In a heterogeneous sample. Those factors which best discriminate among subgroups in the sample will be extracted later in the analysis. It will not change the amount of variance extracted or the number of factors extracted.30 — minimum consideration level. o QUARTIMAX attempts to simplify the rows of the matrix so that variables load highly on a single factor. This tends to create a large general factor. 2. There are two general types of rotation. o + .Stage 5: Interpreting the Factors 1.40 — more important o + . o EQUIMAX is a compromise between VARIMAX QUARTIMAX in that it simplifies both the rows and columns. Orthogonal rotation maintains the axes at 90 degrees thus the factors are uncorrelated. Interpretation is assisted through selection of a rotational method. • Rotation redistributes the variance from earlier factors to later factors by turning the axes of the factors about the origin until a new position is reached. Rotation is used as an aid in explaining the factors by providing a more simple structure in the factor matrix.50 — practically significant (the factor accounts for 25% of variance in the variable). o VARIMAX simplifies the columns of the factor matrix indicating a clearer association and separation among variables and factors. . • and • Oblique rotation methods such as OBLIMIN (SPSS) and PROMAX (SAS) allow correlated factors. orthogonal and oblique. There are three orthogonal approaches that operate on different aspects of the factor matrix. Criteria for Practical and Statistical Significance of Factor Loadings • Magnitude for practical significance: Factor loadings can be classified based on their magnitude: o Greater than + .  Orthogonal rotation — each variable's loading on each factor is independent of its loading on another factor. • Variables that load across factors or that have low loadings or communalities may be candidates for deletion.55 and above are significant. o Increases in the sample size.• Power and statistical significance: Given the sample size. decreases the level necessary to consider a loading significant. The variables' magnitude and strength provide meaning to the factors. o Impact of the Rotation: The selection of a rotation method affects the interpretation of the loadings. For example. Oblique rotation — independence of the loadings is not preserved and interpretation then becomes more complex. the researcher may determine the level of factor loadings necessary to be significant at a predetermined level of power. in a sample of 100 at an 80% power level. 3. o Increases in the number of factors extracted. increases the level necessary to consider a loading significant. factor loadings of . decreases the level for significance. Interpreting a Factor Matrix: • Look for clear factor structure indicated by significant loadings on a single factor and high communalities. • Naming the factor is based on an interpretation of the factor loadings. o Significant loadings: The variables that most significantly load on each factor should be used in naming the factors. • Necessary loading level to be significant varies due to several factors: o Increases in the number of variables.  . Analyses can be run using a split sample or another new data set. 1. Confirmatory factor analysis is the most commonly used replication technique. the researcher may wish to use the factor analysis results in subsequent analysis. Some methods are: • Deletion of a variable(s) from the analysis • Employing a different rotational method for interpretation • Extraction of a different number of factors Stage 6: Validating the Results • Employing a different extraction method 1. The impact of outliers should be determined by running the factor model with and without the influential observations. Validation assesses 1) the degree of generalizability of the findings and 2) the degree to which the results are influenced by individual cases. . Stability is highly dependent on sample size and the number of observations per variable. or (3) replacement of the factor with a factor score. (2) creation of a new variable with a summated scale. Factor analysis may be used to reduce the data for further use by (1) the selection of a surrogate variable. • Results should be replicable. The factor structure should be stable across additional analyses. A surrogate variable that is representative of the factor may be selected as the variable with the highest loading. Respecification should always be considered. • • 0 Stage 7: Additional Uses of the Factor Analysis Results Beyond the interpretation and understanding of the relationship among the variables.4. All the variables loading highly on a factor may be combined (the sum or the average) to form a replacement variable. and reliability are established other forms of scale validity should be assessed. . may also be used as a composite replacement for the original variable. Nomological validity refers to the degree that the scale makes accurate predictions of other concepts. Another form of reliability is the internal consistency of the items in a scale. Discriminant validity is the extent that two measures of similar but different concepts are distinct. • Basic Issues of Scale Construction: o A conceptual definition is the starting point for creating a scale. computed using all variables loading on a factor. • Factor scores may not be easy to replicate. Measures of internal consistency include item-to-total correlation. • Advantages of the summated scale: o Measurement Error is reduced by multiple measures. meaning that all items are strongly associated with each other and represent a single concept. 3. Reliability is the degree of consistency between multiple measurements of a variable. The scale must appropriately measure what it purports to measure to assure content or face validity. Factor scores. • Factor scores are computed using all variables that load on a factor. and the reliability coefficient. inter-item correlation.2. o Reliability of the scale is essential. Testretest reliability is one form of reliability. o Once content or face validity. o Taps all aspects or domains of a concept with highly related multiple indicators. o A scale must be unidimensional. unidimensionality. ANSWERS TO END-OF-CHAPTER QUESTIONS (1) WHAT ARE THE DIFFERENCES BETWEEN THE OBJECTIVES OF DATA SUMMARIZATION AND DATA REDUCTION? Answer The basic difference between the objectives of data summarization and data reduction depends upon the ultimate research question. In data summarization the ultimate research question may be to better understand the interrelationship among the variables. This may be accomplished by condensing a large number of respondents into a smaller number of distinctly different groups with Q-type factor analysis. More often data summarization is applied to variables in R-type factor analysis to identify the dimensions that are latent within a dataset. Data summarization makes the identification and understanding of these underlying dimensions or factors the ultimate research question. Data reduction relies on the identification of the dimensions as well, but makes use of the discovery of the items that comprise the dimensions to reduce the data to fewer variables that represent the latent dimensions. This is accomplished by either the use of surrogate variables, summated scales, or factor scores. Once the data has been reduced to the fewer number of variables further analysis may become easier to perform and interpret. (2) HOW CAN FACTOR ANALYSIS HELP THE RESEARCHER IMPROVE THE RESULTS OF OTHER MULTIVARIATE TECHNIQUES? Answer Factor analysis provides direct insight into the interrelationships among variables or respondents through its data summarizing perspective. This gives the researcher a clear picture of which variables are highly correlated and will act in concert in other analysis. The summarization may also lead to a better understanding of the latent dimensions underlying a research question that is ultimately being answered with another technique. From a data reduction perspective, the factor analysis results allow the formation of surrogate or summated variables to represent the original variables in a way that avoids problems associated with highly correlated variables. In addition, the proper usage of scales can enrich the research process by allowing the measurement and analysis of concepts that require more than single item measures. (3) WHAT GUIDELINES CAN YOU USE TO DETERMINE THE NUMBER OF FACTORS TO EXTRACT? EXPLAIN EACH BRIEFLY. Answer The appropriate guidelines utilized depend to some extent upon the research question and what is known about the number of factors that should be present in the data. If the researcher knows the number of factors that should be present, then the number to extract may be specified in the beginning of the analysis by the a priori criterion. If the research question is largely to explain a minimum amount of variance then the percentage of variance criterion may be most important. When the objective of the research is to determine the number of latent factors underlying a set of variables a combination of criterion, possibly including the a priori and percentage of variance criterion, may be used in selecting the final number of factors. The latent root criterion is the most commonly used technique. This technique is to extract the number of factors having eigenvalues greater than 1. The rationale being that a factor should explain at least as much variance as a single variable. A related technique is the scree test criterion. To develop this test the latent roots (eigenvalues) are plotted against the number of factors in their order of extraction. The resulting plot shows an elbow in the sloped line where the unique variance begins to dominate common variance. The scree test criterion usually indicates more factors than the latent root rule. One of these four criterion for the initial number of factors to be extracted should be specified. Then an initial solution and several trial solutions are calculated. These solutions are rotated and the factor structure is examined for meaning. The factor structure that best represents the data and explains an acceptable amount of variance is retained as the final solution. (4) HOW DO YOU USE THE FACTOR-LOADING MATRIX TO INTERPRET THE MEANING OF FACTORS? Answer The first step in interpreting the factor-loading matrix is to identify the largest significant loading of each variable on a factor. This is done by moving horizontally across the factor matrix and underlining the highest significant loading for each variable. Once completed for each variable the researcher continues to look for other significant loadings. If there is simple structure, only single significant loadings for each variable, then the factors are labeled. Variables with high factor loadings are considered more important than variables with lower factor loadings in the interpretation phase. In (5) general, factor names will be assigned in such a way as to express the variables which load most significantly on the factor. HOW AND WHEN SHOULD YOU USE FACTOR SCORES IN CONJUNCTION WITH OTHER MULTIVARIATE STATISTICAL TECHNIQUES? Answer When the analyst is interested in creating an entirely new set of a smaller number of composite variables to replace either in part or completely the original set of variables, then the analyst would compute factor scores for use as such composite variables. Factor scores are composite measures for each factor representing each subject. The original raw data measurements and the factor analysis results are utilized to compute factor scores for each individual. Factor scores may replicate as easily as a summated scale, therefore this must be considered in their use. (6) WHAT ARE THE DIFFERENCES BETWEEN FACTOR SCORES AND SUMMATED SCALES? WHEN ARE EACH MOST APPROPRIATE? Answer The key difference between the two is that the factor score is computed based on the factor loadings of all variables loading on a factor, whereas the summated scale is calculated by combining only selected variables. Thus, the factor score is characterized by not only the variables that load highly on a factor, but also those that have lower loadings. The summated scale represents only those variables that load highly on the factor. Although both summated scales and factor scores are composite measures there are differences that lead to certain advantages and disadvantages for each method. Factor scores have the advantage of representing a composite of all variables loading on a factor. This is also a disadvantage in that it makes interpretation and replication more difficult. Also, factor scores can retain orthogonality whereas summated scales may not remain orthogonal. The key advantage of summated scales is, that by including only those variables that load highly on a factor, the use of summated scales makes interpretation and replication easier. Therefore, the decision rule would be that if data are used only in the original sample or orthogonality must be maintained, factor scores are suitable. If generalizability or transferability is desired then summated scales are preferred. (7) WHAT IS THE DIFFERENCE BETWEEN Q-TYPE FACTOR ANALYSIS AND CLUSTER ANALYSIS? Answer Both Q-Type factor analysis and cluster analysis compare a series of responses to a number of variables and place the respondents into several groups. The difference is that the resulting groups for a Q-type factor analysis would be based on the intercorrelations between the means and standard deviations of the respondents. In a typical cluster analysis approach, groupings would be based on a distance measure between the respondents' scores on the variables being analyzed. (8) WHEN WOULD THE RESEARCHER USE AN OBLIQUE ROTATION INSTEAD OF AN ORTHOGONAL ROTATION? WHAT ARE THE BASIC DIFFERENCES BETWEEN THEM? Answer In an orthogonal factor rotation, the correlation between the factor axes is arbitrarily set at zero and the factors are assumed to be independent. This simplifies the mathematical procedures. In oblique factor rotation, the angles between axes are allowed to seek their own values, which depend on the density of variable clusterings. Thus, oblique rotation is more flexible and more realistic (it allows for correlation of underlying dimensions) than orthogonal rotation although it is more demanding mathematically. In fact, there is yet no consensus on a best technique for oblique rotation. When the objective is to utilize the factor results in a subsequent statistical analysis, the analyst may wish to select an orthogonal rotation procedure. This is because the factors are orthogonal (independent) and therefore eliminate collinearity. However, if the analyst is simply interested in obtaining theoretically meaningful constructs or dimensions, the oblique factor rotation may be more desirable because it is theoretically and empirically more realistic. Multiple regression analysis has two possible objectives: • prediction — attempts to predict a change in the dependent variable resulting from changes in multiple independent variables. Multiple regression may be used any time the researcher has theoretical or conceptual justification for predicting or explaining the dependent variable with the set of independent variables. • explanation — enables the researcher to explain the variate by assessing the relative contribution of each independent variable to the regression equation." "Why?. 2. When do you use multiple regression analysis? 1. Multiple regression analysis is the technique of choice when the research objective is to predict a statistical relationship or to explain underlying relationships among variables. 3. 2.CHAPTER FOUR MULTIPLE REGRESSION ANALYSIS The following presentation will address the basic questions of "What?. with applications across all types of problems and all disciplines. Common applications include models of: * business forecasting * consumer decision-making or preferences * new products * firm performance * consumer attitudes * quality control . Why do we use multiple regression analysis? 1." and "How?" as applied to Multiple Regression Analysis. 2. Multiple regression analysis is applicable in almost any business decisionmaking context. Multiple regression analysis is the most widely-used dependence technique. What is multiple regression analysis? 1. This is accomplished by a statistical procedure called ordinary least squares which minimizes the sum of squared prediction errors (residuals) in the equation. We also use multiple regression analysis because it enables the researcher to utilize two or more metric independent variables in order to estimate the dependent variable." "When?. An exact estimate is made. points for each step of the process. not functional relationships. with no error. (i. Multiple regression analysis is appropriate for statistical relationships.e. As such. • Statistical relationships assume that more than one value of the dependent value will be observed for any value of the independent variables. An average value is estimated and error is expected in prediction. correlations) 2. two objectives are associated with prediction: o Maximization of the overall predictive power of the independent variables in the variate. Functional relationships assume that a single value of the dependent value will be observed for any value of the independent variables. (i. • . o Comparison of competing models made up of two or more sets of independent variables to assess the predictive power of each. linearity) o Insight into the interrelationships among the independent variables and the dependent variable. The following are important 1.e. Multiple Stage 1: Objectives of Multiple Regression regression appropriate for two types of research problems: explanation.How do you use multiple regression analysis? Multiple regression analyses follow six stages. As such. o Assessment of the nature of the relationships between the predictors and the dependent variable. three objectives are associated with explanation: o Determination of the relative importance of each independent variable in the prediction of the dependent variable. • Explanation — explain the degree and character of the relationship between dependent and independent variables. • analysis is prediction and Prediction — predict a dependent variable with a set of independent variables. • Selecting the dependent variable: dictated by the research problem. but has no theoretical or managerial relationship with the dependent variable is of no use to the researcher in explaining the phenomena under observation. o Minimum level is 5 to 1 (i. o Small samples (less than 20 observations). Stage 2: Research Design of a Multiple Regression Analysis 1. The researcher must seek parsimony in the regression model.3.e. with concern for measurement error. • The power (probability of detecting statistically significant relationships) at specified significance levels is related to sample size. 4. . will detect only very strong relationships with any degree of certainty. The sample size used will impact the statistical power and generalizability of the multiple regression analysis. • Selecting the independent variables: The inclusion of an independent variable must be guided by the theoretical foundation of the regression model and its managerial implications. o Large samples (1000 or more observations) will find almost any relationship statistically significant due to the over sensitivity of the test. • The generalizability of the results is directly affected by the ratio of observations to independent variables. The fewest independent variables with the greatest contribution to the variance explained should be selected. A variable that by chance happens to influence statistical significance. Researchers must be concerned with specification error. The selection of dependent and independent variables for multiple regression analysis should be based primarily on theoretical or conceptual meaning. or whether the variable is an accurate and consistent measure of the concept being studied. 5 observations per independent variable in the variate). or the inclusion of irrelevant variables or the omission of relevant variables. o moderators change the interpretation of the regression coefficients. Most regression models for survey data are random effects models. square root or logarithm) and polynomials are most often used to represent nonlinear relationships. In a random effects model. . When a nonlinear relationship exists between the dependent and the independent variables or when the analyst wishes to include nonmetric independent variables in the regression model. 3. 2.e. the levels of the predictor are selected at random and a portion of the random error comes from the sampling of the predictors. To determine the total effect of an independent variable. • Moderator effects: o reflect the changing nature of one independent variable's relationship with the dependent variable as a function of another independent variable. the separate and the moderated effects must be combined. o represented as a compound variable in the regression equation. transformations of the data should be computed. o The resulting coefficients represent the differences in group means from the comparison group and are in the same units as the dependent variable.o Desired level is 15 to 20 observations for each independent variable. also known as dummy variables may be used to replace nonmetric independent variables. • Nonlinear relationships: o Arithmetic transformations (i. • Nonmetric variable inclusion: o Dichotomous variables. . If a curvilinear relationship is anticipated between the criterion variable and one or more of the predictor variables. • Implied relationship: Regression attempts to predict one variable from a group of other variables. the resulting relationship will be one of averages. 2. Thus. 1. we have a condition referred to as heteroscedasticity. Homoscedasticity — constant variance of the dependent variable. When the distributions are not equal. In special cases. the researcher must feel that some relationship will be found between the single criterion variable and the predictor group of variables. the anticipated relationship should be linear. there can be no prediction. • If variances are unequal. the dependent variable may be dichotomous. When no relationship between the criterion variable and predictor variables exists. such as logistic regression. a number of data transformations may be used to regain a linear relationship. the dependent variable should have a constant variance. • Assumption is that the variance of the dependent variable values will be equal. Therefore. • The assumption of a statistical relationship means that the dependent variable will have a number of different values at each level of the independent variables. • Linearity: In order to justify the use of multiple regression analysis. they may be corrected through data transformations.Stage 3: Assumptions in Multiple Regression Analysis Multiple regression analysis assumes the use of metric independent and dependent variables. for each level of the independent variable. or a statistical relationship. A statistical linear relationship must exist between the independent and dependent variables. • Statistical relationship: Since the single criterion variable will have a number of different values over the range of values for the predictor variables. • Tolerance value or the variance inflation factor (VIF) can be used to detect multicollinearity. Multicollinearity — multiple regression works best with no collinearity among the independent variables. Normality — dependent and independent variables are normally distributed.3. there are data transformations that may restore normality and allow the use of the variable(s) in the regression equation. 4. A normal distribution is necessary for the use of the F and t statistics. since sufficiently large violations of this assumption invalidate the use of these statistics. • Once detected. • The data distribution for all variables is assumed to be the normal distribution. This correlation among the predictor variables prohibits assessment of the contribution of each independent variable. Diagnostics for normality are: o histograms of the residuals (bell-shaped curve) o normal probability plots (diagonal line) o skewness  Shapiro-Wilks test  Kolmogorov-Smirnov test • If violations of normality are found. the analyst may choose one of four options: o omit the correlated variables. • The researcher should expect the same chance of random error at each level of prediction. . Uncorrelated error terms. • 5. • Any errors encountered in multiple regression analysis are expected to be completely random in nature and not systematically related. • The presence of collinearity or multicollinearity suppresses the R2 and confounds the analysis of the variate. the analyst has total control over variable selection. This is repeated until the F-to-enter test finds no variables to enter. The same repetition of estimation is performed as with forward estimation. backward. Model selection — accomplished by any of several methods available to aid the researcher in selecting or estimating the best regression model.o use the model for prediction only. • Confirmatory specification o The analyst specifies the complete set of independent variables. Backward elimination begins with all variables in the regression equation and then eliminates any variables with the F-to-remove test. or o use a different method such as Bayesian Stage 4: Estimating the Regression Model and Assessing Overall Fit regression. Thus. o assess the predictor-dependent variable relationship with simple correlations. Variable entry may be done in a forward. • Sequential search approaches o Sequential approaches estimate a regression equation with a set of variables and by either adding or deleting variables until some overall criterion measure is achieved.  Forward method begins with no variables in the equation and then adds variables that satisfy the F-to-enter test. 1.  . Then the equation is estimated again and the F-to-enter of the remaining variables is calculated. or stepwise manner. 3. A value of one (1) means perfect explanation and is not encountered in reality due to ever present error. constant variance. 2. one would have to estimate 1024 regression equations. At each re-estimation stage. . Assessment of the regression model fit is in two parts: examining overall fit and analyzing the variate. o Examine the variate's ability to predict the criterion variable and assess how well the independent variables predict the dependent variable. for even 10 independent variables. This repetition continues until both F tests are not satisfied by any of the variables either in or out of the regression equation. • Examine the overall model fit. The most common procedure is known as all-possible-subsets regression. A value of . however. the variables already in the equation are also examined for removal by the appropriate F test. For example. Stepwise estimation is a combination of forward and backward methods. • Combinatorial Methods o The combinatorial approach estimates regression equations for all subset combinations of the independent variables. The equation is estimated again and additional variables that satisfy the F test are entered. independence and normality along with the individual variables in the analysis.91 means that 91% of the variance in the dependent variable is explained by the independent variables. o Combinatorial methods become impractical for very large sets of independent variables. o Several statistics exist for the evaluation of overall model fit  Coefficient of determination (R2) • The coefficient of determination is a measure of the amount of variance in the dependent variable explained by the independent variable(s). It begins with no variables in the equation as with forward estimation and then adds variables that satisfy the F test. The variate must meet the assumptions of linearity. R2 should be greater than zero. o the number of independent variables included in the analysis. This test indicates whether a significant amount (significantly different from zero) of variance was explained by the model. As you increase the number of independent variables in the model. o The variate is the linear combination of independent variables used to predict the dependent variables. R2 is impacted by two facets of the data: o the number of independent variables relative to the sample size. It represents an estimate of the standard deviation of the actual dependent values around the regression line. (see sample size discussion earlier) For this reason.  Standard error of the estimate • Standard error of the estimate is another measure of the accuracy of our predictions. which adjusts for inflation in R2 from overfitting the data. • Analyze the variate. analysts should use the adjusted coefficient of determination. Analysis of the variate relates the respective contribution of each independent variable in . Since this is a measure of variation about the regression line.• • The amount of variation explained by the regression model should be more than the variation explained by the average. the better. •  F-test • The F-test reported with the R2 is a significance test of the R2. the smaller the standard error. Thus. you increase the R2 automatically because the sum of squared errors by regression begins to approach the sum of squared errors about the average. o Regression coefficients are tested for statistical significance.   4. which is explainable by an extraordinary situation. leverage points. The objective of this analysis is to determine the extent and type of effect. o a valid but exceptional observation. The estimated coefficients should be tested to ensure that across all possible samples. • An F-test may be used to test the appropriateness of the intercept and the regression coefficients. o an exceptional observation with no likely explanation. The researcher should examine the data for influential observations. it cannot be used for predictive purposes. If the constant is not significantly different from zero. One of four conditions gives rise to influential observations: o an error in observation or data entry. The size of the sample will impact the stability of the regression coefficients. or o an ordinary observation on its individual characteristics. but exceptional in its combination of characteristics.the variate to the regression model.  The researcher is informed as to which independent variable contributes the most to the variance explained and may make relative judgments between/among independent variables (using standardized coefficients only). and outliers all have an effect on the regression results. the coefficient would be different from zero.  The intercept (or constant term) should be tested for appropriateness for the predictive model. • Detection of problem cases is accomplished with: o examine residuals  studentized residuals > 2 (+ or -) • . the more generalizable the estimated coefficients will be. The larger the sample size. • Influential observations. Stage 5: Interpreting the Regression Variate o With justifiable reasoning. o For prediction. computed on the same unit of measurement) in order to be able to directly compare the contribution of each independent variable to explanation of the dependent variable. use the hat matrix. o Beta coefficients (the term for standardized regression coefficients) enable the researcher to examine the relative strength of each variable in the equation. the researcher evaluates the estimated regression coefficients for their explanation of the dependent variable and evaluates the potential impact of omitted variables.  dummy variable regression (n + 1 df) partial regression plots o identify leverage points  with two predictor variables. the observation(s) may be discarded and the regression equation estimated again. • Partial correlations o Partial correlation: correlation between each independent variable and the dependent variable with the effects of the other independent variables removed. Cook's distance. . plot the two variables as the axes of a two-dimensional plot  with three or more variables. Leverage values: p>10 and n>50 then use 2p/n. such as DFBETA. If deletion is not warranted. regression coefficients are not standardized and. p<10 or n<50 then use 3p/n o examine single case diagnostics. • Standardized regression coefficients (beta coefficients) o Regression coefficients must be standardized (i. are in their original units of measurement. and DFFIT. 1.e. therefore. then more "robust" estimation techniques must be used. When interpreting the regression variate. 1.  o T-values indicate the significance of the partial correlation of each variable. • Tolerance is a measure of collinearity between two independent variables or multicollinearity among three or more independent variables. o VIF values of just over 1. The researcher must ensure that the regression model represents the general population (generalizability) and is appropriate for the applications in which it will be employed (transferability). are used to assess the degree of collinearity among independent variables.5 indicates a collinearity or multicollinearity problem. It is the proportion of variance in one independent variable that is not explained by the remaining independent variables. This may be compared against the researcher's a priori standard for significance. This correlation provides the researcher with a more pure correlation between the dependent and independent variables. 2. 'Partial Correlation': removes their effect from both sides of the regression equation.0. A VIF value of 1.0 are desirable.007 would be considered very good and indicative of no collinearity. Multicollinearity is a data problem that can adversely impact regression interpretation by limiting the size of the R-squared and confounding the contribution of independent variables. o Each independent variable will have a tolerance measure and each of measure should be close to 1. with the floor VIF value being 1. A tolerance of less than . 'Part Correlation': removes the effect of the remaining independent variables from the independent side of the equation. tolerance and VIF. For this reason. • Variance inflation factor (VIF) is the reciprocal of the tolerance value Stage 6: Validation of the Results and measures the same problem of collinearity. two measures. • Collection of additional samples or split samples . 2. whereby one observation is omitted in the estimation of the regression model and the omitted observation is predicted with the estimated model.o The regression model or equation may be retested on a new or split sample. the researcher should consider the following: • Predictions have multiple sampling variations. Before applying an estimated regression model to a new set of independent variable values. Conditions and relationships should not have changed materially from their measurement at the time the original sample was taken. then prediction of the dependent variable when 200 tires per month are sold is invalid. The regression equation is only valid for prediction purposes within the original range of magnitude for the prediction variables. Do not use the model to estimate beyond the range of the independent variables in the sample. we do not know the form of the relationship between the predictor variable and the criterion variable. At 200 tires per month the relational form may become curvilinear or • • . but also those of the newly drawn sample. not only the sampling variations from the original sample. Results cannot be extrapolated beyond the original range of variables measured since the form of the relationship may change. o Example: If a predictor variable is the number of tires sold and in the original data the range of this variable was from 30 to 120 tires per month. The residuals for the observations are summed to provide an overall measure of predictive fit. We are outside the original range of magnitude. o No regression model should be assumed to be the final or absolute model of the population. • Comparison of regression models o The adjusted R-square is compared across different estimated regression models to assess the model with the best prediction. • Calculation of the PRESS statistic o Assesses the overall predictive accuracy of the regression by a series of iterations. .quadratic. the regression results are not valid. or serial correlation (due to nonindependence or error terms). . When the assumption of linearity is violated. being particularly concerned with the problems caused by multi-collinearity. Either approach must be used cautiously.ANSWERS TO END-OF-CHAPTER QUESTIONS (1) HOW WOULD YOU EXPLAIN THE "RELATIVE IMPORTANCE" OF THE PREDICTOR VARIABLES USED IN A REGRESSION EQUATION? Answer Two approaches: (a) beta coefficients and (b) the order that variables enter the equation in stepwise regression. All of these conditions require correction before statistical inferences of any validity can be made from a regression equation. they reflect the impact on the criterion variable of a change of one standard deviation in any predictor variable. They should be used only as a guide to the relative importance of the predictor variables included in your equation. This gives the model the properties of additivity and homogeneity. Their value is basically that we no longer have the problem of different units of measure. When using stepwise regression. (2) WHY IS IT IMPORTANT TO EXAMINE THE ASSUMPTION OF LINEARITY WHEN USING REGRESSION? Answer The regression model is constructed with the assumption of a linear relationship among the predictor variables. heteroscedasticity. a variety of conditions can occur such as multicollinearity. Thus. Hence coefficients express directly the effect of changes in predictor variables. the partial correlation coefficients are used to identify the sequence in which variables will enter the equation and thus their relative contribution. With regard to beta coefficients. they are the regression coefficients which are derived from standardized data. the linearity assumption should be examined because if the data are not linear. and only over the range of sample data included. Basically. Another method of representing nonlinear relationships is through the use of an interaction or moderator term for two independent variables. One of the assumptions is that the conditions and relationships existing when sample data were obtained remain unchanged. cubic. This range is determined by the predictor variable values used to construct the model. but a very small coefficient of determination—too small to be of value. In using the model. One way is through a direct data transformation of the original variable as discussed in Chapter 2. If changes have occurred they should be accommodated before any new inferences are made. with a sufficiently large sample size you could obtain a significant relationship. there are statistical considerations. For example. For example. In addition. or higher order polynomials in the regression equation. Two additional ways are to explicitly model the nonlinear relationship in the regression equation through the use of polynomials and/or interaction terms. Inclusion of this type of term in the regression equation allows for the slope of the relationship of one independent variable to change across values of a second dependent variable. The advantage of polynomials over direct data transformations in that polynomials allow testing of the type of nonlinear relationship. Polynomials are power transformations that may be used to represent quadratic. could make any obtained results at best spurious. predictor values should fall within this relevant range. Finally. . (4) COULD YOU FIND A REGRESSION EQUATION THAT WOULD BE ACCEPTABLE AS STATISTICALLY SIGNIFICANT AND YET OFFER NO ACCEPTABLE INTERPRETATIONAL VALUE TO MANAGEMENT? Answer Yes.(3) HOW CAN NONLINEARITY BE CORRECTED OR ACCOUNTED FOR IN THE REGRESSION EQUATION? Answer Nonlinearity may be corrected or accounted for in the regression equation by three general methods. the effects of multicollinearity among predictor variables is one such consideration. Another is that there is a "relevant range" for any regression model. there are some basic assumptions associated with the use of the regression model. which if violated. Thus. but over another part of the relevant range the second predictor variable may become the more important. then. These coefficients (b1-bk) then. the intercept (constant) coefficient (bo) estimates the average effect of the omitted dichotomous variables. This variable has a value of one or zero depending on the category being expressed (e. the intercept (bo) serves to locate the point where the regression equation crosses the Y axis. When interactive effects are encountered. The effect of this interaction is that over part of the relevant range one predictor variable may be considerably more important than the other.(5) WHAT IS THE DIFFERENCE IN INTERPRETATION BETWEEN THE REGRESSION COEFFICIENTS ASSOCIATED WITH INTERVAL SCALED PREDICTOR VARIABLES AS OPPOSED TO DUMMY (0. In the equation. . the dichotomous variable will be included when its value is one and omitted when its value is zero. and the other coefficients (b1-bk) indicate the effect on the predictor variable(s) on the criterion variable (if any).1) PREDICTOR VARIABLES? Answer The use of dummy variables in regression analysis is structured so that there are (n-1) dummy variables included in the equation (where n = the number of categories being considered). When dichotomous predictor variables are used. With metric predictors. b1 through bk. discrete ranges of influence can be misinterpreted as continuous effects.g. Coefficients bo through bk serve a different function when metric predictors are used. male = 0.. (6) WHAT ARE THE DIFFERENCES BETWEEN INTERACTIVE AND CORRELATED PREDICTOR VARIABLES? DO ANY OF THESE DIFFERENCES AFFECT YOUR INTERPRETATION OF THE REGRESSION EQUATION? Answer The term interactive predictor variable is used to describe a situation where two predictor variables' functions intersect within the relevant range of the problem. The other coefficients. female = 1). represent the average importance of the two categories in predicting the dependent variable. represent the average differences between the omitted dichotomous variables and the included dichotomous variables. since n = 2. the coefficients actually represent averages of effects across values of the predictors rather than a constant level of effect. In the dichotomous case. there is one variable in the equation. there can be no real gain in adding both of the variables to the predictor equation. In the remaining two instances (an ordinary observation exceptional in its combination of characteristics or an exceptional observation with no likely explanation). data entry). In this case. Theoretical or conceptual justification is much preferable to a decision based solely on empirical considerations.When predictor variables are highly correlated.g. if any. . the case of an observation with some form of error (e. thus a representative observation. Omission or correction is easily decided upon in one case. (7) ARE INFLUENTIAL CASES ALWAYS TO BE OMITTED? GIVE EXAMPLES OF WHEN THEY SHOULD AND SHOULD NOT BE OMITTED? Answer The principal reason for identifying influential observations is to address one question: Are the influential observations valid representations of the population of interest? Influential observations. with the other causes. The researcher must decide if the situation is one which can occur among the population. The objective is to assess the likelihood of the observation occurring in the population. Since the direction and magnitude of change is highly related for the two predictors. A valid but exceptional observation may be excluded if it is the result of an extraordinary situation. the predictor with the highest simple correlation to the criterion variable would be used in the predictive equation. gain in predictive power. the addition of the second predictor will produce little. the answer is not so obvious. whether they be "good" or "bad. However. In this case.." can occur because of one of four reasons. the researcher has no absolute guidelines. the coefficients of the predictors are a function of their correlation. When correlated predictors exist. little value can be associated with the coefficients since we are speaking of two simultaneous changes. identify variables with variance proportions above .50%.90 or above) for two or more coefficients. The following discussion includes the assessment of multicollinearity and the identification of influential observations. o A collinearity problem is present when a condition index identified in part one accounts for a substantial proportion of variance (. • Identifying Influential Observations Influential observations may be identified in a four-step process. The condition index represents the collinearity of combinations of variables in the data set. Step two: For all condition indices above 30. these terms will be referred to in later chapters.ADVANCED DIAGNOSTICS FOR MULTIPLE REGRESSION ANALYSIS Additional material posted on the Companion Web site – MVSTATS. o The regression coefficient variance-decomposition matrix shows the proportion of variance for each regression coefficient attributable to each eigenvalue (condition index). .COM Overview This material reviews additional diagnostic procedures available to the research analyst to assess the underlying assumptions of multiple regression analysis. Assessing Multicollinearity Multicollinearity can be diagnosed in a two-step process: • Step one: Identify all condition indices above 30. Although these concepts are particularly important in multiple regression analysis. the model is reestimated. With large sample sizes. and those dummy variables with significant coefficients are outliers. o Dummy-variable regression may also be used to identify outliers on the dependent variable when the researcher has at least N + 1 (where N is the sample size) degrees of freedom available. A dummy variable is added for each observation.  Threshold values: When the number of predictors is greater than 10 and the sample size exceeds 50. individual observations. is also used to identify outliers. Outliers can be identified as those cases which impact the regression slope and the corresponding regression equation coefficients. o Studentized deleted residual (the studentized residual for observation i when observation i is deleted from calculation of the regression equation) may be used to identify extremely influential.  Interpretation: Large values indicate that the observations carry disproportionate weight in determining the dependent variable. studentized residuals of greater than + 2. • Step 3: Identify single case influential observations. o Mahalanobis distance. Values greater than + 2.• Step 1: Examine the residuals. otherwise. use 3p/n. a value is large if it exceeds 2p/n (where p is the number of predictors and n is the sample size).  Diagonal of the matrix represents the distance of the observation from the mean center of all other observations on the independent variables. . o Partial regression plots visually portray all individual cases. as discussed in Chapter 2.0 indicate an influential case. o Studentized residuals identify observations which are outliers on the dependent variable. • Step 2: Identify leverage points from the predictors. o Hat matrix represents the combined effects of all independent variables for each case and enables the researcher to identity multivariate leverage points.0 are substantial. Values greater than 1. values exceeding 2 ÷ sqrt(n) are influential.1 which exceed + 3p ÷ n are indicative of influential observations. The researcher should never classify an observation based on a single measure. for large sample sizes. o Remedy: Once identified. o COVRATIO is similar to DFBETA. values greater than 1. For small or medium sample sizes.0 should be considered influential. If the researcher is unable to delete the observations. • Step 4: Select and accommodate influential observations. Values exceeding 2 * square root of (p/n) are indicative of influential observations. o Use of multiple measures: The above steps should converge on the identification of observations which may adversely affect the analyses. o DFFIT measures the degree to which fitted values change when the case is deleted. a more robust estimation technique should be used.0 may be considered influential. influential observations may be deleted where justified. o Cook's distance measures the influence of an observation based on the size of changes in the predicted values when the case is omitted (residuals) and the observation's distance from the other observations (leverage). but is different in that it considers all coefficients collectively rather than individually. Values of the COVRATIO .o DFBETA reflects the relative change in the regression coefficient when an observation is deleted. but should always conduct a number of diagnostics to identify truly influential observations. . or variates of the predictor variables." "Why?." "When?. that are each appropriate when encountering a research question with a categorical dependent variable and several metric dependent variables. The following presentation will first address the basic questions of "What?. 4. 2. multiple discriminant analysis and logistic regression. which maximize the between-group variance and minimize the within-group variance on the discriminant function score(s). Why do we use multiple discriminant analysis? 1. Classification is accomplished by a statistical procedure.CHAPTER FIVE MULTIPLE DISCRIMINANT ANALYSIS AND LOGISTIC REGRESSION Chapter Five discusses two multivariate techniques." and "How?" as applied to Multiple Discriminant Analysis. This is followed by a brief overview of the differences and similarities of the two techniques. Further analysis of the variate reveals between which groups and by which independent variables there are significant differences. thus a significant model would indicate that the group means are not equal. What is multiple discriminant analysis? 1. Researchers use multiple discriminant analysis to help them understand: • • • group differences on a set of independent variables the ability to correctly classify statistical units into groups or classes the relative importance of independent variables in the classification process . The null hypothesis is that the two or more group means are equal on the discriminant function(s). 3. which derives discriminant functions. Discriminant analysis is a dependence technique that forms variates (linear combinations of metric independent variables). which are used to predict the classification of a categorical dependent variable. 2.) · selecting returns for IRS audits How do you use multiple discriminant analysis? Multiple discriminant analyses follow the six stages in model building. . Multiple discriminant analysis may be considered a type of profile analysis or an analytical predictive technique which is most appropriate when there is a single categorical dependent variable and multiple metric independent variables. Common applications include: · assessing credit risk · profiling market segments · predicting failures (firm. • establish the number and composition of the dimensions of discrimination between groups that are formed from the set of independent variables. The appropriate applications are those in which respondents are profiled and/or classified into groups. The Stage 1: Objectives of Discriminant Analysis 1. etc. product. Discriminant analysis may address a number of common objectives: • determine the statistical significance of differences between the average score profiles on a set of variables for two (or more) a priori defined groups. • identify the independent variables accounting most for the differences in the average score profiles of the two or more groups. • establish procedures for classifying statistical units (individuals or objects) into groups on the basis of their scores on a set of independent variables. following are important points for each step of the process.When do you use multiple discriminant analysis? 1. thus. 3. However.Stage 2: Research Design of a Multiple Discriminant Analysis 1. • For smaller samples or lower ratios of independent variables to sample size. a polar extremes approach which omits all middle categories may be used. cluster analysis). the basis for group division must be theory. • Number of categories: Any number of categorical groups can be developed. This means that for every independent variable used. 4. • Examination of the group histograms will provide the researcher with evidence that sample size effects may be operating. Selection of the independent variables for use in the analysis may be based on previous research. Discriminant analysis is not an exploratory procedure that is used to define groups. intuition. indiscriminate variables may yield a significant difference between groups. the gross amount of overlap between the groups remains the same. Discriminant analysis is very sensitive to the ratio of sample size to the number of predictor variables. • A 20 to 1 ratio of the sample size to the number of independent variables is needed to maintain the integrity of discriminant analysis. but no practical difference. This occurs because as the sample size increases.g. the researcher must keep an especially sharp eye on the assumptions and the stability of the discriminant function.. a theoretical model. or some other technique (e. previous knowledge. . Sample size may influence the results. 2. or intuition. there should be twenty respondents. resulting in a statistically significant difference between the groups. • With very large sample sizes. • Two group dependent variable: If the analyst is only concerned with the two extreme groups. the percentage of overlap between the groups declines. The dependent variable selected for use in the analysis must be categorical and the categories must be mutually exclusive and exhaustive. the sizes of the groups will be proportionate to the total sample distribution. there is no rule specifying a certain division. 6. with at least two groups. then the validation of that function on the same data is upwardly biased. the validation sample is not biased and also possesses the same properties as the analysis sample. The sample should be divided into an estimation sample and a validation sample. • Each group should have at least 20 observations. • The data distribution for the independent variables is assumed to be the normal distribution. Thus. • The dependent variable must be categorical.5. Stage 3: Assumptions in Multiple Discriminant Analysis 1. • At minimum. • Data used to derive the discriminant function cannot also be used to validate the function. Discriminant analysis is also sensitive to the sample sizes of each group. Discriminant analysis requires nonmetric dependent variables and multiple metric independent variables. however. a requirement for use of the F test. smallest group size must exceed the number of independent variables.e. . 2. • By dividing the sample into two parts. • A proportional stratified sampling procedure is used to select the observations in the validation sample. • Groups of similar relative size avoid adverse effects on estimation and classification. Discriminant analysis assumes multivariate normality of the independent variables. 50% estimation / 50% validation). If the researcher uses all of the available data to derive the discriminant function. Division of the sample is most often equal (i. Discriminant analysis assumes unknown. o using quadratic classification techniques. Unequal covariance matrices can have an adverse affect on classification. • When selecting independent variables with a stepwise method. o computing group-specific covariance matrices. Outliers adversely impact the classification accuracy of discriminant analysis.3. 6. remedies for violations of this assumption may be: o increasing sample size. but equal dispersion and covariance structures (matrices) for the groups as defined by the dependent variable. . and outliers which are not representative of the population should be deleted. Based on the problem's origin. multicollinearity may impede inclusion of certain variables in the variate and impact interpretation. Discriminant analysis implicitly assumes that all relationships are linear. Equality is assessed by the Box's M test for homogeneity of dispersion matrices. • • 5. 4. • The analyst should complete diagnostics for influential observations. • Nonlinear relationships can be remedied with the appropriate transformations. Discriminant analysis is adversely affected by collinearity of the independent variables. • This assumption is necessary for the maximization of the ratio of variance between groups to the variance within groups. • Significance level: The conventional criterion of . analysts must: o carefully evaluate the statistical significance of each discriminant function. • Criteria: Wilks' lambda. the researcher must choose a computational method. Mahalanobis' distance. • 3 or more groups: If the dependent variable has three or more groups. only those variables that provide the unique discriminatory power will be included in the analysis. • Simultaneous — all of the independent variables.Stage 4: Estimation of the Discriminant Model and Assessing Overall Fit 1. o Best use: when the researcher has some theoretical reason to include all of the independent variables. Pilliai's criteria. and Rao's V measures. are used to compute the discriminant function(s). Roy's greatest characteristic root. Through a series of "step" iterations similar to stepwise regression. 2. To derive discriminant functions. Two methods from which to choose: simultaneous and stepwise methods.05 or beyond is most often used. After the discriminant function is computed. o Best use: when the researcher has a large number of independent variables and wishes to select the best combination of predictor variables. . Hotelling's trace. o some discriminant functions may not contribute significantly to overall discriminatory power. regardless of discriminatory power. • Stepwise — independent variables are chosen one at a time on the basis of their discriminating power. the researcher must assess its level of statistical significance. • Group sizes: When developing a classification matrix. The default assumption is that the population group sizes are assumed to have an equal chance of occurring. o Percentage correctly classified is shown for:  each group  overall or hit ratio: The overall percentage correctly classified is shown at the bottom.3. with the following chance criteria available: o Equal chance criterion: When the sample sizes of the groups are equal. or holdout. Overall fit of the discriminant function may be assessed by constructing classification matrices. sample are compared to the cutting score and thereby classified into a group. To accurately predict based on this criterion. percentage of correct classification by chance is equal to 1 divided by the number of groups. accuracy of the discriminant function may be assessed via comparison to several criteria. • Chance criteria — the hit ratio may be compared to the percentage of respondents who would be correctly classified by chance. The optimal cutting score (critical Z value) is dependent on whether or not the sizes of the groups are equal or unequal. the hit ratio should exceed the percentage equal to the proportional size of the largest group in the sample. The acceptable level of predictive. the analyst must decide whether or not the observed group sizes in the sample are representative of the group sizes in the population. Individual discriminant scores for the observations in the validation. o Maximum chance criterion: Percentage of correct classification by chance is based on the sample size of the largest group. or classification. 4. • Cutting Score: Used to construct a classification matrix. o Off-diagonal values represent the incorrect classifications. • Interpretation: o Diagonal of the classification matrix represent the number of respondents correctly classified. . Larger F values indicate greater discriminating power. percentage of correct classification by chance is equal to the sum of the proportion of respondents in each group squared. . Examine the discriminant functions to determine the relative importance of each independent variable in discriminating between groups by one of the following methods: • Discriminant weights — The sign and magnitude of discriminant weights (also called discriminant coefficients) represent the relative contribution of each variable to the function. the total sample size and the number of groups. • Press's Q statistic — A statistical test that computes a value based on the number of correct classifications. and compares this value to a critical value (chi-square value for 1 degree of freedom at the desired confidence level). o Reflect the variance shared by the independent variables and the discriminant function. o Sensitive to sample size. Large samples are more likely to show significance than small sample sizes of the same classification rate. • Discriminant loadings — measure the simple linear correlation between each independent variable and the discriminant function. o Can be interpreted like factor loadings when assessing the relative contribution of each independent variable to the discriminant function. Stage 5: Interpretation of the Variate 1. • Partial F values — Used when the stepwise method is selected. o Classification accuracy should be at least one fourth greater than that achieved by chance.o Proportional chance criterion: When the sample sizes of the groups are unequal. partial F values reflect the relative contribution of each independent variable to the discriminant function. do not change the structure of the solution. The length of the vector is indicative of the relative importance of each variable in discriminating among the groups. Rotations. but make the functions easier to interpret. Stage 6: Validation of the Results 1. Group may also be profiled on other independent variables which were not included in the analysis. • Profiling group differences: Groups are profiled on the independent variables in order to ensure their correspondence to the conceptual bases of the model. or using group profiling. collecting a new data set. This procedure may be completed several times by randomly dividing the total sample into analysis and holdout samples. similar to those done in factor analysis. Accomplished by splitting the original data set. Vectors are created by drawing a line from the origin to a point representing each discriminant loading multiplied by its respective univariate F value. Interpretation of two or more discriminant functions is somewhat more complicated. This . The discriminant functions are derived from the analysis sample and the results are validated with the holdout sample.2. The analyst must now determine the relative importance of the independent variables across all the discriminant functions. • Split sample: The data set is split into two parts: an analysis sample and a holdout sample. • Variable Importance Across Multiple Functions: The relative importance of each independent variable across all significant discriminant functions can be determined with: o Potency Index is a composite or summary measure that indicates which independent variables are most discriminating across all discriminant functions. The hit ratios obtained from the various analyses can be averaged. The discriminant functions derived from a previous sample are validated with new data. • New sample: A new data set is gathered. • Rotation: Simplifies the profiling of each discriminant function. o Stretching the vectors is one approach used to identify the relative importance of independent variables. however. The coefficients in the variate can be interpreted in a manner quite similar to regression. 2. Even when the statistical assumptions are met. 2. Logistic Regression: Regression with a Binary Dependent Variable What is logistic regression? 1. classification matrices and hit ratios). in that it can only accommodate a binary or two-group categorical dependent variable. there are substantive differences in .e.. Logistic regression may be preferred over discriminant analysis in such instances since it is not as impacted by violations of statistical assumptions and is much more robust when these assumptions are not met. Researchers use logistic regression to understand any of the research questions appropriate for discriminant analysis as long as the nonmetric dependent measure has only two groups. It is similar to discriminant analysis in its classification approach that produces predictions of group membership for each observation. Why do we use multiple discriminant analysis? 1. When do you use multiple discriminant analysis? 1. How do you use multiple discriminant analysis? While logistic regression addresses many of the same research questions as discriminant analysis and measures predictive accuracy in the same manner (i.approach provides external validity to the findings. Logistic regression is another multivariate technique that forms a variate (linear combinations of metric independent variables) used to predict the classification of a categorical dependent variable. It is different. 3. many researchers prefer logistic regression since its interpretation and diagnostic measures are quite similar to those found in multiple regression. Logistic regression is thus comparable to a twogroup discriminant analysis. 4. It operates in a manner quite similar to multiple regression although it utilizes a maximum likelihood estimation procedure rather than the ordinary least squares approach used in multiple regression. The logistic curve also meets the assumptions assuming a normal distribution for the error terms and homoscedasticity by equalizing variance. • • 2. which is a S-shaped curve that levels off as it approaches zero and one.several areas. it allows for a “regression-like” procedure to be used rather than the “means difference” approach seen in discriminant analysis. • . it avoids the problems found in multiple regression where the predicted values of a binary measure can go lower than zero and higher than one. The following discussions highlight these differences in the following areas: • Unique nature of the dependent variable as represented by the logistic curve • • • Estimating the model using the Odds and Logit values Differing measures of model estimation fit Interpreting the different types of estimated logistic coefficients 1. The logistic curve represents the probability that an observation is in one group or another (let’s call them groups 0 and 1). What are the odds ratio and logit values used in model estimation? • Since logistic regression uses a logistic curve as the predicted value of the logistic regression variate. If the probability is less than 50% then the observation is classified in group 0 and if it is greater than 50% it is classified in group 1. Likewise any estimated odds value can be converted back to a probability value. to portray the predicted value for any observation. any probability is stated in a metric form that can be directly estimated. How does logistic regression represent the dependent measure in a form amenable to model estimation in a manner similar to regression? • Logistic regression uses the logistic curve. In doing so. In this manner. In this form. The objective is to estimate a logistic variate that predicts low probability values for all the observations in group 0 and high probability values from group 1. This transformation is achieved by expressing the probability as odds – the ratio of the probabilities of being in groups 0 and 1. it must find a way to “transform” the probability value so that the predicted value can never fall below 0 or above one. . then it is 4.25 (i. In the last section we will examine how to interpret each form of the coefficient and what diagnostic information is best derived from each. • But there is one additional problem: How do we keep the odds value from going below zero. then they would b ..e. it is generally estimated as a model with no independent variables. No matter how low the logit value goes. Odds less than 1.e. A typical comparison is: o Estimate a null model: acting as the baseline. how is overall model fit evaluated? • The maximum likelihood method fits the likelihood value and the measure of fit is -2 time the log of the likelihood value.0 will have negative values and those above 1. it can still be transformed back to an acceptable odds value by taking the antilog. the estimated logistic coefficients can be stated in terms of estimating the odds value and the logit value and providing two forms of the coefficients.0 will have positive values. 80% ÷ 20%) or it is four times as likely to be in Group 1. Since logistic regression used maximum likelihood rather than ordinary least squares used in multiple regression. then we can assume that the overall model fit is statistically significant. • Model comparisons are made by comparing the difference in-2LL. The minimum value is 0 corresponding to perfect fit.o Example: Assume that the probability of being in Group 1 was 80%. often expressed as -2LL.0 (i. If we were expressing the odds of being in Group 0. If we are expressing the odds of being in Group 1. which is the theoretical minimum? The is solved by taking the logarithm of the odds and creating the logit value. making the probability of being in Group 0 equal to 20%. o Estimate the proposed model: Add the independent variables in the specified model o Assess the -2LL difference: If the -2LL difference is statistically significant. any improvements in fit can be attributed to the independent variables added to the variate. • 3. As such. As will be seen. 20% ÷ 80%) or there is one-fourth the chance of being in Group 0 compared to Group 1.. the amount of change in the dependent variable for a unit change in the independent variable depends where on the logistic curve the value of the independent variable occurs. As such.0) * 100. it is essential to understand what each form of coefficient represents in terms of interpretation of the variate: o Logistic coefficient – estimated in the original model form where the logit value acts as the dependent variable. • Since the relationship between independent and dependent variables is nonlinear (i. it is easiest to use separate forms of the logistic coefficient for each assessment: o Directionality of the relationship – can be determined directly from the logistic coefficients. using classification ratios and evaluating the hit ratio. o Magnitude of the relationship: This is best determined with the exponentiated coefficient.. 4. • While each form of coefficient can be used to assess both directionality and magnitude of the relationship. • Predictive accuracy is assessed in the same manner as discriminant analysis. where the signs (positive or negative) represent the type of relationship between independent and dependent variable.e. where the percentage change in the dependent variable (the odds value) is shown by the calculation (Exponentiated coefficient – 1. o If the independent variable value falls close to the 50% . o Exponentiated logistic coefficient – a transformation of the logistic coefficient (antilog of the logistic coefficient) that reflects changes in the odds value. the S-shaped logistic curve). How do the two forms of logistic coefficients represent the relationship between the independent and dependent variables? • The two forms of logistic coefficients relate to the two ways in which the dependent variable can be represented – as odds or as the logit value.• There are also “pseudo R2” measures which are calculated to represent the percent explanation similar to R2 in multiple regression. which is the percentage correctly classified. But if the probability is close to 50%. then it is much harder to increase the probability value since this is in the “flatter” portion of the curve. o For example. then the impact is less. it is much easier to increase or decrease the probability since the curve is much “steeper” in this area. . if the value of the independent variable is associated with a 90% probability value. o But if the independent variable value falls in the high or low probability ranges. then one can expect a larger impact on probability for a change in the independent variable.probability value. the single dependent (criterion) variable is nonmetric and the independent (predictor) variables are metric. Thus.the multiple dependent variables are metric and the single independent variable is nonmetric. the difference lies in the number of independent and dependent variables and in the way in which these variables are measured. When the basic assumptions of both methods are met. Logistic regression is limited though to the prediction of only a two-group dependent measure. Logistic regression has the advantage of being less affected than discriminant analysis when the basic assumptions of normality and equal variance are not met. Note the following definitions: • Multiple discriminant analysis (MDA) . AND ANALYSIS OF VARIANCE? Answer Basically. REGRESSION ANALYSIS. It also can accommodate nonmetric dummy-coded variables as independent measures. • Analysis of Variance (ANOVA) . each gives comparable predictive and classificatory results and employs similar diagnostic measures. In the case of a two-group dependent variable either technique might be applied. (2) WHEN WOULD YOU EMPLOY LOGISTIC REGRESSION RATHER THAN DISCRIMINANT ANALYSIS? WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF THE DECISION? Answer Both discriminant analysis and logistic regression are appropriate when the dependent variable is categorical and the independent variables are metric. .ANSWERS TO END-OF-CHAPTER QUESTIONS (1) HOW WOULD YOU DIFFERENTIATE BETWEEN MULTIPLE DISCRIMINANT ANALYSIS. discriminant analysis is required. • Regression Analysis .both the single dependent variable and the multiple independent variables are metric. but only discriminant analysis is capable of handling more than two groups. when more than two groups are involved. or 75-25) so long as each "half" is proportionate to the entire sample. b. This is because there is little likelihood that the function will classify more accurately than would be expected by randomly classifying individuals into groups (i. The minimum acceptable percentage of correct classifications usually is predetermined. Criterion for stopping after derivation. but a cut-off value of 100 units is often used. a proportionately stratified sampling procedure is usually followed.. 50-50 analysis/hold-out. . . Comparison of "hit-ratio" to some criterion. If the function is not significant at a predetermined level (e.g.. Criterion for stopping after interpretation.. (4) WHAT PROCEDURE WOULD YOU FOLLOW IN DIVIDING YOUR SAMPLE INTO ANALYSIS AND HOLDOUT GROUPS? HOW WOULD YOU CHANGE THIS PROCEDURE IF YOUR SAMPLE CONSISTED OF FEWER THAN 100 INDIVIDUALS OR OBJECTS? Answer When selecting individuals for analysis and holdout groups. There is no minimum sample size required for a sample split.g.e.05). by chance). The level of significance must be assessed. 60-40.(3) WHAT CRITERIA COULD YOU USE IN DECIDING WHETHER TO STOP A DISCRIMINANT ANALYSIS AFTER ESTIMATING THE DISCRIMINANT FUNCTION(S)? AFTER THE INTERPRETATION STAGE? Answer a. The result is an upward bias in statistical significance which should be recognized in analysis and interpretation. Many researchers would use the entire sample for analysis and validation if the sample size were less than 100. The split in the sample typically is arbitrary (e. then there is little justification for going further. This is usually a fairly straight-forward function of the classifications used in the model and of the sample size. For unequal group sizes. Another test would be to use a test of proportions to examine for significance between the chance criterion proportion and the obtained hitratio proportion. For equal group sizes. . The authors then suggest the following criterion: the classification accuracy (hit ratio) should be at least 25 percent greater than by chance. the optimum cutting score is defined by: ZA + ZB ZCE = —————————— N ZCE =critical cutting score value for equal size groups ZA = centroid for group A ZB = centroid for Group B N = total sample size b.(5) HOW DO YOU DETERMINE THE OPTIMUM CUTTING SCORE? Answer a. the optimum cutting score is defined by: NAZA + NBZB ZCU = ———————————NA + NB ZCU =critical cutting score value for unequal size groups NA = sample size for group A NB = sample size for Group B (6) HOW WOULD YOU DETERMINE WHETHER OR NOT THE CLASSIFICATION ACCURACY OF THE DISCRIMINANT FUNCTION IS SUFFICIENTLY HIGH RELATIVE TO CHANCE CLASSIFICATION? Answer Some chance criterion must be established. This stretches the group centroids along the axis in the discriminant plot that provides more of the accounted-for variation. Frequently. Plots are usually produced for the first two significant functions. the technique is referred to as twogroup discriminant analysis. When three or more classifications are identified. In this case stretching the discriminant loadings and centroid data. medium. In other instances. and high classifications. Discriminant analysis is capable of handling either two groups or multiple groups (three or more).(7) HOW DOES A TWO-GROUP DISCRIMINANT ANALYSIS DIFFER FROM A THREE-GROUP ANALYSIS? Answer In many cases. more than two groups are involved. for example. male versus female. prior to plotting the discriminant function. When two classifications are involved. aids in detecting and interpreting differences between groups. such as a three-group classification involving low. (8) WHY SHOULD A RESEARCHER STRETCH THE LOADINGS AND CENTROID DATA IN PLOTTING A DISCRIMINANT ANALYSIS SOLUTION? Answer Plots are used to illustrate the results of a multiple discriminant analysis. plots are less than satisfactory in illustrating how the groups differ on certain variables of interest to the researcher. the technique is referred to as multiple discriminant analysis. Stretching the discriminant loadings by considering the variance contributed by a variable to the respective discriminant function gives the researcher an indication of the relative importance of the variable in discriminating among the groups. By using the statistically significant discriminant functions. the group centroids can be plotted in the reduced discriminant function space so as to show the separation of the groups. the dependent variable consists of two groups or classifications. Group centroids can be stretched by multiplying by the approximate Fvalue associated with each of the discriminant functions. . A discriminant (z) score is then calculated for each observation. the probability increases. At very low levels of the independent variables. Group means (centroids) are calculated and a test of discrimination is the distance between group centroids.(9) HOW DO LOGISTIC REGRESSION AND DISCRIMINANT ANALYSES EACH HANDLE THE RELATIONSHIP OF THE DEPENDENT AND INDEPENDENT VARIABLES? Answer Discriminant analysis derives a variate. the probability approaches zero. . logistic regression assumes the relationship between the independent and dependent variables resembles an S-shaped curve. the linear combination of two or more independent variables that will discriminate best between the dependent variable groups. (10) WHAT ARE THE DIFFERENCES IN ESTIMATION AND INTERPRETATION BETWEEN LOGISTIC REGRESSION AND DISCRIMINANT ANALYSIS? Answer Estimation of the discriminant variate is based on maximizing between group variance. Logistic regression uses a maximum likelihood procedure to fit the observed data to the curve. To define the probability. Logistic regression may be comfortable for many to interpret in that it resembles the more commonly seen regression analysis. Logistic regression is estimated using a maximum likelihood technique to fit the data to a logistic curve. Logistic regression forms a single variate more similar to multiple regression. It differs from multiple regression in that it directly predicts the probability of an event occurring. As the independent variable increases. Discrimination is achieved by setting variate weights for each variable to maximize between group variance. Both techniques produce a variate that gives information about which variables explain the dependent variable or group membership. (11) EXPLAIN THE CONCEPT OF ODDS AND WHY IT IS USED IN PREDICTING PROBABILITY IN A LOGISTIC REGRESSION PROCEDURE. Yet we would like for a straight-forward method of estimating the probability values without having to utilize some form of nonlinear estimation. The odds value provides a convenient transformation of a probability value into a form more conducive to model estimation. Since we only use logistic regression for two-group situations. Answer One of the primary problems in using any predictive model to estimate probability is that is it difficult to “constrain” the predicted values to the appropriate range. Probability values should never be lower than zero or higher than one. we can always calculate the odds ratio knowing just one of the probabilities (since the other probability is just 1 minus that probability). . The odds value is simply the ratio of the probability of being in one of the groups divided by the probability of being in the other group. The odds ratio is a way to express any probability value in a metric value which does not have inherent upper and lower limits. " "When?. This provides the researcher a much more reliable and valid result. It derives its name from its ability to perform a series of univariate analysis of variance (ANOVA) tests while maintaining a specified overall error rate for all tests combined. 2) The null hypothesis is that the vectors of dependent variable means are equal across groups formed by the categorical independent variable(s)." "Why?. MANOVA accomplishes the task of multiple significance tests without a loss of control of the error level. It is misleading to think that it would be simpler to run multiple ANOVAs instead of using MANOVA. What is multivariate analysis of variance? 1) Multivariate analysis of variance (MANOVA) is used to assess group differences across multiple metric dependent variables simultaneously. By using MANOVA. Therefore. MANOVA is the technique of choice when a researcher wishes to predict metric dependent variables with multiple categorical independent variables." and "How?" as applied to multivariate analysis of variance (MANOVA). the researcher avoids an inflated error level which occurs when alpha is not controlled during multiple ANOVA analyses. 2. 3) The number of independent variables and the number of categories in each defines the number of group vectors to examine.CHAPTER SIX MULTIVARIATE ANALYSIS OF VARIANCE The following presentation will address the basic questions of "What?. • two or more independent variables — each combination of independent variable values forms a separate group for which the vector of dependent variable means is calculated. • single independent variable — each value of the independent variable becomes a group for which a vector of dependent variable means are calculated. . Why do we use multivariate analysis of variance? 1. but needs some control over the experimentwide error rate. How do you use multivariate analysis of variance? Multivariate analysis of variance follows a six stage model building perspective. MANOVA is used to assess whether an overall difference is found between groups. Only those dependent variables which have a sound conceptual or theoretical basis should be selected for inclusion in the analysis. • Inclusion of irrelevant variables may adversely affect the resulting conclusions of an analysis. 2.When do you use multivariate analysis of variance? MANOVA is the appropriate statistical technique when the researcher wishes to control for the experimentwide error rate or to test for differences among a combination of dependent variables. The collective effect of several variables is of interest. then the separate univariate tests are employed to address each dependent variable. • Intrinsically Multivariate Research Questions: The researcher wishes to address how a set of dependent measures differ as a whole across groups. . • Structured Multivariate Research Questions: The researcher gathers data which have two or more dependent measures that have specific relationships between them. not the individual effects. The following three types of questions are appropriate objectives for MANOVA • Multiple Univariate Research Questions: The researcher identifies a number of separate dependent variables that are to be analyzed separately. This is especially so when the researcher's objective is to learn about the collective effect. Stage 1: Objectives of Multivariate Analysis of Variance 1. A common type of structured question would be a repeated measures design. • Blocking factors are treatments added after the analysis design. 2. • Treatments (independent variables) will be specified in the design of the experiment. These treatments enable the researcher to further segment the respondents in order to obtain greater within-group homogeneity and to reduce the mean square within source of variance. • Types of interactions: nonparallel. The sample size in each cell must be greater than the number of dependent variables included in the analysis. The interaction is represented by differing patterns of dependent variable means for combinations of the values of two or more independent variables. Treatments in a factorial design should be selected based on the research question. o Ordinal: the effects of a treatment are not equal across all levels of another treatment. Examine interaction effects before main effects with multiple independent variables. ordinal interaction (acceptable) or a disordinal interaction (unacceptable). the interaction effect must be interpreted before any possible main effects. Post hoc tests may be used to assess the significance of the differences (Scheffe's or contrast test). 3. • Interactions are how one independent variable affects (interacts with) another independent variable. • Impact of significant interaction: When a significant interaction is present.Stage 2: Issues in the Research Design of MANOVA 1. • Interpretation of significant interactions: Ordinal interactions may be interpreted (when the results are conceptually . but the magnitude is always the same direction o Disordinal: effects of one treatment are positive for some levels and negative for other levels of the other treatment • Diagnosis: Interactions may be found by plotting the means of each dependent variable for each combination of two or more independent variables. 10 * sample size) . o Must have a homogeneity of regression effect. 4. which may bias the results. • Covariate is a metric independent variable that is regressed on the dependent variables to eliminate its effect before using the dependent variable in the analysis. Significant disordinal interactions require the redesign of the study. Covariates are not entered into MANOVA as independent variables (factors) because they are metric and we would lose too much information if they were made categorical. • Number of covariates included in the analysis should be less than (.acceptable) and the effects of each treatment may be described. . meaning that they have equal effects on the dependent variable across groups. but not correlated with the independent variables. • Requirements for Use of a Covariate: o Must have some relationship with the dependent measures. Thus. the main effects cannot be interpreted. covariates remove effects on the dependent variable before assessing any main effects from the independent variables. • Use of covariate to either: o eliminate some systematic error outside the control of the researcher.1). The researcher must decide on the use of a covariate.(number of groups . • Nature of covariate: Highly correlated with the dependent variable. or o account for differences in the responses due to unique characteristics of the respondents. • Violation of this assumption has minimal impact if the groups are of approximately equal size. if a violation occurs. In addition. • Violation of this assumption is the most serious. • Each observation or subject's response must be independent from all others. • Equality of variance assumed across the dependent variables for each group. the researcher can combine observations with a group and analyze the group's average score instead of the scores of separate respondents. • Testing for multivariate normality: Since there is no test for multivariate normality. The variance-covariance matrices must be equal for all treatment groups.Stage 3: Assumptions of Multivariate Analysis of Variance 1. If any situation arises in which some connection is made between observations and not accounted for in the procedures. 4. significant biases can occur. • Box test may be used to test equality of covariance matrices. . A large sample size may increase the sensitivity to assumption violations. • Multivariate normality assumes that the joint effect of two variables is normally distributed. 3. the researcher may employ a covariate to account for the dependence. If violations are detected. The observations must be independent. • Nonnormal dependent variables should be transformed. The dependent variables must follow a multivariate normal distribution. the researcher should conduct univariate analyses with the idea that univariate normality is indicative of multivariate normality. 2. ratios in this area will normally perform well with MANOVA. must be linearly related and exhibit low • Nonlinear relationship: if detected among the dependent variables. there should be at least one Chi-square for each degree of freedom. • Multicollinearity among the dependent redundancy and decreases statistical efficiency. • Pillai's criterion or Wilks' lambda are the most immune to violations of the assumptions and maintain the greatest power. The level of power of a statistical test is based on the alpha level. variables indicates 6. Wilks' lambda. .• Impact of large samples: The equal variance-covariance matrices test most likely will be violated with a very large sample size. an appropriate transformation should be conducted. • As a rule of thumb. Hotellings' trace and Pillai's criterion. While this is only a rule of thumb. Stage 4: Estimation of the MANOVA Model and Assessing Overall Fit 1. 2. • Pillai's criterion is more robust if the sample size is small. Criteria to assess multivariate differences across groups: Roy's greatest characteristic root. unequal cell sizes are present. or homogeneity of covariances is violated. MANOVA is especially sensitive to outliers and their impact on Type 1 error. Dependent variables multicollinearity. the effect size of the treatment and the sample size of the groups. 5. As an alternative. while Roy's gcr is most powerful if all assumptions are met. examine F statistic and Chi square for additional information. and the Newman-Kuels test. determination of which dependent variables exhibited differences across groups. Tukey's extension of the Fisher least significant difference approach. Duncan's multiple range test. the power of the test may become too sensitive. • Assessment of the dependent variable is done through statistical tests to determine which dependent variables contribute the most to overall group differences. Stage 5: Interpretation of Results 1. If the alpha level is set too conservatively. However. • Effect size is directly related to the power of the statistical test for a given sample size. . these methods suffer from low power. the greater the power of the test (i. identifying almost any difference as significant. sample sizes greater than 150 per group do not contribute greatly to increasing the power of the test. To be considered effective. the covariate should improve the statistical power of the tests and reduce within-group variance. the more probable that the statistical test will identify a treatment's effect if it exists). This may be done by running the analysis with and without covariates. Interpreting MANOVA results involves three stages: interpretation of the covariate. Since post hoc tests examine every possible combination of groups.e. • Interpreting the Covariates by evaluating the effectiveness of including a covariate. and identification of which groups differed on the dependent variate or a single dependent variable. the greater the standardized differences between groups.• Power is inversely related to alpha. Tukey's honestly significant difference method. The larger the effect size. the power of the test may be too low for valid results to be identified. • Increasing the sample size increases the power by reducing sampling error. in very large sample sizes. In fact. The Scheffe test is most conservative with respect to type 1 error. o Post hoc tests are the most common post hoc procedures and include Scheffe's test. These tests may be post hoc tests and a priori tests. o A priori tests. Replication is the primary means of validation of MANOVA results. • Causation is based on several criteria must be met before the researcher can suggest causation. 2. The analyst specifies which group comparisons are to be made instead of testing all possible combinations. Stage 6: Validation of the Results 1. such as univariate tests. a priori tests are more powerful than post hoc tests. • Identification of differences between groups by post hoc or a priori statistical tests. Causation can never be proved. o Single dependent variable contribution assessment has the potential to inflate Type 1 error when running several consecutive a priori tests.  Context for use: A priori tests are most appropriate when the analyst has conceptual bases which support the selection of specific comparisons. . A priori comparisons should not be used as an exploratory technique. o Adjustment for potential Type 1 error inflation involves the use of the Bonferroni inequality or a stepdown analysis. Significant MANOVA results do not necessarily support causation. • Exact replication may be difficult in certain research contexts (such as survey research). Thus. • Covariate usage is dictated when the researcher is knowledgeable of characteristics of the population which may affect the dependent variables. 3) Subjects are assigned at random. In MANOVA. the set of metric variables now act as dependent variables and the objective becomes finding groups of respondents that exhibit differences on the set of dependent variables. totals for both (all) dependent variables b. Moreover. (2) DESIGN A TWO-WAY MANOVA EXPERIMENT. Discriminant analysis employs a single nonmetric variable as the dependent variable. The single nonmetric dependent variable of discriminant analysis becomes an independent variable in MANOVA. but in equal numbers to each of the cells. sums of squares for both (all) dependent variables c. Requirements for two-way MANOVA: 1) Two (or more) metric dependent variables. MANOVA and discriminant analysis are mirror images. 4) Statistics are calculated for each cell: a. The objective is to determine the independent variables that discriminate between groups. The experimental design is a 2 x 2 (n x n) matrix of independent nonmetric variables. The dependent variables in MANOVA ( a set of metric variables) are the independent variables in discriminant analysis.ANSWERS TO END-OF-CHAPTER QUESTIONS (1) WHAT ARE THE DIFFERENCES BETWEEN MANOVA AND DISCRIMINANT ANALYSIS? WHAT SITUATIONS BEST SUIT EACH MULTIVARIATE TECHNIQUE? In a way. The independent metric variables are used to form variates that maximize differently between groups formed by the dependent variable. 2) Two (or more) nonmetric experimental (treatment) variables. Use of one technique over the other primarily depends upon the research objective. sums of products of dependent variables . both use the similar methods in forming the variates and assessing statistical significance between groups. WHAT ARE THE DIFFERENT SOURCES OF VARIANCE IN YOUR EXPERIMENT? WHAT WOULD THE INTERACTION TEST TELL YOU? Answer a. 5) Marginals are computed a. • Analogous to step-wise regression in concept. (B) STEP-DOWN ANALYSIS. Answer a. (3) BESIDES THE OVERALL. c. A disadvantage in using the Scheffe' test is that it requires the use of the gcr distribution. Multiple Discriminant Analysis of the SSCP matrix • The relative importance of each independent variable can be identified by deriving correlations between each original dependent variable and the discriminant function. WHICH IS SIMILAR TO STEPWISE REGRESSION IN THAT EACH SUCCESSIVE F-STATISTIC IS COMPUTED AFTER ELIMINATING THE EFFECTS OF THE PREVIOUS DEPENDENT VARIABLES. SIGNIFICANCE.05 (or at the level specified by the researcher). THERE ARE AT LEAST THREE APPROACHES TO DOING FOLLOW-UP TESTS: (A) USE OF SCHEFFE' CONTRAST PROCEDURES. the joint effect of treatment variables in addition to the individual main effects on the dependent variable(s). . In other words. Step-Down Analysis • Similar to F-tests but allows for correlation among dependent variables. There are four sources of variance: 1) between columns (treatments) 2) between rows (factors) 3) interactions between factors and treatments 4) residual error In factorial designs (n x n) the interaction test would aid in discovering an interaction effect. then the most appropriate overall test would be the gcr-statistic in MANOVA. Scheffe' Contrast procedures • Tests for differences between groups on any dependent variable. • These procedures ensure that the probability of any Type I error across all comparisons will be held to d = . May overlook a significant dependent (independent) variable due to its high correlation with another dependent (independent) variable. OR GLOBAL. If the Scheffe' test is to be used. AND (C) EXAMINATION OF THE DISCRIMINANT FUNCTION(S). b. NAME THE PRACTICAL ADVANTAGES AND DISADVANTAGES OF EACH OF THESE APPROACHES. especially if they are correlated. Examples of the use of these techniques in these two fields may be found in the selected readings at the end of the chapter. WHAT TYPES OF UNCONTROLLED VARIABLES OR COVARIATES MIGHT BE OPERATING IN EACH OF THESE SITUATIONS? Answer There are a wide variety of applications possible in the areas of psychology and education. In the design of the study. To ensure adequate power. . the researcher should estimate the effect size and the needed sample size to achieve the desired level of power given the alpha required. Covariates might include frequency of shopping trips and readership of both newspapers. These could be controlled for after the experiment if these variables did indeed have an effect on the outcome of the test. the number of dependent variables. Similar problems of interest might occur in any discipline where experimental design is of concern. Another type of experiment might be to test the effect of a point of purchase display (present or absent) against newspaper advertising.(4) • Major areas of differences between groups can be identified. A wide variety of applications are also possible in the area of marketing. estimated effect size. One type of experiment which might be carried out in advertising research would be to test the effects of two broadcast communications media at three different times of the day on consumer knowledge and intention to buy simultaneously. The local newspaper in one city only would carry ads about the specific product. (5) DESCRIBE SOME DATA ANALYSIS SITUATIONS IN WHICH MANOVA AND MANCOVA WOULD BE APPROPRIATE IN YOUR AREAS OF INTEREST. desired alpha level. and sample size. the researcher should consider the use of as few dependent variables as possible. HOW IS STATISTICAL POWER AFFECTED BY STATISTICAL AND RESEARCH DESIGN DECISIONS? HOW WOULD YOU DESIGN A STUDY TO ENSURE ADEQUATE POWER? Answer The primary factors affecting power can be assessed prior to a study. Two cities could be selected which possess similar demographic profiles. Some stores would be selected in each city for the point of purchase displays and some selected for observation without the displays. age. Covariates in such an experiment might include sex. Dependent variables to be observed might include levels of traffic on the aisles containing the product and the proportion of purchases containing the item of interest. or education level of the respondents. CHAPTER SEVEN CONJOINT ANALYSIS The following presentation will address the basic questions of "What?," "Why?," "When?," and "How?" as applied to Conjoint Analysis. What is conjoint analysis? 1. Objective: Conjoint analysis is a decompositional dependence technique that infers the importance of attributes used by the consumer in the decision making. 2. Difference from other dependence techniques: It is in direct contrast to the other dependence techniques in that the values for the independent variables are prespecified by the researcher, who creates objects and, from the consumers' responses, infers the elements used in the thought process. 3. Difference from other decompositional techniques: This also differs from the other decompositional techniques we have discussed, multidimensional scaling (MDS), in that in MDS the objects are existing objects (products, persons, etc.) that are evaluated. 4. Type of results: Conjoint does not try to determine the dimensions upon which a decision is made, rather the dimensions are specified a priori and conjoint attempts to determine each dimension's influence (and also each level of each dimension) in the decision process. 5. Level of results: Conjoint results are obtained for each respondent in the sample. Why and when do we use conjoint analysis? Conjoint analysis is the technique of choice when the objective is any of the following: • Define the object or concept with the optimum combination of features. • Show the relative contributions of each attribute and each level of each attribute to the overall evaluation of the object. • Use estimates of purchaser or customer judgments to predict market shares among objects with differing sets of features (other things held constant). • Isolate groups of potential customers who place differing importance on the features in order to define high and low potential segments. • Identify marketing opportunities by exploring the market potential for feature combinations not currently available. How do you use conjoint analysis? Conjoint analyses follow the same six stages of all multivariate techniques discussed in the text. The following are important points for each step of the design process Stage 1: The Objectives of Conjoint Analysis 1. Conjoint Analysis has two primary objectives. • Determine attributes' contribution to determining consumer preferences. • Create a valid model of consumer preference judgments which will predict consumer acceptance of any combination of attributes. 2. The total worth of the object defines the specificity of the model. • Objects are defined in terms of factors and levels, much like in an experiment, to define stimuli or treatments. o Factors – represent specific attributes or characteristics that impact utility for the object. o Levels – the possible values for each factor. For example, if the factor is price, then levels may be 50 cents, 75 cents or 99 cents. o Stimulus – a combination of levels (one for each factor) that define an object which will be evaluated by a respondent. Also known as a treatment. • All positive and negative factors/attributes which impact (add to or detract from) the overall worth of the product / service should be included in the model. • Limited to making statements pertaining to the variables and levels used in the analysis. We cannot interpolate between variables or levels of variables. In its most general form of part-worth utilities, the conjoint procedure uses categorical relationships between variables so there is no assumption of a linear relationship. • Assumption is that the model contains all the needed dimensions to make the choice (i.e., inclusion of all determinant attributes), so the researcher must ensure that the specified attributes define the total worth of the products. 3. Determinant factors can be specified and are limited to what we specify as the basis for decision. • Implicit is that the researcher can specify the dimensions or variables upon which a decision is based, and even further that the dimensions we specify are the only dimensions used. • Conjoint analysis requires some a priori basis for selection of variables. The justification may be theoretical or derived from other research, such as a survey to determine the appropriate variables to include. Stage 2: The Design of a Conjoint Analysis 1. Three conjoint methodologies are available. • Traditional conjoint analysis – characterized by simple additive model with up to nine factors/attributes • Adaptive conjoint method – designed to accommodate up to 30 factors and typically done through a computerized process that generates stimuli shown to the respondent • Choice-based approach – presents sets of stimuli (choice sets) to the respondent for choice of one stimuli from the set. Limited in number of factors to be included (generally up to 6 factors), but considered more realistic 2. Factors must be actionable and communicable. • Factors must be precise and perceptually distinct. In other words, the variables must be singular, concrete attributes that illicit the same interpretation from all respondents. • Descriptions of factors must be easily understood by respondents. The dimensions upon which choice is made should be stated in very tangible terms. Factors must be capable of being verbalized or written in order to be operationalized in conjoint analysis. 3. Dimensionality and number of attributes. • Unidimensionality is required of all variables used in conjoint analysis. Variables that are multidimensional may lead to interpretation problems. One respondent may respond to the variable with low importance, while another considers the same variable with high importance because the two respondents were considering two different dimensions of the same variable. • The number of variables used in the analysis must balance the needs of complexity and specificity. Including too many variables results in too complex a design, which requires consumers to make endless hypothetical preference judgments and researchers to complete intricate analyses. However, too few variables in the design will not provide the level of specificity needed to bring any validity to the model. 4. Number and range of levels. • Reasonability: The number of levels for each variable should reflect the most reasonable levels expected by the consumer. • Believability: Levels outside the range of believability only weaken the model and provide spurious results. • Balance in number of levels: The researcher should balance the number of levels across variables. Unequal levels across variables may adversely impact the consumer's perception of relative importance of the variables. • Complexity of design: As you increase the number of variables and the levels of each variable, you will rapidly reach a very complex design. The researcher should use only the necessary variables and required levels for each variable. 5. Attributes should not exhibit high levels of interattribute correlation. This may impact orthogonality of stimuli design as well as design efficiency.g. o construct the set of stimuli to exclude the unacceptable stimuli. typically known as prohibited pairs. such a choice would produce less efficient and less reliable estimates. The researcher must consider the trade-offs of selecting a type of relationship that is most like the preference formations of consumers and of producing reliable estimates. 6. the independent variables should not have any substantial degree of collinearity. When a conjoint analysis is to be performed the researcher must establish a priori the choice rule to be used for the entire group. • Interattribute correlation has the tendency to result in unbelievable or otherwise unacceptable stimuli. . miles per gallon and acceleration are combined in new attribute terms performance). • The solutions are: o eliminate one of the attributes. The researcher must determine how the levels of a factor are related. This type of model is widely used in consumer research and simplifies the implementation process. but this impacts validity when deleted attributes impact utility of object o create "super attributes" that are combinations of the correlated variables (e. o constrain estimation process to be sure part-worths correspond to prespecified relationship.• Just as found in other multivariate techniques. • Each type of relationship can be specified separately. • All consumers are assumed to use the same choice rule. Conjoint does not make allowances for multiple choice rules (composition rules) in the same data set. however. 7. • Additive model with no interactions is normally assumed. The researcher must determine whether an additive or an interactive composition rule is appropriate. Collinearity typically results from the basic character of the variable itself.. not from the levels of a variable. This design is many times impractical.8. all the most preferable) which are less useful in choice decisions o Violations of constraints of combinations of levels 10. • Dependent on choice of presentation method. which can be the result of: o Unbelievable combinations of levels o Obvious combinations (e. popular forms of presentation are: The three most • Full Profile: By far the most widely used. 9. o Trade-off method: employs only ranking data o Pairwise comparison method and the full-profile method: either rating or ranking. This design is used most often and is the chosen method for researchers interested in larger numbers of variables and levels. It has the advantage of being more representative of the actual decisionmaking process followed by respondents. • Pairwise comparison: Pairwise comparison of profiles with a complete or reduced set of attributes allows the respondent to make simple judgments about the profiles. unless the researcher is interested in a very small number of variables and levels. While attempting to approximate the consumer's true nature. the researcher must define the set of stimuli to present to the respondent. • The stimuli should always be examined for the presence of unacceptable stimuli. it too often presents an artificial decision-making context. each derived from a combination of a level from each specified independent variable.. The number of stimuli is dependent on the composition rule. full profile presents to the respondent a series of hypothetical objects. The researcher must select a measure of preference. The two options are: • Factorial design: all combinations of levels are utilized. The researcher must select a presentation method. In data collection. • Fractional factorial design: a subset of the combinations are employed. .g. • Trade-off: Trade-off considers each pair of attributes and asks respondents to indicate the preference order for each combination of attribute levels. • Metric ratings data can be analyzed by ordinary least squares approaches. • Conjoint analysis is extremely dependent on the conceptual assumptions underlying the design. mail or telephone with the proper planning and technical support. Stage 3: Assumptions of Conjoint Analysis 1. Theory driven design. 3. Most of the tests performed in other multivariate techniques are unnecessary. • Rank order data requires the use of a modified analysis of variance technique which is designed for ordinal data. • Primary advantage is that model specifications that must be estimated at only the aggregate level with traditional techniques can now be estimated at individual level. The researcher must choose a means to administer the stimuli. • Goodness of fit involves comparison of actual preference measures (rankings or ratings) with predicted values from estimated model. Testing of Statistical Assumptions • Least restrictive of the dependence techniques.11. 2. Estimation technique must be appropriate for the type of data collected. • Administration: successfully performed by person. Stage 4: Estimating the Conjoint Model and Assessing Overall Fit 1. . Traditional estimation techniques (MONANOVA for rankings and regression-based methods for metric ratings) are being supplemented by Bayesian estimation. 2. Should assess the overall fit of the model at both the individual and the aggregate levels. or . • The results of conjoint analysis provide information pertaining to each factor as a whole and to each level of each factor. Can also be combined across individuals for aggregate assessment. • Validation profiles: In order to test for overfitting of the model. Analysis begins with a comparison of each variable (factor or attribute) and then an examination of the levels of each variable. researchers should always plan for a validation sample of stimuli. • Assess theoretical consistency and avoid reversals – patterns of part-worths for a factor which are contrary to acceptable relationships. the researcher employs more stimuli than necessary to fit the model and uses the extra data to test model accuracy. To do so. o Any number of issues contribute to reversals  Inadequate respondent effort  Data collection failures  Research context limits o Identifying reversals requires researcher judgment as no absolute criteria.  Apply constraints – estimation techniques allow for specifications in allowable part-worth patterns. A comparison between factors may be performed and then an examination of the levels of each factor allows for understanding of relative influences. Stage 5: Interpreting the Results 1. Usually involves examination of part-worth patterns for each respondent o Typically.• Individual or aggregate level assessments: The correlations between a person's actual response and his / her predicted response should be tested for statistical significance. especially if aggregate results are primary interest. one of three remedies for reversals are used:  Do nothing – small number of reversals can be ignored. Examples such as higher preference for lower quality objects or less convenient stores. o Part-worth utilities are expressed as differences from the overall average utility (similar in concept to the intercept term in regression analysis). 2. 3. • Importance of variable/factor: For each respondent. Delete respondents – identify and delete respondents with inappropriate or large number of reversals. The researcher should internally and externally validate conjoint analysis results. o Percentage value based on a range of zero to 100 percent. Unless the researcher has reason to assume the population is homogeneous with respect to the factors being measured. When summed. A large negative value would mean that this level of the variable was associated with lower levels of preference. conjoint analysis determines an importance value for each variable used in the analysis. 1. o Expressed in raw form (utility) with a sign indicating the relationship (positive or negative) with the dependence variable. • Importance of level: There is also a utility value for each level of each variable. . • Level of analysis: All of the measures mentioned above are provided for each respondent in the sample (disaggregate) and also for the sample as a whole (aggregate). disaggregate analysis is most appropriate. disaggregate analysis should be used to interpret conjoint results. interpret the relative importance of each attribute and each level of each attribute. while a positive value increases Stage 6: Validation of the Conjoint Results preference. the importance values for all variables will total 100 percent. In most cases. Next. providing a measure of the influence of each level of each variable. • Choice simulators enable the researcher to predict consumer response to market questions. The three most common applications of conjoint results are: • Segmentation is the grouping of individuals with similar part-worths or importance values. 1. varying in both the type and number of products in a set. For example. . The sample should always be evaluated for population representation. the market shares among any set of products can be estimated. • A marginal profitability analysis aids in the product design process by predicting the viability of each hypothetical product. Any number of product sets can be evaluated. the researcher provides the market stimuli and the simulator predicts consumer response.• Internal validation: the researcher confirms that the choice of composition rule is most appropriate. • External validation: corresponds to a test of sample representativeness. In any application. This is usually completed in a pretest. given the cost of each product and its expected market share and sales volume. There are two alternative conjoint methodologies that have been developed to supplement the traditional methodology in specific research situations. . This approximates realistic choice situations for respondents and generally provides greater respondent involvement. Choice set design must balance not only among factors and levels within factors. All of the basic issues discussed impacting the traditional methodology applies to the alternative methodologies. primarily due to the realistic choice context for respondents – choosing from among a set of stimuli rather than rating or ranking individual stimuli. 2. Many conjoint analyses may need to incorporate more that the smaller number of factors (up to 10) that the traditional methods reliably accommodate. What varies is the type of stimuli presentation and decision process that the respondents partake in. Choice-based conjoint has quickly become one of the most widely used conjoint methodologies. As a result. thus limiting “pencil and paper” administration 3. • Adaptive techniques have gained widespread acceptance due to ease of handling large number of factors while reducing the required respondent effort through simplified stimuli designs and presentation. • A primary element in a choice-based design is the choice set which represents a set of full-profile stimuli from which the respondents choose one (or None in some instances). but among stimuli as well.Alternative Conjoint Methodologies 1. Most often associated with Sawtooth Technologies Adaptive Conjoint Analysis (ACA) has potential limitation in that must b administered interactively with respondents. the adaptive or self-explicated methods have been developed to allow for considerably more factors (up to 30 or so) in a conjoint analysis • Self-explicated models allow respondents to provide direct evaluations of attribute/factor importance as well as part-worth utilities which can then be incorporated into the estimation process. Adaptive/Self-Explicated Conjoint Approaches are most suited for dealing with large numbers of factors. Has been found to have acceptable predictive validity but is inconsistent in some regards since respondent may never complete a choice task. choice-based analyses were typically estimated only at the aggregate level since each respondent could not provide enough responses for individual level partworth estimates. • All of the issues discussed earlier in terms of factor and level characteristics and design consideration are still applicable. by any researcher. • Until the recent development of Bayesian estimation.• Given the comparison task for the respondent. typically no more than six attributes are included in the analysis. • Widespread adoption has increased the availability of software for choice-based designs. Now individual and aggregate estimates are possible. along with the need for validation tasks independent from the estimation procedure. which now can be completely designed and estimated. . even with Bayesian methods. Illustrations: a. describing the thickness of a hand lotion in terms of viscosity does not help the average respondent. b. HOW DIFFICULT WAS IT FOR RESPONDENTS TO HANDLE THE WORDY AND SLIGHTLY ABSTRACT CONCEPTS THEY WERE ASKED TO EVALUATE? HOW WOULD YOU IMPROVE ON THE DESCRIPTIONS OF THE FACTORS OR LEVELS? WHICH PRESENTATION METHOD WAS EASIER FOR THE RESPONDENTS? Answer Students will have a difficult time with abstract notions presented as general concepts.ANSWERS TO END-OF-CHAPTER QUESTIONS (1) ASK THREE OF YOUR CLASSMATES TO EVALUATE CHOICE COMBINATIONS BASED ON THESE VARIABLES AND ON LEVELS RELATIVE TO THE CHOICE OF A TEXTBOOK FOR A CLASS AND SPECIFY THE COMPOSITIONAL RULE YOU THINK THEY WILL USE. Each chapter contains graphics to illustrate the numeric issues. the three levels are not really levels. b. Depth: a. The pairwise design is sometimes seen as easier to evaluate simply because the information on each concept pair is easier to process. Each chapter includes humorous pictures. A final concern: just because levels of a factor are discrete and precise does not mean the respondent will always be capable of dealing with the information. Illustrative topics are presented. c. For example. Each chapter includes specific references for topics covered. Introduces each subject in a general overview.g. Students must be careful that levels of an attribute are truly discrete levels. Goes into great depth on each subject. References: a. Some people try to quantify abstract concepts only to further confuse the respondent. General references are included at the end of the textbook. e. b. You can point out that for the factor: Illustrations. A goal should be to make the concepts easy to perceive regardless of which design is used. a book could have both humorous illustrations and graphics to illustrate numeric topics. . You can just let them pour the lotion at different viscosities without defining a quantification for thickness. with some respondents. when stress is not 0 and regression is not 1. lack of attention or consistent evaluation procedures produces poor fits that cannot be explained as interaction. An example follows for a 3 level factor with a 2 level (with one other 2 level factor as in problem 1). When stress=0 or R-square = 1. A crude but effective way to look for interactions is the method shown in the text. No Interaction A1 A2 B1 B2 B3 1+4=5 2+5=7 3+6=9 7+10=17 8+11=19 9+12=21 B1 B2 B3 Interaction A1 A2 1+4=5 2+5=7 3+6=9 9+12=21 8+11=19 7+10=17 Just looking at the linear component shows: 5+21=9+17 and 5+17<9+21 So there is likely only a simple interaction between A & B. EXAMINE THE INTERACTIONS. ANALYZE THE DATA FROM THE PRECEDING EXPERIMENT. Unfortunately.(2) USING EITHER THE DIFFERENCES MODEL OR A CONJOINT PROGRAM. Answer You must look for interactions on a respondent by respondent basis. you need not look. . it does not mean that you necessarily have an interaction. However. USE AT LEAST FIVE RESPONDENTS TO SUPPORT YOUR LOGIC. the product must be viewed as comprised of separate attributes and not really valued by "the whole is greater than the sum of its parts" axiom. since other attributes are not easily accommodated in either of the presentation methods. The linear model is a good starting point to suggest the direction for augmentation (which may not be obvious before the initial linear experiment). multiplicative. Moreover. If inspection of the data from a design based on a linear model suggests another decision model was used by the respondent. it lends itself to classical experimental designs for administration and interpretations. . IN DOING SO. THE EXPERIMENTAL DESIGN FOR CREATING STIMULI. it is usually easier to augment the original design rather than start over. If the student uses a scale for obtaining evaluations in the experiment. For those respondents for whom the model fits. Moreover. • Conjoint analysis is best suited to examining the choice of hypothetical objects which have easily quantifiable characteristics. DEFINE THE COMPOSITIONAL RULE YOU WILL USE. Perhaps more limiting is the fact that only tangible and easily communicated attributes are feasible. such that a choice object must be characterized on a small number of dimensions. In addition to its naive simplicity. AND THE ANALYSIS METHOD. (4) WHAT ARE THE PRACTICAL LIMITS OF CONJOINT ANALYSIS IN TERMS OF VARIABLES OR TYPES OF VALUES FOR EACH VARIABLE? WHAT TYPE OF CHOICE PROBLEMS ARE BEST SUITED TO ANALYSIS WITH CONJOINT ANALYSIS? WHICH ARE LEAST WELL SERVED BY THE USE OF CONJOINT ANALYSIS? Answer • Conjoint analysis is limited in terms of both the type and number of attributes that can be used to describe the choice objects.) but will typically find that the simple linear additive model gives a good starting point. the analysis task equates to just classifying the respondents into the appropriate pattern of coefficients.(3) DESIGN A CONJOINT ANALYSIS EXPERIMENT WITH AT LEAST FOUR VARIABLES AND TWO LEVELS OF EACH VARIABLE THAT IS APPROPRIATE TO A MARKETING DECISION. etc. Answer The student can use any number of rules found in the literature (threshold. The student should quickly see (as pointed out in the example problems in this manual) that with rank order data. the number of solutions are bounded and easily estimated. then the rank order assumptions of MANOVA are not necessary. as the data can be assumed to not clearly represent only order of choice. the number of attributes is usually limited to less than ten. sensory-based attributes or "images" which convey an emotional appeal). A choice-based conjoint method employs a unique form of presenting stimuli in sets rather than one-by-one.g. Choice of a method should be made based on the number of factors and the need to represent interaction effects. (2) level of analysis and (3) the permitted model form.. . The adaptive conjoint method.• It is ill-suited to examine existing objects (since it is hard to describe them in simple terms) and objects which have intangible attributes (e. can accommodate up to 30 factors for each individual. HOW WOULD YOU ADVISE A MARKET RESEARCHER TO CHOOSE AMONG THE THREE TYPES OF CONJOINT METHODOLOGIES? WHAT ARE THE MOST IMPORTANT ISSUES TO CONSIDER. Traditional conjoint analysis is characterized by a simple additive model containing up to nine factors for each individual. It also differs in that it directly includes interaction and must be estimated at the aggregate level. also an additive model. ALONG WITH EACH METHODOLOGY’S STRENGTHS AND WEAKNESSES? The choice of a conjoint methodology revolves around three basic characteristics of the proposed research: (1) the number attributes. 5. " and "How?" as applied to Cluster Analysis. As such. Why do we use cluster analysis? 1. understandable description of the observations with minimal loss of information. The result is a more concise. 3. • Hypotheses development: enables development of hypotheses about the nature of the data or examination of previously stated hypotheses. The result of cluster analysis is a number of heterogeneous groups with homogeneous contents. Cluster analysis classifies objects or variables on the basis of the similarity of the characteristics they possess. or consumption pattern groups." "When?. or classified into similar demographic. What is cluster analysis? 1. 2. smaller subgroups. ." "Why?. Cluster analysis seeks to minimize within-group variance and maximize between-group variance.CHAPTER EIGHT CLUSTER ANALYSIS The following presentation will address the basic questions of "What?. Cluster analysis is the technique of choice when the objective is one of the following: • Data reduction: reduces the information from an entire population or sample to information about specific. When do you use cluster analysis? Cluster analysis is used any time the analyst wishes to group individuals or objects. • Classification: sample respondents may be profiled. cluster analysis is currently used in a wide variety of disciplines. including the hard and soft sciences. psychographic. • A graphic profile diagram may be used to identify outliers. While the primary objective of cluster analysis is to partition a set of objects into two or more groups based on the similarity of the objects on a set of specified characteristics. • Irrelevant variables will have a substantive detrimental effect on the results. • Outliers should be assessed for their representativeness of the population and deleted if they are unrepresentative. Results from cluster analysis are only as good as the variables included in the analysis. other uses of cluster analysis include: • exploratory analysis to develop a classification system. 2. Stage 1: Objectives of Cluster Analysis 1. • Each variable should have a specific reason for being included. . the researcher should conduct a preliminary screening of the data. Cluster analysis is very sensitive to outliers in the dataset therefore. and • generating hypotheses and confirmatory analysis to test a proposed structure. Stage 2: Research Design in Cluster Analysis 1.How do you use cluster analysis? Cluster analyses follow the six stages of all multivariate techniques discussed previously. • The variable should be excluded if the researcher cannot identify why it should be included in the analysis. • Outliers are either observations which are truly nonrepresentative of the population or observations which are representative of an undersampling of an actual group in the population. 1. by classifying as similar those cases which are close to each other. Multicollinearity among the variables may have adverse effects on the analysis. thereby receiving improper emphasis in the analysis. and thus are rarely used. nonmetric. • Multicollinearity causes the related variables to be weighted more heavily. The researcher must specify the interobject similarity measure and the characteristics which will define similarity among the objects clustered. only the patterns. Data may be metric. • Outliers which are not representative of the population should be deleted. is the most commonly used measure. These measures focus on the magnitude of the values. compensating for intercorrelation among the variables. or a combination of both. But note that the use of a combination of data types will make the interpretation of the cluster analysis very tentative. Cluster analysis assumes that the sample is truly representative of the population. o Presence of intercorrelation among clustering variables: Preferred measure — Mahalanobis distance.3. which is the length of the hypotenuse of a right triangle formed between the points. These measures do not consider the magnitude of the variable values. . Often simple association measures are used to determine the degree of agreement or disagreement between a pair of cases. • Association measures are used to represent similarity among objects measured by nonmetric terms (nominal or ordinal measurement). 3. • Correlational measures represent similarity by the analyzing patterns across the variables. • All scales of measurement may be used. The researcher should be cautious interpreting these conditions. • Distance measures represent similarity as the proximity of observations to each other across the variables. o Euclidean distance. 2. o Standardization: Used when the range or scale of one variable is much larger or different from the range of others. which standardizes the data and also sums the pooled within-group variance-covariance matrices. 2. • Cluster analysis cannot confirm the validity of these groupings. Hierarchical clustering has two approaches: agglomerative or divisive methods. Naturally-occurring groups must be present in the data.• One or more of the highly collinear variables should be deleted or use a distance measure. • Divisive method begins with one large cluster and proceeds to split into smaller clusters items that are the most dissimilar. This role must be performed by the researcher by: o ensuring that theoretical justification exists for the cluster analysis. Stage 4: Deriving Clusters and Assessing Overall Fit • Agglomerative clustering starts with each observation as a cluster and with each step combines observations to form clusters until there is only one large cluster. There are five measures of forming clusters in hierarchical clustering: • Single linkage — based on the shortest distance between objects • Complete linkage — based on the maximum distance between objects • Average linkage — based on the average distance between objects • Ward's method — based on the sum of squares between the two clusters summed over all variables . • Cluster analysis assumes that partitions of observations in mutually exclusive groupings do exist in the sample and population. which compensates for this correlation. 4. such as Mahalanobis distance. and o performing follow-up procedures of profiling and discriminating among groups 1. o Coefficient size indicates the homogeneity of objects being merged. it is suggested that both hierarchical and nonhierarchial clustering algorithms be used. The centroid method requires metric data and is the only method to do so. In this case. The actual values will depend on the clustering method and measure of similarity used. 3. • First stage — a hierarchical cluster analysis is used to generate and profile the clusters. • Second stage — a nonhierarchical cluster analysis is used to finetune the cluster membership with its switching ability. There is no generally accepted procedure for determining the number of clusters to extract. Nonhierarchical clustering assigns all objects within a set distance of the cluster seed to that cluster instead of the tree-building process of hierarchical clustering. Some of the most common methods used include the following: • clustering coefficient — a measure of the distance between two objects being combined. 5. While there is no set rule as to which type of clustering to use.• Centroid method — based on the distance between cluster centroids. This decision should be guided by theory and the practicality of the results. Nonhierarchical clustering has three approaches: • sequential threshold — based on one cluster seed at a time selected and membership in that cluster fulfilled before another seed is selected • parallel threshold — based on simultaneous cluster seed selection and membership threshold distance adjusted to include more or fewer objects in the clusters • optimizing — same as the others except it allows for membership reassignment of objects to another cluster based on some optimizing criterion 4. Small coefficient indicates fairly homogeneous objects are being . Several items in the output are available to help the analyst determine how many clusters to extract. the centroids from hierarchical clustering are used as the seeds for nonhierarchical clustering. while a large coefficient is the result of very different objects being combined. The blanks represent clusters and the X's indicate the members per cluster. 7.merged. Respecification may be needed if widely varying cluster sizes or clusters with only one to two observations are found. As the lines joining clusters become longer. o Large increase (absolute or percentage) in the clustering coefficient is an indication of the joining of two diverse clusters. • dendrogram — pictorial representation of the clustering process which identifies how the observations are combined into each cluster. examine the structure of each cluster and determine whether or not the solution should be respecified. Cluster centroids on each variable are a common basis of interpretation. • Statistical tests (F statistic and significance level of each variable) are provided to denote significant differences across the clusters. These scores may be used to assign labels to the clusters. 6. • Profiling of the clusters may be computed with discriminant analysis. • vertical icicle — pictorially represents the number of objects across the top and the number of clusters down the side. . Only significant variables should be considered in interpreting and labeling the clusters. by utilizing those variables which were not used in the cluster analysis. Stage 5: Interpretation of the Clusters 1. The solution's appropriateness must be confirmed with additional analyses. which denotes that a possible "natural grouping" existed before the clusters were joined. the clusters are becoming increasingly more dissimilar. o Researcher must then examine the possible solutions identified from the results and select one as best supportive of the research objectives. When cluster solution is reached. • Cluster centroids represent the average score for each group. This then becomes one potential cluster solution. separate sample is cluster analyzed and compared. Among the available methods are: • New. Validation involves analyzing the cluster solution for representativeness of the population and for generalizability. Profiling involves assessing how each cluster differs from the other clusters on relevant descriptive dimensions. or consumption patterns. • Obtain cluster centers from one group and use them with the other groups to define clusters. . 3. Predictive or criterion validity of the clusters may be tested by selecting a criterion variable that is not used in the cluster analysis and testing for its expected variability across clusters. • Only variables not used in the cluster analysis are used in profiling. 2. variables used in this step are demographics. • Split the sample into two groups and cluster analyze each separately. • Discriminant analysis is technique often used.Stage 6: Validation and Profiling of the Clusters 1. Often. psychographics. ANSWERS TO END-OF-CHAPTER QUESTIONS (1) WHAT ARE THE BASIC STAGES IN THE APPLICATION OF CLUSTER ANALYSIS? Answer • Partitioning . Cluster analysis identifies and classifies objects or variables so that each object is very similar to others in its cluster with respect to some predetermined selection criteria. As you may recall. Factor analytic approaches to clustering respondents are based on the intercorrelations between the means and standard deviations of the respondents resulting in groups of individuals demonstrating a similar response pattern on the variables included in the analysis.the process of understanding the characteristics of each cluster and developing a name or label that appropriately defines its nature. In a typical cluster analysis approach. factor analysis is also a data reduction technique and can be used to combine or condense large numbers of people into distinctly different groups within a larger population (Q factor analysis). .stage involving a description of the characteristics of each cluster to explain how they may differ on relevant dimensions. • Profiling .the process of determining if and how clusters may be developed. • Interpretation . groupings are devised based on a distance measure between the respondent's scores on the variables being analyzed. (2) WHAT IS THE PURPOSE OF CLUSTER ANALYSIS AND WHEN SHOULD IT BE USED INSTEAD OF FACTOR ANALYSIS? Answer Cluster analysis is a data reduction technique that’s primary purpose is to identify similar entities from the characteristics they possess. Cluster analysis should then be employed when the researcher is interested in grouping respondents based on their similarity/dissimilarity on the variables being analyzed rather than obtaining clusters of individuals who have similar response patterns. . error disturbances of the distance measure. but they can be misleading because undesirable early combinations may persist throughout the analysis and lead to artificial results. If the analyst is concerned with the cost of the analysis and has an a priori knowledge as to initial starting values or number of clusters.(3) WHAT SHOULD THE RESEARCHER CONSIDER WHEN SELECTING A SIMILARITY MEASURE TO USE IN CLUSTER ANALYSIS? Answer The analyst should remember that in most situations. However. Finally. Also. one should standardize the data before performing the cluster analysis. objective. Hierarchical procedures do have the advantage of being fast and taking less computer time. and theoretically sound approach can be developed to select the seeds or leaders. If a practical. then a hierarchical method should be employed. and the choice of a distance measure. and it is advisable to use several measures and compare the results to theoretical or known patterns. the K-means procedure appears to be more robust than any of the hierarchical methods with respect to the presence of outliers. To reduce this possibility. the Mahalanobis distance measure is likely to be the most appropriate because it adjusts for intercorrelations and weighs all variables equally. different distance measures lead to different cluster solutions. (4) HOW DOES THE RESEARCHER KNOW WHETHER TO USE HIERARCHICAL OR NONHIERARCHICAL CLUSTER TECHNIQUES? UNDER WHICH CONDITIONS WOULD EACH APPROACH BE USED? Answer The choice of a hierarchical or nonhierarchical technique often depends on the research problem at hand. then a nonhierarchical method can be used. the analyst may wish to cluster analyze the data several times after deleting problem observations or outlines. In the past. The choice of the clustering algorithm and solution characteristics appears to be critical to the successful use of CA. when the variables are intercorrelated (either positively or negatively). hierarchical clustering techniques were more popular with Ward's method and average linkage being probably the best available. when the variables have different units. (5) HOW CAN YOU DECIDE HOW MANY CLUSTERS TO HAVE IN YOUR SOLUTION? Answer Although no standard objective selection procedure exists for determining the number of clusters. pp. In the final analysis. Also. Profile analysis focuses on describing not what directly determines the clusters but the characteristics of the clusters after they are identified. the analyst may use the distances between clusters at successive steps as a guideline. or theoretical foundation. The emphasis is on the characteristics that differ significantly across the clusters. common sense. (6) WHAT IS THE DIFFERENCE BETWEEN THE INTERPRETATION STAGE AND THE PROFILING STAGE? Answer The interpretation stage involves examining the statements that were used to develop the clusters in order to name or assign a label that accurately describes the nature of the clusters. "Cluster Analysis in Marketing Research: Review and Suggestions for Application. the analyst may choose to stop when this distance exceeds a specified value or when the successive distances between steps make a sudden jump. Punj. it is probably best to compute solutions for several different numbers of clusters and then to decide among the alternative solutions based upon a priori criteria. 134-148. some intuitive conceptual or theoretical relationship may suggest a natural number of clusters. and in fact could be used to predict membership in a particular attitude cluster. obtain centroids. Girish and David Stewart. 20 (May 1983). The profiling stage involves describing the characteristics of each cluster in order to explain how they may differ on relevant dimensions.Punj and Stewart (1983) suggest a two-stage procedure to deal with the problem of selecting initial starting values and clusters. practical judgment. however. use an iterative partitioning algorithm using cluster centroids of preliminary analysis as starting points (excluding outliers) to obtain a final solution. . and eliminate outliers. The first step entails using one of the hierarchical methods to obtain a first approximation of a solution." Journal of Marketing Research. Then select candidate number of clusters based on the initial cluster solution. In using this method. Finally. nested groupings.(7) HOW DO RESEARCHERS USE THE GRAPHICAL PORTRAYALS OF THE CLUSTER PROCEDURE? Answer The hierarchical clustering process may be represented graphically in several ways. or a dendogram. Specifically. . the graphics might provide additional information about the number of clusters that should be formed as well as information about outlier values that resist joining a group. a vertical icicle diagram. The researcher would use these graphical portrayals to better understand the nature of the clustering process. it employs a global measure of evaluation. Two objectives of the visual display: • Portrayal of the perceptual dimensions used by the respondents when evaluating the stimuli. In this manner it "decomposes" the overall evaluation into dimensions. In many cases identification can be made." and "How?" as applied to multidimensional scaling. • This strength also gives rise to its primary disadvantage: the inability to precisely define the perceptual dimensions of evaluation. 3. From this. What is multidimensional scaling? 1. 2. MDS does not require the specification of the attributes used in evaluation. such as similarity among objects. Rather." "Why?. • Assessment of individual objects for their perceptual location and their relative location to other objects. Why do we use multidimensional scaling? • Its primary strength is its decompositional nature. we have a better understanding of the similarities and dissimilarities between objective and perceptual dimensions. The researcher must attempt to identify these dimensions with additional analyses." "When?.CHAPTER NINE MULTIDIMENSIONAL SCALING The following presentation will address the basic questions of "What?. and then infers the dimensions of evaluation that constitute the overall evaluation. but it is possible that MDS will result in perceptual dimensions that cannot be identified in terms of existing attributes. Multidimensional scaling (MDS) is essentially an exploratory technique designed to identify the evaluative dimensions employed by respondents and represent the respondents' perceptions of objects spatially. These visual representations are referred to as spatial maps. . 2. How do you use multidimensional scaling? Multidimensional scaling analyses follow a six stage model building perspective. • Selection of objects — All relevant objects must be included in the analysis. and choice of individual or group level analysis. Stage 1: Objectives of Multidimensional Scaling Analysis 1. Multidimensional scaling has two primary objectives: • Identify unrecognized dimensions affecting behavior. Multidimensional scaling is defined through three decisions: selection of the objects to be evaluated.When do you use multidimensional scaling? Multidimensional scaling is best used as an exploratory tool in identifying the perceptual dimensions used in the evaluation of a set of objects. . • Aggregate versus disaggregate analysis — the researcher must decide whether he or she is interested in producing output on a per subject basis or on a group basis. choice of similarity or preference data. Relevancy is determined by the research questions. • Choice of similarity or preference data — the researcher must evaluate the research question and decide whether he or she is interested in respondents' evaluations of how similar one object is to another or of how a respondent feels (like / dislike) about one object compared to another. The omission of relevant objects or the inclusion of irrelevant objects will greatly influence the results. Stage 1: Objectives of Multidimensional Scaling • Obtain comparative evaluations of objects when the specific bases of comparison are unknown or indefinable. Its use of only global judgments and ability to be "attribute-free" provide the researcher with an analytical tool minimizing the potential bias from improper specification of the attributes characteristic of the objects. . they are not required to detail the attributes used in evaluation.  Solutions require substantial researcher judgment.  Each respondent gives a full assessment of similarities among all objects. or a global measure of similarity. o Disadvantages:  Research has no objective basis provided by the respondent which identifies the dimensions of evaluation. o Advantages:  Explicit descriptions of the dimensions underlying the perceptual space. therefore maps can be developed for each respondent or aggregated to a composite map. and then derives spatial positions in multidimensional space to reflect these perceptions. and derives evaluative dimensions for object positioning. o Recommendation: disaggregate method of analysis due to Stage 2: Research Design of Multidimensional Scaling Analysis representation problems in attempting to combine respondents for aggregate analysis." o Disaggregate analysis examines each respondent separately. with two approaches available: the decompositional (attribute-free) and compositional (attribute-based) approaches. Assessing similarity is the most fundamental decision in perceptual mapping. resulting in an analysis of the "average respondent. o Advantages:  Requires only that respondents give overall perceptions of objects. • Compositional: measures an impression or evaluation for each combination of specific attributes. • Decompositional: measures the overall impression or evaluation of an object.o Aggregate analysis creates a single map for the group. Stage 2: Research Design of Multidimensional Scaling 1. creating a separate perceptual map for each respondent. combines the set of specified attributes in a linear combination.  Able to represent both attributes and objects on a single perceptual map.  Results are not available for the individual respondent. • Input measures of similarity may be metric or nonmetric. meaning that the researcher's choice is dependent mostly on the preferred mode of data collection. 3.  The research must assume some method of combining these attributes to represent overall similarity. Thus. Multidimensional scaling produces metric output for both metric and nonmetric input. Just asking respondents for comparative responses between objects does not mean that underlying evaluative dimensions exist. The number of objects must be determined while balancing two issues: a greater number of objects to ensure adequate information for higher dimensional solutions versus the increased effort demanded of the respondent as the number of objects increases. • Violating the rule of thumb: having less than the suggested number of objects for a given dimensionality causes an inflated estimate of fit and may adversely impact validity of the resulting perceptual maps. • Rule of thumb: more than four objects for each derived evaluative dimension. 5. 2. The analyst must ensure that the objects selected for analysis do have some basis of comparison. 4. Choice of using either similarity or preference data based on research objectives. at least five objects are required for a one dimensional perceptual map. This chosen method may or may not represent the respondent's thinking. The results from both types are very similar.  The data collection effort is substantial. o Disadvantages:  The similarity between objects is limited to only the attributes which are rated by the respondents. . o Data collection modes:  direct ranking — objects ranked from most preferred to least preferred  paired comparisons (when presented with all possible pair combinations. o Nature of preference data: arrange the stimuli in terms of dominance relationships. Stage 3: Assumptions of Multidimensional Scaling Analysis 1. but in each instance the analyst must assume the inference can be made between attributes and preference without being directly assessed. the analyst must be able to assume that all pairs of stimuli may be compared by the respondents. All consumers will not use the same dimensions for evaluating stimuli.• Similarity data — represent perceptions of attribute similarities of the specified objects. o Comparability of objects: Since similar data investigate the question of which stimuli are the most similar and which are the most dissimilar. Not all respondents will perceive a stimulus to have the same dimensionality. . A dimension of the stimuli that one person feels is quite important may be of little consequence to another. since we are not able to demonstrate the correspondence among attributes and choice. but do not offer any direct insights into the determinants of choice among the objects. o Three procedures for data collection:  comparison of paired objects — rank or rate similarity of all object pair combinations  confusion data — subjective clustering of objects  derived measures — scores given to stimuli by respondents • Preference data — reflect the preference order among the set of objects. but are not directly related to attributes. The stimuli are ordered according to the preference for some property of the stimuli. the most preferred object in each is chosen) • Combination approach: Methods are available for combining the two approaches. Stage 4: Deriving the MDS Solution and Assessing Overall Fit NOTE: Given the wide number of techniques encompassed under the general technique of perceptual mapping. How does MDS determine the optimal positioning of objects in perceptual space? • Five-step procedure: Most MDS programs follow a five-step procedure which involves selection of a configuration. this is not the case. • Primary criterion for determining an optimal position is the preservation of the ordered relationship between the original rank data and the derived distances between points. • Degenerate solutions: The researcher should be aware of degenerate solutions. 1. Not all respondents will attach the same level of importance to a dimension. Over time. Changes in consumers' lives are reflected in their evaluation of stimuli. and reduction of dimensionality. 2. comparison to fit measures. we are unable to discuss each separately. Although two consumers may be aware of an attribute of a product. even if all respondents perceive the dimension.2. they may not both attach equal importance to this dimension. Respondents' dimensions and level of importance will change over time. Thus. While it would be very convenient for marketers if consumers would always use the same decision process with the same stimuli dimensions. which are inaccurate perceptual maps. The following discussion will center on general issues in perceptual mapping. Degenerate solutions may be identified by a circular pattern of objects or a clustered pattern of objects at two ends of a single dimension. consumers may view different dimensions as important. 3. How are the number of dimensions to be included in the perceptual map determined? . consumers will assign different levels of importance to the same dimensions of a stimuli or may even change the dimensions of the stimuli that they evaluate completely. This is a measure of how well the model fits the data. 2) use of an internal or an external analysis. o Stress measurement: measures the proportion of variance in the data that is not accounted for by the model. the analyst must make a trade-off between the fit of the solution and difficulty of interpretation due to the number of dimensions. three additional issues are 1) estimation of the ideal point explicitly or implicitly. With preference data. The interpretation of this plot is the same. • Three approaches for determining the number of dimensions: o Subjective evaluation: The researcher evaluates the spatial maps and determines whether or not the resulting configuration looks reasonable.  Desire a low stress index. Desired levels of the fit index are similar to those desired when using regressions R2. the fit index always improves and stress always decreases.• Trade-off of best fit with the smallest number of dimensions possible. Overall fit index: a squared correlation index which indicates the amount of variance in the data that can be accounted for by the model. 3.   • Parsimony: Parsimony should be sought in selecting the number of dimensions. Interpretation of more than three dimensions is difficult. Scree plot of stress index: The stress measure for models with varying numbers of dimensions may be plotted to form a scree plot as in factor analysis. As you add dimensions. The stress measure and the overall fit index react much the same as R2 in regression. • Ideal point estimation: ideal points (preferred combination of perceived attributes) may be determined by explicit or implicit estimation procedures. Thus. It is the opposite of the fit index. since stress is minimized when the objects are placed in a configuration such that the distances between the objects best matches the original distances. where the analyst looks for the bend in the plot line. and 3) portrayal of the ideal point. o Explicit estimation: respondents are asked to identify or rate a hypothetical ideal combination of attributes . Stage 5: Interpreting the MDS Results Stage 5: Interpreting the 1. the Multidimensional Scaling Results analyst must identify and describe the perceptual dimensions. o Recommendation: external analysis be performed due to computational difficulties with internal analysis and the fact that perceptual space (preference) and evaluative space (similarities) may not contain the same dimensions with the same salience. Procedures for identification of dimensions may be objective or subjective. o Vectors are lines extended from the origin of the graph toward the point which represents the combination of dimensions specified as ideal. o External analysis fits ideal points based on preference data to a stimulus space developed from similarities data. o Point representation is location of most preferred combination of dimensions from the consumer's standpoint. deviance in any direction leads to a less preferred object. • Vector or point representation of ideal point — ideal point is the most preferred combination of dimensions. This is the best approach when the dimensions are highly intangible or affective/emotional. • Subjective procedures: the researcher or the respondent visually inspects the perceptual map and identifies the underlying dimensions. while with a vector. For decompositional methods. less preferred objects are those located in the opposite direction from which the vector is pointing. o Difference in representation: with a point representation. .o Implicit estimation: an ideal combination of attributes is empirically determined from respondents' responses to preference measure questions • Internal versus external analysis o Internal analysis develops spatial maps solely from preference data. Underlying dimensions across analyses cannot be compared. Validation will help ensure generalizability across objects and to the population.• Objective procedures: formal methods. • Only the relative positions of objects can be compared across MDS analyses. • Perceptual map positions are totally defined by the attributes specified by the researcher. are used to empirically derive underlying dimensionality from attribute ratings. When using compositional methods. the analyst should compare the perceptual map against other measures of perception for interpretation. Stage 6: Validating the MDS Results Stage 6: Validating the Multidimensional Scaling Results 1. • Multi-approach method: applying both decompositional and compositional methods to the same sample and looking for convergence. Stage 1: Objectives of Correspondence Analysis 1. • Bases of comparisons across analyses: visual or based on a simple correlation of coordinates. Correspondence analysis will accommodate both nonmetric data and nonlinear relationships. both practitioner and academic. 2. CORRESPONDENCE ANALYSIS Correspondence analysis (CA) is another form of perceptual mapping that involves the use of contingency or cross-tabulation data. • Split-samples or multi-samples may be utilized to compare MDS results. The following sections detail some of the unique aspects of correspondence analysis. such as PROFIT. Its application is becoming widespread within many areas. . • Dimensional reduction is performed in a manner similar to factor analysis. • Performs perceptual mapping represented in multidimensional space. Correspondence analysis requires only a rectangular data matrix of nonnegative data. Stage 4 Deriving the CA Results and Assessing Overall Fit 2. 1. correspondence analysis does not have a strict set of assumptions. Unlike other multivariate techniques. Stage 3: Assumptions of Correspondence Analysis Stage 3: Assumptions in Correspondence Analysis 1. but represent responses to categorical variables. The researcher must identify the number and importance of the dimensions. where categories may be Stage 2: Research Design of Stage 2: Research Design of Correspondence Analysis Correspondence Analysis 1. . The researcher need only be concerned with including all relevant attributes. • Eigenvalues are provided to aid the researcher in determining the appropriate number of dimensions to select and in evaluating the relative importance of each dimension. • Rows and columns do not have predefined meanings. • The categories for a row or column may be a single variable or a set of variables. Correspondence analysis derives a single representation of categories (both rows and columns) in the same multidimensional space.• Contingency tables are used to transform nonmetric data to metric form. The researcher should also assess the sensitivity of the analysis to the addition or deletion of certain objects and/or attributes. 2. Although much debate centers around the issue. Stage 5: Interpretation of the Results . Generalizability of the results may be confirmed by split-sample or multisample analyses. The degree of similarity among Stage 5: Interpretation of Results Stage 6: Validation of the Results categories is directly proportional to the proximity of categories in perceptual space. proximities should only be compared within rows or within columns.1. 1. AND HOW DOES IT IMPACT THE RESULTS OF MDS PROCEDURES? Answer In obtaining preference data from respondents.. to create visual displays that represent the dimensions perceived by the respondents when evaluating stimuli (e. Therefore. the researcher is trying to determine which items are most similar to each other and which are most dissimilar. (2) WHAT IS THE DIFFERENCE BETWEEN PREFERENCE DATA AND SIMILARITIES DATA. However.g. while cluster and factor analysis provide a classification of objects or variables so that each object is very similar to others in its cluster. The choice of input data is important to the researcher when using MDS since an individual's perception of objects in a preference context may be different from that in a similarity context.ANSWERS TO END-OF-CHAPTER QUESTIONS (1) HOW DOES MDS DIFFER FROM OTHER INTERDEPENDENCE TECHNIQUES (CLUSTER AND FACTOR ANALYSIS)? Answer Multidimensional scaling (MDS) is a family of techniques which helps the analyst to identify key dimensions underlying respondents' evaluations of objects. that is. the stimuli are ordered in terms of the preference for some property. a particular dimension may be very useful in describing the differences between two objects but is of no consequence in determining preferences. For instance. . In employing MDS the researcher should identify which type of output information is needed before deciding on the form of input data. then preference data is required. then similarities data would be required. objects). For example. When collecting similarities data. if a company is interested in determining how similar/dissimilar their product is to all competing products. MDS techniques enable the researcher to represent respondents' perceptions spatially. brands. MDS differs from cluster and factor analysis in that it provides a visual representation of individual and group respondents' perceptions of the object(s). two objects could be perceived as different in a similarities-based map but similar in a preference-based spatial map. if the researcher is interested in respondent's preferences for one brand over another. Preference data allow the researcher to view the location of objects in a spatial map when distance implies differences in preference. e. the higher the R-square. Metric methods assume that input as well as output are metric. which allows the researcher to strengthen the relationship between the final output dimensionality and the input data.. In determining ideal points.60 or better are considered acceptable. an ideal product). The analyst may also use an index of fit to determine the number of dimensions.(3) HOW ARE IDEAL POINTS USED IN MDS PROCEDURES? Answer Ideal points may be used to represent the most preferred combination of perceived attributes (i. (4) HOW DO METRIC AND NONMETRIC MDS PROCEDURES DIFFER? Answer Nonmetric methods assume ordinal input and metric output. One method is to locate the ideal point as close as possible to the most preferred object and as distant as possible from the least preferred object. There are several procedures for implicitly positioning ideal points. Explicit estimation involves having the respondents rate a hypothetical ideal on the same attributes on which the other stimuli were rated. which requires a trade-off between the fit of the solution and the number of dimensions. Stress measures the proportion of the variance of the disparities that is not accounted for by the MDS model. Measures of . The index of fit (or R-square) is a squared correlation index that can be interpreted as indicating the proportion of variance of the disparities that can be accounted for by the MDS procedure. Interpretation of solutions derived in more than three dimensions is extremely difficult and is usually not worth the improvement in fit. Implicit estimation positions an ideal point in a defined perceptual space such that the distance from the ideal conveys changes in preference. (5) HOW CAN THE ANALYST DETERMINE WHEN THE "BEST" MDS SOLUTION HAS BEEN OBTAINED? The objective of the analyst should be to obtain the best fit with the smallest number of dimensions. or may be used to define relative preference so that products further from the ideal should be less preferred. A third approach is to use a measure of stress. . the better the fit. the researcher may use either explicit or implicit estimation. and the distance output by the MDS procedure may be assumed to be approximately internally scaled. • Attribute or ratings data may be collected in the original research to assist in labeling the dimensions. the researcher may rely on "rules of thumb" for use in determining which items load on which factor. Measures of similarity are based on the chi-square metric derived from a cross tabulation table. the respondents may be asked (after stating the similarities and/or preferences) to identify the characteristics most important to them in stating these values. such as PROFIT (PROPerty FITting). The set of characteristics can then be screened for values that match the relationships portrayed in the maps. • The subjects may be asked to evaluate the stimuli on the basis of research-determined criteria (usually objective values) and researcher perceived subjective values. (7) COMPARE AND CONTRAST CORRESPONDENCE ANALYSIS TO THE MDS TECHNIQUES. Correspondence analysis is a compositional perceptual mapping technique which relies on the association among nominally scaled variables. It has the unique feature of spatially representing both objects and attributes on the same spatial map.(6) HOW DOES THE RESEARCHER GO ABOUT IDENTIFYING THE DIMENSIONS IN MDS? COMPARE THIS PROCEDURE WITH THAT FOR FACTOR ANALYSIS. the number of factors to extract. Specific programs. . are available for this specific purpose. While the labeling of factors is also met with its share of subjectivity. The procedure for labeling the dimensions in MDS is much more subjective and requires additional skill and experience in analyzing the results. Answer 1. These evaluations can be compared to the stimuli distances on a dimension-by-dimension basis for labeling the dimensions. • If the data were obtained directly. 2. The analyst can adopt several procedures for identifying the underlying dimensions: • Respondents may be asked to interpret the dimensionality subjectively by inspecting the maps. and the naming of factors. Using the totals for each category an expected value is calculated for each cell. The similarity measure provides a standardized measure of association that can be plotted in an appropriate number of dimensions (number of rows or columns minus one). The chi-square values can be converted to similarity measures by applying the opposite sign of their difference. a chi-square statistic is formed for each cell as the squared difference divided by the expected value.(8) DESCRIBE HOW “CORRESPONDENCE” OR ASSOCIATION IS DERIVED FROM A CONTINGENCY TABLE. Then the difference between the expected and actual is calculated. Using this value. . Correspondence analysis allows the representation of the rows and columns of a contingency table in joint space. A focus on explaining the covariance among the measured items. This allows an assessment of fit and provides a better tool for assessing the construct validity of a set of measures. What is structural equations modeling (SEM)? SEM is a multivariate technique combining aspects of factor analysis and multiple regression that allows an investigation of the structure of relationships among the: • measured variables representing latent constructs and among the latent constructs themselves • latent constructs are variates of the measured variables just as factors were described as variates in exploratory factor analysis. The overview emphasizes a few key points that help understand SEM.CHAPTER TEN STRUCTURAL EQUATIONS MODELING (SEM): AN INTRODUCTION This presentation will approach the general idea of structural equations modeling (SEM) by attempting to answer the basic questions of "What?. One point involves the distinction between exogenous and endogenous constructs." "When?. (3) Pages 711-718 provide an introductory overview. 144 . The distinguishing features of SEM include: (1) (2) The estimation of multiple and interrelated dependence relationships in a single analysis." and "How?". An ability to represent unobserved concepts in these relationships and correct for measurement error in the estimation process thus providing for more accurate relationships." "Why?. The assessment of fit allows a better examination of the accuracy of a model. Correlational. The measurement model represents the theory which specifies how measured variables come together to represent latent factors.e. SEM tests a researchers’ model. Endogenous constructs must have an error term associated with each. Dependence relationships imply that a relationship acts in a cause and effect manner. they are not explained by any other construct or variable in the model). they use a variate of measures to represent the construct which acts as an independent variable in the model. From this definition. the model implies that variates represent the factors. 2. or covariance. Students are often confused about the distinctions between latent and measured variables. The error lies in the measured variables that indicate the construct.• Exogenous constructs are the latent. That is. 145 . relationships simply exist to represent association between two constructs in which one variable is not dependent upon another. multi-item equivalent of independent variables. • Why does one use SEM? SEM is used to test theoretical models. The structural model represents the theory specifying how constructs are related to other constructs in the model.e. A conventional “model” SEM models really consist of two models: 1. That is. Exogenous constructs are represented without error. As such. the measured variables that indicate the construct also have an error term. we see that theory is not the exclusive domain of academia but can be rooted in experience and practice obtained by observation of real-world behavior. They are determined by factors outside of the model (i. In addition. Theory can be thought of as a systematic set of relationships providing a consistent and comprehensive explanation of phenomena. (i. which represents some explanatory theory. Figure 10-1 is important in illustrating the components of an SEM.. Endogenous constructs are the latent. between independent and dependent variables). there is no distinction of the type made typifying a dependence technique (i. They are constructs that are theoretically determined by factors within the model. Use this figure to help make the difference clear. multi-item equivalent to dependent variables. a variate of individual dependent variables)... The material around Figure 10-3 illustrates the different types of relationships that are involved in constructing a model.e. The closer the estimated covariances come to the observed covariances. Theoretical support – there must be a compelling rationale for linking a “cause” to an “effect. It is important to distinguish between measured variables and latent variables and between the types of relationships. This means. Nonspurious association – the observed covariance must be true. SEM is a particularly useful technique because it allows researchers to do more than simply test the significance of relationships.” 3. The equations for the measurement model contain parameters used in the equation for the structural parameters. it is a good idea to emphasize that SEM is a confirmatory technique. They are: 1. On page 720. SEM does not seek to only explain variance. Spuriousness is evident when another predictor is added to a model and the relationship between the original two variables disappears.” Figure 10-4 illustrates these points. Figure 10-5 illustrates a simple path diagram. It covers the conditions necessary to establish causality. SEM is sometimes referred to as causal modeling because the models SEM tests often propose causal relationships. Sequence – a “cause” has to occur before an “effect. That is.The relationships are represented by parameters in a set of structural equations. It explains covariance. Covariance – a change in one thing corresponds proportionately to a change in another thing. a discussion of causality begins. 2. Therefore. 146 . the equations are interrelated. SEM models are often illustrated with a path diagram. fit is assessed by how well the structural equations can be used to reproduce the observed covariance among measured items. When do you use SEM? SEM is used when the researcher wants to test theory. It allows researchers to assess the overall validity of a proposed model by assessing its fit. 4. Above all. It is not the most appropriate technique to simply empirically assess relationships. whether testing the structural model or the measurement model. it is useful in testing some proposed theory. the better the fit. SEM is useful when the researcher is either trying to confirm a single model or trying to test the relative fit of two or more competing models. then the required sample size is more on the order of 200. • • How do you use multivariate data analysis techniques? The remainder of this chapter provides a brief overview and introduction of these six stages. The stages are introduced beginning on page 734. and SEM is used to assess how well the model fits the data based on a comparison of the observed covariance (S) with the estimated covariance (Σ). then minimum sample sizes of 300 or more are needed to be able to recover population parameters.• Confirmatory modeling strategy .45–. These same six stages will be discussed in greater detail over the next two chapters. If the communalities are lower or the model includes multiple underidentified (fewer than3 items) constructs. The stages are: 147 .The most direct application of structural equation modeling is a confirmatory modeling strategy. Based on the discussion of sample size. When the number of factors is larger than six.55). each with more than three items(observed variables). Competing modeling strategy .and with high item communalities (.” which is a much stronger test than just a slight modification of a single “theory. sample size requirements may exceed 500. In recent years. When comparing these models. The researcher specifies a single model (set of relationships). The strongest test of a proposed model is to identify and test competing models that represent truly different hypothetical structural relationships. research has suggested that simple cutoffs such as a sample of 300 are needed are too simplistic.As a means of evaluating the estimated model with alternative models. overall model comparisons can be performed in a competing models strategy.•If any communalities are modest (.” • SEM is also inappropriate when sample size considerations are not met.or the model contains constructs with fewer than three items.can be adequately estimated with samples as small as 100–150. the following suggestions are offered: • SEM models containing five or fewer constructs. some of which use fewer than three measured items as indicators and multiple low communalities are present. the researcher comes much closer to a test of competing “theories.6 or higher). the estimated covariance matrix and the actual observed covariance matrix would be equal. and the model fit compares the theory to reality as represented by the data. The closer the values of these two matrices are to each other. it is important to stress the importance of face validity – or the fact that the definitions of constructs match the content of the item indicators. 2. Thus. Here. Designing a study to produce empirical results 4. A researcher’s theory is used to specify a model. Assessing the measurement model validity a. If a researcher’s theory were perfect. the better the model is said to fit. Developing the overall measurement model a. once a researcher specifies a model. EQS or AMOS estimates parameters and provides an assessment of overall model fit. It is also important to point out the differences in the types of fit indices: i. Defining individual constructs a. 3. the estimated covariance matrix (Σk) is compared mathematically to the actual observed covariance matrix (S) to provide an estimate of model fit. Relative fit indices iii. Indeed. a SEM program such as LISREL. Parsimony fit indices 148 . Here. Absolute fit indices ii. it is important to consider the number of indicators used for each construct and whether or not the measured variables indicate a latent construct or form some factor. A more detailed discussion of fit is included.1.Σk) is a key value in SEM. Basically. A Chi-square (χ2) test provides a statistical test of the resulting difference. b. c. SEM estimation procedures like maximum likelihood produce parameter estimates that mathematically minimize this difference for a specified model. The difference in the covariance matrices (S . This will be discussed more in the next chapter. On page 758. both of which vary from situation to situation. Specifying the structural model a.5. They describe the difference between dependence relationships and correlational relationships and summarize the conditions necessary for causal inferences to be made. the size and significance of the parameters representing hypotheses must be examined. these rules of thumb apply to stages 1-4 of the SEM process. On page 744. They concern important issues including handling missing data and appropriate sample sizes for valid SEM analysis. 6. In addition. the evaluation of fit depends very much on the characteristics of the model and the sample. rules of thumb discussing the evaluation of an SEM model are provided. these rules of thumb apply to the basics of constructing a theoretical model. • • 149 . There is no golden rule! In other words. Assessing structural model validity a. Using the Rules of Thumb The chapter’s Rules of Thumb are very helpful in quickly referencing guidelines that help answer basic questions for students. • On page 734. Figure 10-8 demonstrates a complete structural path model including both the measurement parameters and the structural parameters. b. b. but emphasize the fact that no single criterion or benchmark exists that distinguishes good from poor models. Each hypotheses means that a path must be freed between constructs. They include guidelines for fit. The fit must be assessed again just as it was done for the measurement model. the estimated covariance matrix is computed so it can be compared to the observed covariance matrix. WHAT ARE THE DISTINGUISHING CHARACTERISTICS OF SEM? Three key distinguishing characteristics of SEM are (1) the estimation of multiple and interrelated dependence relationships. In survey research. DESCRIBE HOW THE ESTIMATED COVARIANCE MATRIX IN AN SEM ANALYSIS (Σk) CAN BE COMPUTED? WHY DO WE COMPARE IT SO S? The estimated covariance matrix can be computed by using the parameter estimates provided by SEM to reconstruct the correlations or covariances between the measured variables following the rules of path analysis. 150 . It is represented by multiple measured variables. Then. A latent construct is not directly observable. The latent constructs are represented as variates of the measured variables. The closer together they come. Alternatively. and (3) a focus on explaining the covariance among the measured items. the measured variables are often responses to survey items. WHAT IS THE DIFFERENCE BETWEEN A LATENT CONSTRUCT AND A MEASURED VARIABLE? Answer Latent constructs are the way constructs are represented in a SEM. 2. the better the fit.ANSWERS TO END-OF-CHAPTER QUESTIONS 1. 3. (2) an ability to represent unobserved concepts in these relationships and correct for measurement error in the estimation process. In either case. the parameter estimates could be used to construct predicted values for each observation similar to the way predicted values are found in multiple regression analysis. SEM algorithms are designed to find parameter estimates which will minimize the difference between the two given the specifications that match a researcher’s model. an estimated covariance matrix can be found by computing the covariance among the measured items. An assessment of fit provides an assessment of the accuracy of some theoretical model. Using SEM. a model including the cause and effect can be modified by adding the third variable/construct as an additional predictor of both the cause and effect. 5. 6. 7. WHAT IS A THEORY? HOW IS A THEORY REPRESENTED IN A SEM FRAMEWORK? Theory can be defined as a systematic set of relationships providing a consistent and comprehensive explanation of a phenomenon. If the relationship between the cause and effect becomes insignificant. The equations for SEM and a multiple regression analysis are similar in form. The models specify relationships. Unlike EFA however. WHAT IS FIT? Fit indicates how well a specified model reproduces the covariance matrix among the measured items. the researcher must specific the number of factors and the variables that load on it prior to conducting a CFA. we see that theory is not the exclusive domain of academia but can be rooted in experience and practice obtained by observation of realworld behavior. Theories are represented by models. However. The measured variables form variates that create the constructs. Multiple regression is also used to test dependence relationships. One way a relationship can be spurious is when there is a really another event that explains both the cause and effect. HOW IS STRUCTURAL EQUATION MODELING SIMILAR TO THE OTHER MULTIVARIATE TECHNIQUES DISCUSSED IN THE EARLIER CHAPTERS? SEM shares much similarity with multiple regression and factor analysis. 151 . WHAT IS A SPURIOUS CORRELATION? HOW MIGHT IT BE REVEALED USING SEM? A spurious relationship is one that is false or misleading. it cannot assess relationships for multiple dependent variables simultaneously. From this definition.4. there is evidence of spuriousness in the relationship. The relationships correspond to parameter estimates that are represented in equations for each construct or measured variable. SEM also shares similarity with exploratory factor analysis in that each is capable of representing latent factors by creating variates using measured variables. the baseline model is a “null” model specifying that all measured variables are unrelated to each other. Therefore. may be in appropriate for a model with only four constructs and 12 measured variables and a sample size of 125. Additionally. Sample size is included in the equation for many fit indices. Relative fit indices are incremental fit indices. 9. They do not explicitly compare the fit of a specified model to any other model as do other types of fit indices. including the chi-square likelihood test. five constructs and a sample size over 250.97. The chapter provides guidelines for assessing whether the sample size is sufficient to get reliable SEM results. HOW DOES SAMPLE SIZE AFFECT STRUCTURAL EQUATION MODELING? Sample size affects structural equation modeling in several ways. WHAT IS THE DIFFERENCE BETWEEN AN ABSOLUTE AND A RELATIVE FIT INDEX? Absolute fit indices are based on how well a specified model reproduces the observed covariance matrix. The factors that influence the appropriateness include the number of constructs and measured variables in a model and the communalities of the latent constructs. which may be acceptable for a model with 24 measured variables. these fit indices increase in a way that may not reflect inaccuracy in the model. WHY ARE THERE NO ‘MAGIC VALUES’ THAT DESIGNATE GOOD FIT FROM POOR FIR ACROSS ALL SITUATIONS? Because there are too many factors that influence fit. 10. the number of constructs. Sample size influences whether or not SEM is appropriate. with large samples. These include the sample size.8. Most commonly. a CFI of . So for example. Incremental fit indices are a group of statistical fit indices that assess how well a specified model fits relative to some alternative baseline model. sample size affects SEM through its impact on fit. the number of measured items and the number of measured items representing each construct. 152 . It should look something like this (the student can fill in the names of the constructs as he/she wishes): E X1 E X2 E X3 E X4 E X5 Price (ξ1) - Atmosphere Customer Share (η1) (ξ3) X6 X7 X8 X9 X10 Y1 Y2 Y3 Y4 E E E E E E E E E 153 . THE EXOGENOUS CONSTRUCTS ARE EACH MEASURED BY 5 ITEMS AND THE ENDOGENOUS CONSTRUCT IS MEASURED BY 4 ITEMS.11. DRAW A PATH DIAGRAM WITH TWO EXOGENOUS CONSTRUCTS AND ONE ENDOGENOUS CONSTRUCT. BOTH EXOGENOUS CONSTRUCTS ARE EXPECTED TO BE RELATED NEGATIVELY TO THE ENDOGENOUS CONSTRUCT. With CFA. the researcher must specify both the number of factors that exist within a set of variables and which factors will load highly on each factor before statistics can be computed. The chapter illustrates this by showing how CFA compares to other multivariate techniques and by providing several illustrations. The researcher cannot control which variables “load” on which factor. the researcher uses CFA to provide a thorough examination of construct validity for a set of measures." "When?. all measured variables are related to every latent factor by a factor loading estimate. 154 . unlike EFA. but often even this is left up to the statistical procedure. A measurement theory specifies how measured variables logically and systematically represent constructs involved in a theoretical model. Why does one use SEM? Following from the discussion above. Simply. EFA explores the data and provides the researcher with information about how many factors are needed to best represent the data. CFA is similar in many respects to EFA. Measurement theory first requires that a construct can be defined. Therefore." and "How?". What is confirmatory factor analysis (CFA)? This chapter begins by providing a description of confirmatory factor analysis (CFA). with CFA a researcher uses measurement theory to specify a priori the number of factors as well as which variables load on those factors. Therefore. more than any other multivariate technique. CFA. CFA is the tool that tests measurement theories. CFA is a way of testing how well measured variables represent a smaller number of constructs. SEM is then applied to calculate statistics the researcher can use to test the extent to which a researcher’s a-priori pattern of factor loadings represents the actual data. With EFA." "Why?. CFA is a tool allowing us to either “confirm” or “reject” our preconceived theory. facilitates the examination of construct validity. The researcher can control the number of factors that are extracted.CHAPTER ELEVEN SEM: CONFIRMATORY FACTOR ANALYSIS This presentation will approach the general idea of using structural equations modeling (SEM) to conduct confirmatory factor analysis (CFA) by attempting to answer the basic questions of "What?. Nomological validity is tested by examining whether or not the correlations between the constructs in the measurement theory make sense. o Standardized factor loadings should be at least . A more conservative test is to compare the variance extracted percentages for any two constructs with the square of the correlation estimate (Ф) between these two constructs. Evidence of construct validity provides confidence that item measures taken from a sample represent the actual “true score” that exists in the population.7 or greater. CFA provides evidence in two ways: • First. o Variance extracted estimates for a construct should be . . it deals with the accuracy of measurement. If the fit of the two construct model is not significantly better than that of the one construct model. In other words. 2.5 or greater but preferably. high discriminant validity provides evidence that a construct is unique and captures some phenomena other measures do not. the construct should “fit” with other theoretical concepts as theory would suggest that it does. then there is insufficient discriminant validity. • CFA provides evidence in the form of measurement parameter coefficients or factor loadings. 155 . Construct validity is made up of four important components: 1. The Ф matrix of construct correlations can be useful in this assessment.5 or greater. the items making up two constructs could just as well make up only one construct. Thus.Construct validity is the extent to which a set of measured items actually reflect the theoretical latent construct they are designed to measure. Convergent validity is the extent to which items that are indicators of a specific construct ‘converge’ or share a high proportion of variance in common. The variance extracted estimates should be greater than the squared correlation estimate. which can be used to compute variance extracted estimates. competing CFA models could be set up comparing the fit of a CFA assuming the items make up one construct with that of a CFA assuming they make up two constructs. Discriminant validity is the extent to which a construct is truly distinct from other constructs. • 3. Thus. So. Figures 11. When do you use SEM? CFA is used when the researcher wants to test a measurement theory. Generally.• • • Things that are expected to be unrelated should show no correlation. Informed judgment is used to provide evidence of face validity (common sense logic). Things that are opposites should produce negative correlations.4 can be used to help illustrate the difference between an underidentified model. o Over-identified models are those with more unique covariance and variance terms than parameters to be estimated. In other words. 4. Face validity is sometimes called content validity. CFA provides a valid test of measurement theory when certain conditions are met. CFA is also not diagnostic in this case. They fit perfectly by definition and thus are not very interesting. It is the extent to which the content of the items match the construct definition. These include: • • Adequate sample characteristics as discussed in Chapter 10. this means there must be a sufficient number of measured items per construct to make sure that the model has a net positive number of degrees of freedom. • CFA is used prior to testing a theoretical model proposing relationships between constructs. a just-identified model and an over-identified model. Things that coincide to some degree should show positive correlations. prior to conducting a structural model. Over-identified construct/measurement model. o An under-identified model is one with more parameters to be estimated than there are item variance and covariances. 156 . This can be and should be examined prior to conducting a CFA. • • • CFA does not provide direct evidence of face validity.3 and 11. o A just-identified model has just enough degrees of freedom to estimate all parameters required for a unique solution. Judge the items for content. Developing scale items that match the definition. A confirmatory test of the measurement model is conducted using Stage 2: Developing the Overall Measurement Model CFA. How do you use confirmatory factor analysis? The remainder of this chapter provides a brief overview and introduction of the first four stages of the SEM process described in the previous chapter. Scale modifications are made if necessary. • 157 . are discussed more specifically here in reference to the analysis of measurement models. described beginning on page 779. Stage 1: Defining Individual Constructs The steps in developing a multi-item scale are discussed in detail. Figure 11-2 illustrates the concepts of between construct error variance and within construct error variance. These stages comprise the CFA process. The stages. Conduct a pretest to evaluate the items. • • • • • • Define the construct theoretically.• CFA is not used to examine measurement properties if constructs are represented by single items or by composite indicators. • The concept of unidimensionality is discussed. Both are threats to the unidimensionality of a CFA model. Unidimensionality means that a set of observed variables (indicators) has only one underlying construct. The researcher should design a study that will minimize the likelihood of identification problems. the value of one loading estimate per construct is set to one. When a measurement model hypothesizes no covariance between or within construct error variances. o A reflective measurement theory is modeled based on the idea that latent constructs cause the measured variables and that the error results in an inability to fully explain these measures. o A formative measurement theory is modeled based on the assumption that the measured variables cause the construct. • • Stage 3: Designing a Study to Produce Empirical Results 1. Most commonly. Two ways to do this are discussed. good practice suggests four or more constructs are needed. The measurement model must be specified in this stage.• Good practice suggests that a measurement model have congeneric properties. The error in formative measurement models is an inability to fully explain the construct. the arrows are drawn from latent constructs to measured variables. meaning they are all fixed at zero. A key assumption is that formative constructs are not considered latent. Two conditions that help establish identification are the: 158 . Thus. Congeneric measurement models are considered to be sufficiently constrained to represent good measurement properties. 2. Three items can be acceptable for a given construct if other constructs have more than three item indicators. the measurement model is said to be congeneric. The arrows are drawn from the measured variable to the construct. As such. One aspect in doing this is setting the scale for each construct. Largely as a result of the identification issue. Reflective constructs must be treated differently from formative constructs. reflective measures are consistent with classical test theory. The appropriate number of items per construct is discussed. it is likely due to a problem with the rank condition. CFA allows us to test or ‘confirm’ whether or not a theoretical measurement model is valid by applying using fit statistics as described in the previous chapter. The researcher compares the theoretical measurement model against reality as represented by the sample. When an SEM program provides a message about linear dependence. 2. The rank condition can be difficult to verify. This is done by examining how well the ‘theory’ fits the data. That is. the number of unique covariance and variance terms less the number of free parameter estimates must be positive. It requires that each parameter estimated be algebraically defined. Advanced CFA Topics 1. • Stage 4: Assessing Measurement Model Validity 1. 3. This can be much more difficult to establish. The CFA model results must demonstrate all characteristics associated with construct validity.• The order condition refers to the requirement discussed earlier that the net degrees of freedom for a model be greater than zero. Higher-Order Factor Analysis • • • Used when a measurement theory involves constructs that have different levels of abstraction Most often. this involves a second-order factor analysis with constructs existing at one of two levels of abstraction When should a higher-order factor analysis be used? When the answer to each of these questions is yes: 159 . • Including partial scalar invariance (at least two intercept terms per measured variable equal for all constructs). Multiple Group Analysis • • From a CFA standpoint.   Loose-cross-validation is usually sufficient to support the same measurement model in each group. 160 . this often means that some type of crossvalidation is in order.821. o Refer to the different types of cross-validation on pages 820 . Cross-validation is an attempt to reproduce the results found in one sample using data from a different sample.o Is there a theoretical reason to expect conceptual layers of constructs (each layer reflecting a different level of abstraction)? o Are all first-order factors expected to influence other nomologically related constructs in the same way? o Are the higher-order factors going to be used to predict other constructs of the same level of abstraction? o Are the minimum conditions for identification and good measurement practice present in both the first-order and higherorder layers of the measurement theory? 2. Table 11-8 contrasts different tests of invariance and the results that go along with each. Factor-loading equivalence is sufficient to support comparisons of relationships between groups.  o Also. • Measurement Bias o Constant methods bias implies that the covariance among measured items is caused in part by the fact that a common type of scale is used across a set of items. • Including partial metric invariance (equivalence and invariance can be used interchangeably). Scalar invariance is sufficient to support comparisons of construct means between groups. Page 781 lists the rules of thumb dealing with creating a scale to represent a construct. Using the Rules of Thumb The chapter’s Rules of Thumb are very helpful in quickly referencing guidelines that help answer basic questions for students. It deals with the number of measured items needed per construct and the direction of causality between measured items and the construct. • • • Page 779 describes rules of thumb dealing with construct validity. They emphasize steps for insuring content validity.o CFA can be used to test this by creating methods factors for each scale type or type of measurement device. They emphasize the usefulness of congeneric measurement structures and a way to avoid Heywood cases. • • 161 . fit and residuals. Page 800 lists rules of thumb used to assess the results of a CFA. Page 794 lists rules of thumb that help the researcher design the study. Page 790 lists rules of thumb helpful in developing the overall measurement model. It lists guidelines for convergent validity. If the measurement model fits well. the better the fit. overall. More importantly however. WHAT DOES “FIT” REFER TO WHEN USING CFA? • Technically. LIST THE WAYS IN WHICH CFA DIFFERS FROM EFA. However. fit means that the measurement equations conforming to the model specifications for each construct can be used to compute an estimated covariance matrix that can be compared to the actual observed covariance matrix of all measured items. The closer together they become. fit is also suggested of construct validity. measurement model fit goes along with assessing how valid the measures taken together are. • • 162 . • CFA and EFA differ in several respects. the researcher can proceed to examine a structural model representing specific theoretical relationships among the constructs. If a “measurement model” fits well its overall fit statistics are consistent with those discussed in Chapter 10. • • • • EFA Explores the data Determines the number of factors (in most applications) Computes a factor loading for every item on every factor Overall model fit is not a concern • • • • CFA Confirms (tests) a measurement model The number of factors must be specified Only computes a factor loading for “free” parameter estimates Overall model fit is an important concern 2.ANSWERS TO END-OF-CHAPTER QUESTIONS 1. a measurement model that fits well also displays the empirical characteristics of construct validity suggested in the chapter. So. Face validity is sometimes called content validity.3. Developing scale items that match the definition. Those parameters that are assumed to be zero. are also considered fixed. Thus. 163 . LIST AND VALIDITY. • • WHAT IS THE DIFFERENCE BETWEEN A FIXED AND FREE PARAMETER IN CFA? A free parameter is one for which a value is estimated using SEM procedures. WHAT ARE THE STEPS IN DEVELOPING A NEW CONSTRUCT MEASURE? • • • • • • Define the construct theoretically. A fixed parameter is one for which a value is specified prior to estimating a model using SEM procedures. Discriminant validity is the extent to which a construct is truly distinct from other constructs. such as cross-construct loading estimates. • DEFINE THE COMPONENTS OF CONSTRUCT 4. Convergent validity is the extent to which items that are indicators of a specific construct ‘converge’ or share a high proportion of variance in common. It is the extent to which the content of the items match the construct definition. Judge the items for content Conduct a pretest to evaluate the items. Nomological validity is tested by examining whether or not the correlations between the constructs in the measurement theory make sense. high discriminant validity provides evidence that a construct is unique and captures some phenomena other measures do not. • • • 5. A confirmatory test of the measurement model is conducted using CFA. Scale modifications are made if necessary. o Formative items cause the factor. WHY IS A FOUR-ITEM FACTOR OVER-IDENTIFIED? • A four-item factor is over identified because it has net positive degrees of freedom after estimation.6. 164 . WHAT ARE THE PROPERTIES OF A CONGENERIC MEASUREMENT MODEL? WHY DO THEY REPRESENT THE PROPERTIES OF GOOD MEASUREMENT? • Congeneric measurement models have the following properties: o All cross-construct loading estimates are fixed to zero (assumed not to exist) o This means that every measured item loads on exactly one latent construct o All between-construct error covariance terms are fixed to zero o All within-construct error covariance terms are fixed to zero • Congeneric measurement models are considered to be sufficiently constrained to represent good measurement properties because a congeneric measurement model meets requirements associated with construct validity. 7. 8. WHAT ARE THE CONSIDERATIONS IN DETERMINING WHETHER OR NOT INDICATORS SHOULD BE MODELED AS FORMATIVE OR REFLECTIVE? • What is the direction of causality between the multiple indicators and the factor (construct)? o Reflective items are caused by the factor. A four-item congeneric measurement model requires estimation of eight estimates using eight out of a possible ten degrees of freedom ([4 * 5]/2). then the measurement model is best considered reflective. formative indicator items are not expected to “move together. o Formative indicators of a factor are not expected to show high covariance. dropping an item produces a material change in the construct. High inter-item covariance provides evidence consistent with reflective indicators. then they are still acceptable as formative indicators. For formative measurement models. • How do the indicators relate to other variables? o All of the indicators of a single construct relate to other variables in a similar way in a reflective measurement model.” • Is there high duplicity in the content of the items? o If all of the indicator items share a common conceptual basis. Thus.• What is the nature of the covariance among the indicator items? o If the items are expected to covary highly with each other. Therefore. then the reflective model is more appropriate. all of the indicators will tend to “move together” meaning that changes in one will be associated with changes in the others. if it appears the indicators cause the formative construct. If an indicator should not be highly related to the others. Since all the items represent the same concept. the researcher would expect one indicator to produce a different pattern of relationships with an outside variable than would another indicator. but they share nothing in common conceptually. a key point is that with reflective models. o With formative indicator models. dropping an item does not materially change a construct’s meaning. As a result. you probably should delete it. an index may be comprised of numerous measures for which there is no common basis. 165 . o Formative items need not share a common conceptual basis. Thus. o The indicators of a formative construct need not relate to other variables in a similar way. meaning they all indicate the same thing. • WHAT IS A HEYWOOD CASE AND HOW IS IT TREATED USING SEM? The term Heywood case refers to a factor solution that produces an error variance estimate of less than 0 (a negative error variance). In particular. • 11. may enable a CFA solution. the number of constructs. measured items and the sample size involved in a CFA all can affect common fit indices. works in a way such that smaller values represent better fit. So. with the resulting identification issues. Heywood cases are particularly problematic in CFA models with small samples or when the three indicator rule is not followed. there are too many things that can affect the various indices from situation to situation to expect a “one-size fits all” fit index value. like the CFI. In this case. avoiding small samples and one and two item indicators is a way to avoid Heywood cases. the better is the fit. works in a way such that higher values represent better fit. if it can be isolated. A badness of fit index. as in tau-equivalence. The same fit criteria apply for CFA as for SEM in general. WHAT IS THE DIFFERENCE BETWEEN A GOODNESS OF FIT AND A BADNESS OF FIT INDEX? • A goodness of fit index. like the RMSEA. For the RMSEA. the offending variable. values that become progressively close to zero provide better fit. IS IT POSSIBLE TO ESTABLISH PRECISE CUT-OFFS FOR CFA FIT INDICES? EXPLAIN YOUR ANSWER. it isn’t possible to establish precise cut-offs. an added constraint such as fixing factor loadings to be equal. 166 . Alternatively. • 10. can be deleted. If a Heywood case occurs.9. • • No. Recalling from the previous chapter. the closer the values are to one. 167 . DESCRIBE THE STEPS IN A SPECIFICATION SEARCH. • A specification search is an empirical trial and error approach that may lead to sequential changes in the model based on key model diagnostics. WHAT CONDITIONS MAKE A SECOND ORDER FACTOR MODEL APPROPRIATE? • • • Is there a theoretical reason to expect two conceptual ‘layers’ of a construct exist? Are all the first-order factors expected to influence other nomologically related constructs in the same way? Are the higher-order factors going to be used to predict other constructs of the same general level of abstraction (i. global personality – global attitudes)? Are the minimum conditions for identification and good measurement practice present in both the first-order and higher order layers of the measurement theory? • If the answer to all of these questions is yes. The steps involved in conducting one are: o Examine factor loadings – are any too low? o Examine residuals – are any too high? o Examine modification indices – do any suggest a significant improvement in fit if freed? o Estimate the model after making the changes suggested through these stages – is the fit significantly better? If so. a second order factor model can be justified. stop.e. o Otherwise.. repeat the process until an ‘acceptable’ fit is found.12. 13. ONE FROM ITALY AND ONE FROM JAPAN? EXPLAIN YOUR RESPONSE. To make valid mean comparisons. AN INTERVIEWER COLLECTS DATA ON AUTOMOBILE SATISFACTION. 168 . If the introduction of the two “method” factors changes the parameter estimates and fit of the original automobile satisfaction model significantly. HOW CAN CFA BE USED TO TEST WHETHER OR NOT THE QUESTION FORMAT HAS BIASED THE RESULTS? • Two additional factors can be introduced to the CFA. 15. • Measurement Invariance Must Exist in Two Forms: o Metric Invariance – the loading estimates do not vary between Canada and Japan. THE RESPONDENT RESPONDS TO ANOTHER 20 ITEMS BY MARKING THE ITEMS USING A PENCIL. WHAT CONDITIONS MUST BE SATISFIED IN ORDER TO DRAW VALID CONCLUSIONS ABOUT DIFFERENCES IN RELATIONSHIPS AND DIFFERENECES IN MEANS BETWEEN THREE DIFFERENT GROUPS OF RESPONDENTS – ONE FROM CANADA. at least partial scalar invariance must exist. THEN. This means at least two intercept terms have to be the same among both Canadian and Japanese respondents. This means at least two loading estimates have to be the same in both Canada and the Japan for each construct.14. o Scalar Invariance – the measured variable intercept terms must be the same among the Canadian sampling units as they are among the Japanese sampling units. One should represent the influence of the paper and pencil instrument. Valid relationship comparisons can be made if at least partial metric invariance exists. The other should represent the personal interview format. then the methods are biasing the results. TEN QUESTIONS ARE COLLECTED VIA A PERSONAL INTERVIEW. " "When?. Why does one use SEM to test a structural model? CFA cannot really be used to test a structural model because in the conventional CFA applications. all constructs are allowed to correlate with each other. If we follow from the corresponding figure in the previous chapter (Figure 11. including a theoretical model or occasionally as a causal model.1 illustrates a simple structural model. The researcher makes changes to the CFA model to represent the corresponding structural model. 169 . Figure 12. SEM procedures allow the relationships making up the theory to be estimated taking into consideration the measurement error that exists. is related to customer commitment in a way that the relationship can be captured by a regression coefficient. Thus. Structural models are referred to by several terms. The researcher uses SEM to test a structural model because it will provide a way of assessing how well this theory “fits” reality. the assumption now is that the first construct. In addition. A causal model infers that the relationships meet the conditions necessary for causation. What is a structural model? This chapter builds off of the previous chapter to demonstrate how the SEM process can be completed by testing a full structural model. It can be expressed in terms of a structural model which represents the theory with a set of structural equations and is usually depicted with a visual diagram. the researcher can proceed to this step. it is good to review some of the advantages of SEM in general discussed in chapter 10. customer share. there is no theory specifying that some constructs are related to one another while others are not." "Why?.1).CHAPTER TWELVE SEM: TESTING A STRUCTURAL MODEL This presentation will approach the general idea of using structural equations modeling (SEM) to test a structural model by attempting to answer the basic questions of "What?. This allows the researcher to more accurately assess each relationship. Before moving on." and "How?". Once a measurement is shown valid using CFA. A structural theory is a conceptual representation of the relationships between constructs. Constructs with arrows from other constructs are considered endogenous. depicted with one-headed arrows on a path diagram. direction and significance of the structural parameter estimates. The researcher must first understand the unit of analysis involved in a model. The size. Do the constructs involve individual people.When do you use SEM to test a structural theory? SEM is used to test a structural theory when the researcher is using multiple-item measures for multiple constructs. Stage 5: Specifying the Structural Model 1. individual brands. 170 • • . The process for computing the degrees of freedom for a model are covered on page 852. How do you use confirmatory factor analysis? The remainder of this chapter provides a brief overview and introduction of the remaining stages of the SEM process described in chapter 10. the relationship is not hypothesized to exist. Chapter 11 covered the first four stages in some detail. The structural model is then specified following certain conventions: • • • No arrows from a construct can enter an exogenous construct. If no arrow is drawn between two constructs. SEM is particular useful when a researcher is interested in both: • • The overall model fit and the relative model fit compared to other plausible models. Each hypothesized relationship uses one degree of freedom. stores. This often implies causality from a cause to an effect. This chapter focuses on the final two stages. A single-headed arrow is used to represent a hypothesized structural relationship between one construct and another. or companies? 2. If the SEM model specifies the same number of structural relationships as there are possible construct correlations in the CFA. the model is considered saturated. Therefore. • • • Recalling from Chapter 10. • A model is considered recursive if the paths between constructs all proceed only from the predictor (antecedent) construct to the dependent or outcome construct (consequences). A nonrecursive model contains “feedback” loops. Models can be either recursive or nonrecursive. Even a CFI equal to 1. good practice dictates that more than one fit statistic be used. we recommended one absolute index. 3.3. one can conclude that the structural theory lacks validity if the structural model fit is substantially worse than the CFA model fit. Since a recursive structural model cannot fit any better (have a lower χ2) than the overall CFA. 171 . only general guidelines are given for different situations. The structural model fit is assessed as was the CFA model. Once again. one incremental index and the model χ2 be used at a minimum. The feedback loop can either involve a direct or an indirect relationship. Also. Saturated theoretical models are not generally interesting since they usually cannot reveal any more insight than the CFA model. 2. The fit statistics for a saturated theoretical model should be the same as those obtained for the CFA model. Therefore.0 and an insignificant χ2 may not have a great deal of meaning in a very simple model. A feedback loop exists when a construct is seen as both a predictor and an outcome of another single construct. The CFA fit provides a useful baseline to assess the structural or theoretical fit. Those guidelines remain the same for evaluating the fit of a structural model. there is no magic set of numbers that suggests good fit in all situations. • Stage 6: Examining Structural Model Validity 1. one of the indices should be a badness of fit index. Indirect effects are those relationships that involve a sequence of relationships with at least one intervening construct involved. A moderator means that the relationship between two variables changes with the level of another variable/construct. any relationship revealed in a post hoc analysis provides only empirical evidence – not theoretical support. Direct effects are the relationships linking two constructs with a single arrow.4. • • Advanced Structural Model Topics 1. post hoc identified relationships should not be relied upon in the same way as the original theoretical relationships. A moderating effect occurs when a third variable or construct changes the relationship between two related variables/constructs. For this reason. Nontrivial. That is. • 172 . they are greater than zero for a positive relationship and less than zero for a negative relationship. Theoretical validity increases to the extent that the parameter estimates are: • Statistically significant and in the predicted direction. The guideline here is the same as in other multivariate techniques. This can be checked using the completely standardized loading estimates. Relationship Types – diagrams are used to help illustrate different types of relationships • • A mediating effect is created when a third variable/construct intervenes between two other related constructs. Therefore. Recall that SEM provides an excellent tool for testing theory. This frequently involves a test of moderation Using the Rules of Thumb The chapter’s Rules of Thumb are very helpful in quickly referencing guidelines that help answer basic questions for students. These rules point out a few aspects associated with advanced SEM topics. These rules of thumb concentrate on issues associated with specifying the structural model. Researchers often develop a theory that predicts that one or more structural relationships vary between groups. They provide particularly useful advice for dealing with moderating and mediating relationships.2. Page 880. they focus on how a CFA model can be converted to a structural model. The interest now focuses on similarities and differences between structural parameters indicating differences in relationships between the groups. • Page 856. Multi-Group Analysis • • Multiple group analysis for structural models is an extension of the multiple group CFA case. In particular. • 173 . it also specifies specific types of relationships.ANSWERS TO END-OF-CHAPTER QUESTIONS 1. HOW CAN A MEASURED VARIABLE REPRESENTED WITH A SINGLE ITEM BE INCORPORATED INTO AN SEM MODEL? • The relationship between the variable and the latent construct is set (fixed) to the square root of the estimated reliability. Exogenous constructs can retain their two-headed arrow to represent simple correlation. HOW IS THE VALIDITY OF AN SEM MODEL ESTIMATED? • • By assessing the overall fit of the model. • 2. size and direction of the relationships between constructs. In doing so. meaning the two constructs are not hypothesized to be related. Visually. WHAT IS THE DISTINGUISHING NONRECURSIVE SEM MODEL? • One construct can be thought of as causing another construct and at the same time. The visual diagram replaces all two-headed arrows indicating covariance with either a one-headed arrow indicating an hypothesized relationship from one construct to another or with no arrow. By assessing the statistical significance. It also generally assumes that all constructs are related to one another – at least with respect to a conventional CFA analysis. 4. The corresponding error term is set to 1 – the reliability estimate. CHARACTERISTIC OF A 3. The structural theory specifies that some constructs are related to one another and others are not. Are they consistent with predictions and nontrivial? 174 . IN WHAT WAYS IS A MEASUREMENT THEORY DIFFERENT FROM A STRUCTURAL THEORY? WHAT IMPLICATIONS DOES THIS HAVE FOR THE WAY AN SEM MODEL IS TESTED? HOW DOES THE VISUAL DIAGRAM FOR A MEASUREMENT MODEL DIFFER FROM THAT OF AN SEM MODEL? • A measurement model specifies the correspondence rules between measured items and constructs. an arrow can be drawn from one construct to another and a separate arrow can be drawn in the opposite direction. be considered as an outcome of that same construct. 6. • • • • • • • 175 . SEM attempts to reproduce observed covariance between measured items. PLS models have fewer problems with statistical identification and fatal errors that prevent solutions. PLS finds solutions based on minimizing the variance in endogenous constructs. WHAT IS A MAJOR CONCERN WHEN USING SEM TECHNIQUES WITH LONGITUDINAL DATA? • One of the key issues in modeling longitudinal data with SEM involves added sources of covariance associated with taking measures on the same units over time. Degrees of freedom do not have a meaningful role in PLS as in SEM. it does not try to recreate the covariance between measured item scores. WHAT IS PLS AND HOW IS IT DIFFERENT FROM SEM? • PLS treats the factors as individual composite scores. PLS cannot distinguish formative and reflective indicators. In other words. PLS does not generally rely on optimization procedures as does SEM. PLS is less sensitive to sample size considerations.5. PLS does not require the characteristics of good measurement to produce results. P A ND R.7. HOW CAN SEM TEST FOR A MODERATING EFFECT? • Multi-group SEM is often used to test moderating effects. X P M Y Z R 8. The model fit can be compared to an alternative model containing the same pattern of free and fixed elements with the exception that the relationship representing the moderating effect is made equal in both groups. This is the less preferred application in most research contexts. continuous variable interactions can be computed and used with SEM. WHY IS IT SO IMPORTANT TO FIRST EXAMINE THE RESULTS OF A MEASUREMENT MODEL BEFORE PROCEEDING TO TEST A STRUCTURAL MODEL? • To assess the construct validity of a set of measures. A multigroup SEM model is conducted as described in the previous chapter for multi-group CFA. EACH AFFECTING A MEDIATING CONSTRUCT. Significant structural relationships between constructs that lack validity would have little meaning. M. then evidence suggests there is no need for a higher order model. X. The procedures that are used for testing moderation in this manner follow closely along with the tests of invariance performed with CFA. Alternatively. 176 . the same SEM model structure is used with both groups. Y AND Z. WHICH IN TURN DETERMINES TWO OTHER OUTCOMES. • 9. That is. If the model holding the relationship equal is not significantly worse. DRAW A STRUCTURAL MODEL HYPOTHESIZING THAT 3 EXOGENOUS CONSTRUCTS. The number of dependent variables and the measurement scale of both independent and dependent variables. The successful categorization of dependence methods requires two characteristics: a. Multivariate analysis is best defined as an extension of univariate analysis and/or bivariate analysis. Factor analysis and cluster analysis b. 3. Multivariate analysis is difficult to define. b. c.SAMPLE MULTIPLE CHOICE QUESTIONS (correct answers indicated by bolded answer) Chapter 1: Introduction Circle the letter of the item which best answers the question. b. d. d. From the list below pick the pair of techniques that can both be classified under the dependence methods. b. The number and measurement scale of the independent variables. 1. Examining relationships between or among more than two variables. c. c. Which of the below statements most adequately defines this type of analysis? a. d. 177 2. Cluster analysis and conjoint analysis Multiple discriminant analysis is useful in situations where: a. The number of independent and dependent variables. a. Multiple discriminant analysis and multivariate analysis of variance c. 4. The number and measurement scale of the dependent variables. The total sample can be divided into groups based on independent variables. The total sample can be divided into groups based on a combination of independent and dependent variables. An analysis in which all variables are interrelated in such ways that their different effects cannot be easily studied separately. The total sample can be divided into groups based on a dependent variable. Multiple regression analysis and multidimensional scaling d. . Multivariate techniques can be classified as either dependence or interdependence methods. The total sample can be divided into groups based on descriptive variables. Simultaneously analyzing multiple measurements using statistical methods. and measuring information. interval scale. c. identifying. If you had nonmetric or qualitative data.5. Analyze the interrelationships among a large number of variables. 7. Assigning. d. b. Testing. 6. Partitioning. c. e. labeling. canonical correlation analysis. multiple discriminant analysis. c. Showing the relation to the amount of the attribute possessed. d. nominal scale. ordinal scale. 9. multivariate analysis of variance. d. Metric dependence method. Multiple regression is best described as a: a. ratio scale. d. b. c. 10. b. one would be using: a. b. Develop meaningful subgroups of individuals or objects. Nonmetric independence method. Data analysis involves the basic nature of: a. and manipulating information. 8. d. Nonmetric dependence method. Assigning numbers which are used to label subjects or objects. Transform consumer judgments of similarity or preference into distances represented in multidimensional space. multiple regression analysis. Cluster analysis is used to: a. An absolute zero point. Metric independence method. evaluating. b. d or a An example of a multivariate technique that would be appropriate for use with a nonmetric dependent variable and a metric independent variable is: a. The most precise measurements available in data analysis. 178 . c. Explore simultaneously the relationship among several independent variables. Measurements with a nominal scale involve: a. b. c. and gathering information. Chapter 2: Examining Your Data Circle the letter of the item that best answers the question. A plot of the residuals. Multiple imputation. c. Identifiable as distinctly different from other observations. A comparison of the missing data with what would be expected if the missing data was totally randomly distributed. pick the item(s) which are not characteristic of a missing data process. the following test is not appropriate: a. A scatterplot. c. c. Missing at random. the following imputation method is most appropriate. Data entry errors. To examine the patterns of missing data and to determine if the missing data are distributed randomly. Censored data. To be considered ignorable. missing data must be: a. Badly designed research questions. b. 6. 5. b. b. From the list below. d. A box plot. d. Categorized as neither beneficial. A stem-and-leaf diagram. d. No imputation method should be used. a. Comparison of the observations with and without missing data for each variable on the other variables. . The following item may be classified as characteristic of outliers: a. Identifiable and able to be accommodated. When missing data are missing at random. c. b. b. A review of each case individually. Case substitution. An analyst can examine the shape of the distribution of a variable through the use of: a. 1. 179 2. Respondent mis-interpretation of the question. Mean substitution. Influential observations which distort multivariate analyses. nor problematic. b. a. 4. c. c. Respondent's refusal to complete an entire questionnaire. d. Analysis of the correlations between dichotomous variables. 3. d. Researcher imposed. The most fundamental assumption in multivariate analyses is: a. Simple transformations change the interpretation of the variables. b. d. b. 9. Independence of error terms. Heteroscedasticity can only be remedied by transformation of the independent variable.0. All of the following statements are true. c. the ratio of a variable's mean divided by its standard deviation should be less than 4. Homoscedasticity. When choosing between the transformation of two variables. Extraordinary observations associated with a specific event. c. d. Linearity. d. 180 . An ordinary value which is unique when combined with other variables. 8. the variable with the smallest ratio of a variable's mean divided by its standard deviation should be chosen. Normality. Normality. 10. Independence of error terms. c. Outliers are not the result of: a. c. Linearity.7. Procedural errors. b. Homoscedasticity. d. except: a. To conduct a transformation. An unrepresentative sample. b. The Box's M test is used to test for the assumption of: a. The primary purpose of factor analysis is: a. nonmetric measurement and nominal. metric measurement and qualitative. b. canonical roots. Factor analysis can identify the: a. and subsequent regression variables. and the residual variance. d. monotone model and the preference model. c. data reduction and summarization of characteristics. b.Chapter 3: Factor Analysis Circle the letter of the item which best answers the question. Raw data variables for factor analysis are generally assumed to be of: a. 181 . 5. extraneous variable(s). nonmetric measurement and ordinal. 100 or more observations. d. chance classifications of the sample size. 25 or less observations. c. to analyze the relationship between a single dependent variable and several independent variables. 1. to transform unidimensional expressions of relationships into multidimensional expressions of the same relationships. b. b. c. 10 or less observations. c. b. linear model and the collinear model. 3. Variance in factor analysis consists of: a. a. metric measurement and quantitative. d. common variance. specific variance. e. error variance. 4. discriminant model and the centroid model. d. 50 or less observations. to study the effect of multiple independent variables measured on two or more dependent variables simultaneously. only 2. d. c. common factor model and the component model. Two basic models the analyst can utilize to obtain factor solutions are the: a. and determine factor loadings. Factor analysis is most preferable with sample signs containing a. d. b. residual variance. c. separate dimensions. c. 6. b. c. orthogonal rotation. data reduction. Unrotated factor solutions always achieve the objective of: a. the larger the residual variance. b. parameter rotation. c. b. The most commonly used technique in factor extraction is called the: a. the larger the loading to be considered significant. In testing the significance of factor loadings. b. factor rotation. centroid rotation. 8. scree test criterion. 10. treatment reduction. d. the smaller the residual variance. main rotation. c. orthogonal rotation. a priori criterion.: a. latent root criterion. d. the larger the sample size. linear rotation. b. percentage of variance criterion. oblique rotation. data summarization. d.7. bivariate reduction. d. 182 . c. the smaller the loading to be considered significant. 9. Two ways to rotate factors in factor analysis are: a. 3. the examination of the errors in prediction. nonmetric statistical technique used to analyze the relationship between a single dependent variable and several independent variables. predict the values of one variable from the values of others. 4. d. colluminar. c. answers a. b. remove the effect of any uncontrolled independent variables on the dependent variables. examine the strength of association between the single dependent variable and the one or more independent variables. c. b. d. determine the appropriateness of using the regression procedure with the problem. b. b. d. homoscedastic. Multiple regression analysis can be used to: a. 2. metric statistical technique used to analyze the relationship between a single dependent variable and several independent variables. c. c. e. c. the transformation of the data to the appropriate formula When the variance of the error terms appears constant over a range of x values the data are said to be: a. b.Chapter 4: Multiple Regression Circle the letter of the item which best answers the question. standardized. 183 . Multiple regression analysis is a: a. looking at the relative size of the sample in comparison to the population size. procedure which involves obtaining a set of weights for the dependent and independent variables to provide the maximum simple correlation. linear. 1. metric statistical technique used to analyze the relationship between a single independent variable and several dependent variables. d. only One way to determine the appropriateness of our predictive model is through: a. the examination of the coefficients of the dummy variables. as a prediction in estimating the size of the confidence interval. d. d. looking at the appropriateness of the F-statistic. determination of the standard deviation of the data. alpha coefficients. confidence level measurement. c. c. c. variable coefficients. multiple estimation. d. 7. c. examining the t-value for the original variables in the equation. 10. ratio scale. to assess the relationship between the dependent and the independent variables. d. testing the criterion variables for slack. The simplest method is: a. measure the variance of the plot of residuals. The coefficient of determination is used: a. parameter elimination. b. 6. beta coefficients. construction of histograms. The two most common approaches to regression analysis are: a. residual plot. b. Backward elimination involves: a. c. backward elimination. to test the different coefficients of each independent variable. the interval scale. b. In testing the normality of error term distribution one can use three procedures. backward elimination. c. as a guide to the relative importance of the predictor variables. background elimination. 8. 9. d. b. d. correlation matrix. computing a regression equation with all the variables.5. b. and then going back and deleting those independent variables which are nonsignificant. examining the collinearity diagnostics. looking at each variable for consideration in the model prior to the inclusion of the variable into the developing equation. The method of elimination of the variables from the regression model is done through a: a. b. The coefficients resulting from standardized data are called: a. 184 . stepwise forward estimation. freedom coefficients. b. unequal costs of misclassifications. standard analysis. a metric dependent variable and several metric independent variables. ordinal and metric dependent variables. normality of the distributions and known high-low ratio. a categorical dependent variable and several metric independent variables. d. d. cutting score. 185 . abnormality of the distributions and a positive centroid. One assumption in deriving discriminant functions is the: a. b. normality of the distributions and unknown depression and covariance structures.Chapter 5: Multiple Discriminant Analysis and Logistic Regression Circle the letter of the item which best answers the question. c. 3. d. ratio and quantitative dependent variables. c. 1. interval and nonmetric dependent variables. Multiple discriminant analysis is a statistical technique which involves: a. b. c. one can arrive at what is called a: a. 2. nominal and qualitative dependent variables. Multiple discriminant analysis is an appropriate analytical technique for using: a. high ratio. an ordinal dependent variable and several nonmetric independent variables. By averaging the discriminant scores in a particular analysis. c. centroid. a categorical dependent variable and several nonmetric independent variables. 4. d. b. correlation. 7. d. to measure the linear correlation between independent variables. to determine the correlations between a single dependent variable and several independent variables. is to estimate the hold-out samples. d. useful in developing a classification matrix that discriminates significantly. to determine the model which possesses the property of additivity and homogeneity. analyzing. to determine if statistically significant differences exist between the high ratios of the defined priori groups. develop a classification matrix to assess the predictions of the function. c. c. rotation. extraction. b. measure the simple linear correlation between each independent variable and the discriminant function. is to maximize the percentage correctly classified. c. calibration. predicting. c. to determine which independent variables account for the most difference in the average score profiles of the groups being analyzed. c. b. with the highest R2 value. a computational method utilized in deriving discriminant functions. Discriminant loadings are most commonly used to: a. necessary in order to clarify the usefulness Mahalanobis D statistic. d. 6. derivation. The simultaneous method is: a. The three stages of discriminant analysis include: a. 9. d. d. b. One objective for applying discriminant analysis is: a. b. The maximum chance criterion should be used when the sole objective: a. b. 186 . 8. and projection. and interpretation. evaluate the group differences. useful when the analyst wants to consider a relatively large number of independent variables for inclusion. and application. derivation. examine the sign and magnitude of the standardized discriminant weights. and rotation.5. is to interpret the magnitude of the standardized discriminant weights. c. validation. d. 187 . and stepwise method. and partial F-values. standardized discriminant weights.10. and standardization. chance models. The interpretation phase of discriminant analysis involves the three methods of: a. cutting score determination. deviation. and partial F-values. discriminant structure. discriminant structure correlations. simultaneous method. b. only metric independent variables but nonmetric dependent variables. b. multiple independent variables measured on two or more dependent variables simultaneously. d. only nonmetric independent variables but metric dependent variables. the effect of any uncontrolled independent variables on the independent variables. d. d. two or more dependent variables on one independent variable simultaneously. multiple independent variables measured on two or more dependent variables separately. b. only nonmetric independent variables and nonmetric dependent variables. 4. 188 . c.Chapter 6: Multivariate Analysis of Variance (MANOVA) Circle the letter of the item which best answers the question. c. the effect of any autocorrelation. the effect of any bivariate factor tendencies. only looking at the centroid. one independent variable on one dependent variable. MANOVA makes use of: a. only looking at the population standard deviation. When using Univariate Analysis of Variance the true population differences can be estimated from: a. c. 2. d. only looking at the whole population. Multivariate Analysis of Variance used in conjunction with covariance analysis is helpful to remove: a. Multivariate Analysis of Variance is a statistical technique which can be used to study the effect of: a. 3. only a sample group. the effect of any nonlinear interaction effects. b. b. only metric independent variables and metric dependent variables. c. 1. random effect factorial design. the dummy variables used in the regression analysis. he has what is known as a: a. Residual or error variance should be: a. unknown mass and variance. b. 8.5. b. nonmetric independent variables are used with nonmetric dependent variables. d. fixed-effect factorial design. unknown standard deviation and mean. metric dependent variables are used with metric independent variables. The primary function of an experimental design is to serve as: a. c. with unequal error variance among the cells. normally distributed. Random effects designs assume the groups being studied are a random sample from a larger population with a: a. c. metric independent variables are used with nonmetric independent variables. 189 . with dependent variables. d. a control mechanism to provide more confidence in your relationships among variables. full factorial design. d. both the Hotelling's T2 statistic and the Mahalanobis D2 statistic. d. 9. In "analysis of variance" designs: a. b. only the Hotelling's T2 statistic. with dependent variables. nonmetric dependent variables are used with nonmetric independent variables. each at two or more levels. When one has two or more factors. with equal error variance among the cells. d. c. unknown mean and known variance. c. 6. he must use: a. 7. known mean and variance. bimodally distributed. only the Mahalanobis D2 statistic in conjunction with Wilk's lambda. b. normally distributed. 10. an analysis sample taken from the whole population. When one is testing the significance of difference among three or more treatment groups. b. a hypothesis for the experiment in general. c. the Wilk's lambda. d. c. not normally distributed. bivariate factorial design. b. b. c. c. factors stimulus. nonmetric. the predictors can be: a. the complexity of the variables d. metric. 190 . Which of the following is not a key issue in designing a conjoint analysis experiment? a. There is a common composition rule for all respondents in the experiment. e. Conjoint measurement attempts to: a. 4. the number of variables b. both metric and nonmetric. d. determine the underlying dimensions of the predictor and response variables. factors factor. d.Chapter 7: Conjoint Analysis Circle the letter of the item which best answers the question. c. c. The variables and their levels are easily communicated. d. There is no measurement error. 5. None of the above. is one of the object's attributes which has several stimulus. or values. group the respondents according to their similarity or dissimilarity. Which of the following is not an assumption of conjoint analysis? a. 2. There is stability of evaluation across all variables and all levels of variables. 3. b. the style of presentation c. b. b. 1. d. interaction terms None of the above. levels part-worth. In conjoint analysis. None of the above. find scales that relate the predictor variables to the response variable using a selected composition rule. the research setting A a. Conjoint analysis is most closely like which of these multivariate techniques? a. linear. a. e. full profile c. quadratic. multidimensional scaling Which of the following is not a feature which distinguishes conjoint analysis from other multivariate techniques? a. assessing the importance of new attributes. predicts relationships for each respondent c.6. The method of presentation of the objects is presumed to be most realistic. a and b only e. c. Realism is not a consideration in presentation methods. cluster analysis d. factor analysis b. analysis of variance (ANOVA) c. d. 10. 8. The part-worth relationship which is most similar to the relationships found in past multivariate techniques is: a. defining segments of consumers with similar part-worth profiles. nonlinear. trade-off b. b. b. e. a and c only 7. can be estimated in which each level of a variable has no relationship to other levels d. separate. assessing preferences for a specified set of objects. incorporating new choice objects into the estimation of the partworths. d. 9. both are equally realistic d. conjoint not comparable in this regard The most appropriate use of a conjoint analysis choice simulator is for: a. discriminant analysis e. can predict nonlinear as well as linear relationships b. 191 . c. All of the above are appropriate uses for choice simulators. d. single linkage. identify the most distinguishing characteristics of the sample. divisive procedure. c. b. c. Euclidean or Mahalanobis distances are among the most common measures of: a. b. 4. The process of combining observations in a stepwise process until only a single cluster remains is: a. complete linkage.Chapter 8: Cluster Analysis Circle the letter of the item which best answers the question. interobject similarity. 2." if any. b. linkage methods. define similarity based on a series of characteristics of the observations. nonhierarchical algorithms. centroid method. hypothesis testing of differences among groups of observations. 192 . d. Ward's method. c. d. among observations based on their characteristics. identify the similarities among the observations of a sample. c. parallel threshold procedure. sequential threshold procedure. d. d. b. 1. b. 5. define "natural groupings. 3. The agglomerative procedure most likely to produce equal size clusters is: a. assess the characteristics of the sample most important in defining similarities among the sample. define a relationship between group membership and characteristics of the observations. c. agglomerative procedure. hierarchical algorithms. An objective of cluster analysis is to: a. Cluster analysis is used for: a. develop a representative profile of the population in terms of characteristics. b. In the absence of absolute guidelines. a and c e. The linkage technique that minimizes the effects of extreme observations is: a. Ward's method. 10. The validation stage involves: a. Which of the following is of most concern when using a nonhierarchcial technique? a. average linkage. d. 193 . complete linkage. c. assessing the accuracy of the similarity measure used. the number of clusters selected to represent the "natural groupings" of the sample are determined by: a. represents similarity in a single multivariate measure. c. 8. determining that significant differences do exist between the clusters based on the characteristics used to define similarity d. single linkage.6. only can make the decision based on all of these measures. practical considerations. the specification of seed points d. 9. All of the above. the measure of similarity b. defines the sample in a small number of clusters. identifies the most distinguishing characteristics of the differences among clusters. ensuring that the cluster solution is representative of the population. c. Cluster analysis is best seen as a data reduction technique when it: a. None of the above. d. d. measures of internal consistency. the linkage method c. construction of the dendogram 7. b. b. theoretical or conceptual guidelines. c. b. preference data. optimal stimuli. 5. underlying factors. transforming unidimensional expressions of relationships into multidimensional expressions of these same relationships. dependence method. Multidimensional scaling is used for: a. d. 4. a. main stimuli. component stimuli. d. Multidimensional scaling is an: a. 194 . collinear method. 1. c. In MDS. and then explaining these variables in terms of their common. c. c. differentiating multivariate from univariate variables. testing the hypothesis that the group means of two or more groups are equal. c. only 2.Chapter 9: Multidimensional Scaling Circle the letter of the item which best answers the question. Multidimensional scaling categorizes variables according to the: a. if we locate the point which represents the most preferred combination of perceived attributes. predictor data. d. c. d. classification method. b. one would have a spatial representation of a subject's: a. b. b. interval versus external data. 3. level of measurement assessed for input and output. b. b. ideal stimuli. d. c. aggregate versus disaggregate data. treatment data. b. The data that was gathered by having subjects evaluate stimuli and order them in preference according to some property is called: a factor data. interdependence method. analyzing the interrelationships among a large number of variables. e. 8. c. metric output.6. a. b. metric input and metric output. One procedure for gathering preference data is to rate each stimuli on a(n): a. b. disjoint scale. nonmetric methods assume the use of: a. perceptions are not stable over time d. b. metric methods assume the use of: a. b only In MDS. External preference analysis Which of the following is not an assumption of MDS? a. nominal input and ordinal output. Correspondence analysis b. nonmetric input and nonmetric output. c. all of the above are assumptions of MDS 7. Internal preference analysis e. explicit scale. implicit scale. respondents place different levels of importance on the dimensions of an object c. ordinal input. d. 9. c. nonmetric output. nominal input. 195 . is the perceptual mapping technique based on the association of nominal data. factor input and trace output. Individual difference analysis (INDSCAL) c. In MDS. factor scale. Property fitting (PROFIT) d. d only f. d. a. 10. all respondents will perceive objects in the same dimensionality b. d. c. d. b. Constructs Measured variables Latent factors Composite factors Parameter coefficients SEM can be thought of as a combination of which two techniques? MANOVA and ANOVA MANOVA and Exploratory factor analysis Factor analysis and multiple regression analysis MANOVA and multiple regression analysis not a. b. e. 196 . d. c. Which of the following is not a distinguishing characteristic of SEM? The estimation of multiple and interrelated dependence relationships in a single analysis. e. c. observation or some other measurement device. c and d 1) a. e. The path analysis between employee pay satisfaction and employee loyalty is .5. 2) a. b. 4) _______________ are the actual items that are measured using a survey. b. c.Chapter 10: Structural Equation Modeling Circle the letter of the item which best answers the question. The ability to identify multiple cluster solutions in a single analysis a. b. 3) a. d. An ability to represent unobserved concepts in these relationships and correct for measurement error in the estimation process thus providing for more accurate relationships. This allows an assessment of fit and provides a better tool for assessing the construct validity of a set of measures and a better examination of the accuracy of a model. b. c or d A researcher tests a structural model. What does this provide evidence of? sequentially nonspurious association covariance theoretical support interpretational confounding c. a. d. A focus on explaining the covariance among the measured items. e. 97 There is no single accepted cut-off value. c. d. e.90 . The construct is represented by a box with no arrows. c or d 6) 7) Which is not an absolute fit index? CFI SRMR RMSEA chi-square GFI Which of the following is the accepted cut-off value for the TLI and CFI that distinguishes a good from a poor fit? . b. e. a.07 . b. The construct is represented by an oval with arrows going into it. How is an exogenous construct represented in an SEM path model? The construct is represented by a box with arrows going into it. a. minimize the RMSEA. b. d. 5) SEM algorithms are designed to: maximize the chi-square likelihood function. c.95 . e. not a. d. The construct is represented by an oval with arrows going into it from other constructs (ovals). 8) a. minimize the covariances between constructs. The construct is represented by an oval with no arrows going into it. c. 197 . d. b. c. e.a. b. minimize the difference between the observed and estimated covariance matrices. RMSEA. If a researcher uses four fit indices to assess a CFA fit. Discriminant validity d. c. 4. Which is a way to help make sure that the rank and order conditions for identification are satisfied? a. the specification of the number of factors c. RMSEA. SRMR.Chapter 11: Confirmatory Factor Analysis Circle the letter of the item which best answers the question. TFI c. the formative indicator rule c. all between-construct error covariance terms are estimated. CFI. χ2. Convergent validity e. all measured variables must load on exactly 1 construct. all measured variables are free to load on any factor. Construct validity b. a. the three indicator rule b. If a researcher draws a path model and wishes it to be congeneric. use of a Heywood case d. a CFI of . TFI. d. whether the technique is confirmatory or exploratory e. GFI 198 . the size of the loading estimates considered valid d. all measured variables must load on each factor.95 or higher e. χ2 d. b. Face validity c. b.: a. 1. which of the following sets is best to use? a. RMSEA. c and d each contrast EFA with CFA 2. GFI. GFI. Which of the following is not a difference contrasting EFA with CFA? a. GFI b. Test-retest validity 3. SRMR. all latent constructs load on one factor. larger sample size 5. a.R . the specification of which variables load on which factors b. CFI. _________ is established when the measured items are conceptually consistent with a construct definition. e. 60 c. a standardized factor loading estimate of . What type of cross-validation is sufficient to draw this conclusion? a. AGFI 6. loose cross-validation b. a standardized residual of 3. b and c are each sufficient 7. SRMR c.e. CFI. χ2. partial scalar invariance 8. GFI. Δχ2 b. theoretical inspection suggesting it does not match the definition as well as do other items d. GFI 199 . full metric invariance c. Which statistic is most useful for testing invariance and drawing conclusions about differences between groups? a. What conditions below are sufficient to justify deleting a scale item from analysis? a.0 b. a and b e. a. R2 d. The researcher is specifically interested in whether or not the same scales can be used to capture the same constructs in each country. AFGI e. factor loading equivalence e. strong factorial invariance d. Suppose a researcher is interested in comparing measurement theory results from respondents in South Korea with respondents from Australia. d. Hypothesized relationships between constructs are tested using SEM much the same as they are tested using: cluster analysis. one-headed straight arrows two-headed straight arrows two-headed curved arrows ovals one-headed dashed arrow A proposed structural model involves six constructs. which in turn causes D and E. exploratory factor analysis. multiple discriminant analysis. c. 3. e. two of which are exogenous. Dependence relationships are represented with __________ in a path model. multiple regression analysis. The overall fit is kept the same. where E causes both D and F. d. b. multidimensional scaling. 200 . 4. c. a. b. e. A and B together cause C. b. c. The endogenous constructs become exogenous. e. d. The correlation estimates between constructs are replaced with either a one-headed arrow or simply deleted depending on the theory. What term describes this model? confirmatory theoretical moderating nonrecursive recursive a. e. d.Chapter 12: Testing a Structural Model Circle the letter of the item which best answers the question. a. b. 1. Which of the following is a change made to convert a measurement model into a hypothesized structural model? The error terms are correlated with each other. a. 2. The exogenous constructs each become endogenous. c. a. e. c. e. Moderation Mediation Partial mediation Post hoc analysis Structural theory test When a theory hypothesizes that two constructs in a six construct model are unrelated to each other. e. 201 a. d. The sample size is probably too large. d. c. c. c. a path is tested where the original theory did not indicate a path. be assigned a value of 1. b. Management dismisses the finding. . The model fit is poor. 7. be drawn as a double headed arrow. b. b. a. a. the path between those two constructs should: be “free. The researcher determines that a significant relationship exists between salesperson intelligence and sales volume. d. 8. b. e. b.” be “fixed. 6. An indirect effect A mediating effect A moderating effect A direct effect Correlation ________ occurs when the relationship between a predictor and an outcome is reduced but remains significant when a mediator is also entered as an additional predictor. c. In other words. Moderation Mediation Partial moderation Partial mediation Parsimony After-the-fact tests of relationships for which no hypothesis was theorized. What is a reason? The relationship is trivial.5. a.” be drawn as a single headed arrow. 9.06. The sample size is probably too small. d. The standardized path estimate is significant and equal to . d. e. A researcher tests a structural model of sales performance using a sample of 760 professional salespeople. A and C ________ occurs when a third variable or construct changes the relationship between two related variables/constructs.