APPLIED UNIVARIATE, BIVARIATE, AND MULTIVARIATE STATISTICS

Preface xix About the Companion Website xxxiii 1 Preliminary Considerations 1 1.1 The Philosophical Bases of Knowledge: Rationalistic versus Empiricist Pursuits, 1 1.2 What is a “Model”?, 4 1.3 Social Sciences versus Hard Sciences, 6 1.4 Is Complexity a Good Depiction of Reality? Are Multivariate Methods Useful?, 8 1.5 Causality, 9 1.6 The Nature of Mathematics: Mathematics as a Representation of Concepts, 10 1.7 As a Social Scientist, How Much Mathematics Do You Need to Know?, 11 1.8 Statistics and Relativity, 12 1.9 Experimental versus Statistical Control, 13 1.10 Statistical versus Physical Effects, 14 1.11 Understanding What “Applied Statistics” Means, 15 Review Exercises, 15 2 Mathematics and Probability Theory 18 2.1 Set Theory, 20 2.1.1 Operations on Sets, 22 CONTENTS vi CONTENTS 2.1.2 Denoting Unions and Intersections of Many Sets, 23 2.1.3 Complement of a Set, 24 2.2 Cartesian Product A × B,24 2.3 Sets of Numbers, 26 2.4 Set Theory Into Practice: Samples, Populations, and Probability, 27 2.5 Probability, 28 2.5.1 The Mathematical Theory of Probability, 29 2.5.2 Events, 29 2.5.3 The Axioms of Probability: And Some of Their Offspring, 30 2.5.4 Conditional Probability, 31 2.5.5 Mutually Exclusive versus Independent Events, 32 2.5.6 More on Mutual Exclusiveness, 34 2.6 Interpretations of Probability: Frequentist versus Subjective, 35 2.6.1 Law of Large Numbers, 36 2.6.2 Problem with the Law of Large Numbers, 37 2.6.3 The Subjective Interpretation of Probability, 37 2.7 Bayes’ Theorem: Inverting Conditional Probabilities, 39 2.7.1 Decomposing Bayes’ Theorem, 40 2.7.2 A Medical Example—Probability of HIV: The Logic of Bayesian Revision, 41 2.7.3 Recap of Bayes’ Theorem: The Idea of Revising Probability Estimates and Incorporating New Data, 42 2.7.4 The Consideration of Base Rates and Other Information: Why Priors Are Important, 42 2.7.5 Conditional Probabilities and Temporal Ordering, 43 2.8 Statistical Inference, 44 2.8.1 Shouldn’t the Stakes Matter?, 45 2.9 Essential Mathematics: Precalculus, Calculus, and Algebra, 48 2.9.1 Polynomials, 48 2.9.2 Functions, 48 2.9.3 What is a Mathematical Function?, 49 2.9.4 Spotting Functions Graphically: The Vertical-Line Test, 50 2.9.5 Limits, 52 2.9.6 Why Limits? How Are Limits Useful?, 54 2.9.7 Asymptotes, 55 2.9.8 Continuity, 56 2.9.9 Why Does Continuity Matter? Leaping from Rationalism to Empiricism, 58 2.9.10 Differential and Integral Calculus, 59 2.9.11 The Derivative as a Limit, 61 2.9.12 Derivative of a Linear Function, 62 2.9.13 Using Derivatives: Finding Minima and Maxima of Functions, 63 2.9.14 The Integral, 64 2.9.15 Calculus in R, 65 2.9.16 Vectors and Matrices, 66 2.9.17 Why Vectors and Matrices?, 66 2.9.18 Solving Systems of Linear Equations, 70 2.10 Chapter Summary and Highlights, 72 Review Exercises, 74 3 Introductory Statistics 78 3.1 Densities and Distributions, 79 3.1.1 Plotting Normal Distributions, 82 3.1.2 Binomial Distributions, 84 3.1.3 Normal Approximation, 87 3.1.4 Joint Probability Densities: Bivariate and Multivariate Distributions, 88 3.2 Chi-Square Distributions and Goodness-of-Fit Test, 91 3.2.1 Power for Chi-Square Test of Independence, 96 3.3 Sensitivity and Specificity, 98 3.4 Scales of Measurement: Nominal, Ordinal, and Interval, Ratio, 98 3.4.1 Nominal Scale, 99 3.4.2 Ordinal Scale, 100 3.4.3 Interval Scale, 100 3.4.4 Ratio Scale, 100 3.5 Mathematical Variables versus Random Variables, 101 3.6 Moments and Expectations, 103 3.6.1 Sample and Population Mean Vectors, 104 3.7 Estimation and Estimators, 106 3.8 Variance, 108 3.9 Degrees of Freedom, 110 3.10 Skewness and Kurtosis, 111 3.11 Sampling Distributions, 113 3.11.1 Sampling Distribution of the Mean, 113 3.12 Central Limit Theorem, 116 3.13 Confidence Intervals, 117 3.14 Bootstrap and Resampling Techniques, 119 3.15 Likelihood Ratio Tests and Penalized Log-Likelihood Statistics, 121 3.16 Akaike’s Information Criteria, 122 3.17 Covariance and Correlation, 123 3.17.1 Covariance and Correlation Matrices, 127 3.18 Other Correlation Coefficients, 128 3.19 Student’s t Distribution, 131 3.19.1 t-Tests for One Sample, 132 3.19.2 t-Tests for Two Samples, 136 3.19.3 Two-Sample t-Tests in R, 137 3.20 Statistical Power, 139 3.20.1 Visualizing Power, 140 3.20.2 Power Estimation Using R and G∗Power, 141 3.20.3 Estimating Sample Size and Power for Independent Samples t-Test, 144 3.21 Paired Samples t-Test: Statistical Test for Matched Pairs (Elementary Blocking) Designs, 146 3.22 Blocking with Several Conditions, 149 3.23 Composite Variables: Linear Combinations, 149 3.24 Models in Matrix Form, 151 3.25 Graphical Approaches, 152 3.25.1 Box-and-Whisker Plots, 153 3.26 What Makes a p-Value Small? A Critical Overview and Simple Demonstration of Null Hypothesis Significance Testing, 155 3.26.1 Null Hypothesis Significance Testing: A History of Criticism, 156 3.26.2 The Makeup of a p-Value: A Brief Recap and Summary, 159 3.26.3 The Issue of Standardized Testing: Are Students in Your School Achieving More Than the National Average?, 159 3.26.4 Other Test Statistics, 161 3.26.5 The Solution, 161 3.26.6 Statistical Distance: Cohen’s d, 162 3.26.7 What Does Cohen’s d Actually Tell Us?, 163 3.26.8 Why and Where the Significance Test Still Makes Sense, 163 3.27 Chapter Summary and Highlights, 164 Review Exercises, 167 4 Analysis of Variance : Fixed Effects Models 173 4.1 What is Analysis of Variance? Fixed versus Random Effects, 174 4.1.1 Small Sample Example: Achievement as a Function of Teacher, 175 4.1.2 Is Achievement a Function of Teacher?, 176 4.2 How Analysis of Variance Works: A Big Picture Overview, 178 4.2.1 Is the Observed Difference Likely? ANOVA as a Comparison (Ratio) of Variances, 179 4.3 Logic and Theory of ANOVA: A Deeper Look, 180 4.3.1 Independent Samples t-tests versus Analysis of Variance, 181 4.3.2 The ANOVA Model: Explaining Variation, 182 4.3.3 Breaking Down a Deviation, 184 4.3.4 Naming the Deviations, 185 4.3.5 The Sums of Squares of ANOVA, 186 4.4 From Sums of Squares to Unbiased Variance Estimators: Dividing by Degrees of Freedom, 189 4.5 Expected Mean Squares for One-Way Fixed Effects Model: Deriving the F-Ratio, 190 4.5.1 Expected Mean Squares Between, 192 4.5.2 Expected Mean Squares Within, 194