Summary of the lectures of Measurement theory & assessment 2. Also summarised most of the chapters from Psychometrics: An introduction 3rd edition R. Micheal Furr (chapters required by the course).
INDEX
1. Psychometrics & The importance of Psychological Measurement 2
2. Scaling 4
3. Individual Differences & Correlations 6
® Individual Differences & Interpreting Test Scores 6
® Correlations (Linear & Logistics Regression) 8
4. Test Dimensionality & Factor Analysis 10
® Exploratory Factor Analysis 11
® Confirmatory Factor Analysis 14
5. Reliability 15
® A Conceptual Basis 15
® Empirical Estimates of Reliability 17
® The Importance of Reliability 19
6. Validity 21
® A Conceptual Basis 23
® Estimating and Evaluating Convergent and Discriminant Validity Evidence 25
7. Test Bias 29
® Detecting Construct Bias 29
® Detecting Prediction Bias 30
8. Item Response Theory (IRT) and Rasch Models 32
This summary includes (almost) everything from the lectures & the book
This summary includes quite some formulas/calculations. Most of the time, these are just used to make things a little
clearer. The lecturer stated that the questions in the exam will mostly be conceptual à so no calculations etc. You do not
have to learn all the formulas by heart. Make sure you understand the underlying conceptual foundation that leads to the
formulas. The Q&A (13th dec.) lecture is not included
CLAIMER
This summary is made by a student!
Studying from it and relying on it for 100% is your own responsibility.
THANKS & GOOD LUCK!!! J
J YOU CAN DO IT !!!!
, 2
Psychometrics & The importance of Psychological Measurement (Furr, Ch. 1)
- Psychological testing is important in many careers! Not e.g. just in clinical psychology or in research
- Measuring behavior can be for 2 purposes
o To actually measure observable events in the physical world
o To make inferences/assess unobservable psychological states (e.g. intelligence)
§ 3 things to keep in mind!
1. You make an inference from an observable behavior to an unobservable entity (à
important for validity…)
This does not mean that suddenly the results can never be valid! Many other branches
of science make inferences about unobservable entities (e.g. physicists about gravity!)
2. The task must be theoretically linked to the concept you are trying to measure
3. The theoretical concept is a hypothetical construct or a latent variable, measured
using operational definitions
Cronbach:
Psychological tests: definition and types “a test is a systematic
- 3 important components of a test procedure to compare
§ They involve behavioral samples of some kind the behavior of two or
§ They must be connected in some systematic way more people.”
§ The purpose of a test is to compare behavior of 2 or more people
- A general definition of the construct you are measuring is important
o There can be many “types of output” (qualitative/quantitative)
o General purpose either to compare the behavior of different people (interindividual) or to
compare the behavior of the same person at different points in time (intraindividual)
- Tests can vary on many different dimensions:
o Content (e.g. intelligence vs personality tests) The distinction is often
o Type of response required (e.g. open vs closed questions) blurred. Criterion referenced
o Intended purpose of score (criterion vs norm) tests are always normed in
§ Criterion referenced – used when decision must be made about some sense… However, there
are different methods to
a person’s skill level
assess the quality of criterion
§ Norm reference – used to understand how a person compared and norm reference tests.
with other people (by comparing a person to a reference
sample or a norm sample).
o Type of assessment (e.g. speed vs power tests)
Psychometrics
- Psychometrics – assessing the attributes of psychological tests. Three important attributes are 1) the
type of information generated by the test, 2) the reliability, and 3) the validity.
- Brief history
o The two foundations of psychometrics are the practice of psychological testing and measurement,
and the development of particular statistical concepts and procedures
o In the 30s/40s the field of psychometrics started to develop
§ Thurstone & Guilford
o 70s à Cronbach à Classical Test Theory (CTT)
Measurement in psychology
- Measurement – the assignment of numerals (symbols) to characteristics of individuals
- Purpose of measurement = to identify individual differences
o Difference in test-scores must arguably be related to the difference on the relevant underlying
psychological attribute
o Differential psychology – the study of individual differences
, 3
§ all psychologists should be concerned with psychometrics, no matter their specialty!
- Challenges to measurement: All measures are imperfect, because …
o Psychological constructs cannot be measured directly (they are complex)
o No single approach to measure any construct is universally accepted
o (psychological) measurement is subject to error à reliability
o Well defined units on measurement scales are lacking
o Participant reactivity
§ Demand characteristics – changing behavior to accommodate the researcher
§ Social desirability – change behavior to try and impress the researcher
§ Malingering – change behavior to make a poor impression
o Expectation & bias effect
o Psychologists rely on composite scores – a total score resulting from adding-up of subscores
o Score sensitivity – the ability of a measure to discriminate adequately between meaningful
amounts or units
- Lack of awareness of important psychometric info!!
Path diagrammatic representation
- Latent variables (unobservable) is shown in a circle/oval shape
o Examples are psychopathology, personality traits, cognition,
developmental stage etc.
o Error also in circle à latent!!
- Observable (e.g. item score) are shown in squares
3 important “roles”: the role of Statistics, Theory and Causality
- The role of Statistics
o Using a linear regression to relate ¡ à o (spot the b1) and to determine
how much of the item responses is explained by the latent variable
o Using a logistic regression to do this for binary/dichotomous variables
o Using correlation (Pearson’s r) to express the linear relationship between the items (e.g. item 1, 2,
3 are subject to the same source)
o Using probability for responses (how likely is it that you will answer yes to this question?)
§ Correlation & probability can be conditional (will be explained more later)
- The role of Theory
o Psychological theory informs statistics
o Latent variable à you have to know the distribution of the variable in the population in order to
use statistics à theoretical distribution affects what analysis to use or what tests
§ Not all latent variables are continuous (e.g. Piaget’s stages)
§ Is the distribution normal? Is it skewed?
o Theory is used to decide what is relevant to measure, e.g. for depression à what behavior/items is
related to depression?
§ Conceptual definition of the latent variable
§ Determine relevant observable variables
- The role of Causality
o My position on the latent variable (e.g. depression continuum) determines the probability of my
response to the (depression) item
§ The items are reflective – they reflect the essence of the latent variable
§ Another example is a medical test for virus
o The virus itself is not directly observable… but we can make up items/causal
indicators of the influential virus (e.g. runny nose (yes/no), sneezing (yes/no).
o The virus causes the runny nose. The runny nose does not cause the virus.
o Not all items on all tests are/have to be reflective!
§ E.g. the Apgar score (used to determine babies health directly after birth)
, 4
o The score is not causal for appearance, but appearance contributes to the score
o The sub-concepts are causally related but they have no relation with the overall index
o These items are formative in a formative model – the item scores determine the
APGAR score, the APGAR score does not cause the item scores
§ There can be a lot of debate on whether an indicator is formative or reflective.
o Causal interpretation of group differences
§ E.g. sex differences on spatial ability
§ When someone states there is a difference in spatial ability between males and females, we
assume that this difference lies in the latent variable and not in the items.
o But what if that assumption is false, and all the items have equal chances for each sex
to answer it correctly, except for one item which favors males hugely compared to
females? Then there is not a difference in the actual latent variable but in the item
that is supposedly representative of the latent variable.
o We would say that there is a bias in the items. Hypothetically, there is a possibility to
have a woman who has greater visuospatial abilities, while we have a man who has
higher probability to answer a specific item correctly (maybe with lower visuospatial
ability).
o This is related to the issue of dimensionality (à what factors influence the item
results?)
Scaling (Furr, Ch. 2)
Fundamental issues with numbers
- 3 important numerical properties
Property of identity
- Def. – the ability to reflect sameness vs differentness
- Differentiate between categories of people
o E.g. ill vs not ill female vs male
- Rules of categorizing
o People within a category must satisfy the condition of that category
o Categories must be mutually exclusive – you can only be in 1 category at a time
o Categories must be exhaustive – you need to “cover” each individual
- Numerals serve as labels for categories
Property of order
- Def. – when numerals indicate the rank order of people relative to each other along some dimension
o Again, numerals are essentially labels!
- Does not tell us anything about the actual degree of difference
o i.e., does not show how much difference there is between rank no. 1 and 2
Property of quantity
- Def. – when numerals provide information about the magnitude of differences between people
- Each numeral represents a count of basic units
The number 0
- 2 meanings
o Absolute zero – denying the existence of an attribute (e.g. Reaction Time)
o Arbitrary zero – when the 0 is an arbitrary quantity of the attribute (e.g. 0 degrees Celsius)
- What does it mean to score 0 on e.g. a depression test?
- Problem for 0 – Thorndike example
o Child gets spelling test of 10 words, spells everything wrong
o Test-score = 0 (test itself has absolute 0)
o But likely the child’s spelling abilities are not 0!!! à so is the test score a relative or absolute 0?
Voordelen van het kopen van samenvattingen bij Stuvia op een rij:
Verzekerd van kwaliteit door reviews
Stuvia-klanten hebben meer dan 700.000 samenvattingen beoordeeld. Zo weet je zeker dat je de beste documenten koopt!
Snel en makkelijk kopen
Je betaalt supersnel en eenmalig met iDeal, creditcard of Stuvia-tegoed voor de samenvatting. Zonder lidmaatschap.
Focus op de essentie
Samenvattingen worden geschreven voor en door anderen. Daarom zijn de samenvattingen altijd betrouwbaar en actueel. Zo kom je snel tot de kern!
Veelgestelde vragen
Wat krijg ik als ik dit document koop?
Je krijgt een PDF, die direct beschikbaar is na je aankoop. Het gekochte document is altijd, overal en oneindig toegankelijk via je profiel.
Tevredenheidsgarantie: hoe werkt dat?
Onze tevredenheidsgarantie zorgt ervoor dat je altijd een studiedocument vindt dat goed bij je past. Je vult een formulier in en onze klantenservice regelt de rest.
Van wie koop ik deze samenvatting?
Stuvia is een marktplaats, je koop dit document dus niet van ons, maar van verkoper evabus. Stuvia faciliteert de betaling aan de verkoper.
Zit ik meteen vast aan een abonnement?
Nee, je koopt alleen deze samenvatting voor €8,49. Je zit daarna nergens aan vast.
