Chapter 1: Basic Concepts in Psychological Measurement
1.1 Some simple statistical ideas
1.1.1 Levels of Measurement
● It is more difficult to measure someone’s psychological ability because it does not
include a true zero
● Someone’s psychological measurement is different from physical measurements such as
height and weight (height and weight can be 0 and the 0 is meaningful)
● In psychological measurement, someone might score 0 on an intelligence test but it is not
very meaningful to think that someone actually has 0 intelligence
○ Because psychological measurements do not usually have a true zero level, the
ratios between measurements are not meaningful
■ We cannot really say that one person is twice as intelligent of 50 percent
more unconventional than another person
● So in sum, psychological measurements do not have a true 0 and
the 0 is not meaningful
● However, there are still ways in which we can meaningfully compare people’s levels of
psychological characteristics
○ One way is simply to rank people according to their scores
■ We could measure people’s ;levels of a characteristic by whatever method,
and then record their positions relative to each other
● For example, 1st, 2nd, 3rd etc
● Ranks cannot always be useful either because there might be a big
difference between the first and second rank but a smaller
difference between the third and fourth difference
● In measuring people’s characteristics therefore psychologists
would like to obtain scores that have meaningful differences
between them
■ For example, if we are trying to measure the trait of assertiveness, we
would like to be confident that a score of 60 rally does mean a level of
assertiveness that is halfway between the levels indicated by 50 and 70
● The important thing for comparing people and calculating averages
is simply that equal differences or intervals between scores
represent roughly equal differences in the level of the trait
○ For example, in an IQ test, psychologists do hope that a
difference between 110 and 120 is really the same as
between 130 and 140
1.1.2 Standard Scores
● Psychological characteristics are not measured in any particular units and it does not
matter how high or low the scores on a characteristic tend to be as long as the differences
1
, between scores are meaningful
○ But differences in the numbers used for measuring variables might cause
difficulties when we want to compare someone’s scores across two or more traits
■ When you are comparing two different scores that both have different
averages, it is important to turn the raw score into standard scores
■ The first step in calculating a standard score is to take an individuals
original (raw) score on a given scale and then subtract the mean score (the
average score) for the persons who have been measured
■ This difference between the individual’s score and the mean score tells us
whether the person is above the average (if the difference is positive) or
below the average (if the difference is negative)
1.1.4 Sample Representativeness and Sample Size
● To gain accurate estimates of avengers or of correlations, researchers must measure
variables in groups of people: that is, in samples of people that meet 2 important
requirements
○ 1. The first requirement is that the sample of people should be reasonably
representative of the population that the researcher wants to learn about
■ If a researcher wants to learn about the typical political attitudes in a
sample that resembles the population in all the ways that might be related
to political attitudes
○ 2. The second requirement is that the sample of people should be reasonably
large. Even if samples are representative of the general population, they will not
all give the same results
■ If the sample size is larger than it is more reliable
○ Some results from computer simulations suggest that correlations compared from
samples of 250 people usually are reasonably close to the population correlations
and these results correspond well with researcher’s experience
■ So 250 people seems to be the kind of sweet spot
1.2 Assessing quality of measurement: reliability and validity
1.2.1 Reliability
● The reliability of a measurement is the extent to which it is consistent with similar
measurements of the same characteristics
○ When there is a good agreements between measurements, this tells us that they
are in large part assessing some real characteristic, rather than just being
meaningless random numbers
1.2.1.1 Internal-Consistency Reliability
● If we average a person’s score across many items, then the error associated with any
2
, single item will tend to be canceled out
● The overall average score will therefore have less error and it will be a more reliable
indication of whatever the items have in common
● To maximize reliability, in other words, to obtain a very strong measure of the common
characteristic measured by a set of items - it is better to have both a large number of items
and high correlations between the items
○ To some extent, a large number of items can compensate for low correlations
between items and high correlations between items can compensate for a small
number of items
○ This kind of reliability is called internal-consistency reliability, because it depends
in part on the extent to which the items in a test are correlated with each other or
“internally consistent”
1.2.1.2 Interrater Reliability
● Another kind of reliability that is not based on the items of test but instead on the
observers or raters of a characteristic
● Interrater reliability is about there being different/multiple observers/raters
○ For example clinical interviews might be conducted and there are 3 different
psychologists who are each given an interpretation of the person’s personality
■ We will assume that all the psychologists will come more or less to the
same conclusions
■ If we have more psychologists and more observers and take the average
then it is more reliable and it is also more reliable if the psychologists are
closer to each other in terms of their skills
■ If they are all skilled then they will hopefully provide similar
interpretations of the people they are observing
1.2.1.3 Test-Retest Reliability
● In a test-retest situation it is possible that we see some fluctuations in how people are
performing from the first test to the next
1.2.2 Validity
● The validity of a measurement is the extent to which it assesses the same characteristic
that it is supposed to assess
○ When we evaluate validity, we are concerned with precisely which characteristics
is being assessed
1.2.2.1 Content Validity
● Content validity is literally as it states in the name about measuring the content of a study
and it is about whether or not the study is measuring what it is supposed to be measuring
3
, ● If we have a questionnaire with questions on it that aims to measure depression, you
shouldn't be asking questions about their personals levels of anxiety for example
1.2.2.2 Construct Validity: Convergent and Discriminant
● The word “construct” indicates that the property being measured is imagined or
constructed not something concrete or tangible
○ Therefore, characteristics such as intelligence are constructs
○ When we say that a measurement has a construct validity, we are saying that this
measurement really does assess the same construct that we are trying to assess
○ Researchers evaluate the construct validity of a measurement by examining its
correlations with other variables
● One aspect of construct validity is called convergent validity
○ When we examine the convergent validity of a scale, we investigate how strongly
it correlates with variables that measure characteristics very similar (or opposite)
to the one that you scale is intended to measure
○ Either there is a strong positive correlation between two ways of testing or strong
negative but since your measurements are assessing the same thing there should
always be a strong correlation whether positive or negative
● Another aspect of construct validity is called discriminant validity
○ When we look at the discriminant validity, we are looking at how strong the
correlation is with variables that measure characteristics unrelated to the one our
scale is intended to measure
○ For example students performance on a math test and their math aptitude should
not be correlated with their self-esteem
■ These things are unrelated !
1.2.2.3 Criterion Validity
● Criterion validity is also called predictive validity
● Can we use an assessment to then predict a diagnosis later in life
1.3 Methods of measurement: self and observer reports, direct observations and biodata
1.3.1 Self-Reports
● This method is known as self-report and it is the most widely used method of measuring
personality characteristics
● Perhaps the main reason for the popularity of self report is the belief that it usually
provides a fairly accurate measurement and does so cheaply and efficiently
● People evaluate themselves
1.3.2 Observer reports
● Observer reports can be good in the sense that an observer might be able to judge the
4
