PsychologicalAssessment PYC4807
THEME 1
Psychometric theory
Psychometric properties
• The purpose of testing needs to be specified before an assessment battery can be assembled
• Assessment battery: the combination of psychological assessment measures that will be used
• A measure is evaluated in terms of a number of characteristics before including it in the battery:
1. Consider what attribute, characteristic or construct it measures.
2. Its appropriateness for an individual, group or organisation should be determined.
3. Determine if the measure is psychometrically sound (validity, reliability and equivalence).
The Employment Equity Act
• The new Constitution and the Labour Relations Act in 1996, worker unions and individuals forbids any
discriminatory practices in the workplace and includes protection for applicants as they have all the rights of
current employees in this regard.
• To ensure that discrimination is addressed within the testing arena, the Employment Equity Act No. 55 of
1998, Section 8) refers to psychological tests and assessment specifically and states:
• Psychological testing and other similar forms of assessments of an employee are prohibited unless the test or
assessment being used:
(a) Has been scientifically shown to be valid and reliable
(b) Can be applied fairly to all employees
(c) Is not biased against any employee or group
(d) Has been certified by the Health Professions Council of South Africa or any other body which may be
authorised by law to certify those tests or assessments' (added as an amendment – but later removed).
• In South Africa because many of the measures currently in use, whether imported from the US and Europe or
developed locally, have not been investigated for bias and have not been cross-culturally validated.
• It would be useful if test publishers and distributors could certify a measure as being 'Employment Equity Act
Compliant' as this will aid assessment practitioners when selecting measures.
• However, given the variety of cultural and language groups in this country, bias investigations would have to
be conducted for all the subgroups on whom the measure is to be used before approval. Alternatively, it
would have to be clearly indicated for which subgroups the measure has been found to be unbiased and that
it is only for these groups that the measure complies with the act.
• The advent of the Employment Equity Act has forced assessment practitioners to take stock of the available
measures in terms of their quality, cross-cultural applicability, the appropriateness of their norms, and the
availability of different language versions.
,• To this end, the Human Sciences Research Council (HSRC) conducted a survey of test use patterns and
needs of practitioners in South Africa. It was found that most of the tests being used frequently are in need of
adapting for our multicultural context or require updating, and appropriate norms and various language
versions should be provided.
• For at least three decades the HSRC almost exclusively developed and distributed tests In South Africa.
However, at the start of the 1990s the HSRC was restructured, became unsure about the role that it should
play in psychological and educational test development, and many staff with test-development expertise left
the organisation.
• Consequently, since the South African adaptation of the Wechsler Adult Intelligence Scales-Ill (WAIS-Ill) in
the mid-1990s, the HSRC has not developed or adapted any other tests and its former tests that are still in
circulation are distributed by Mindmuzik Media now.
• Some international test development and distribution agencies such as SHL and Psytech have agencies in
South Africa.
• Local test agencies Jopie van Rooyen & Partners SA and Mindmuzik Media. Each of these agencies has a
research and development section, and much emphasis is being placed on adapting international measures
for the South African context.
• There is a greater involvement of universities in researching and adapting tests, developing local norms and
undertaking local psychometric studies, and developing indigenous tests.
• Some organisations such as the South African Breweries and the South African Police Services (SAPS)
undertake large-scale testing have undertaken numerous studies to provide psychometric information on the
measures that they use, investigate bias, and adapt measures on the basis of their findings.
Norm-referenced and criterion-referenced measures
Norms
• A norm is a measurement against which the individual’s raw score is evaluated so that the
individual’s position relative to that of the normative sample can be determined.
• The normative sample refers to the group of people on whom the test was initially standardised during the
test development process.
• With norm-referenced measures each test-taker’s performance is interpreted with reference to a relevant
standardization sample or norm group.
• Criterion-referenced measures compare the test-taker’s performance to the attainment of a defined skill
or content.
The standard normal distribution
• Many human characteristics measured in psychology are assumed to be normally distributed in the
population.
,• The normal distribution is the standard normal (bell-shaped) distribution, which has a mean of zero and a
standard deviation of 1.
• Raw scores obtained by test-takers on psychological measures are converted to normal (standardized)
scores through statistical transformation.
Establishing norm groups
• Norm groups consist of two subgroups
1. The applicant pool
2. The incumbent population.
• The choice of a norm group has to be:
1. Representative of both the applicant pool and the incumbent population
2. Appropriate for the position for which the assessment is conducted.
• The similarities between the norm group and the applicant (not yet competent) and incumbent groups
(competent) are established by means of comparative aspects such as ethnicity, gender, age and
educational background.
• Standardised or normal scores are calculated for each of the tests in the battery.
• New applicants’ test scores can now be compared to these newly established standardised (normal)
scores.
Co-norming of measures
• Co-norming: the process where two or more related but different measures are administered and
standardised as a unit on the same norm group.
• Addresses potential issues such as test-order effects; learning and memory effects; gender, age, and
education effects; and variations of scaling format effects.
• A range of different effects can be tested for such as test-order, learning, and memory effects.
• These issues cannot be addressed if different norm groups are used.
Types of Test norms
Developmental Scales
• The rationale of developmental scales is that certain human characteristics increase with age and
experience.
• Mental age scales: A basal age is computed (the highest age at which a measure was passed). A child’s
mental age on a measure is the sum of the basal age plus additional months of credit earned at higher age
levels. The development of a child with a mental age of 10 years corresponds to the mental development of
, the average 10-year-old child, no matter what his/her chronological age.
• Grade equivalents: Scores on educational achievement measures (grade equivalents). Used in a school
setting, a pupil’s grade value is described as equivalent to a specified grade. It is used especially for
expressing performance on standardised scholastic measures. Drawbacks of this scale are that scale units
are not necessarily equal and that they represent the median performance with overlapping from one grade
to another.
Percentiles
• A percentile rank score is the percentage of people in a normative standardisation sample who fall at or
below a given raw score.
• Example: If an individual obtains a percentile score of 70, it means that 70 per cent of the normative
population obtained a raw score lower than the individual.
• Used for describing individual test performance.
• The 50th percentile corresponds to the median.
• The 25th and 75th percentiles are known as the first (Q1) and third (Q3) quartiles respectively as they cut
off the lowest and highest quarters of the normal distribution.
• Percentile rank scores can be obtained from frequency tables or from using conversion tables.
• These tables estimate the placement of raw scores in the score distribution by relying on a raw score
conversion to standard scores (e.g. z-scores, which will be discussed later).
• Disadvantages:
1. The inequality of their scale units, especially at the extreme ends of the distribution.
2. Percentile ranks are ordinal level measures, and cannot be used for normal arithmetic calculations.
3. Percentile ranks calculated for various variables and on various populations are not directly
comparable.
Standard scores
Z-scores:
• Expresses an individual's distance from the mean in terms of standard deviation units.
• Calculated by subtracting the mean from the raw score and dividing the result by the standard deviation (s).
• A raw score equal to the mean is equal to a z-score of zero.
• Positive z-scores indicate above-average performance and negative z-scores below-average performance.
• Normally range between approximately -3.0 to +3.0.
• Advantages:
1. They represent interval level measurements and may thus be statistically manipulated.
2. The frequency distribution has the same form as the raw scores on which they are based.
• Disadvantage: half of the z-scores in a distribution have negative values.
Linearly transformed standard scores:
