Bornstein (2017): Evidence-Based Psychological Assessment
There has been increasing emphasis on evidence-based practice in psychology (EBPP)
- Definition: integration of the best available research (component 1) with clinical expertise
(component 2) in the context of patient characteristics, culture, and preferences (component
3)
However, emphasis in EBPP has been on treatment; less attention paid to its application in
assessment
- When psychological assessment is discussed in this context it is typically with respect to
refining diagnosis or evaluating the effectiveness of intervention
- However, assessment also plays a role in:
- Diagnosis and classification in mental health settings
- Qualification of aptitude. intelligence, and achievement
- Documentation of neuropsychological impairment and recovery
- Competency and capability assessment in forensic settings
- Program and policy evaluation
- Personnel selection, placement, and evaluation in organizational contexts
Component 1: Best Available Research
Distinction between testingand assessment(by Meyer):
- Testing: a particular test is administered to obtain a specific score (+ descriptive meaning
applied)
- Academic training is not necessary
- (Psychological) assessment is concerned with the clinician who takes a variety of test scores,
generally obtained from multiple test methods, and considers the data in the context of
history, referral information, and observed behavior to understand the person being
evaluated, to answer the referral questions, and then to communicate findings to the patient,
his/her significant others, and referral sources
Best Available Research: Testing
To obtain a complete picture of a test’s psychometric soundness and consequential validity,
traditional outcome-focused validity assessment must be complemented by process-focused validity
assessment
- Definition: assessment of the degree to which a given instrument engages in the respondent
those psychological processes that the test is expected to engage, given the construct being
measured and the format of items and and tasks that comprise the question
- Processed-focused assessment of test score validity (1) optimizes the utility of test scores
and (2) helps understand test bias (e.g., faking good)
- E.g., a self-report test requires the following steps:
1. Retrospection(turning attention inward to determine if the statement captures
some aspect of one’s feelings, thoughts, motives, or behaviors
2. Retrospective memory search(retrieving instances wherein a person
experienced/exhibited the response(s) described in the test item)
1
, 3. Deliberate self-representation(deciding whether to share the retrieved
information with the researcher or to engage in ‘faking good’)
- Different types of measures elicit different processes
- Table below classifies instruments into 6 categories based on the mental activities
and behaviors involved in responding to these tests (three types: observational tests,
informant-report tests, and self-attribution)
- NOTE: mix methods, mix informants, mix context ⇒ best way to collect information
about the client
- The process-focused model argues that test bias can be traced in part to differential
psychological processes engaged by similar test items in different groups
- E.g., narcissism items may evoke greater defensiveness in women than men, but
dependency items may produce an opposite effect ⇒ different responses
- That’s why every instrument needs a thorough validity + reliability
investigation
- Therefore, multiple approaches need to be used to account for these
contextual differences
Best Available Research: Assessment
Validating psychological assessment results is far more challenging (no established norms). Steps of
a well-constructed assessment battery:
1. Each test must add incremental validity to the battery
2. Results must be combined in a meaningful way (and contextualized appropriately)
3. Findings must be framed and communicated in a useful way
Operationalizing assessment effectiveness entails a shift in emphasis from validity to clinical utility
(i.e., the relevance and usefulness of an intervention in patient care). Clinical utility can be quantified
in multiple ways
- E.g., asking the referent/patient/assessment expert to make judgements on the clarity,
helpfulness, and usefulness of the assessment
Component 2: Clinical Expertise
Problematic pillar; no single definition of ‘expert’
- Best attempt: clinical exercise refers to competence attained by psychologists through
education, training, and experience that results in effective practice
Competence ⇒ proficiency ⇒ expertise
Competence includes domain-specific knowledge, skills, and attitudes
2
