Experts and their Biases
Problem 1: Experts
Schweitzer (2018): What evidence matters to jurors?
Usually 90-95% of the cases end in a plea deal and only 5-10% goes to a jury trial.
Knowing what type of evidence is valued in court has empirical implications for
researchers, but also practical implications for attorneys.
There are three types of evidence
1. Eyewitness testimony, in which the confidence of the witness is the most
influential part
2. Expert testimony, for which the weight differs for every juror. It is a prevalent
type of evidence; 69-80% of the civil and criminal cases contain expert
testimony.
3. Visual evidence, in which photos influence the jury to give a more punitive
sentence.
Aim was to extend the current literature regarding the prevalence and importance of
trial evidence
Study 1
Aim was to assess what legal professionals report as being common evidence in
homicide trials
Method Attorneys and judges from 6 states completed an online survey in which they
had to rate how common a piece of evidence was and if they encountered this type of
evidence in their last trial. There were four domains of evidence: Biological/physical,
documentary, demonstrative and testimony.
Results
The most common types of evidence were as follows:
- Crime scene photos
- Witness to the crime
- Diagram injuries
- Police expert
- Forensic expert
- Weapon
- Fingerprints
- Maps
- Audio or video confession
1
,Study 2
Aim was to assess how important mock jurors find the types of evidence for their
verdicts
Method Mock jurors completed an online survey in which they rated the importance
of the type of evidence for their verdict without any context.
Results
- All evidence was assessed as ‘somewhat important’
- Most important were:
o DNA
o Fingerprints
o Weapon
o Video records
o Crime scene photos
o Gunshot residue
o Bodily secretions
o Video confession
o Forensic expert
o Eyewitness
Study 3
Aim was to examine whether or not the perceptions of mock jurors would be altered
if they were only presented with the top 10 most important types of evidence from
study 2
Method The researchers first conducted a pilot study that left them with 4 types of
evidence instead of 10. A process-tracing method was used to assess the order in
which the mock jurors viewed the evidence. The evidence that was viewed first was
deemed as most important. Again, mock jurors completed an online survey, which
presented DNA, eyewitness testimony, fingerprints and a video confession. All the
evidence was incriminating.
Results
- DNA was most likely to be viewed first; then video confession; then
eyewitness testimony and last fingerprints.
Limitations
- In the last study, people might have viewed DNA first, not because it was
deemed most important, but because they did not have prior knowledge about
this topic and wanted to learn more about this.
2
, - There was no inclusion of exculpatory evidence
- The types of evidence were only specific to homicide cases
- The researchers did not present the mock jurors with the types of evidence that
were used most often in court.
Thomas (2018): Assessment of expert performance compared across professional
domains
Aim was to explain variability in the performance of experts across different domains
Professional competence is expert performance across the domain level
Expertise is performance at the individual level
There are different techniques to identify professional competence. It is unnecessary
to assess professional competence when the correct answer or ground truth is known.
- Consensus is when domain experts agree with each other (inter-rater
reliability)
- Consistency is when domain experts agree with themselves when making
repeated judgements (intra-rater reliability)
These two constructs differ per domain. For example, stockbrokers and clinical
psychologists have low consistency, whereas weather forecasters and auditors have
high consistency.
Measure Description Limitation
Experience Amount of year on the job Not correlated with expertise
Accreditation Accreditation or Reflects more of job-relevant
certification of certain skill experience than expertise
Peer Being nominated as expert Popularity is not a proxy for
identification by peers expertise
3
Problem 1: Experts
Schweitzer (2018): What evidence matters to jurors?
Usually 90-95% of the cases end in a plea deal and only 5-10% goes to a jury trial.
Knowing what type of evidence is valued in court has empirical implications for
researchers, but also practical implications for attorneys.
There are three types of evidence
1. Eyewitness testimony, in which the confidence of the witness is the most
influential part
2. Expert testimony, for which the weight differs for every juror. It is a prevalent
type of evidence; 69-80% of the civil and criminal cases contain expert
testimony.
3. Visual evidence, in which photos influence the jury to give a more punitive
sentence.
Aim was to extend the current literature regarding the prevalence and importance of
trial evidence
Study 1
Aim was to assess what legal professionals report as being common evidence in
homicide trials
Method Attorneys and judges from 6 states completed an online survey in which they
had to rate how common a piece of evidence was and if they encountered this type of
evidence in their last trial. There were four domains of evidence: Biological/physical,
documentary, demonstrative and testimony.
Results
The most common types of evidence were as follows:
- Crime scene photos
- Witness to the crime
- Diagram injuries
- Police expert
- Forensic expert
- Weapon
- Fingerprints
- Maps
- Audio or video confession
1
,Study 2
Aim was to assess how important mock jurors find the types of evidence for their
verdicts
Method Mock jurors completed an online survey in which they rated the importance
of the type of evidence for their verdict without any context.
Results
- All evidence was assessed as ‘somewhat important’
- Most important were:
o DNA
o Fingerprints
o Weapon
o Video records
o Crime scene photos
o Gunshot residue
o Bodily secretions
o Video confession
o Forensic expert
o Eyewitness
Study 3
Aim was to examine whether or not the perceptions of mock jurors would be altered
if they were only presented with the top 10 most important types of evidence from
study 2
Method The researchers first conducted a pilot study that left them with 4 types of
evidence instead of 10. A process-tracing method was used to assess the order in
which the mock jurors viewed the evidence. The evidence that was viewed first was
deemed as most important. Again, mock jurors completed an online survey, which
presented DNA, eyewitness testimony, fingerprints and a video confession. All the
evidence was incriminating.
Results
- DNA was most likely to be viewed first; then video confession; then
eyewitness testimony and last fingerprints.
Limitations
- In the last study, people might have viewed DNA first, not because it was
deemed most important, but because they did not have prior knowledge about
this topic and wanted to learn more about this.
2
, - There was no inclusion of exculpatory evidence
- The types of evidence were only specific to homicide cases
- The researchers did not present the mock jurors with the types of evidence that
were used most often in court.
Thomas (2018): Assessment of expert performance compared across professional
domains
Aim was to explain variability in the performance of experts across different domains
Professional competence is expert performance across the domain level
Expertise is performance at the individual level
There are different techniques to identify professional competence. It is unnecessary
to assess professional competence when the correct answer or ground truth is known.
- Consensus is when domain experts agree with each other (inter-rater
reliability)
- Consistency is when domain experts agree with themselves when making
repeated judgements (intra-rater reliability)
These two constructs differ per domain. For example, stockbrokers and clinical
psychologists have low consistency, whereas weather forecasters and auditors have
high consistency.
Measure Description Limitation
Experience Amount of year on the job Not correlated with expertise
Accreditation Accreditation or Reflects more of job-relevant
certification of certain skill experience than expertise
Peer Being nominated as expert Popularity is not a proxy for
identification by peers expertise
3
