Arguinis et al. – Best-Practice Recommendations for Defining, Identifying, and
Handling Outliers
Outliers are data points that deviate markedly from others. They usually exert
disproportionate influence on substantive conclusions regarding relationships among
variables.
Researchers are faced with multiple (and often conflicting) definitions of outliers, techniques
to identify outliers, and suggestions on what to do with outliers once they are found.
Choices about outliers change substantive conclusions
Decisions on how to define, identify and handle outliers influence conclusions, including the
presence or absence, direction, and size of an effect or relationship.
First of all, the way in which outliers are identified is often inconsistent with how outliers are
defined.
The writers note that lack of transparency and incongruence in how outliers are defined,
identified, and handled are quite pervasive in articles published in some of the most
prestigious and influential journals.
Third, there is little discussion on the subject of studying outliers that are found to be
interesting and worthy of further examination. A pervasive view is that outliers among
substantive researchers is that outliers are problems that must be fixed, usually by
removing particular cases from the analyses.
Making decisions on how to define, identify, and handle outliers
The first principle is that choices and procedures regarding the treatment of outliers should
be described in detail to ensure transparency including a rationale for the particular
procedures that have been implemented. Second, researchers should clearly and explicitly
acknowledge the type of outlier in which they are interested, and then use an identification
technique that is congruent with the outlier definition.
The best practice recommendation is built around a sequence of steps:
1. Error outliers: the data point that lie at a distance from other data points because
they are the result of inaccuracies, they are nonlegitimate observations.
Using more than one technique is necessary to identify as many potential error
outliers as possible. It is then necessary to determine the cause of the identified
outlying observations. If caused by an error in recording, coding or data collection,
then an outlying observation is an error outlier. All remaining outlying data points
whose cause is unclear are treated as interesting outliers.
Once identified, the correct procedure is to either adjust the data points to their
correct values or remove such observations.
2. Interesting outliers: accurate data points that lie at a distance from other data points
and may contain valuable or unexpected knowledge.
The first step is to identify potential interesting outliers by using the same techniques
as identifying error outliers, and second to identify which are actually interesting.
Handling those outliers could be done by a quantitative approach, such as empirically
analyzing differences between the manufacturing synergies of high and low outlier
performers. Also qualitative approaches can be used.
3. Influential outliers: accurate data points that lie at a distance from other data points,
are not error or interesting, and also affect substantive conclusions. There are two
types of influential outliers, namely model fit outliers (data points whose presence
alters the fit of a model) and prediction outliers (data points whose presence alters
parameter estimates.
Handling Outliers
Outliers are data points that deviate markedly from others. They usually exert
disproportionate influence on substantive conclusions regarding relationships among
variables.
Researchers are faced with multiple (and often conflicting) definitions of outliers, techniques
to identify outliers, and suggestions on what to do with outliers once they are found.
Choices about outliers change substantive conclusions
Decisions on how to define, identify and handle outliers influence conclusions, including the
presence or absence, direction, and size of an effect or relationship.
First of all, the way in which outliers are identified is often inconsistent with how outliers are
defined.
The writers note that lack of transparency and incongruence in how outliers are defined,
identified, and handled are quite pervasive in articles published in some of the most
prestigious and influential journals.
Third, there is little discussion on the subject of studying outliers that are found to be
interesting and worthy of further examination. A pervasive view is that outliers among
substantive researchers is that outliers are problems that must be fixed, usually by
removing particular cases from the analyses.
Making decisions on how to define, identify, and handle outliers
The first principle is that choices and procedures regarding the treatment of outliers should
be described in detail to ensure transparency including a rationale for the particular
procedures that have been implemented. Second, researchers should clearly and explicitly
acknowledge the type of outlier in which they are interested, and then use an identification
technique that is congruent with the outlier definition.
The best practice recommendation is built around a sequence of steps:
1. Error outliers: the data point that lie at a distance from other data points because
they are the result of inaccuracies, they are nonlegitimate observations.
Using more than one technique is necessary to identify as many potential error
outliers as possible. It is then necessary to determine the cause of the identified
outlying observations. If caused by an error in recording, coding or data collection,
then an outlying observation is an error outlier. All remaining outlying data points
whose cause is unclear are treated as interesting outliers.
Once identified, the correct procedure is to either adjust the data points to their
correct values or remove such observations.
2. Interesting outliers: accurate data points that lie at a distance from other data points
and may contain valuable or unexpected knowledge.
The first step is to identify potential interesting outliers by using the same techniques
as identifying error outliers, and second to identify which are actually interesting.
Handling those outliers could be done by a quantitative approach, such as empirically
analyzing differences between the manufacturing synergies of high and low outlier
performers. Also qualitative approaches can be used.
3. Influential outliers: accurate data points that lie at a distance from other data points,
are not error or interesting, and also affect substantive conclusions. There are two
types of influential outliers, namely model fit outliers (data points whose presence
alters the fit of a model) and prediction outliers (data points whose presence alters
parameter estimates.
