Summary
Statistics summary A2
- Course
- Institution
This document contains the summary of literature A2: Bayesian data analysis for newcomers - John K. Kruschke, and Torrin M. Liddell
[Show more]
Seller
Follow
jlmkuipers
Reviews received
Content preview
Bayesian data analysis for newcomers – John K. Kruschke, and Torrin M. LiddellAbstract
This article explains the foundational concepts of Bayesian data analysis using virtually no
mathematical notation. Bayesian ideas already match your intuitions from everyday reasoning and
from traditional data analysis. Simple examples of Bayesian data analysis are presented that illustrate
how the information delivered by a Bayesian analysis can be directly interpreted. Bayesian
approaches to null-value assessment are discussed. The article clarifies misconceptions about
Bayesian methods that newcomers might have acquired elsewhere. We discuss prior distributions
and explain how they are not a liability but an important asset. We discuss the relation of Bayesian
data analysis to Bayesian models of mind, and we briefly discuss what methodological problems
Bayesian data analysis is not meant to solve. After you have read this article, you should have a clear
sense of how Bayesian data analysis works and the sort of information it delivers, and why that
information is so intuitive and useful for drawing conclusions from data.
The first section of the article explains the foundational ideas of Bayesian methods, and shows how
those ideas already match your intuitions from everyday reasoning and research. The next sections
show some simple examples of Bayesian data analysis, for you to see how the information delivered
by a Bayesian analysis can be directly interpreted. We discuss Bayesian parameter estimation,
Bayesian model comparison, and Bayesian approaches to assessing null values. The final sections
focus on disabusing possible misconceptions that newcomers might have.
The main idea: Bayesian analysis is reallocation of credibility across possibilities
The main idea of Bayesian analysis is simple and intuitive. There are some data to be explained, and
we have a set of candidate explanations. Before knowing the new data, the candidate explanations
have some prior credibilities of being the best explanation. Then, when given the new data, we shift
credibility toward the candidate explanations that better account
for the data, and we shift credibility away from the candidate
explanations that do not account well for the data. A
mathematically compelling way to reallocate credibility is called
Bayes’ rule.
You already are Bayesian in everyday reasoning
Sherlock Holmes’ reasoning is Bayesian
The fictional detective Sherlock Holmes was famous for remarking
to his sidekick, Dr. Watson: “How often have I said to you that
when you have eliminated the impossible, whatever remains,
however improbable, must be the truth?” That simple reasoning is
Bayesian. From the data, suspicion is reallocated across the
suspects. If the data eliminate some suspects, the remaining
suspects must be more suspicious, even if their prior probability
was small. Bayesian analysis does exactly the same reallocation,
but using precise mathematics. Figure 1 illustrates the reallocation
,graphically. The horizontal axis denotes the range of possibilities, and the vertical axis denotes the
credibility, or probability, of each possibility.
In Holmes’ reasoning, a tacit premise is that the actual culprit is included in the set of suspects. A
more accurate phrasing for Holmesian reasoning is this: When you have eliminated the impossible,
whatever remains, however improbable, must be the least bad option from among the possibilities
you are considering. In general, Bayesian reasoning provides the relative credibilities within the set of
considered possibilities.
Exoneration is Bayesian
The logic of exoneration is Bayesian: Reasoning starts with a set of candidate causes of the event,
then collects new data such as a confession, and then reallocates credibility accordingly. If the data
fully implicate one suspect, the remaining (unaffiliated) suspect must be less suspicious. Bayesian
analysis does the same reallocation, but with exact mathematics.
The possibilities are parameter values
In data analysis, the candidate explanations are values of parameters in mathematical descriptions.
We conceive of each randomly polled response as a flip of a coin that has some underlying
probability of coming up yes. We start with some prior allocation of credibilities across the
continuum of possible parameter values. The prior allocation could be quite vague and spread evenly
across the range of candidate values from 0 to 1, or the prior could give some candidate proportions
higher credibility than others if previous knowledge recommends it. Then we collect data and re-
allocate credibility to parameter values that are consistent with the data.
Parameter values are meaningful in the context of their model
We care about parameter values because they are meaningful. To illustrate, suppose we collect the
weights of all the children in third grade of a particular school. We describe the set of data as being
randomly generated from a normal distribution with mean μ and standard deviation σ. If we tell you
that certain values of μ and σ are good descriptions of the data, then you have a pretty good idea of
what the data look like. The values of the parameters are meaningful in the context of the model.
Bayesian analysis tells us the relative credibilities of the parameter values. That’s why the
information provided by a Bayesian analysis is so useful and intuitive.
Parameters can be discrete or continuous
In many mathematical descriptions of data, the parameters are continuous, such as means, standard
deviations, and regression coefficients. The posterior distribution on continuous parameters is a
continuous distribution, rising and falling smoothly across the range of the joint parameter space. As
nearly all statistical models involve continuous parameters, it is continuous parameter distributions
that dominate Bayesian data analysis.
But descriptive parameters can also be discrete, not continuous. E.g., having a disease or not having a
disease.
Another important case of a discrete parameter, that we will revisit later, is model comparison. For a
given set of data, there can be multiple models. Each model involves its own parameters and prior
distribution over its parameters. The models are labeled by a discrete indexical parameter (“1” for
the first model, “2” for the second model, and so on). When new data are considered, credibility
shifts over the parameter distributions within each model, andcredibility simultaneously shifts over
, the discrete indexical parameter. The re-allocated posterior probabilities of the model indices are the
relative degrees to which we believe each model, given the data. In particular, when one model
represents a null hypothesis and a second model represents an alternative hypothesis, this discrete
model comparison is one way of implementing hypothesis testing in a Bayesian framework.
The more general application is for continuous parameters; discrete parameters are just a special
case.
Bayesian analysis provides the relative credibilities of parameter values
The goal of Bayesian data analysis is to provide an explicit distribution of credibilities across the range
of candidate parameter values. This distribution, derived after new data are taken into account, is
called the posterior distribution across the parameter values. The posterior distribution can be
directly examined to see which parameter values are most credible, and what range of parameter
values covers the most credible values.
The posterior distribution can be directly interpreted. We can “read off” the most credible parameter
values and the range of reasonable parameter values. Unlike in frequentist statistical analysis, there
is no need to generate sampling distributions from null hypotheses and to figure out the probability
that fictitious data would be more extreme than the observed data. In other words, there is no need
for p values and p value based confidence intervals. Instead, measures of uncertainty are based
directly on posterior credible intervals.
You already are Bayesian in data analysis
Bayesian reasoning is so intuitive that it’s hard to resist spontaneously giving Bayesian
interpretations to results of traditional frequentist data analysis. Consider, for example, the t test for
a single group of 50 people to whom we administered a “smart drug” and then an
intelligencequotient (IQ) examination. We would like to know if the mean IQ score of the group
differs from the general population average of 100. Suppose the t test yields t (49) = 2.36, p = 0.023,
with 95% confidence interval on μ extending from 100.74 to 109.26. What does this mean?
Your intuitive interpretation of p values is Bayesian
Consider the result that p = 0.023. This means that the probability that μ equals the “null” value of
100 is only 0.023, right? This is a natural, intuitive interpretation and is the one that many or most
people give. Unfortunately, it is not the correct interpretation of the p value. A Bayesian hypothesis
test provides the probability that μ equals the null value relative to an alternative hypothesis that μ
could span a wide range of values. In other words, we naturally interpret a frequentist p value as if it
were some form of Bayesian posterior probability.
But a frequentist p value is not a Bayesian posterior probability. The p value is the probability that
the observed data summary (such as its t value), or something more extreme than observed would
be obtained if the null hypothesis were true and the data were sampled according to the same
stopping and testing intentions as the observed data. In other words, the p value is the probability
that fictional, counterfactual data from the null hypothesis would be more extreme than the
observed data, when those data were sampled and tested as intended by the current researchers.
Different stopping and testing intentions therefore yield different p values.
Your intuitive interpretation of confidence intervals is Bayesian
Consider the 95% confidence interval from 100.74 to 109.26. That means there is a 95% probability
that the mean μ falls between 100.74 and 109.26, right? This is a natural, intuitive interpretation, and
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller jlmkuipers. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $3.20. You're not tied to anything after your purchase.
Can Stuvia be trusted?
4.6 stars on Google & Trustpilot (+1000 reviews)
53068 documents were sold in the last 30 days
Founded in 2010, the go-to place to buy study notes for 14 years now