MRCPsych - Basic Statistics: Questions And A+ Answers
The prevalence of depression in patients with mild cognitive impairment is
10%. On applying a depression rating scale with the likelihood ratio of a
positive test (LR+) equal to 10, a patient with mild cognitive impairment
becomes test positive. The probability that this patient is depressed is equal
to: Right Ans - This question tests one's ability to calculate post-test
probability from likelihood ratios.
The probability of having a disease after testing positive with a diagnostic test
depends on two factors:
(a) the prevalence of the disease
(b) the likelihood of a positive test result using the instrument.
It is important to remember that baseline prevalence of a disease for which a
diagnostic instrument is being tested is taken as the pretest probability.
So pretest probability = 10%
Now, post-test odds = likelihood ratio × pretest odds
From a given probability odds can be calculated using the formula odds =
(probability)/(1 - probability)
Here pretest odds = (10%)/(1 - 10%) = 10/90 = 1/9.
Now post-test odds = likelihood ratio × pretest odds = 10 × 1/9 = 10/9
Using the formula probability = odds/(1 + odds)
Post-test probability = (10/9)/[1 + (10/9)] = 10/19 = 52.3%
Note: the post test probability for a positive results equals the positive
predictive value (PPV).
The post test probability for a negative results equals 1 minus the negative
predictive value (1 minus NPV). Thus, to calculate the NPV you need to
subtract that result from one.
,If prevalence in your practice is lower that the study, the PPV will be lower
and the NPV will be higher.
While adapting the results of an RCT into clinical practice, a clinician wants to
calculate the new NNT values for his own clinical population using the results
of the RCT. Apart from the reported RCT which of the following is needed to
carry out the calculation of the new NNT? Right Ans - The expected rate of
spontaneous resolution of the treated condition in the clinical population:
Published RCTs may quote impressive outcomes in terms of NNT. Applying
principles of evidence-based medicine, one must check for the internal
validity of a study and the degree of generalizability before adapting the
results to clinical practice.
One must also be aware of the fact that though clinically more meaningful,
NNTs quoted in RCTs may not translate to the same extent in actual clinical
practice.
One way of appreciating the usefulness of a newly introduced drug is to
calculate the NNT for one's own clinical population (target population).
To enable this one may estimate the patient expected event rate (PEER),
which is given by the expected spontaneous resolution rate or the response
rate for an existing standard treatment. This can be obtained from the local
audit data or clinical experience.
The product of PEER and relative benefit increase from the published RCT
gives the new absolute benefit increase (ABI new) value for the target
population. The inverse of the new ABI gives the new NNT for the target
population.
In an attempt to ensure equivalent distribution of potential effect-modifying
factors in treating refractory depression, a researcher weighs the imbalance
that might be caused whenever an individual patient enters one of the two
arms of the study. Every patient is assigned to the group where the least
amount of imbalance will be caused. This method is called: Right Ans -
Minimization.
,In most treatment trials interventions are allocated by randomization. Block
randomization and stratified randomization can be used to ensure the balance
between groups in size and patient characteristics. But it is very difficult to
stratify using several variables in a small sample.
A widely acceptable alternative approach is minimization. This method can be
used to ensure very good balance between groups for several confounding
factors irrespective of the size of the sample. With minimization the treatment
allocated to the next participant enrolled in the trial depends (wholly or
partly) on the characteristics of those participants already enrolled. This is
achieved by a simple mathematical computation of magnitude of imbalance
during each allocation.
The effectiveness of an intervention is measured by using pragmatic trials.
Which trial design is normally employed when carrying out a pragmatic trial?
Right Ans - RCT.
RCTs provide high-quality evidence for or against proposed interventions. But
RCTs have a major limitation in terms of generalizability. This is because the
trials are conducted in a somewhat artificial experimental setting that is
different from clinical practice. So RCTs have high internal validity due to
rigorous methodology but poor external validity. Pragmatic RCTs are a type of
RCTs introduced with the intention of increasing external validity, i.e.
generalizability of RCT results. But this takes place at the expense of internal
validity. In pragmatic RCTs the trial takes place in a setting as close as possible
to natural clinical practice, i.e. the inclusion and exclusion criteria are less
fastidious, often 'treatment as usual' is employed for comparisons, instead of
placebos and real world, functionally significant outcomes are considered.
Power of a study Right Ans - The power of a study refers to the ability of
the study to show the difference in outcome between studied groups if such a
difference actually exists.
Power can also be defined as probability of avoiding type 2 error. (It is 1-beta,
where beta = type 2 error)
To calculate power, one needs to know four variables.
1. sample size
2. magnitude of a clinically significant difference
, 3. probability of type 1 error (significance level from which p value is derived)
4. variance of the measure in the study sample.
Underpowered trials are those that enrol too few participants to identify
differences between interventions (arbitrarily taken as at least 80% of the
time) when such differences truly exist. Underpowered RCTs are prone to
false-negative conclusions (type 2 errors). Somewhat controversially,
underpowered trials are considered to be unethical, as they expose
participants to the ordeals of research without providing an adequate
contribution to clinical development.
Power calculation Right Ans - The term power calculation is often used
while referring to sample size estimation before a study is undertaken. In
order to carry out power calculation one has to know the expected precision
and variance of measurements within the study sample (obtained from a
literature search or pilot studies), the magnitude of a clinically significant
difference, the certainty of avoiding type 1 error as reflected by the chosen p
value, and the type of statistical test one will be performing. There is no point
in calculating the statistical power once the results of a study are known. On
completion of trials, measures such as confidence intervals indicate the power
of a study and the precision of results.
A new diagnostic test detects 60 out of 100 schizophrenia patients correctly. It
does not wrongly diagnose anyone in a sample of 100 controls. How accurate
is this test in detecting schizophrenia? Right Ans - Accuracy = all true
observations / total population studied
False negative rate: Right Ans - False negative rate = false negative/ total
diseased. OR 1 minus sensitivity.
Similarly false-positive rate (FPR) is same as (1 - specificity).
Which of the following properties of a screening test increases with increasing
disease prevalence in the population? Right Ans - Positive predictive value.
Sensitivity, specificity, and accuracy are measures that reflect the
characteristics of the test instrument. These measures do not vary with
changes in the disease prevalence.
The prevalence of depression in patients with mild cognitive impairment is
10%. On applying a depression rating scale with the likelihood ratio of a
positive test (LR+) equal to 10, a patient with mild cognitive impairment
becomes test positive. The probability that this patient is depressed is equal
to: Right Ans - This question tests one's ability to calculate post-test
probability from likelihood ratios.
The probability of having a disease after testing positive with a diagnostic test
depends on two factors:
(a) the prevalence of the disease
(b) the likelihood of a positive test result using the instrument.
It is important to remember that baseline prevalence of a disease for which a
diagnostic instrument is being tested is taken as the pretest probability.
So pretest probability = 10%
Now, post-test odds = likelihood ratio × pretest odds
From a given probability odds can be calculated using the formula odds =
(probability)/(1 - probability)
Here pretest odds = (10%)/(1 - 10%) = 10/90 = 1/9.
Now post-test odds = likelihood ratio × pretest odds = 10 × 1/9 = 10/9
Using the formula probability = odds/(1 + odds)
Post-test probability = (10/9)/[1 + (10/9)] = 10/19 = 52.3%
Note: the post test probability for a positive results equals the positive
predictive value (PPV).
The post test probability for a negative results equals 1 minus the negative
predictive value (1 minus NPV). Thus, to calculate the NPV you need to
subtract that result from one.
,If prevalence in your practice is lower that the study, the PPV will be lower
and the NPV will be higher.
While adapting the results of an RCT into clinical practice, a clinician wants to
calculate the new NNT values for his own clinical population using the results
of the RCT. Apart from the reported RCT which of the following is needed to
carry out the calculation of the new NNT? Right Ans - The expected rate of
spontaneous resolution of the treated condition in the clinical population:
Published RCTs may quote impressive outcomes in terms of NNT. Applying
principles of evidence-based medicine, one must check for the internal
validity of a study and the degree of generalizability before adapting the
results to clinical practice.
One must also be aware of the fact that though clinically more meaningful,
NNTs quoted in RCTs may not translate to the same extent in actual clinical
practice.
One way of appreciating the usefulness of a newly introduced drug is to
calculate the NNT for one's own clinical population (target population).
To enable this one may estimate the patient expected event rate (PEER),
which is given by the expected spontaneous resolution rate or the response
rate for an existing standard treatment. This can be obtained from the local
audit data or clinical experience.
The product of PEER and relative benefit increase from the published RCT
gives the new absolute benefit increase (ABI new) value for the target
population. The inverse of the new ABI gives the new NNT for the target
population.
In an attempt to ensure equivalent distribution of potential effect-modifying
factors in treating refractory depression, a researcher weighs the imbalance
that might be caused whenever an individual patient enters one of the two
arms of the study. Every patient is assigned to the group where the least
amount of imbalance will be caused. This method is called: Right Ans -
Minimization.
,In most treatment trials interventions are allocated by randomization. Block
randomization and stratified randomization can be used to ensure the balance
between groups in size and patient characteristics. But it is very difficult to
stratify using several variables in a small sample.
A widely acceptable alternative approach is minimization. This method can be
used to ensure very good balance between groups for several confounding
factors irrespective of the size of the sample. With minimization the treatment
allocated to the next participant enrolled in the trial depends (wholly or
partly) on the characteristics of those participants already enrolled. This is
achieved by a simple mathematical computation of magnitude of imbalance
during each allocation.
The effectiveness of an intervention is measured by using pragmatic trials.
Which trial design is normally employed when carrying out a pragmatic trial?
Right Ans - RCT.
RCTs provide high-quality evidence for or against proposed interventions. But
RCTs have a major limitation in terms of generalizability. This is because the
trials are conducted in a somewhat artificial experimental setting that is
different from clinical practice. So RCTs have high internal validity due to
rigorous methodology but poor external validity. Pragmatic RCTs are a type of
RCTs introduced with the intention of increasing external validity, i.e.
generalizability of RCT results. But this takes place at the expense of internal
validity. In pragmatic RCTs the trial takes place in a setting as close as possible
to natural clinical practice, i.e. the inclusion and exclusion criteria are less
fastidious, often 'treatment as usual' is employed for comparisons, instead of
placebos and real world, functionally significant outcomes are considered.
Power of a study Right Ans - The power of a study refers to the ability of
the study to show the difference in outcome between studied groups if such a
difference actually exists.
Power can also be defined as probability of avoiding type 2 error. (It is 1-beta,
where beta = type 2 error)
To calculate power, one needs to know four variables.
1. sample size
2. magnitude of a clinically significant difference
, 3. probability of type 1 error (significance level from which p value is derived)
4. variance of the measure in the study sample.
Underpowered trials are those that enrol too few participants to identify
differences between interventions (arbitrarily taken as at least 80% of the
time) when such differences truly exist. Underpowered RCTs are prone to
false-negative conclusions (type 2 errors). Somewhat controversially,
underpowered trials are considered to be unethical, as they expose
participants to the ordeals of research without providing an adequate
contribution to clinical development.
Power calculation Right Ans - The term power calculation is often used
while referring to sample size estimation before a study is undertaken. In
order to carry out power calculation one has to know the expected precision
and variance of measurements within the study sample (obtained from a
literature search or pilot studies), the magnitude of a clinically significant
difference, the certainty of avoiding type 1 error as reflected by the chosen p
value, and the type of statistical test one will be performing. There is no point
in calculating the statistical power once the results of a study are known. On
completion of trials, measures such as confidence intervals indicate the power
of a study and the precision of results.
A new diagnostic test detects 60 out of 100 schizophrenia patients correctly. It
does not wrongly diagnose anyone in a sample of 100 controls. How accurate
is this test in detecting schizophrenia? Right Ans - Accuracy = all true
observations / total population studied
False negative rate: Right Ans - False negative rate = false negative/ total
diseased. OR 1 minus sensitivity.
Similarly false-positive rate (FPR) is same as (1 - specificity).
Which of the following properties of a screening test increases with increasing
disease prevalence in the population? Right Ans - Positive predictive value.
Sensitivity, specificity, and accuracy are measures that reflect the
characteristics of the test instrument. These measures do not vary with
changes in the disease prevalence.
