What each row is
One included study. A square marking its effect estimate, a horizontal line showing its confidence interval, and usually the study's weight in the pooled result, sometimes shown by the size of the square.
Larger squares carry more weight, generally because the studies are larger or more precise. That is worth noticing before anything else, because a pooled result can be driven almost entirely by one large study.
The line of no effect
A vertical line at zero for differences, or at one for ratios. Any study whose horizontal line crosses it did not reach significance on its own.
Individual studies crossing that line is entirely normal in a meta-analysis, and it is much of why syntheses exist: several imprecise studies pooled can produce a precise answer none of them reached alone.
The diamond
The pooled estimate, with its width showing the confidence interval. If the diamond crosses the line of no effect, the synthesis did not find a significant overall effect.
Read its ends rather than its centre, exactly as with any interval. Whether the lower end is still large enough to matter is usually the practical question.
Heterogeneity is the real story
Look at how much the study estimates scatter. If they point in different directions, or their intervals barely overlap, pooling them may not be meaningful however tidy the diamond looks.
Reviews report a statistic for this, and the plot shows it visually before any statistic is consulted. A neat diamond under a scattered set of studies deserves scepticism rather than confidence.
Look for the outlier
One study far from the others is worth investigating rather than ignoring. It frequently differs in population, dose, setting or outcome measure, and that difference is often more interesting than the pooled result.
Good reviews discuss it. A review that has an obvious outlier and says nothing about it has left work undone.
Subgroups, with caution
Many plots present subgroup analyses. These are informative and they are also where false findings appear most often, because slicing data enough ways eventually produces something significant by chance.
Subgroups planned in advance carry far more weight than ones that appear only in the results. A review should say which it is doing.
Fixed and random effects
The two pooling models make different assumptions about whether the studies estimate one common effect or a distribution of related ones. Random effects usually gives wider intervals and is the more conservative choice where studies differ.
Which one was used should be stated and justified. Where it is not, that is a gap worth noting if the review matters to your argument.
What to write about it
Report the pooled estimate with its interval, say how many studies and participants it draws on, and say something about heterogeneity. Three clauses, and your reader knows what the synthesis found and how firmly.
That is also what distinguishes citing a meta-analysis from citing its abstract, which is what most references to one actually amount to.
Why this is worth learning
Meta-analyses sit at the top of most evidence hierarchies, which means they carry disproportionate weight in student writing and are disproportionately cited from their abstracts.
Ten minutes with the plot tells you whether the synthesis deserves the weight the hierarchy assigns it, and occasionally it does not.
Questions this raises.
What does the size of each square mean?
The study's weight in the pooled estimate, which usually reflects its precision and therefore largely its sample size. It is worth checking before reading the diamond, because a pooled result dominated by one very large study is a different kind of evidence from one drawing evenly on ten.
What is heterogeneity and why does it matter?
The extent to which the included studies disagree beyond what chance would explain. High heterogeneity means the studies may not be estimating the same thing, so pooling them produces a number that describes none of them well. The plot shows it visually before any statistic is quoted.
Should I trust a meta-analysis over a single trial?
Usually, and not automatically. A synthesis of poor studies is a poor synthesis, and a well-conducted large trial can be stronger evidence than a pooled set of small ones. What matters is the quality of what went in and whether pooling was appropriate, both of which the plot helps you judge.
What if the diamond crosses the line?
The synthesis did not find a significant overall effect, which is a finding rather than a failure. Look at the width of the diamond as well: a narrow one centred on no effect suggests there genuinely is little effect, while a wide one suggests the evidence was simply insufficient to tell.
Do I need to understand the statistics to use one?
You need to read the plot, which is a different and much smaller thing. Weights, the line of no effect, the spread of the studies and the width of the diamond are all visual. That is enough to cite a meta-analysis responsibly without being able to reproduce the pooling yourself.
Warren Cobbett
PhD, Biostatistics, on the statistics and analysis side of the hall. Writes for The Footnote and coaches the students who bring this work to the desk. The rest of the specialists.
Elsewhere in the hall.
Send the assignment and the deadline.
A specialist credentialed in your own field reads what you actually have and replies within two hours with a plan and a price. The reading costs nothing, and if the honest answer is that we are not the right hall for it, you will get that instead.
