Ask seven questions, in order: what design was used, how many people took part, who they were, what was actually measured, how large the effect was, how certain the authors are, and who paid. A randomised trial outranks an observational study; a meta-analysis is only as good as its trials; a blood marker is not a tendon; and an effect size means little until its certainty rating is printed beside it.
Each question below is worked through on studies this site already cites, including the ones that cut against what we sell. None of it needs a statistics course. It needs the methods section, the funding statement and the habit of reading the size of a finding alongside its certainty. Terms are defined in the glossary, and the forms the studies compare are tabled on the forms reference.
| The question to ask | What to look for | An example from this site |
|---|---|---|
| 1. What design was used? | Randomised, controlled and blinded; a crossover counts | Shaw 2017: a randomised, double-blind crossover — a strong design, in eight people |
| 2. How many people took part? | Participants analysed per group, and whether a sample size was calculated | Balshaw 2023: 39 men, in groups of 19 and 20, over 15 weeks |
| 3. Who were they? | Age, sex, training status, medical history | JFMK 2026: 246 of 257 participants were men, aged 18 to 52 and mostly young |
| 4. What was measured? | The outcome itself, or a marker standing in for it | Shaw 2017 measured a blood marker; Balshaw 2023 measured tendon by MRI |
| 5. How large was the effect? | An effect size with its confidence interval, not only a p-value | Bischof 2024: maximal strength SMD 0.19, p < 0.01 |
| 6. How certain is it? | A certainty rating, and which scale the review used | Bischof 2024 graded that strength figure low certainty; JFMK 2026 prints letter grades |
| 7. Who paid? | Funding, competing interests, donated product | A 2025 meta-analysis of 23 trials: the pooled effect vanished in trials without company funding |
What is the strongest type of study for a supplement?
For the question of whether a supplement does something, a randomised controlled trial: participants are allocated by chance, so the groups differ, on average, only by what they took. The GRADE system for rating evidence, as the Institute of Medicine’s 2011 standards for systematic reviews set it out, starts randomised trials at high certainty and observational studies at low. A crossover trial, in which each person takes every arm in turn, is a randomised design that works with small numbers.
Three other kinds of paper sit around the trial, and each answers a narrower question. An observational study can show that two things travel together; it cannot show that one caused the other. A systematic review searches for every trial on a question by a stated method and assesses each for bias — the 2026 review of collagen and tendon in the Journal of Functional Morphology and Kinesiology screened several hundred citations to find eight eligible randomised trials. A narrative review does neither, which makes it the weakest of the four as evidence, however often it is cited.
What is a meta-analysis?
A meta-analysis pools the findings of several studies — usually every eligible trial a systematic review found — into one estimate with a confidence interval. It adds precision and inherits every weakness of what it pools. Bischof and colleagues pooled 19 collagen studies with 768 participants in 2024 and found a maximal-strength SMD of 0.19, p < 0.01, on evidence they graded low certainty themselves.
Pooling is not always possible, and declining to pool can be the more honest choice. The 2026 JFMK review synthesised its eight trials in words rather than numbers because, in the authors’ account, supplementation protocols, training and outcome measures differed too much to combine. When a meta-analysis does pool, look for how much its studies disagree — heterogeneity — because one pooled number can hide trials that point in opposite directions.
What does GRADE evidence mean?
GRADE rates confidence in a body of evidence for each outcome, not in a single study, on four levels: high, moderate, low and very low. Randomised trials start high and observational studies low, and evidence loses levels for study limitations, inconsistency, indirectness, imprecision and publication bias. At the top, further research is “very unlikely to change our confidence in the estimate of effect”; at the bottom, “any estimate of effect is very uncertain”.
Two cautions before comparing grades across papers. Not every review uses the same ladder: the 2026 JFMK review prints its verdicts as letter grades of recommendation — “strong evidence (GRADE A)”, “conflicting evidence (GRADE C)” — with each trial ranked on the Oxford levels of evidence, while the GRADE Working Group’s own certainty scale is the four words above. And size and certainty are separate axes. The 2024 meta-analysis that found SMD 0.19 for strength found SMD 0.67 for tendon morphology and graded it very low certainty. The JFMK review graded the evidence for tendon size and stiffness as strong, and graded the evidence against an effect on muscle strength as strong as well. A 15 g trial that review did not include — Balshaw and colleagues, 39 men over 15 weeks, tendon measured by MRI — found no between-group difference on any tendon measure. Two reviews, two grades, one null trial: all three belong in the same paragraph, every time.
What does an effect size of 0.19 mean?
It means the difference between groups was 0.19 standard deviations — about a fifth of the ordinary spread between people. On the benchmarks Cohen proposed, 0.2 is small, 0.5 medium and 0.8 large, so 0.19 sits just below small. That is why this site prints the 2024 collagen strength figure as SMD 0.19 with its low certainty beside it, and never alone.
Two refinements. The benchmarks are rules of thumb: Lakens, in a methods primer, calls them “arbitrary” and says they “should not be interpreted rigidly”, and a 2022 review of sports-science methods notes that they were derived from effects observed in behavioural science. And a p-value answers a different question. Bischof’s p < 0.01 says a difference that size is unlikely to be chance; it says nothing about whether the difference matters to anyone lifting weights. Read the effect size for how much, the confidence interval for how precisely, and the p-value last.
What is a surrogate marker in a supplement study?
A measurement standing in for the outcome that matters — a blood marker in place of a tendon. The 2017 study behind this site’s 15 g collagen protocol found that 15 g of vitamin C-enriched gelatin roughly doubled blood PINP, a marker of collagen synthesis, in eight healthy men. It measured blood, not tendon, and the later 15-week trial that measured tendon by MRI found no difference from placebo.
The same check applies across supplements. A plasma curve after a single dose is a marker of absorption, not of what reaches muscle or how much of a mineral the body holds; a laboratory rate of protein synthesis is a marker, not strength. Markers are where research starts, because they respond quickly and cheaply. They become a problem only when the movement of a marker is sold as the outcome. Shaw’s crossover is a careful study of eight people and a blood marker, and it should be quoted as exactly that.
How many participants does a supplement study need?
Enough to detect the effect it is looking for, which for small effects is far more than many supplement trials enrol. A 2022 review of sports-science methods cites a median sample size of 19 in the Journal of Sports Sciences, and shows that with 10 people per group a real medium-sized effect reaches significance only 18% of the time. Small trials miss real effects and, when only significant findings get published, overstate the ones they find.
The trials this site leans on make the point on their own. Shaw’s crossover had eight participants, so one person was an eighth of the sample. Jager and colleagues’ 2007 plasma study of two creatine salts had six. Balshaw’s null tendon trial had 39, in groups of 19 and 20, more than the 2026 review’s average of about 32 per trial and still small. Combining studies helps: the 2024 meta-analysis pooled 768 participants across 19 studies, and the 2026 systematic review gathered 257 across eight trials. Neither number rescues a weak design, but both make a single unusual participant matter less.
Does a supplement study apply to someone like you?
Only as far as the people studied resemble you. The 2026 collagen and tendon review covered 257 participants aged 18 to 52, 246 of them men and most of them young; a 2007 plasma study of creatine forms enrolled six adults, three men aged 21 to 29 and three women aged 35 to 55. Check the age, sex, training status and medical history of the participants before applying a finding: a finding in 20-year-olds is a hypothesis for a 50-year-old.
For the men this site is written for — over 45 and still lifting — the fit is usually partial. Balshaw’s tendon trial enrolled men aged 18 to 40, with group means of 24 and 27, and the creatine form trials mostly enrolled young adults. That does not make the findings wrong for an older lifter; it makes them less direct, which is exactly the indirectness GRADE subtracts for. The honest move is to say which population a number came from every time it is quoted, and this site tries to.
How do you check who paid for a supplement study?
Read three places: the funding statement, the competing-interests declaration and the author affiliations, and count donated product as funding. It matters because sponsorship tracks findings: in a 2007 analysis of nutrition articles on soft drinks, juice and milk, none of the intervention studies funded wholly by industry reached an unfavourable conclusion, against 37% of those with no industry money. And in a 2025 meta-analysis of 23 randomised collagen trials, the pooled effect disappeared in the trials without company funding.
That beverage figure comes from Lesser and colleagues, who classified 206 articles published from 1999 to 2003. Setting neutral conclusions aside, the odds of a favourable rather than an unfavourable conclusion were 7.61 times higher with all-industry funding than with none, with a 95% confidence interval from 1.27 to 45.73 — a large effect measured imprecisely. The collagen meta-analysis made the same point inside a single supplement, by reporting its pooled effect separately for trials with and without company funding.
On this site the funding cuts both ways, and both directions are printed. The null tendon trial at 15 g a day for 15 weeks — Balshaw and colleagues in Medicine & Science in Sports & Exercise — prints no funding statement of its own, but its companion paper from the same project states: “This study was funded by the Collagen Research Institute (CRI); Kiel, Germany.” Both papers name a collagen manufacturer’s branded peptides as the supplement. A sponsored trial that found nothing is a reminder that a funding line is a fact to record, not a verdict. The 2025 meta-analysis whose pooled effect vanished without company money is the reason this site makes no claim on that outcome at all; the full account is on does collagen actually work and on the four questions any document should answer. Naming a funder is not the same as dismissing a paper. It is one more column to read.
What the research does not show
The research on research has limits of its own, and they belong here. The nutrition estimate rests on 206 articles about three drinks, with a confidence interval wide enough to hold a modest effect and a very large one, and the collagen split is one meta-analysis of one supplement. Neither shows that any particular sponsored paper is wrong. They show that sponsorship is a variable worth recording, and only a paper’s own methods can settle the rest.
Certainty ratings are judgements, and two competent reviews of the same question can reach different grades, as the two collagen reviews above did for tendon outcomes. Effect-size benchmarks were borrowed from behavioural science and fit sports nutrition loosely. The median sample size of 19 describes one journal’s papers, not every supplement trial. And no checklist, this one included, replaces reading the methods section: who was enrolled, what they were given, what was measured, and what was left out.
Where TARE sits: our evidence pages are written to these seven questions, including when the answer costs us. Under Load’s protocol is 15 g — a direction, not a serving — because it is the bottom of the 15–30 g daily range the 2026 tendon review recommends with training, and the trial at that dose that found nothing is printed wherever the favourable evidence is. The evidence tables for creatine, collagen and magnesium apply the seven questions one study at a time.
Sources
- Institute of Medicine. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: National Academies Press; 2011. Chapter 4. nationalacademies.org
- Buchalski A, Jeanfavre M, Altorelli C, Leff G. J Funct Morphol Kinesiol 2026;11(1):130. PubMed 41900537
- Bischof K, Moitzi AM, Stafilidis S, Konig D. Sports Med 2024;54(11):2865-2888. doi:10.1007/s40279-024-02079-0
- Balshaw TG, Funnell MP, McDermott EJ, et al. Med Sci Sports Exerc 2023;55(11):2083-2095. PubMed 37436929
- Lakens D. Front Psychol 2013;4:863. doi:10.3389/fpsyg.2013.00863
- Mesquida C, Murphy J, Lakens D, Warne J. R Soc Open Sci 2022;9(12):220946. PMC9748505
- Shaw G, Lee-Barthel A, Ross ML, Wang B, Baar K. Am J Clin Nutr 2017;105(1):136-143. PMC5183725
- Jager R, Harris RC, Purpura M, Francaux M. J Int Soc Sports Nutr 2007;4:17. jissn.biomedcentral.com
- Myung SK, Park Y. Am J Med 2025;138(9):1264-1277. PubMed 40324552
- Lesser LI, Ebbeling CB, Goozner M, Wypij D, Ludwig DS. PLoS Med 2007;4(1):e5. PMC1764435