Stack of medical journals and research papers on a desk with organized scientific literature and healthcare publications for professional study and medical research
Généré par l'IA / Generated using AI
π Science and technology π Health and biotech π Neuroscience
Behind the scenes of the scientific method: when research corrects itself

Medical research: why so many studies rely on data no one can actually check

with Saul Martin Rodriguez, Postdoctoral Researcher in the Space and Environmental Physiology Group at the Karolinska Institute
On October 28th, 2026 |
5 min reading time
Saul Martin Rodriguez_VF
Saul Martin Rodriguez
Postdoctoral Researcher in the Space and Environmental Physiology Group at the Karolinska Institute
Key takeaways
  • Restricted access to empirical data undermines the validity of certain meta-analyses and, consequently, the reliability of medical studies.
  • The demand for transparency often comes up against the need to protect participants’ data and regulatory constraints.
  • This lack of accurate data has led to tens of millions of women being prescribed treatments that cause harm.
  • These limitations also affect government decisions, with considerable financial consequences.
  • Improving data transparency remains the most appropriate response to these uncertainties.

Every year, thou­sands of clin­ic­al recom­mend­a­tions, drug reim­burse­ment decisions, and pub­lic health policies rest on meta-ana­lyses. These stud­ies pool res­ults from many indi­vidu­al reports to draw a con­clu­sion more robust than any single one could offer on its own. Although pil­lars of evid­ence-based medi­cine, they may be con­trib­ut­ing to a sys­tem­ic and widely under­es­tim­ated prob­lem because most rely on data that no one out­side the ori­gin­al authors has ever been able to see or verify.

The idea behind a meta-ana­lys­is is simple enough. Research­ers gath­er res­ults from sev­er­al stud­ies on the same ques­tion to identi­fy a more reli­able over­all trend. In doing so, they almost nev­er have access to the raw data behind the ori­gin­al stud­ies such as indi­vidu­al res­ults for each patient or par­ti­cipant. Instead, they must work from the sum­mary stat­ist­ics pub­lished in the art­icles such as means, medi­ans, stand­ard devi­ations, which are often repor­ted incom­pletely or incon­sist­ently from one study to the next.

This lack of access forces research­ers to rely on estim­a­tion and imputa­tion meth­ods to fill in the gaps. For instance, guess­ing at miss­ing val­ues based on stat­ist­ic­al assump­tions. Even when car­ried out care­fully, these approx­im­a­tions intro­duce addi­tion­al uncer­tainty that can dis­tort the final res­ults. Moreover, when sev­er­al stud­ies with­in the same meta-ana­lys­is require this kind of estim­a­tion, small errors can accu­mu­late and, in some cases, com­pletely change the con­clu­sions, some­times even revers­ing a res­ult altogether.

A fragile link at the heart of evidence-based medicine

The prob­lem isn’t a lack of rules. For sev­er­al years now, many sci­entif­ic journ­als and fund­ing bod­ies have required research­ers to com­mit to shar­ing their data. Although, in prac­tice, these com­mit­ments largely go unful­filled. A study of 7,750 bio­med­ic­al research papers pub­lished between 2015 and 2020 found that only 9% of papers pledged to make their data pub­licly avail­able and that among those, just 3% actu­ally fol­lowed through.

The pic­ture becomes even more strik­ing when you look closely at what hap­pens once a paper offi­cially prom­ises to share its data. In anoth­er study ana­lys­ing 3,556 art­icles, 1,792 manu­scripts included an expli­cit pledge to share data. Among the authors who were then con­tac­ted for that data, 93% either did not respond or declined. In the end, only 6.8% of authors provided the reques­ted data­sets. In oth­er words, the data-shar­ing state­ment now found in nearly every pub­lic­a­tion almost nev­er trans­lates into real access.

In some set­tings, leg­al and reg­u­lat­ory con­straints imposed by eth­ics com­mit­tees, gov­ern­ments, or data gov­ernance frame­works genu­inely lim­it what can be shared, par­tic­u­larly when data span mul­tiple coun­tries or come from large cohort stud­ies and biobanks.

The reas­ons behind this lack of trans­par­ency are var­ied, and they don’t always reflect bad faith. In some set­tings, leg­al and reg­u­lat­ory con­straints imposed by eth­ics com­mit­tees, gov­ern­ments, or data gov­ernance frame­works genu­inely lim­it what can be shared, par­tic­u­larly when data span mul­tiple coun­tries or come from large cohort stud­ies and biobanks. Pro­tect­ing par­ti­cipants’ pri­vacy and hon­our­ing the terms of their ori­gin­al con­sent can also restrict what can be made pub­lic. On top of that, there are more sub­ject­ive factors at play: some research­ers worry about los­ing own­er­ship of their data, are uncer­tain how it might be reused, or fear intense scru­tiny of their ori­gin­al ana­lyses. Lim­ited data avail­ab­il­ity, then, reflects a tangled mix of struc­tur­al, eth­ic­al, and cul­tur­al con­straints, not simply a refus­al to cooperate.

Missing data can change the course of medicine

The pub­lic­a­tion in question1 draws on a par­tic­u­larly strik­ing his­tor­ic­al example to illus­trate the real-world con­sequences of this prob­lem: men­o­paus­al hor­mone replace­ment ther­apy. For nearly two dec­ades, meta-ana­lyses based on aggreg­ate data from obser­va­tion­al stud­ies sug­ges­ted that this ther­apy reduced the risk of coron­ary heart dis­ease in men­o­paus­al women by almost 50%. That con­clu­sion shaped clin­ic­al guidelines world­wide and influ­enced the care of tens of mil­lions of women. In the most influ­en­tial meta-ana­lys­is of the peri­od, the pooled rel­at­ive risk of coron­ary heart dis­ease among hor­mone ther­apy users was estim­ated at 0.56 — an impress­ive fig­ure that rein­forced the nar­rat­ive of cardio-protection.

That nar­rat­ive col­lapsed, how­ever once detailed, par­ti­cipant-level data finally became access­ible through the Women’s Health Ini­ti­at­ive ran­dom­ised tri­al in the United States. With full access to adju­dic­ated out­comes, con­found­ing factors, and time-to-event data, the tri­al revealed that com­bined oes­tro­gen-pro­gestin ther­apy increased the risk of coron­ary events, stroke, pul­mon­ary embol­ism, and breast can­cer. The reversal was so dra­mat­ic that the tri­al had to be hal­ted early on safety grounds.

What makes this epis­ode par­tic­u­larly instruct­ive is that it was​n’t a case of unpub­lished stud­ies or select­ive report­ing: every major obser­va­tion­al study involved was, in fact, pub­licly avail­able. What was miss­ing were the under­ly­ing data­sets needed to detect con­found­ing, test altern­at­ive mod­els, and exam­ine effects across dif­fer­ent sub­groups of women. Work­ing only from pub­lished sum­mary stat­ist­ics, research­ers had pro­duced estim­ates that looked con­fid­ent and repro­du­cible, but that turned out to be incon­sist­ent with evid­ence from far more rig­or­ous ran­dom­ised tri­als after shap­ing treat­ment decisions for dec­ades. Hor­mone ther­apy became one of the most widely pre­scribed treat­ments in high-income coun­tries dur­ing the 1990s, rep­res­ent­ing bil­lions of dol­lars in annu­al spend­ing. When the Women’s Health Ini­ti­at­ive res­ults came out, pre­scrib­ing rates col­lapsed with­in months.

Consequences that go well beyond academic debate

Far from being a purely meth­od­o­lo­gic­al dis­pute among research­ers, this lack of trans­par­ency has very real con­sequences for health policy. Gov­ern­ment agen­cies and pub­lic health bod­ies fre­quently base reim­burse­ment and cov­er­age decisions on meta-ana­lyt­ic find­ings, which dir­ectly shapes how health­care resources (and some­times entire mar­kets) are alloc­ated. For instance, accord­ing to the Insti­tute for Clin­ic­al and Eco­nom­ic Review, unjus­ti­fied price increases on cer­tain drugs in the United States, made without any new evid­ence of clin­ic­al bene­fit, added an estim­ated $815 mil­lion to health­care spend­ing in 2023 alone.

The long-run­ning debate over stat­in safety offers a telling illus­tra­tion. These widely pre­scribed cho­les­ter­ol-lower­ing drugs have been at the centre of per­sist­ent dis­agree­ments over how often side effects occur and how clin­ic­ally sig­ni­fic­ant they really are. Such dis­agree­ments have fuelled con­flict­ing inter­pret­a­tions of the same evid­ence and eroded pub­lic trust in treat­ment recom­mend­a­tions, along­side com­plic­at­ing clin­ic­al prac­tice. Address­ing this calls for a pro­gress­ive roadmap toward genu­ine data trans­par­ency in research. The most imme­di­ate and low-cost step is simply to enforce rules that already exist: data and ana­lyt­ic code should be made avail­able at the time of pub­lic­a­tion, and manu­scripts that fail to meet this com­mit­ment should be rejec­ted out­right, bar­ring a clearly jus­ti­fied leg­al or eth­ic­al constraint.

Bey­ond this first step lie changes to the broad­er research eco­sys­tem. Data shar­ing needs to be embed­ded into how research insti­tu­tions are eval­u­ated, fun­der require­ments need strength­en­ing, and the pos­sib­il­ity of future data shar­ing and reuse should be built into the eth­ics and con­sent pro­cess from the very start of a study, not bolted on after­ward. On the infra­struc­ture side, data and code should be sys­tem­at­ic­ally depos­ited in recog­nised, trust­worthy repos­it­or­ies (Zen­odo, the Open Sci­ence Frame­work, or Dry­ad among them) with doc­u­ment­a­tion thor­ough enough to allow mean­ing­ful reuse rather than mere archiv­ing. A more ambi­tious step still would intro­duce ded­ic­ated edit­or­i­al pro­cesses to veri­fy the avail­ab­il­ity and qual­ity of depos­ited data before pub­lic­a­tion: essen­tially, a form­al qual­ity check woven into peer review itself.

None of these meas­ures, on their own, would have pre­ven­ted every ques­tion­able price hike or resolved every stat­in debate. But taken togeth­er, they would shift the bur­den of proof back to where it belongs: on the evidence.

1Mar­tin-Rodrig­uez, S., Fernan­dez-Gonzalo, R., & Moher, D. (2026). “The data trans­par­ency crisis in research: Les­sons from sys­tem­at­ic reviews and meta-ana­lyses.” PLOS Medi­cine, 23(6), e1005145.↑

Our world through the lens of science. Every week, in your inbox.

Get the newsletter