However, researchers’ concerns about missing opportunities to publish their real-world evidence-based papers in high-ranked scientific journals are justified to some extent due to lack of ‘best practices’ from publishers and suitable peer reviewers for these study designs
In a 2018 investigation (Int J Technol Assess Health Care. 2018;34(1):11-119) into peer-reviewed journal editors’ views on real-world evidence (RWE) stemming from RWD, respondents from 15 scientific publications generally expressed favourable attitudes towards observational studies as being complementary to randomised clinical trials (RCTs), but reported that the greater scarcity of clinical trial results submitted, combined with their prestige factor, tends to skew review processes in their favour. Diana Romero, Chief Editor of Nature Reviews Clinical Oncology, shares the view that although RCTs remain the best way to test most hypotheses related to defining the best care in clinical oncology, RWE can also constitute the basis for high-quality research papers. “There are some questions, pertaining to treatment toxicity or patient preferences, for example, that may be more effectively addressed using RWD,” she says. “If the authors of a study have a strong rationale to explain why this was the best method to use, if their analysis is sound and the results support their conclusions, then it is a high-quality study in my perspective.”
According to Javier Carmona Sanz, Deputy Editor at Nature Medicine, there are some rare conditions for which RWD can provide a faster and more efficient avenue to finding effective treatments, because setting up a clinical trial would be difficult given the low number of patients and their geographic dispersion. “In a study we published last year, for instance, the authors intersected genomics data with clinical records containing information about real-world outcomes for patients with angiosarcoma, of which as little as 300 cases are diagnosed in the USA each year,” Carmona Sanz reports. “This allowed them to identify treatment strategies that may benefit patients who otherwise have limited therapeutic options available to them” (Nat Med. 2020 February 10;26:181-187).
RWE publication on the rise despite persisting obstacles
The potential for RWE to deliver meaningful advances in patient care has attracted significant interest by regulatory bodies both in the USA (Food and Drug Administration) and in Europe, where the European Medicines Agency (EMA) is actively developing technical and methodological guidance for the generation of RWE, with recent additions including recommendations for registry-based research (Guideline on registry-based studies). Romero has seen these shifts in attitudes translated into an uptick in the number of RWE-focused papers published in medical journals: “In one of our review articles on the use of RWD in oncology, the authors tracked the evolution of PubMed search results for the terms ‘real-world data’, ‘real-world evidence’ and ‘registry’ over time and found that the number of publications had increased sixfold – from about 2,500 to almost 15,000 – between 2002 and 2016,” she recalls (Nat Rev Clin Oncol. 2019;16:312–325). “In comparison, a control search for studies on colon cancer revealed a much smaller relative increase, from 3,000 publications to about twice as many over the same time period.”
If, in absolute terms, the number of real-world research papers published in highly ranked journals remains low, according to Carmona Sanz, this may be due partly to the fact that the field is still in its infancy. “However, there are also some practical hurdles to overcome for studies using RWD, which rely on access to large patient datasets that contain confidential patient information, as well as robust computational models with which to analyse it,” he explains, reporting that editors like himself often come up against restrictions in making the relevant data and algorithms from these papers available to the community upon publication. This, he says, may limit the ability of other researchers to replicate their findings and thus hamper the wider adoption of these approaches.
Lessons learned from sharing RWD in a pandemic
The advantages of using RWD to obtain quick answers to pressing clinical questions have seemed particularly self-evident in the context of the coronavirus pandemic – yet the flurry of research on COVID-19 being put out rapidly has yielded mixed results. “A number of questionable articles published during the pandemic attracted public attention and highlighted the limitations of poorly-designed observational studies,” Romero recalls. “Of course, this is not a new problem for journals, but as editors we have had to become even more vigilant about the quality of RWE that is submitted to us.”
Carmona Sanz sees in these events the exemplification of why transparent access to both the raw data and the models underpinning its analysis is paramount for testing the validity of an observational study’s findings and thus deciding whether it should be published. However, because verifying this type of data and complex analytics requires very specific skills and often a familiarity with the relevant data sources, finding suitable peer reviewers for RWE-based papers may remain an intractable problem for journal editors.
At the same time, Romero believes the pandemic has clearly illustrated the fact that different research methods should be used to address different questions, and that they are not interchangeable. “RCTs have been very important in determining the best therapies to use against the novel coronavirus – be they vaccines or other agents,” she says. “Registry studies, meanwhile, have helped us understand the epidemiology of COVID-19 and its risks for patients with cancer. Had it not been for extensive data sharing between different hospitals, our knowledge of the disease epidemiology would have been much more limited.”
According to Romero, there are many other instances in oncology where cancer registries are proving crucial to broadening knowledge of the disease and patient outcomes beyond what is known from studies conducted at a few major cancer centres, in tightly controlled clinical settings. “There is a real need for research into disparities in treatment response that may exist among different real-world populations (JCO Glob Oncol. 2020;6:462-470), as well as assessments of how cancer patients are actually treated in routine practice (JCO Glob Oncol. 2020;6:1286-1297) to better understand the therapeutic needs in geographical zones that may not be leaders in oncology research, but which are certainly home to people with cancer who deserve to receive high-quality care,” she says.
Taking the guesswork out of publishing real-world research
What does this mean for the publication opportunities available to scientists interested in conducting real-world research in the future? Both Romero and Carmona Sanz believe that clear instructions from journals about their publication criteria and quality standards for this category of papers would help to alleviate investigators’ concerns in this regard. “For instance, a set of ‘best practices’ for submitting RWE, endorsed by both editors and researchers, to safeguard the confidentiality of patients’ data while also encouraging transparency about computational models, may facilitate the proliferation of these studies in the scientific literature,” Carmona Sanz suggests.
He also cautions that potential outcomes related to the publication of a paper should not influence decisions about how a study is conducted, and concludes: “The main editorial criteria that determine the suitability of a paper for our journal are the clinical relevance of the research question that is addressed, and whether the use of RWD has allowed researchers to attain knowledge that may advance the management of a specific group of patients but which would have been inaccessible through more conventional approaches.”
