Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a free and non-commercial open data platform and infrastructure that facilitates research on pragmatic trials. It gathers and distributes clean trial data, ratings, and evaluations using PRECIS-2. This allows for a variety of meta-epidemiological analyses that compare treatment effect estimates across trials of various levels of pragmatism.
Background
Pragmatic studies provide real-world evidence that can be used to make clinical decisions. However, the use of the term "pragmatic" is inconsistent and its definition and assessment requires clarification. The purpose of pragmatic trials is to guide the practice of clinical medicine and policy decisions rather than confirm a physiological hypothesis or clinical hypothesis. A pragmatic trial should try to be as close as it is to real-world clinical practices, including recruitment of participants, setting, design, implementation and delivery of interventions, determining and analysis outcomes, and primary analysis. This is a major difference between explanatory trials, as described by Schwartz & Lellouch1 which are designed to test the hypothesis in a more thorough way.
The most pragmatic trials should not blind participants or the clinicians. This can lead to bias in the estimations of treatment effects. The pragmatic trials also include patients from different health care settings to ensure that their outcomes can be compared to the real world.
Additionally the focus of pragmatic trials should be on outcomes that are vital to patients, such as quality of life or functional recovery. This is particularly important for trials that involve the use of invasive procedures or could have dangerous adverse impacts. The CRASH trial29 compared a 2-page report with an electronic monitoring system for hospitalized patients with chronic heart failure. The catheter trial28, however utilized symptomatic catheter-related urinary tract infections as its primary outcome.
In addition to these aspects, pragmatic trials should minimize the requirements for data collection and trial procedures to cut down on costs and time commitments. Furthermore pragmatic trials should strive to make their findings as relevant to actual clinical practice as they can by making sure that their primary method of analysis is based on the intention-to-treat method (as described in CONSORT extensions for pragmatic trials).
Many RCTs that don't meet the criteria for pragmatism but contain features contrary to pragmatism have been published in journals of different types and incorrectly labeled as pragmatic. This can lead to misleading claims about pragmatism, and the term's use should be made more uniform. The development of a PRECIS-2 tool that offers a standardized objective evaluation of pragmatic aspects is the first step.
Methods
In a pragmatic study the aim is to inform policy or clinical decisions by demonstrating how an intervention could be integrated into routine care in real-world situations. This is distinct from explanation trials that test hypotheses about the cause-effect relationship in idealised settings. In this way, pragmatic trials can have a lower internal validity than explanation studies and be more susceptible to biases in their design, analysis, and conduct. Despite these limitations, pragmatic trials may provide valuable information to decisions in the context of healthcare.
The PRECIS-2 tool scores an RCT on 9 domains, with scores ranging from 1 to 5 (very pragmatist). In this study the areas of recruitment, organization, flexibility in delivery, flexible adherence and follow-up were awarded high scores. However, the main outcome and the method for missing data was scored below the pragmatic limit. This indicates that a trial can be designed with well-thought-out practical features, but without harming the quality of the trial.
However, it's difficult to judge how practical a particular trial is, since pragmaticity is not a definite quality; certain aspects of a trial can be more pragmatic than others. A trial's pragmatism could be affected by changes to the protocol or the logistics during the trial. Koppenaal and colleagues discovered that 36% of the 89 pragmatic studies were placebo-controlled, or conducted prior to the licensing. They also found that the majority were single-center. Thus, they are not very close to usual practice and can only be called pragmatic when their sponsors are accepting of the absence of blinding in these trials.
A common aspect of pragmatic research is that researchers attempt to make their findings more relevant by studying subgroups of the trial sample. However, this often leads to unbalanced results and lower statistical power, increasing the chance of not or incorrectly detecting differences in the primary outcome. This was a problem during the meta-analysis of pragmatic trials because secondary outcomes were not adjusted for covariates' differences at the baseline.
Additionally the pragmatic trials may be a challenge in the collection and interpretation of safety data. This is because adverse events are usually self-reported and are prone to delays in reporting, inaccuracies, or coding variations. It is therefore crucial to improve the quality of outcomes assessment in these trials, ideally by using national registry databases instead of relying on participants to report adverse events on a trial's own database.
Results
Although the definition of pragmatism may not require that all trials are 100 percent pragmatic, there are some advantages to including pragmatic components in clinical trials. These include:
Incorporating routine patients, the results of the trial can be translated more quickly into clinical practice. However, pragmatic trials may be a challenge. For example, the right kind of heterogeneity can allow a study to generalize its findings to a variety of settings and patients. However the wrong type of heterogeneity can reduce assay sensitiveness and consequently decrease the ability of a study to detect even minor effects of treatment.
Several studies have attempted to categorize pragmatic trials using various definitions and scoring methods. 프라그마틱 플레이 and Lellouch1 developed a framework to discern between explanation-based studies that support the physiological hypothesis or clinical hypothesis and pragmatic studies that guide the selection of appropriate therapies in real world clinical practice. Their framework included nine domains, each scored on a scale of 1 to 5 with 1 indicating more explanatory and 5 indicating more practical. The domains included recruitment, setting up, delivery of intervention, flex adhering to the program and primary analysis.
The initial PRECIS tool3 had similar domains and an assessment scale ranging from 1 to 5. Koppenaal and colleagues10 developed an adaptation to this assessment, dubbed the Pragmascope that was easier to use in systematic reviews. They found that pragmatic systematic reviews had higher average score in most domains, with lower scores in the primary analysis domain.
This difference in primary analysis domain can be due to the way in which most pragmatic trials analyze data. Some explanatory trials, however do not. The overall score was lower for pragmatic systematic reviews when the domains on the organization, flexibility of delivery and follow-up were combined.
It is important to remember that a pragmatic trial doesn't necessarily mean a low quality trial, and indeed there is an increasing rate of clinical trials (as defined by MEDLINE search, however it is neither specific or sensitive) which use the word "pragmatic" in their abstracts or titles. These terms could indicate a greater appreciation of pragmatism in titles and abstracts, but it isn't clear whether this is evident in the content.
Conclusions
In recent times, pragmatic trials are becoming more popular in research as the value of real-world evidence is increasingly recognized. They are clinical trials that are randomized that evaluate real-world alternatives to care instead of experimental treatments in development. They include patient populations that more closely mirror the patients who receive routine care, they use comparators that are used in routine practice (e.g. existing drugs), and they depend on participants' self-reports of outcomes. This method is able to overcome the limitations of observational research like the biases associated with the reliance on volunteers, and the lack of the coding differences in national registry.
Other advantages of pragmatic trials are the ability to use existing data sources, as well as a higher probability of detecting significant changes than traditional trials. However, they may still have limitations that undermine their validity and generalizability. Participation rates in some trials could be lower than expected due to the healthy-volunteering effect, financial incentives or competition from other research studies. Practical trials are often limited by the need to recruit participants quickly. In addition some pragmatic trials do not have controls to ensure that the observed differences aren't due to biases in trial conduct.
The authors of the Pragmatic Free Trial Meta identified RCTs published up to 2022 that self-described as pragmatism. The PRECIS-2 tool was employed to determine the degree of pragmatism. It covers areas such as eligibility criteria as well as recruitment flexibility, adherence to intervention, and follow-up. They discovered that 14 of these trials scored pragmatic or highly sensible (i.e. scores of 5 or more) in one or more of these domains, and that the majority were single-center.
Studies with high pragmatism scores tend to have more lenient criteria for eligibility than traditional RCTs. They also include patients from a variety of hospitals. According to the authors, can make pragmatic trials more relevant and useful in everyday clinical. However they do not guarantee that a trial will be free of bias. The pragmatism is not a fixed characteristic; a pragmatic test that does not have all the characteristics of an explanation study may still yield valuable and valid results.