Conduct a Pragmatic Intervention Pilot
A rapid and rigorous test of an intervention working with patients, staff, settings, and conditions that are like those intended to use the intervention every day
What is a Pragmatic Pilot Trial?
A pragmatic pilot trial is a rapid and rigorous test of an intervention working with patients, staff, settings, and conditions that are like those intended to use the intervention every day.
Unlike traditional clinical trials, pragmatic pilots are designed to evaluate interventions delivered by non-research staff to the intended population in natural (not controlled) settings.
What are the differences between pragmatic vs. traditional clinical trials?
Prioritize which parts of your trial you will conduct in a pragmatic way [See Pragmatic Priorities Check-In] to decide on what is a priority for your study and what is out of scope.
Designing a pragmatic pilot should follow similar steps to traditional clinical trials. However, to make the trial pragmatic, some key differences (see table on next page) should be considered.
Key Differences between Pragmatic vs. Traditional Clinical Trials
| A traditional efficacy trial tests a hypothesis under ideal conditions | A pragmatic trial tests a hypothesis under everyday conditions | |
| Goal | To determine causes and effects of a treatment. | To improve practice and inform clinical and policy decisions. |
| Intervention and Delivery | Delivered by research staff in a controlled setting outside the usual implementation setting. Delivery is rigid, not allowing for adaptation. | Delivered by clinical staff (not research staff) in ways that are feasible within the settings where implemented. Changes (Adaptations) will likely occur to tailor and optimize intervention delivery. |
| Settings and Participants | Controlled settings outside the target implementation sites with highly defined and carefully selected participants. | Settings closely resemble the target implementation sites and eligibility criteria are minimized to select more representative participants. |
| Study Design | Test the intervention vs. a placebo or alternative using rigid study protocols with minimal variation. | Test the intervention using real-world strategies, flexible protocols and local customization. |
| Measures and Outcomes | Assessments require data collection outside of routine care. Mainly focused on clinical effectiveness. | Brief assessments designed to be collected within routine care. Considers clinical effectiveness, implementation outcomes and sometimes, mechanisms. |
| Results | Rarely directly relevant to everyday practice. | Useful in everyday practice, especially clinical decision-making. |
Why is this step important?
Pragmatic pilots help produce findings that support feasibility, scalability, and sustainability for patients, caregivers/care partners, clinicians, implementation staff, and policymakers.
Pragmatic pilots bridge the gap between clinical research and natural applications while also providing rapid iteration to adapt to local settings and emerging challenges.
Pragmatic trials tend to be more sustainable after research funding runs out.
What are the goals of this step?
Show the intervention CAN and DOES work as intended in real world, usual care conditions.
Test interventions and implementation strategies that are feasible, efficient, generalizable, and sustainable.
Understand when and under what conditions a program works, for whom, and for how long.
In Summary
Consider all the issues mentioned here and described in the detailed guidance below; however, design of a pragmatic pilot often depends on resources available to conduct the intervention and what issues are most important to inform your next steps.
Strong pragmatic studies address the “6 Rs”: They are: Rapid, Rigorous, Relevant (to partners), Recursive (iterative and adaptable), Replicable, and Resource sensitive (affordable and sustainable)
You do not need to include all the issues in the detailed guidance below, but you do need to consider them to make the best decisions for your study.
As a next step, we strongly recommend you complete the Pragmatic Priorities Check-In [see Detailed Guidance].
| Common Pitfalls | Actions to Avoid Them |
| Scope is too large; attempting to do or measure too much for your time and budget | Be modest in what you promise; you can always do more than you planned. |
| Focusing too much on the clinical outcomes and forgetting the importance of the implementation outcomes | Identify key issues, select a relevant assessment, and include implementation and effectiveness outcomes—consider mechanisms if in scope. |
| No quantitative data; or no qualitative data (to understand results) | Using multiple methods may uncover answers to questions that you may not have initially considered. |
| Too little engagement with patients, implementers or decision makers | Include robust, ongoing engagement with people and groups that will be key to implementing your pilot to ensure feasibility and acceptability. [see Engagement Tip Sheet] |