Clinical trial sponsors can collect patient reported outcomes electronically in multinational clinical trials. Advantages include significant improvements in clinic and subject compliance, data quality, data access and monitoring, as well as reduced staff and subject burden. Yet, ePRO clinical solutions can also present challenges. By understanding some important ePRO best practices, however, sponsors can overcome major challenges.
ePRO CLINICAL SOLUTIONS: BRIEF SUMMARY OF THE CHALLENGES
One of the major challenges faced when using Electronic Patient Reported Outcomes (ePRO) in multinational trials is ensuring the quality and reliability of the ePRO systems and data files. After all, even small problems with the systems or data can sometimes damage credibility, which can lead to decreased system usage, increased enrollment, queries, and other trial costs, and at worst, the discrediting of all the data.
Another challenge is the very large number of changing variables that need to be managed. These variables include:
- Complex protocols and lengthy questionnaires
- Long lists of countries, languages and translations
- Complex system requirements
- Interdependent schedules
- Long growing lists of issues
- A multitude of first patient dates
- International user support across different time zones in many languages.
Validity, or measurement equivalence with validated paper questionnaires, can also be a significant challenge. Due to the nature of the hardware and software, some systems cannot fully support existing validated paper questionnaires.
For instance, small hand-held devices generally have very restricted screen real estate. IVR systems are limited because subjects can only remember short questions and answers. In these and other cases, the questions and answers from a validated paper questionnaire often have to be modified in order to accommodate the technology.
In order to then ensure that the data from the electronic systems are equivalent to the paper-based questionnaire, it is often necessary to conduct a validation study comparing the data collected on the new electronic tool with the data collected on the validated paper questionnaire. These validation studies can add significantly to both costs and project timelines.
Finally, a challenge faced by many when using Electronic Patient Reported Outcomes in multinational trials is quality assurance and auditing. Some auditors have limited experience or knowledge of ePRO systems, leading to varied interpretations of industry regulations. These diverse regulatory interpretations can cause confusion, and also have considerable cost and timeline implications.
ePRO CLINICAL SOLUTIONS: BEST PRACTICES
The challenges described above sometimes appear daunting to a study team, even though they appreciate the potential. However, there are solutions. The best practices described below offer a method to manage the challenges.
ePRO CLINICAL SOLUTIONS: SYSTEM QUALITY AND RELIABILITY
All successful systems start with an unambiguous systems requirements document. The purpose of this document is to describe the system(s) to be delivered in detail, so that all team members have exactly the same expectations of what the system will do and what data the system will provide.
The process of writing and reviewing a detailed planning document, whether it be systems requirements, a clinical trial protocol, or otherwise, forces the author and the reviewers to think through options for processes and to discuss and tweak them until there is confidence among all participants that the best solution has been designed.
So, the systems requirements definition process, properly executed, produces quality thinking on system design and is the first step towards producing quality systems. In addition, because the deliverable is a written detailed document describing how the system will work, it prevents misunderstandings, confusion and frustration at a later stage when the systems are implemented.
A systems requirements document must show all screen shots exactly as they will look in the completed system. Every edit for every data entry field, all screen and system validation checks, and all system logic (skips, conditions, calculations, error conditions, messages, and so on) must be described clearly and in detail.
This documented presentation of the system both 'on the surface' and 'beneath the surface' (logic checks, and so on) enables the sponsor team and the ePRO provider to think about the issues carefully before programming starts. It will also make certain that the system design will ensure compliance with protocol, and that the resulting data files will meet the needs of the study.
The final systems requirements document must be signed by the key sponsors and ePRO provider team members, to show agreement and commitment from both parties on the details of all systems functions and operations. When evaluating providers, the sponsor should ask to see systems requirements documents from previous projects.
The sponsor should ensure that the providers have the skills to deliver a thoughtfully prepared systems requirements document which describes a high quality system with the attributes the sponsor desires: ease of use, protocol compliance, data quality controls, rapid data access, and so on.
To deliver consistently reliable systems, the Electronic Patient Reported Outcomes provider must also have the skills to develop and execute high quality system test plans. Test plans must include thoughtful test cases that address every requirement in the systems requirements document.
To ensure every systems requirement is covered, there must be a traceability matrix which matches each requirement in the systems requirements document to the relevant test cases in the test plan. This may seem obvious, but it is often not done.
For each systems requirement there must also be detailed test cases and scenarios for both valid and invalid data, as well as standard and non-standard user actions. The expected results for each test must also be documented in advance of the test cases to ensure the integrity of the test. In addition, the test plan needs to include large data input sets that represent a wide variety of diverse scenarios.
Furthermore, it is vital that the test plan include significant load and stress testing to ensure that the systems will function properly in real world conditions, when an exceptionally large set of data has been collected or when stressful situations are encountered.
When executing the test plan, the testing specialists must document whether the tests pass satisfactorily and ensure that every data file is correct, without exception. When evaluating ePRO providers, the sponsor should also review test plans and traceability matrices from previous projects to ensure that the provider has the skills to develop and execute test plans that meet the standards described above.
In addition to the formal testing performed by the provider, the sponsor must also carefully review the software by performing a user acceptance test. The purpose of this test is to ensure that both the ePRO team and the sponsor team agree that all systems requirements have been implemented as described in the
systems requirements document.
If the provider has executed a quality test plan, this sponsor process should be quick and easy, and should provide the sponsor team final control and peace of mind before delivering systems for the first patient date. However, if the provider has not performed adequate testing, quality issues may be revealed and the sponsor then has the opportunity to delay shipment and require corrective action before the systems are distributed to sites and become a burden in the field.
ePRO CLINICAL SOLUTIONS: COMPETENT PROJECT MANAGEMENT
Because there are so many moving parts in multinational trials, it is critical that the provider can develop and maintain complicated project plans. Project plans must include hundreds, sometimes thousands, of detailed tasks and expected durations, dependencies between them, and contingencies for unexpected events.
Because the ePRO project schedule typically involves hundreds of tasks, the provider must demonstrate competencies with an electronic project management tool so that a changing environment can be accommodated quickly. During the provider evaluation process, sponsors should also look at project schedules and project status reports from previous studies to help determine the providers' project management skills.
In addition, good communication among the project team members needs to be the rule and not the exception. Regularly scheduled project management team meetings with key staff from both the sponsor and the provider are critical to review the status of project tasks, determine if changes are needed, and to resolve any issues that arise. These project status meetings are crucial in order to ensure that sponsor and ePRO team member efforts are complementary and effective.
ePRO CLINICAL SOLUTIONS: CHOOSE SYSTEMS THAT SIGNIFICANTLY REDUCE VALIDATION ISSUES
To resolve the challenge of measurement equivalence between a validated paper questionnaire and an electronic device, sponsors can choose a technology where the validated paper questionnaire text does not need to be changed. Touchscreen tablet or internet-based systems typically offer this kind of flexibility.
Although it is beyond the scope of this article, the validation requirements for these technologies are very minimal.
ePRO CLINICAL SOLUTIONS: MANAGE QUALITY ASSURANCE AUDITORS
Given that many auditors have little experience with Electronic Patient Reported Outcomes, sometimes their statements can be confusing, and even frightening.
If an auditor imposes requirements that seem unreasonable, it is important to review the regulation(s) with the auditor and find more practical solutions. In most cases, there are various avenues to ensure regulatory compliance, and many of them are simple and cost-effective.
ePRO CLINICAL SOLUTIONS: CONCLUSION
The best practices discussed above can provide real tools for addressing the challenges that can arise in the implementation of Electronic Patient Reported
Outcomes in multinational clinical trials. While these best practices do not cover every possible situation that can occur, they will provide significant assistance in addressing key challenges, and put their clinical trials on a solid path to success.
Joy Hebert is a cofounder and COO of .assisTek, the global leader in ePROs on touch-screen tablets. For the last 15 yrs she has managed the development and implementation of ePRO systems in clinical studies worldwide. For more information about ePRO clinical solutions, visit our site.