What means DIY EDC in clinical trials: risks and benefits
In the last few years, many different systems have appeared on the EDC market. One of the most prominent ones is EDC, designed for self-configuration by the sponsor. This system is called DIY (Do It Yourself) EDC. This kind of software has quickly become popular due to its cost and availability.
Unfortunately, there are more and more cases occurring in which the wrong approach to working with such systems leads to significant issues during clinical trials, up to the complete failure of the entire project.
Most commonly, difficulties are caused not by the fact that the DIY systems are faulty, but because setting and validating such systems is an extremely comprehensive task. We know that serious difficulties can be avoided if a skilled multidisciplinary team of experts is engaged and works strictly within standard operating procedures. In this article, we will talk about all the features of DIY systems, their advantages, and disadvantages. We will also consider which projects are suitable for such systems and what to expect when choosing DIY EDC.
What is DIY and why is it called this?
DIY is a general name for systems used to collect data in clinical trials, which are a set of modules ready to be configured for a specific clinical study. The system’s configuration and its validation by the vendor are not provided. The name implies that the sponsor's team or CRO will figure out the features of setting up such an EDC system for the specificity of the particular type of clinical research.
Although these systems have been on the market for more than five years, there is no common name for them. Do It Yourself EDC (DIY) is the name most commonly found in publications, surveys, and independent statistical reports. However, these products are not labeled in the manufacturer's directories and on the websites this way. If you see that this system is described as “an intuitive web form builder,” the "intuitive drug-and-drop interface to create forms," or "you can build edit-checks" it means that the system is a DIY system. When selecting EDC systems, first you should understand whether a vendor configures and validates the system within your project or validation remains your responsibility.
Are there any EDC systems apart from the DIY ones?
As stated earlier, the sponsor configures and validates DIY. Apart from such systems, there are two more types, depending on the setting:
“Fully-configured EDC” is when the vendor is responsible for the configuration all the way through. The sponsor provides the Protocol and paper CRF, and at the end gets access to the system and can begin to collect data.
“Partly-configured EDC” is the system that is partially configured by the vendor, and part of the setting is imposed to the sponsor. Generally, the vendor sets complex parameters (the type of randomization, visit schedule), and the vendor provides all the necessary details (measurement units or field names). The scope of work of each party is negotiated and agreed in advance.
You say that DIY is quite popular among sponsors. Why? After all, they need to adjust these systems themselves, using their own time and resources
The main reason for this choice is the budget limitation. If you buy a DIY system and you are taking care of the setup, you do not need to pay for the system setup services provided by the vendor’s team. At first glance, this is perfect and meets the budget as much as possible.
Other situations occur, for example, when a simple data collection project or an academic project with inadequate funding is planned. In this case, the only possibility of ensuring an adequate level of evidence of efficiency and safety of the drug is to use a DIY system with self-configuration and self-validation performed in a simplified manner. In such projects, the risk of failure is relatively low, and it is with such projects that DIY systems are the most suitable.
What are the risks of using DIY systems?
The difficulties associated with DIY EDC relate to the process of setting up such systems. In some cases, especially if the sponsor did not use EDC before, it would seem that the setting looks rather simple: the team sits down, views the Protocol and the CRF, enters the necessary data into the system, configures the modules and that’s it: EDC is ready for use!
However, this approach is fundamentally wrong.
Let’s start with the first and the primary: team composition. EDC systems are complex and sophisticated technological products. Even if they have a user-friendly interface, a person who does not have experience in clinical data management will spend significant time learning all the modules and the customization principles. Even if a persistent person figures out the tasks, it's not a fact that one will be able to configure the EDC correctly to meet the project requirements the first time.
Therefore, the main thing you need to do if you decide to work with DIY is to form an experienced data management team (composed at least from 3 specialists).
These people have to understand the terminology, be able to work with different types of systems and understand the requirements of international standards. Unfortunately, if there are no such employees, it will be necessary to pay for outsourced additional services or to pay for consultancy services, which are not included in the initial price and are usually charged on an hourly basis. Bear this in mind while planning a budget.
The second stumbling block is working with the Protocol and CRF. Here it is critical to understand that the Protocol and paper CRF are for the doctor, for the project manager, for the clinical operation specialist, but not for the clinical data management team. The Protocol and CRF do not provide detailed instructions on how the CRF should be configured, how to set up a visit matrix, and how to organize access for all types of users. Without this data, it is almost impossible to set up the system correctly.
To make sure everything is done correctly, the data management team creates a separate technical document: System Requirements Specification. It is the link between the Protocol and the system, where technical language and special terminology related to EDC requirements is fixed. Only after the document has been settled and considered, can you start the EDC’s configuration to meet the project requirements.
First of all, it applies to DIY EDC, so once the data management team is formed, one of its first tasks should be to create a System Requirement Specification.
The third stumbling block that we should highlight is the organization of the configuration process itself. Standard operating procedures should regulate it. By mentioning the setup process, we mean not only the creation of the System Requirement Specification but also the configuration of the system and its following validation.
There are two important reasons why it is impossible to begin configuration and validation of the system without the preset procedures:
1. To ensure all processes were completed, and the execution was backed up by a report. Then there is no doubt that the system has been set up correctly. Moreover, even if problems arise, it will be easy to understand at what point something went wrong and who was responsible for it.
2. To get through inspection or audit by the regulator or the sponsor easily. The reports will show the interested party that you and your team have done everything correctly and in line with set procedures. At the same time, it is important to remember that SOPs and reports will only be valid if they are written in accordance with actual international requirements.
So, in general, these are the main pitfalls that the sponsor should be prepared to deal with when setting up DIY EDC.
Bear them in mind and do not forget to budget expenses for a professional team, create SOPs (if you do not have them) and other additional activities to avoid unpleasant surprises.
Moreover, if you compare the cost of the DIY + additional expenses, they can be more than ready-to-go EDC configured and validated by the vendor. There is no catch to this, it is just that in the full-service provider all processes are already set, SOPs are written, and the team is worked out. This setup takes less time physically and consequently requires fewer resources.
So, if the sponsor does not have experience working with DIY EDC, are there quite significant risks?
There are risks for sure, and the better you are informed, the more you will be able to minimize them. All of them originate from the setup procedures that we have discussed above.
It is possible to distinguish two main risks, which users should pay special attention to:
1. Failure to meet the requirements of the Protocol and CRF. This problem occurs quite often if the system was configured without prior detailed audit. This procedure is rather cumbersome as it is necessary to check the conformity of many parameters of the EDC system with the requirements of the Protocol and CRF. If such an audit is not carried out, the errors are detected in the last stage - the statistical processing of data. By this point, the process of data cleaning has been finished and the database has already been closed, investigators have received fees, and it is very difficult and sometimes impossible to fix the situation.
2. Low quality of the collected data. This issue partially comes out of the first one: if the system is not set up accurately, then the clinical data will not be collected properly. At best, this means that the information will be presented in the wrong format or incorrect measurement units. And, at worst, it may turn out that a set of data needed for statistical processing is not collected at all.
Who should use DIY EDC? And is it worth it at all?
Despite all the drawbacks, DIY systems are a beneficial solution for some projects, such as small clinical studies of Phase 1 or academic projects. They need neither huge eCRFs or complicated system architectures, so the EDC team can handle such a test within the EDC setting, especially if there are staff members with data management experience.
However, for complex clinical research or long post-marketing projects, it is better to choose other systems, because, in these cases, the use of DIY can involve very high risks which may jeopardize the whole project. Let's repeat, if you have a well-worked data management team and all the necessary SOPs, then DIY systems are the optimal solution. If you do not have a coherent and professional clinical data management team, the best option is a fully-configured EDC system.