Study design decisions that determine whether your CLIA waiver data will stand up to FDA review.
A previous post in this series covered the intended use decision and what it commits a molecular diagnostic developer to, including the POC/OTC distinction, the three pathways to waived categorization, the dual versus stepwise submission question, and why waiver does not carry over from a predicate or an EUA. This post picks up where that one left off. Once the manufacturer makes the intended use decision and heads down the CLIA waiver route, this is where programs may run into trouble. The first decision that needs to be made is whether to pursue CLIA waiver in initial study and submission or do stepwise process with a CLIA waiver comparison study.
What the CLIA waiver comparison study demonstrates
The intended-user comparison study is the centerpiece of a CLIA waiver application for most molecular diagnostic developers. In this study, untrained operators from the intended use setting perform the test following only the package insert, and their results are compared against those of trained laboratory professionals performing the same test, or under certain study design options against a comparator method. The study is designed to demonstrate that the performance gap between untrained and trained operators is acceptably small: that moving the test out of the laboratory does not introduce a clinically meaningful increase in errors, invalid results, or misclassification. It is one required component of a broader CLIA waiver application package that also includes a device simplicity demonstration, risk analysis, flex studies addressing environmental and usage variation under stress conditions, and failure alert and fail-safe mechanism validation.
FDA’s 2020 guidance on CLIA waiver applications addresses study design requirements in detail, covering minimum site counts, operator counts, and result acceptance criteria for binary qualitative and quantitative tests. For test types outside those categories, including some multiplexed molecular panels, the guidance recommends contacting FDA through a pre-submission to discuss study design before proceeding. This advice should be taken seriously, especially for molecular platforms where the performance characteristics of the test and the pre-analytical variables involved differ from simpler test formats. For products of this nature, a pre-sub conversation about study design is worth the investment.
Who counts as a naive operator, and why it matters
FDA’s definition of an untrained operator goes beyond device-naivety. The guidance calls for operators representative of the intended waived setting who have limited or no hands-on laboratory testing experience generally, and no prior experience with the candidate test. FDA also recommends enrolling operators with the least amount of training likely to be encountered at the intended use sites. A laboratory technologist who happens to work at a waived-site clinic but has substantial laboratory testing experience is not the right recruitment target. The untrained operator population should reflect the worst-case scenario of who will be running the test in the field.
Operator recruitment is one of the most common places where comparison study designs introduce problems. The operators who participate must be representative of the people who will use the test in the intended use setting. If your intended use is urgent care clinics, your naive operators should be urgent care staff. If you enroll research staff who do not normally perform patient care tasks, you are not studying the population your intended use claim covers, and FDA may raise that as a concern during review.
Site selection carries the same logic. The waived sites enrolled in your study should represent the range of intended use environments, including variation in patient population, operator training background, and physical environment. A study conducted entirely at a single site type may not reflect the range of conditions your test will encounter in the field.
Pre-analytical variables: where molecular tests are most exposed
Closed-cartridge molecular systems are designed to contain the most technically demanding steps of amplification-based testing: reagent preparation, amplification, and detection are all instrument-controlled and not accessible to the operator. This is a real advantage in a waiver study context. But closed-cartridge design does not eliminate operator-dependent variables; it shifts them to the steps the operator does handle.
Specimen collection is the step that most directly affects performance in naive operator hands for swab-based molecular tests. For an amplification-based test detecting a pathogen from a nasal or nasopharyngeal swab, collection technique affects the quantity and quality of nucleic acid presented to the assay. An inadequately collected specimen can produce a false result even when the instrument performs exactly as designed. How operators collect specimens, whether they follow the collection instructions in the labeling, and how consistently they do so across operators and sites is data your comparison study should capture.
Cartridge handling and sample introduction are the other operator-dependent steps for most closed-cartridge molecular systems. Sample volume, the mechanics of transferring a specimen to a cartridge, and cartridge loading into the instrument are procedural steps where errors occur. Your instructions for use and the error-detection capability of your system both play a role in how these errors manifest in study results.
For some isothermal amplification platforms, ambient temperature and humidity at the point of care can affect assay performance. If your system has restrictive operating condition requirements, your comparison study sites might not meet the requirements for your product’s performance. It is important, therefore, to test your system in conditions that do not represent a central laboratory’s controlled environment.
Invalid rate: a metric FDA scrutinizes closely for molecular systems
An invalid result is distinct from a negative result. It means the assay did not complete successfully and no result was generated. Internal controls in molecular systems, which may monitor amplification, extraction, or both depending on system design, are intended to detect these failures. A well-functioning internal control system is an asset for a waiver submission because it provides a mechanism to flag when operator or pre-analytical errors have compromised the run, rather than allowing those errors to produce false negatives.
FDA’s concern about invalid rates in the comparison study is whether naive operators generate substantially more invalid results than trained operators. A statistical difference in invalid rate between the two groups raises questions about whether the test meets the simplicity threshold for waiver, because it suggests that untrained operation of the device is associated with a higher rate of test failure. A robust study design should plan for how invalid results will be handled, whether repeat testing is allowed, and how both the invalid rate itself and the outcome of any repeats will be reported.
Your instructions for use must tell operators what to do when they receive an invalid result. This is both a labeling requirement and a human factors consideration.
Designing the study to support both FDA market authorization and the waiver application
For developers pursuing a dual submission, the comparison and reproducibility studies need to be designed to satisfy both market authorization (510(k) or De Novo) and the waiver application simultaneously. These are not identical requirements. The 510(k) comparison study or De Novo reference study is typically performed by trained operators meeting moderate complexity qualifications, while the waiver comparison study uses untrained operators from the intended use setting. FDA notes that an applicant may choose a single study package with untrained operators to support both objectives in some cases. Whether that approach works for a given molecular program, or whether both trained and untrained operator data are needed, is a study design question that belongs in the pre-submission, not one to be resolved mid-enrollment.
FDA’s 2020 dual submission guidance addresses this design requirement directly and strongly recommends a pre-submission to confirm the study design before initiating clinical work. For a molecular POC dual submission, that recommendation should be treated as the practical default: the study design decisions made before enrollment (sites, operators, specimen handling, comparator methods, and how invalid results will be managed) determine what data you collect, and the data you collect determines what you can say in your device labeling. Discovering a design gap mid-enrollment is a serious problem. Discovering it during review is worse.
The OTC path for molecular diagnostics
The CLIA-waived OTC path for molecular diagnostics is not well-traveled. Most of the molecular tests that were authorized for home use during the COVID-19 pandemic were authorized under Emergency Use Authorizations, which do not follow the same evidentiary framework as traditional marketing authorization. Developers now pursuing traditional FDA clearance with OTC intended use for a molecular product are working in a space where the regulatory precedent is still developing.
The human factors burden for OTC is substantially higher than for POC, and for molecular tests it includes challenges that are specific to the platform. Label comprehension studies must demonstrate that lay users correctly understand all possible result states, including invalid results, and know what action to take for each. Simulated use testing must capture real error modes in representative users under realistic home-use conditions. Self-collected specimen adequacy must be studied with lay users and not assumed as the gap between the collection performance of a naive lay user and that of a naive healthcare worker is real. For molecular tests where specimen adequacy directly affects sensitivity, that gap matters.
A companion application that guides users through sample collection and result interpretation can reduce these risks, but it introduces its own validation requirements. Software as a medical device considerations may apply depending on the application’s role in the testing workflow. If the application is integral to generating or interpreting a result, it is part of the device and needs to be treated accordingly.
For developers with serious commercial interest in OTC molecular testing, early engagement with FDA through a pre-submission is strongly advisable. The study design questions, human factors scope, and software classification determinations that are unresolved before you commit to an OTC development program are the ones most likely to surface as obstacles during review. Getting FDA alignment on those questions before enrollment, not after, is what a credible development plan looks like.
This is the second post in a two-part series on CLIA waiver strategy for molecular IVD developers. Part 1 covers the intended use decision and what it commits you to: the POC/OTC distinction, the three pathways to waived categorization, dual versus stepwise submissions, and why waiver does not carry over from a predicate or an EUA. Read Part 1 here.
Talk to DCN Dx’s Regulatory Affairs team
If you are designing a CLIA waiver comparison study for a near-patient molecular test, preparing a pre-submission on study design, or working through the human factors requirements for an OTC molecular program, our Regulatory Affairs team has worked through these questions with molecular IVD developers across diagnostic platforms and submission pathways. The study design decisions that look like operational details are the ones that determine what your data can support when it reaches FDA. It is worth getting them right before enrollment begins.
Contact our Regulatory Affairs team here or visit dcndx.com/regulatory-affairs-services to learn about our services, including CLIA waiver study design.
References
1. U.S. Food and Drug Administration. Recommendations for Clinical Laboratory Improvement Amendments of 1988 (CLIA) Waiver Applications for Manufacturers of In Vitro Diagnostic Devices. February 2020.
2. U.S. Food and Drug Administration. Recommendations for Dual 510(k) and CLIA Waiver by Application Studies for In Vitro Diagnostic Devices. February 2020.
3. U.S. Food and Drug Administration. Applying Human Factors and Usability Engineering to Medical Devices. February 2016.
4. U.S. Food and Drug Administration. Policy for Device Software Functions and Mobile Medical Applications. September 2022.
