Background and objective: The use of in silico clinical trials (ISCTs) to generate clinically-relevant data on new medical devices is an emerging area of regulatory research. Interest in ISCTs stems from recognized challenges in acquiring sufficient clinical data and the continued maturation of in silico technologies. There is currently no guidance in place for evaluating the credibility of ISCT applications. The objective of this work was to adapt an existing risk-based credibility framework specifically for ISCT applications, and demonstrate its utility on a contemporary case study.

Methods: Expanding on guidance currently in place for assessing the risk of traditional modeling applications of medical devices and demonstrating model credibility through benchtop validation activities, a framework is proposed to (1) evaluate the model risk for ISCT applications based on the independent factors of scope, coverage, and severity, and (2) assess the credibility of clinical validation activities based on consideration of the clinical comparator, the validation model, the agreement between the two, and the applicability of the clinical validation activities to the ISCT application.

Results: The resulting framework spans across the range of ISCT applications that may be envisioned, as well as the variety of clinical datasets that can be used to demonstrate model credibility. Credibility factors reflect the expected clinical variability in the validation comparator and validation model, the statistical power of the comparator, the rigor of agreement between the comparator and model in terms of both inputs and outputs, and the overall similarity of the device in the validation activities to the device within the intended ISCT. When applied to a high-risk case study, the framework reveals that planned clinical validation activities require additional rigor in order to achieve the credibility _targets, enabling an assessment of the validation effort relative to the potential benefit prior to investing in the validation studies.

Discussion: An objective and risk-based framework for establishing credibility requirements for ISCT applications is a critical step in advancing ISCT from theory to practice. The proposed framework enforces that appropriate validation of ISCT applications requires evidence that the intended clinical environment is accurately represented. The framework will contribute to reducing uncertainty amongst technical, clinical, and regulatory constituents on ISCT applications, and promote rational adoption.