How should we consider the design types for clinical trials of computer-assisted decision-making products based on deep learning?
06 05 2024
For products that provide decision support in triaging and referring patients based on whether they have a target disease, without specifying the exact lesion conditions, and where a professional physician must review the patient's images regardless of the triage outcome (whether negative or positive), a single-group target value design can be considered. The main evaluation metrics should focus on the diagnostic accuracy of the product's assistance in triaging, such as sensitivity and specificity, typically at the patient level.
For products that assist in detecting specific lesions of a target disease, such as lung nodule detection aids or bone fracture CT image detection aids, a controlled trial design is recommended. In this setup, the experimental group consists of physicians using the product in conjunction with traditional diagnostic methods (like clinical review or comprehensive diagnosis). The main evaluation metrics should consider diagnostic accuracy indicators such as sensitivity, specificity, AFROC curves, and detection rates, with sensitivity generally assessed at the lesion level and specificity at the patient level. The type of clinical trial comparison should reflect the acceptable risk-benefit ratio of the product, and an effectiveness design is suggested. For example, for software assisting in detecting lung nodules larger than 4mm on CT images, consider superiority in specificity at the patient level and non-inferiority in sensitivity at the lesion level.