A0888
Title: Confidence score: A data-driven measure for inclusive systematic reviews considering unpublished preprints
Authors: Jiayi Tong - Johns Hopkins University (United States) [presenting]
Chongliang Luo - Washington University in St Louis (United States)
Yifei Sun - Columbia University (United States)
Rui Duan - Harvard University (United States)
M Elle Saine - University of Pennsylvania (United States)
Lifeng Lin - University of Arizona (United States)
Yifan Peng - Weill Cornell Medicine (United States)
Yiwen Lu - University of Pennsylvania (United States)
Anchita Batra - University of Pennsylvania (United States)
Olivia Wang - University of Pennsylvania (United States)
Ruowang Li - Cedars-Sinai Medical Center (United States)
Arielle Marks-Anglin - University of Pennsylvania (United States)
Yuchen Yang - Johns Hopkins University (United States)
Xu Zuo - The University of Texas Health Science Center at Houston (United States)
Jiang Bian - University of Florida (United States)
Stephen Kimmel - University of Florida (United States)
Keith Hamilton - University of Pennsylvania (United States)
Adam Cuker - University of Pennsylvania (United States)
Rebecca Hubbard - University of Pennsylvania (United States)
Hua Xu - Yale University (United States)
Yong Chen - University of Pennsylvania (United States)
Abstract: COVID-19, since its emergence in December 2019, has globally impacted research, with over 360,000 COVID-19-related manuscripts published on PubMed and preprint servers like medRxiv and bioRxiv, where preprints comprise about 15\% of all manuscripts; yet, the role and impact of preprints on COVID-19 research and evidence synthesis remain uncertain. A novel data-driven method is proposed for assigning weights to individual preprints in systematic reviews and meta-analyses, using a confidence score derived from the survival cure model (also known as the survival mixture model), which accounts for the time elapsed between posting and publication of a preprint, as well as metadata such as the number of first two-week citations, sample size, and study type. Using 146 preprints on COVID-19 therapeutics posted from the beginning of the pandemic through April 30, 2021, the confidence scores are validated, showing an area under the curve of 0.95 (95\% CI, 0.92-0.98), and through a use case on the effectiveness of hydroxychloroquine, demonstrating how these scores can be practically incorporated into meta-analyses to properly weigh preprints.
