Individualizing Risk Prediction for Positive Coronavirus Disease 2019 Testing
MD, MHCDS Lara Jehi, MS Xinge Ji, MS Alex Milinovich, MD Serpil Erzurum, MD Brian P Rubin, MD Steve Gordon, MD James B Young, PhD Michael W Kattan
Chest, doi:10.1016/j.chest.2020.05.580
BACKGROUND: Coronavirus disease 2019 (COVID-19) is sweeping the globe. Despite multiple case-series, actionable knowledge to tailor decision-making proactively is missing. RESEARCH QUESTION: Can a statistical model accurately predict infection with COVID-19? STUDY DESIGN AND METHODS: We developed a prospective registry of all patients tested for COVID-19 in Cleveland Clinic to create individualized risk prediction models. We focus here on the likelihood of a positive nasal or oropharyngeal COVID-19 test. A least absolute shrinkage and selection operator logistic regression algorithm was constructed that removed variables that were not contributing to the model's cross-validated concordance index. After external validation in a temporally and geographically distinct cohort, the statistical prediction model was illustrated as a nomogram and deployed in an online risk calculator. RESULTS: In the development cohort, 11,672 patients fulfilled study criteria, including 818 patients (7.0%) who tested positive for COVID-19; in the validation cohort, 2295 patients fulfilled criteria, including 290 patients who tested positive for COVID-19. Male, African American, older patients, and those with known COVID-19 exposure were at higher risk of being positive for COVID-19. Risk was reduced in those who had pneumococcal polysaccharide or influenza vaccine or who were on melatonin, paroxetine, or carvedilol. Our model had favorable discrimination (c-statistic ¼ 0.863 in the development cohort and 0.840 in the validation cohort) and calibration. We present sensitivity, specificity, negative predictive value, and positive predictive value at different prediction cutoff points. The calculator is freely available at https://riskcalc.org/COVID19. INTERPRETATION: Prediction of a COVID-19 positive test is possible and could help direct health-care resources. We demonstrate relevance of age, race, sex, and socioeconomic characteristics in COVID-19 susceptibility and suggest a potential modifying role of certain common vaccinations and drugs that have been identified in drug-repurposing studies.
Author contributions: L. J. is the guarantor of submission and participated in literature search, figures, study design, data collection, data interpretation, and writing; X. J. participated in data analysis and figures; A. M. participated in data collection and data analysis; S. E. participated in data interpretation, study design, and writing; B. P. R., S. G., and J. B. Y. participated in data interpretation and writing; and M. W. K. participated in literature search, study design, data interpretation, data analysis, and writing. Financial/nonfinancial disclosures: The authors have reported to CHEST the following: A. M. reports grants from Novo Nordisk, Boehringer Ingelheim, Merck, Novartis, and National Institutes of Health (NIH), outside the submitted work. M. W. K. reports grants from Novo Nordisk, Boehringer Ingelheim, Merck, Novartis, and NIH, consulting for Stratify Genomics and RenatlyxAI, outside the submitted work. None declared (L. J., X. J., S. E., B. P. R., S. G., J. B. Y.).
Role of sponsors: The sponsor had no role in the design of the study, the collection and analysis of the data, or the preparation of the manuscript. Additional information: The e-Appendix and e-Figure can be found in the Supplemental Materials section of the online article.
References
Chen, Zhou, Dong, Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study, Lancet
Goff, Hayashi, Martínez-Gil, Synthetic toll-like receptor 4 (TLR4) and TLR7 ligands as influenza virus vaccine adjuvants induce rapid, sustained, and broadly protective responses, J Virol
Harrell, Califf, Pryor, Lee, Rosati, Evaluating the yield of medical tests, JAMA
Harris, Taylor, Minor, The REDCap consortium: building an international community of software partners, J Biomed Inform
Harris, Taylor, Thielke, Payne, Gonzalez et al., Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support, J Biomed Inform
Iqr ¼ Interquartile Range ; Callaway, Cyranoski, Mallapaty, Stoye, Tollefson, The coronavirus pandemic in five powerful charts, Nature
Jin, Lian, Hu, Epidemiological, clinical and virological characteristics of 74 cases of coronavirusinfected disease 2019 (COVID-19) with gastrointestinal symptoms, Gut
Kattan, Gerds, The index of prediction accuracy: an intuitive measure useful for evaluating risk prediction models, Diagn Progn Res
Kattan, Nomograms: introduction, Semin Urol Oncol
Lipsitch, Swerdlow, Finelli, Defining the epidemiology of Covid-19 -studies needed, N Engl J Med
Milinovich, Kattan, Extracting and utilizing electronic health data from Epic for research, Ann Transl Med
Ogbadu, Singh, Gupta, Mehra, Sen, Ageing reduces angiotensin II type 1 receptor antagonism mediated pre-conditioning effects in ischemic kidneys by inducing oxidative and inflammatory stress, Exp Gerontol
Pretesting, dL Missing
Sharfstein, Becker, Mello, Diagnostic testing for the novel coronavirus, JAMA
Shi, Yu, Zhao, Wang, Zhao et al., Host susceptibility to severe COVID-19 and establishment of a host risk score: findings of 487 cases outside Wuhan, Crit Care
Steyerberg, Vickers, Cook, Assessing the performance of prediction models: a framework for traditional and novel measures, Epidemiology
Sun, Koh, Marimuthu, Epidemiological and clinical predictors of COVID-19, Clin Infect Dis
Wang, Xu, Gao, SARS-CoV-2 in different types of clinical specimens, JAMA
Wang, Yang, Li, Wen, Zhang, Clinical features of 69 cases with coronavirus disease 2019 in Wuhan, China, Clin Infect Dis
Weiss, Murdoch, Clinical course and mortality risk of severe COVID-19, Lancet
Wu, Chen, Cai, Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in Wuhan, China, JAMA Intern Med
Wu, Li, Wang, Xiu, Zhang, Carvedilol inhibits angiotensin ii-induced proliferation and contraction in hepatic stellate cells through the RhoA/Rho-Kinase pathway, Biomed Res Int
Yang, Yu, Xu, Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study [published correction appears, Lancet Respir Med
Yoon, Kim, Kim, Ageassociated changes in the vascular reninangiotensin system in mice, Oxide Med Cell Longev
Zhang, Dong, Cao, Clinical characteristics of 140 patients infected with SARS-CoV-2 in
Zhou, Hou, Shen, Networkbased drug repurposing for novel coronavirus 2019-nCoV/SARS-CoV-2, Cell Discov
Zhou, Yu, Du, Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study [published correction appears, Lancet
{ 'indexed': { 'date-parts': [[2023, 12, 21]],
'date-time': '2023-12-21T17:05:47Z',
'timestamp': 1703178347971},
'reference-count': 27,
'publisher': 'Elsevier BV',
'issue': '4',
'license': [ { 'start': { 'date-parts': [[2020, 10, 1]],
'date-time': '2020-10-01T00:00:00Z',
'timestamp': 1601510400000},
'content-version': 'tdm',
'delay-in-days': 0,
'URL': 'https://www.elsevier.com/tdm/userlicense/1.0/'}],
'funder': [ { 'DOI': '10.13039/100006108',
'name': 'National Center for Advancing Translational Sciences',
'doi-asserted-by': 'publisher',
'award': ['UL1TR002548']},
{'DOI': '10.13039/100001003', 'name': 'Boehringer Ingelheim', 'doi-asserted-by': 'publisher'},
{'DOI': '10.13039/100007311', 'name': 'Cleveland Clinic', 'doi-asserted-by': 'publisher'},
{'DOI': '10.13039/100004334', 'name': 'Merck', 'doi-asserted-by': 'publisher'},
{'DOI': '10.13039/501100004191', 'name': 'Novo Nordisk', 'doi-asserted-by': 'publisher'},
{ 'DOI': '10.13039/100000002',
'name': 'National Institutes of Health',
'doi-asserted-by': 'publisher'},
{'DOI': '10.13039/100004336', 'name': 'Novartis', 'doi-asserted-by': 'publisher'}],
'content-domain': { 'domain': [ 'clinicalkey.fr', 'clinicalkey.jp', 'clinicalkey.com.au', 'clinicalkey.es',
'clinicalkey.com', 'journal.chestnet.org', 'elsevier.com',
'sciencedirect.com'],
'crossmark-restriction': True},
'published-print': {'date-parts': [[2020, 10]]},
'DOI': '10.1016/j.chest.2020.05.580',
'type': 'journal-article',
'created': {'date-parts': [[2020, 6, 10]], 'date-time': '2020-06-10T16:39:33Z', 'timestamp': 1591807173000},
'page': '1364-1375',
'update-policy': 'http://dx.doi.org/10.1016/elsevier_cm_policy',
'source': 'Crossref',
'is-referenced-by-count': 134,
'title': 'Individualizing Risk Prediction for Positive Coronavirus Disease 2019 Testing',
'prefix': '10.1016',
'volume': '158',
'author': [ {'given': 'Lara', 'family': 'Jehi', 'sequence': 'first', 'affiliation': []},
{'given': 'Xinge', 'family': 'Ji', 'sequence': 'additional', 'affiliation': []},
{'given': 'Alex', 'family': 'Milinovich', 'sequence': 'additional', 'affiliation': []},
{'given': 'Serpil', 'family': 'Erzurum', 'sequence': 'additional', 'affiliation': []},
{'given': 'Brian P.', 'family': 'Rubin', 'sequence': 'additional', 'affiliation': []},
{'given': 'Steve', 'family': 'Gordon', 'sequence': 'additional', 'affiliation': []},
{'given': 'James B.', 'family': 'Young', 'sequence': 'additional', 'affiliation': []},
{'given': 'Michael W.', 'family': 'Kattan', 'sequence': 'additional', 'affiliation': []}],
'member': '78',
'reference': [ {'key': '10.1016/j.chest.2020.05.580_bib1', 'author': 'Callaway'},
{ 'issue': '15',
'key': '10.1016/j.chest.2020.05.580_bib2',
'doi-asserted-by': 'crossref',
'first-page': '1437',
'DOI': '10.1001/jama.2020.3864',
'article-title': 'Diagnostic testing for the novel coronavirus',
'volume': '323',
'author': 'Sharfstein',
'year': '2020',
'journal-title': 'JAMA'},
{ 'issue': '13',
'key': '10.1016/j.chest.2020.05.580_bib3',
'doi-asserted-by': 'crossref',
'first-page': '1194',
'DOI': '10.1056/NEJMp2002125',
'article-title': 'Defining the epidemiology of Covid-19 - studies needed.',
'volume': '382',
'author': 'Lipsitch',
'year': '2020',
'journal-title': 'N Engl J Med'},
{ 'issue': '18',
'key': '10.1016/j.chest.2020.05.580_bib4',
'first-page': '1843',
'article-title': 'SARS-CoV-2 in different types of clinical specimens',
'volume': '323',
'author': 'Wang',
'year': '2020',
'journal-title': 'JAMA'},
{ 'issue': '2',
'key': '10.1016/j.chest.2020.05.580_bib5',
'first-page': '79',
'article-title': 'Nomograms: introduction',
'volume': '20',
'author': 'Kattan',
'year': '2002',
'journal-title': 'Semin Urol Oncol'},
{ 'key': '10.1016/j.chest.2020.05.580_bib6',
'doi-asserted-by': 'crossref',
'first-page': '14',
'DOI': '10.1038/s41421-020-0153-3',
'article-title': 'Network-based drug repurposing for novel coronavirus '
'2019-nCoV/SARS-CoV-2',
'volume': '6',
'author': 'Zhou',
'year': '2020',
'journal-title': 'Cell Discov'},
{ 'issue': '3',
'key': '10.1016/j.chest.2020.05.580_bib7',
'doi-asserted-by': 'crossref',
'first-page': '42',
'DOI': '10.21037/atm.2018.01.13',
'article-title': 'Extracting and utilizing electronic health data from Epic for research',
'volume': '6',
'author': 'Milinovich',
'year': '2018',
'journal-title': 'Ann Transl Med'},
{ 'issue': '2',
'key': '10.1016/j.chest.2020.05.580_bib8',
'doi-asserted-by': 'crossref',
'first-page': '377',
'DOI': '10.1016/j.jbi.2008.08.010',
'article-title': 'Research electronic data capture (REDCap): a metadata-driven '
'methodology and workflow process for providing translational research '
'informatics support',
'volume': '42',
'author': 'Harris',
'year': '2009',
'journal-title': 'J\xa0Biomed Inform'},
{ 'key': '10.1016/j.chest.2020.05.580_bib9',
'doi-asserted-by': 'crossref',
'unstructured': 'Harris PA, Taylor R, Minor BL, et\xa0al. The REDCap consortium: building '
'an international community of software partners, J Biomed Inform. '
'2019;95:103208.',
'DOI': '10.1016/j.jbi.2019.103208'},
{ 'issue': '18',
'key': '10.1016/j.chest.2020.05.580_bib10',
'doi-asserted-by': 'crossref',
'first-page': '2543',
'DOI': '10.1001/jama.1982.03320430047030',
'article-title': 'Evaluating the yield of medical tests',
'volume': '247',
'author': 'Harrell',
'year': '1982',
'journal-title': 'JAMA'},
{ 'key': '10.1016/j.chest.2020.05.580_bib11',
'doi-asserted-by': 'crossref',
'first-page': '7',
'DOI': '10.1186/s41512-018-0029-2',
'article-title': 'The index of prediction accuracy: an intuitive measure useful for '
'evaluating risk prediction models',
'volume': '2',
'author': 'Kattan',
'year': '2018',
'journal-title': 'Diagn Progn Res'},
{ 'issue': '1',
'key': '10.1016/j.chest.2020.05.580_bib12',
'doi-asserted-by': 'crossref',
'first-page': '128',
'DOI': '10.1097/EDE.0b013e3181c30fb2',
'article-title': 'Assessing the performance of prediction models: a framework for '
'traditional and novel measures',
'volume': '21',
'author': 'Steyerberg',
'year': '2010',
'journal-title': 'Epidemiology'},
{ 'issue': '6',
'key': '10.1016/j.chest.2020.05.580_bib14',
'doi-asserted-by': 'crossref',
'first-page': '1002',
'DOI': '10.1136/gutjnl-2020-320926',
'article-title': 'Epidemiological, clinical and virological characteristics of 74 cases '
'of coronavirus-infected disease 2019 (COVID-19) with gastrointestinal '
'symptoms',
'volume': '69',
'author': 'Jin',
'year': '2020',
'journal-title': 'Gut'},
{ 'key': '10.1016/j.chest.2020.05.580_bib15',
'unstructured': 'Sun Y, Koh V, Marimuthu K, et\xa0al. Epidemiological and clinical '
'predictors of COVID-19. Clin Infect Dis. 2020;ciaa322.'},
{ 'issue': '10229',
'key': '10.1016/j.chest.2020.05.580_bib16',
'doi-asserted-by': 'crossref',
'first-page': '1014',
'DOI': '10.1016/S0140-6736(20)30633-4',
'article-title': 'Clinical course and mortality risk of severe COVID-19',
'volume': '395',
'author': 'Weiss',
'year': '2020',
'journal-title': 'Lancet'},
{ 'issue': '1',
'key': '10.1016/j.chest.2020.05.580_bib17',
'doi-asserted-by': 'crossref',
'first-page': '108',
'DOI': '10.1186/s13054-020-2833-7',
'article-title': 'Host susceptibility to severe COVID-19 and establishment of a host risk '
'score: findings of 487 cases outside Wuhan',
'volume': '24',
'author': 'Shi',
'year': '2020',
'journal-title': 'Crit Care'},
{ 'key': '10.1016/j.chest.2020.05.580_bib18',
'doi-asserted-by': 'crossref',
'unstructured': 'Wang Z, Yang B, Li Q, Wen L, Zhang R. Clinical features of 69 cases with '
'coronavirus disease 2019 in Wuhan, China. Clin Infect Dis. 2020;ciaa272.',
'DOI': '10.1093/cid/ciaa272'},
{ 'key': '10.1016/j.chest.2020.05.580_bib19',
'doi-asserted-by': 'crossref',
'unstructured': 'Zhou F, Yu T, Du R, et\xa0al. Clinical course and risk factors for '
'mortality of adult inpatients with COVID-19 in Wuhan, China: a '
'retrospective cohort study [published correction appears in Lancet. 2020 '
'Mar 28;395(10229):1038]. Lancet. 2020;395(10229):1054-1062.',
'DOI': '10.1016/S0140-6736(20)30566-3'},
{ 'key': '10.1016/j.chest.2020.05.580_bib20',
'unstructured': 'Wu C, Chen X, Cai Y, et\xa0al. Risk factors associated with acute '
'respiratory distress syndrome and death in patients with coronavirus '
'disease 2019 pneumonia in Wuhan, China. JAMA Intern Med. 2020;e200994.'},
{ 'key': '10.1016/j.chest.2020.05.580_bib21',
'doi-asserted-by': 'crossref',
'unstructured': 'Yang X, Yu Y, Xu J, et\xa0al. Clinical course and outcomes of critically '
'ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a '
'single-centered, retrospective, observational study [published '
'correction appears in Lancet Respir Med. 2020 Apr;8(4):e26]. Lancet '
'Respir Med. 2020;8(5):475-481.',
'DOI': '10.1016/S2213-2600(20)30079-5'},
{ 'key': '10.1016/j.chest.2020.05.580_bib22',
'unstructured': 'Zhang JJ, Dong X, Cao YY, et\xa0al. Clinical characteristics of 140 '
'patients infected with SARS-CoV-2 in Wuhan, China. Allergy. In press.'},
{ 'issue': '10223',
'key': '10.1016/j.chest.2020.05.580_bib23',
'doi-asserted-by': 'crossref',
'first-page': '507',
'DOI': '10.1016/S0140-6736(20)30211-7',
'article-title': 'Epidemiological and clinical characteristics of 99 cases of 2019 novel '
'coronavirus pneumonia in Wuhan, China: a descriptive study',
'volume': '395',
'author': 'Chen',
'year': '2020',
'journal-title': 'Lancet'},
{ 'key': '10.1016/j.chest.2020.05.580_bib24',
'unstructured': 'World Health Organization. '
'http://www.euro.who.int/en/health-topics/health-emergencies/coronavirus-covid-19/weekly-surveillance-report. '
'Accessed March 28, 2020.'},
{ 'key': '10.1016/j.chest.2020.05.580_bib25',
'first-page': '6731093',
'article-title': 'Age-associated changes in the vascular renin-angiotensin system in mice',
'volume': '2016',
'author': 'Yoon',
'year': '2016',
'journal-title': 'Oxide Med Cell Longev'},
{ 'key': '10.1016/j.chest.2020.05.580_bib26',
'doi-asserted-by': 'crossref',
'first-page': '110892',
'DOI': '10.1016/j.exger.2020.110892',
'article-title': 'Ageing reduces angiotensin II type 1 receptor antagonism mediated '
'pre-conditioning effects in ischemic kidneys by inducing oxidative and '
'inflammatory stress',
'volume': '135',
'author': 'Ogbadu',
'year': '2020',
'journal-title': 'Exp Gerontol'},
{ 'issue': '6',
'key': '10.1016/j.chest.2020.05.580_bib27',
'doi-asserted-by': 'crossref',
'first-page': '3221',
'DOI': '10.1128/JVI.03337-14',
'article-title': 'Synthetic toll-like receptor 4 (TLR4) and TLR7 ligands as influenza '
'virus vaccine adjuvants induce rapid, sustained, and broadly protective '
'responses',
'volume': '89',
'author': 'Goff',
'year': '2015',
'journal-title': 'J\xa0Virol'},
{ 'key': '10.1016/j.chest.2020.05.580_bib28',
'doi-asserted-by': 'crossref',
'first-page': '7932046',
'DOI': '10.1155/2019/7932046',
'article-title': 'Carvedilol inhibits angiotensin ii-induced proliferation and '
'contraction in hepatic stellate cells through the RhoA/Rho-Kinase '
'pathway',
'volume': '2019',
'author': 'Wu',
'year': '2019',
'journal-title': 'Biomed Res Int'}],
'container-title': 'Chest',
'original-title': [],
'language': 'en',
'link': [ { 'URL': 'https://api.elsevier.com/content/article/PII:S0012369220316548?httpAccept=text/xml',
'content-type': 'text/xml',
'content-version': 'vor',
'intended-application': 'text-mining'},
{ 'URL': 'https://api.elsevier.com/content/article/PII:S0012369220316548?httpAccept=text/plain',
'content-type': 'text/plain',
'content-version': 'vor',
'intended-application': 'text-mining'}],
'deposited': { 'date-parts': [[2022, 11, 16]],
'date-time': '2022-11-16T04:23:43Z',
'timestamp': 1668572623000},
'score': 1,
'resource': {'primary': {'URL': 'https://linkinghub.elsevier.com/retrieve/pii/S0012369220316548'}},
'subtitle': [],
'short-title': [],
'issued': {'date-parts': [[2020, 10]]},
'references-count': 27,
'journal-issue': {'issue': '4', 'published-print': {'date-parts': [[2020, 10]]}},
'alternative-id': ['S0012369220316548'],
'URL': 'http://dx.doi.org/10.1016/j.chest.2020.05.580',
'relation': {},
'ISSN': ['0012-3692'],
'subject': [ 'Cardiology and Cardiovascular Medicine', 'Critical Care and Intensive Care Medicine',
'Pulmonary and Respiratory Medicine'],
'container-title-short': 'Chest',
'published': {'date-parts': [[2020, 10]]},
'assertion': [ {'value': 'Elsevier', 'name': 'publisher', 'label': 'This article is maintained by'},
{ 'value': 'Individualizing Risk Prediction for Positive Coronavirus Disease 2019 Testing',
'name': 'articletitle',
'label': 'Article Title'},
{'value': 'Chest', 'name': 'journaltitle', 'label': 'Journal Title'},
{ 'value': 'https://doi.org/10.1016/j.chest.2020.05.580',
'name': 'articlelink',
'label': 'CrossRef DOI link to publisher maintained version'},
{'value': 'article', 'name': 'content_type', 'label': 'Content Type'},
{ 'value': '© 2020 American College of Chest Physicians. Published by Elsevier Inc. All '
'rights reserved.',
'name': 'copyright',
'label': 'Copyright'}]}
