Journal of Guangdong University of Technology ›› 2021, Vol. 38 ›› Issue (01): 46-53.doi: 10.12052/gdutxb.200082

• Comprehensive Studies • Previous Articles     Next Articles

A Review on ROC Curve and Analysis

Wang Yan-guang, Zhu Hong-bin, Xu Wei-chao   

  1. School of Automation, Guangdong University of Technology, Guangzhou 510006, China
  • Received:2019-06-19 Online:2021-01-25 Published:2020-12-01

HTML

PDF (PC)

3657

Abstract: Receiver operating characteristic curve analysis can be used to evaluate the performance of classifiers and find the optimal segmentation points of different categories. Among them, the non-parametric method has been widely used in practice because of its simplicity and flexibility.How to construct and evaluate receiver operating characteristic curve for the classified data by ROC nonparametric method was introduced .

Key words: binary classification, receiver operating characteristic curve, nonparametric method, ROC curve evaluation

CLC Number: 

  • O212
[1] 孙旭. 接收机工作特性曲线研究[D].广州: 广东工业大学, 2016.
[2] HANLEY J A, OTHERS. Receiver operating characteristic (ROC) methodology: The state of the art [J]. Crit Rev Diagn Imaging, 1989, 29(3): 307-335.
[3] OBUCHOWSKI N A, BULLEN J A. Receiver operating characteristic (ROC) curves: review of methods with applications in diagnostic medicine [J]. Physics in Medicine & Biology, 2018, 63(7): 07TR01.
[4] SWETS J A. The relative operating characteristic in psychology: a technique for isolating effects of response bias finds wide use in the study of perception and cognition [J]. Science, 1973, 182(4116): 990-1000.
[5] SWETS J A, PICKETT R M. Evaluation of diagnostic systems: Methods from signal detection theory [J]. Evaluation of Diagnostic Systems, 1982, 10(2): 1-12.
[6] SPACKMAN K A. Signal detection theory: valuable tools for evaluating inductive learning[C]//Proceedings of the Sixth International Workshop on Machine Learning. San Francisco, USA: Morgan Kaufmann, 1989: 160-163.
[7] HUANG J, LING C X. Using AUC and accuracy in evaluating learning algorithms [J]. IEEE Transactions on Dnowledge and Data Engineering, 2005, 17(3): 299-310.
[8] NORTON M, URYASEV S. Maximization of AUC and buffered AUC in binary classification [J]. Mathematical Programming, 2019, 174(1-2): 575-612.
[9] BRADLEY A P. The use of the area under the ROC curve in the evaluation of machine learning algorithms [J]. Pattern Recognition, 1997, 30(7): 1145-1159.
[10] PROVOST F, FAWCETT T. Analysis and visualization of classifier performance with nonuniform class and cost distributions[C]//Proceedings of AAAI-97 Workshop on AI Approaches to Fraud Detection & Risk Management. California, USA: AAAI Press, 1997: 57-63.
[11] PROVOST F J, TOM F, RON K. The case against accuracy estimation for comparing induction algorithms[C]//Proceedings of the Fifteenth International Conference on Machine Learning. San Francisco, USA: Morgan Kaufmann, 1998: 445-453.
[12] BOWYER K, KRANENBURG C, DOUGHERTY S. Edge detector evaluation using empirical ROC curves [J]. Computer Vision and Image Understanding, 2001, 84(1): 77-103.
[13] SCHMUGGE S J, JAYARAM S, SHIN M C, et al. Objective evaluation of approaches of skin detection using ROC analysis [J]. Computer Vision and Image Understanding, 2007, 108(1-2): 41-51.
[14] FAWCETT T. An introduction to ROC analysis [J]. Pattern Recognition Letters, 2006, 27(8): 861-874.
[15] ALONZO T A, PEPE M S. Distribution-free ROC analysis using binary regression techniques [J]. Biostatistics, 2002, 3(3): 421-432.
[16] MAJNIK M, BOSNIC Z. ROC analysis of classifiers in machine learning: A survey [J]. Intelligent Data Analysis, 2013, 17(3): 531-558.
[17] WU W, LI A D, HE X H, et al. A comparison of support vector machines, artificial neural network and classification tree for identifying soil texture classes in southwest China [J]. Computers and Electronics in Agriculture, 2018, 144: 86-93.
[18] LIELI R P, HSU Y C. Using the area under an estimated ROC curve to test the adequacy of binary predictors [J]. Journal of Nonparametric Statistics, 2019, 31(1): 100-130.
[19] YE L, ZHANG Q, GUAN L. Use hierarchical genetic particle filter to figure articulated human tracking[C]//2008 IEEE International Conference on Multimedia and Expo. New Jersey, USA: IEEE Press, 2008: 1561-1564.
[20] SACCHETTO L, GASPARINI M. Proper likelihood ratio based ROC curves for general binary classification problems[J]. arXiv preprint arXiv: 1809.00694, 2018.
[21] LÓPEZ V, FERNÁNDEZ A, GARCÍA S, et al. An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics [J]. Information Sciences, 2013, 250: 113-141.
[22] 叶枫, 丁锋. 不平衡数据分类研究及其应用[J]. 计算机应用与软件, 2018, 35(1): 132-136.
[23] HAMEL L. Model assessment with ROC curves[C]//Encyclopedia of Data Warehousing and Mining, Second Edition. Pennsylvania, USA: IGI Global, 2009: 1316-1323.
[24] HSIEH F, TURNBULL B W. Nonparametric and semiparametric estimation of the receiver operating characteristic curve [J]. The Annals of Statistics, 1996, 24(1): 25-40.
[25] QIU P, LE C. ROC curve estimation based on local smoothing [J]. Journal of Statistical Computation and Simulation, 2001, 70(1): 55-69.
[26] MOURÃO M F, BRAGA A C, OLIVEIRA P N. CRIB conditional on gender: Nonparametric ROC curve [J]. International Journal of Health Care Quality Assurance, 2014.
[27] ZHOU Y, ZHOU H, MA Y. Smooth estimation of ROC curve in the presence of auxiliary information [J]. Journal of Systems Science and Complexity, 2011, 24(5): 919.
[28] DORFMAN D D, ALF JR E. Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervalsRating-method data [J]. Journal of Mathematical Psychology, 1969, 6(3): 487-496.
[29] PEPE M S, OTHERS. The statistical evaluation of medical tests for classification and prediction[M]. Oxford, UK: Oxford University Press, 2003.
[30] HANLEY J A. The robustness of the "binormal" assumptions used in fitting ROC curves [J]. Medical Decision Making, 1988, 8(3): 197-203.
[31] ZOU K H, HALL W. Two transformation models for estimating an ROC curve derived from continuous data [J]. Journal of Applied Statistics, 2000, 27(5): 621-631.
[32] GNEITING T, VOGEL P. Receiver operating characteristic (ROC) curves[J]. arXiv preprint arXiv: 1809.04808, 2018.
[33] CAI T, MOSKOWITZ C S. Semi-parametric estimation of the binormal ROC curve for a continuous diagnostic test [J]. Biostatistics, 2004, 5(4): 573-586.
[34] ZOU K H, HALL W, SHAPIRO D E. Smooth non-parametric receiver operating characteristic (ROC) curves for continuous diagnostic tests [J]. Statistics in Medicine, 1997, 16(19): 2143-2156.
[35] ZOU K H, TEMPANY C M, FIELDING J R, et al. Original smooth receiver operating characteristic curve estimation from continuous data: statistical methods for analyzing the predictive value of spiral CT of ureteral stones [J]. Academic Radiology, 1998, 5(10): 680-687.
[36] GAO H, YUZHU L I, HAN L, et al. Logistic regression and ROC work curve evaluated the value of five tumor markers in breast cancer diagnosis [J]. China Tropical Medicine, 2018.
[37] YI M, WENLIN C, BAOLONG G, et al. A novel logistic regression model based on density estimation [J]. Acta Automatica Sinica, 2014, 40(1): 62-72.
[38] ALONSO R, NAKAS C T, CARMEN PARDO M. A study of indices useful for the assessment of diagnostic markers in non-parametric ROC curve analysis [J]. Communications in Statistics-Simulation and Computation, 2020, 49(8): 102.
[39] XU W, DAI J, HUNG Y, et al. Estimating the area under a receiver operating characteristic (ROC) curve: parametric and nonparametric ways [J]. Signal Processing, 2013, 93(11): 3111-3123.
[40] RAGHAVAN R, ASHOUR F S, BAILEY R. A review of cutoffs for nutritional biomarkers [J]. Advances in Nutrition, 2016, 7(1): 112-120.
[41] ALSING S. The evaluation of competing classification[D]. Ohio, USA: Air Force Institute of Technology, 2002.
[42] PERKINS N J, SCHISTERMAN E F. The inconsistency of “optimal” cutpoints obtained using two criteria based on the receiver operating characteristic curve [J]. American Journal of Epidemiology, 2006, 163(7): 670-675.
[43] 周萌, 梁涛. 应用ROC曲线探讨心脏术后获得性吞咽障碍相关因素的临界值[J]. 慢性病学杂志, 2018(7): 857-860.
ZhOU M, LIANG T. The ROC curve was used to investigate the critical value of the fac tors associated with postoperative cardiac acquired swallowing disorders [J]. Chronic Pathematology Journal, 2018(7): 857-860.
[44] MACSKASSY S, PROVOST F. Confidence bands for ROC curves: Methods and an empirical study[R]. [2004-08-22]. https://www.researchgate.net/publication/221055307
[45] MACSKASSY S A, PROVOST F, ROSSET S. ROC confidence bands: An empirical evaluation[C]//Proceedings of the 22nd International Conference on Machine Learning. Bonn, Germany: ACM, 2005: 537-544.
[46] FLACH P A. The geometry of ROC space: understanding machine learning metrics through ROC isometrics[C]//Proceedings of the 20th International Conference on Machine Learning (ICML-03). Washington DC, USA: AAAI Press, 2003: 194-201.
No related articles found!
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © 2017 Journal of Guangdong University of Technology
Add：729 East Dongfeng RD., Guangzhou PRC. Postcode: 510090
Tel：020-37626653,020-37626139,020-37626396
Email：xbzrb@gdut.edu.cn
Supported:Beijing Magtech