广东工业大学学报 ›› 2021, Vol. 38 ›› Issue (01): 46-53.doi: 10.12052/gdutxb.200082

• 综合研究 • 上一篇    下一篇

ROC曲线及其分析方法综述

王彦光, 朱鸿斌, 徐维超   

  1. 广东工业大学 自动化学院，广东 广州 510006
  • 收稿日期:2019-06-19 出版日期:2021-01-25 发布日期:2020-12-01
  • 通信作者: 徐维超(1970-)，男，教授，博士，博士生导师，主要研究方向为统计信号处理，E-mail：WCXU@gdut.edu.cn E-mail:WCXU@gdut.edu.cn
  • 作者简介:王彦光(1982-)，男，博士研究生，主要研究方向为统计信号处理，E-mail：yanguang.wang@foxmail.com
  • 基金资助:
    国家自然科学基金资助项目(61771148，61875041)；广州市科技计划项目(201802010037，201902010045)

A Review on ROC Curve and Analysis

Wang Yan-guang, Zhu Hong-bin, Xu Wei-chao   

  1. School of Automation, Guangdong University of Technology, Guangzhou 510006, China
  • Received:2019-06-19 Online:2021-01-25 Published:2020-12-01

HTML

PDF (PC)

3838

摘要： 接收机工作特性曲线分析可用于评价分类器的性能以及寻找不同类别的最优分割点等问题, 其构建方法主要包含参数法及非参数法。其中, 非参数法因其简单、灵活的特性, 在实际应用中得到比较广泛的应用。针对二分类问题, 详细介绍如何通过接收机工作特性曲线非参数法对分类数据进行接收机工作特性曲线构建和评价。

关键词: 二分类, 接收机工作特性曲线, 非参数法, ROC曲线评价

Abstract: Receiver operating characteristic curve analysis can be used to evaluate the performance of classifiers and find the optimal segmentation points of different categories. Among them, the non-parametric method has been widely used in practice because of its simplicity and flexibility.How to construct and evaluate receiver operating characteristic curve for the classified data by ROC nonparametric method was introduced .

Key words: binary classification, receiver operating characteristic curve, nonparametric method, ROC curve evaluation

中图分类号: 

  • O212
[1] 孙旭. 接收机工作特性曲线研究[D].广州: 广东工业大学, 2016.
[2] HANLEY J A, OTHERS. Receiver operating characteristic (ROC) methodology: The state of the art [J]. Crit Rev Diagn Imaging, 1989, 29(3): 307-335.
[3] OBUCHOWSKI N A, BULLEN J A. Receiver operating characteristic (ROC) curves: review of methods with applications in diagnostic medicine [J]. Physics in Medicine & Biology, 2018, 63(7): 07TR01.
[4] SWETS J A. The relative operating characteristic in psychology: a technique for isolating effects of response bias finds wide use in the study of perception and cognition [J]. Science, 1973, 182(4116): 990-1000.
[5] SWETS J A, PICKETT R M. Evaluation of diagnostic systems: Methods from signal detection theory [J]. Evaluation of Diagnostic Systems, 1982, 10(2): 1-12.
[6] SPACKMAN K A. Signal detection theory: valuable tools for evaluating inductive learning[C]//Proceedings of the Sixth International Workshop on Machine Learning. San Francisco, USA: Morgan Kaufmann, 1989: 160-163.
[7] HUANG J, LING C X. Using AUC and accuracy in evaluating learning algorithms [J]. IEEE Transactions on Dnowledge and Data Engineering, 2005, 17(3): 299-310.
[8] NORTON M, URYASEV S. Maximization of AUC and buffered AUC in binary classification [J]. Mathematical Programming, 2019, 174(1-2): 575-612.
[9] BRADLEY A P. The use of the area under the ROC curve in the evaluation of machine learning algorithms [J]. Pattern Recognition, 1997, 30(7): 1145-1159.
[10] PROVOST F, FAWCETT T. Analysis and visualization of classifier performance with nonuniform class and cost distributions[C]//Proceedings of AAAI-97 Workshop on AI Approaches to Fraud Detection & Risk Management. California, USA: AAAI Press, 1997: 57-63.
[11] PROVOST F J, TOM F, RON K. The case against accuracy estimation for comparing induction algorithms[C]//Proceedings of the Fifteenth International Conference on Machine Learning. San Francisco, USA: Morgan Kaufmann, 1998: 445-453.
[12] BOWYER K, KRANENBURG C, DOUGHERTY S. Edge detector evaluation using empirical ROC curves [J]. Computer Vision and Image Understanding, 2001, 84(1): 77-103.
[13] SCHMUGGE S J, JAYARAM S, SHIN M C, et al. Objective evaluation of approaches of skin detection using ROC analysis [J]. Computer Vision and Image Understanding, 2007, 108(1-2): 41-51.
[14] FAWCETT T. An introduction to ROC analysis [J]. Pattern Recognition Letters, 2006, 27(8): 861-874.
[15] ALONZO T A, PEPE M S. Distribution-free ROC analysis using binary regression techniques [J]. Biostatistics, 2002, 3(3): 421-432.
[16] MAJNIK M, BOSNIC Z. ROC analysis of classifiers in machine learning: A survey [J]. Intelligent Data Analysis, 2013, 17(3): 531-558.
[17] WU W, LI A D, HE X H, et al. A comparison of support vector machines, artificial neural network and classification tree for identifying soil texture classes in southwest China [J]. Computers and Electronics in Agriculture, 2018, 144: 86-93.
[18] LIELI R P, HSU Y C. Using the area under an estimated ROC curve to test the adequacy of binary predictors [J]. Journal of Nonparametric Statistics, 2019, 31(1): 100-130.
[19] YE L, ZHANG Q, GUAN L. Use hierarchical genetic particle filter to figure articulated human tracking[C]//2008 IEEE International Conference on Multimedia and Expo. New Jersey, USA: IEEE Press, 2008: 1561-1564.
[20] SACCHETTO L, GASPARINI M. Proper likelihood ratio based ROC curves for general binary classification problems[J]. arXiv preprint arXiv: 1809.00694, 2018.
[21] LÓPEZ V, FERNÁNDEZ A, GARCÍA S, et al. An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics [J]. Information Sciences, 2013, 250: 113-141.
[22] 叶枫, 丁锋. 不平衡数据分类研究及其应用[J]. 计算机应用与软件, 2018, 35(1): 132-136.
[23] HAMEL L. Model assessment with ROC curves[C]//Encyclopedia of Data Warehousing and Mining, Second Edition. Pennsylvania, USA: IGI Global, 2009: 1316-1323.
[24] HSIEH F, TURNBULL B W. Nonparametric and semiparametric estimation of the receiver operating characteristic curve [J]. The Annals of Statistics, 1996, 24(1): 25-40.
[25] QIU P, LE C. ROC curve estimation based on local smoothing [J]. Journal of Statistical Computation and Simulation, 2001, 70(1): 55-69.
[26] MOURÃO M F, BRAGA A C, OLIVEIRA P N. CRIB conditional on gender: Nonparametric ROC curve [J]. International Journal of Health Care Quality Assurance, 2014.
[27] ZHOU Y, ZHOU H, MA Y. Smooth estimation of ROC curve in the presence of auxiliary information [J]. Journal of Systems Science and Complexity, 2011, 24(5): 919.
[28] DORFMAN D D, ALF JR E. Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervalsRating-method data [J]. Journal of Mathematical Psychology, 1969, 6(3): 487-496.
[29] PEPE M S, OTHERS. The statistical evaluation of medical tests for classification and prediction[M]. Oxford, UK: Oxford University Press, 2003.
[30] HANLEY J A. The robustness of the "binormal" assumptions used in fitting ROC curves [J]. Medical Decision Making, 1988, 8(3): 197-203.
[31] ZOU K H, HALL W. Two transformation models for estimating an ROC curve derived from continuous data [J]. Journal of Applied Statistics, 2000, 27(5): 621-631.
[32] GNEITING T, VOGEL P. Receiver operating characteristic (ROC) curves[J]. arXiv preprint arXiv: 1809.04808, 2018.
[33] CAI T, MOSKOWITZ C S. Semi-parametric estimation of the binormal ROC curve for a continuous diagnostic test [J]. Biostatistics, 2004, 5(4): 573-586.
[34] ZOU K H, HALL W, SHAPIRO D E. Smooth non-parametric receiver operating characteristic (ROC) curves for continuous diagnostic tests [J]. Statistics in Medicine, 1997, 16(19): 2143-2156.
[35] ZOU K H, TEMPANY C M, FIELDING J R, et al. Original smooth receiver operating characteristic curve estimation from continuous data: statistical methods for analyzing the predictive value of spiral CT of ureteral stones [J]. Academic Radiology, 1998, 5(10): 680-687.
[36] GAO H, YUZHU L I, HAN L, et al. Logistic regression and ROC work curve evaluated the value of five tumor markers in breast cancer diagnosis [J]. China Tropical Medicine, 2018.
[37] YI M, WENLIN C, BAOLONG G, et al. A novel logistic regression model based on density estimation [J]. Acta Automatica Sinica, 2014, 40(1): 62-72.
[38] ALONSO R, NAKAS C T, CARMEN PARDO M. A study of indices useful for the assessment of diagnostic markers in non-parametric ROC curve analysis [J]. Communications in Statistics-Simulation and Computation, 2020, 49(8): 102.
[39] XU W, DAI J, HUNG Y, et al. Estimating the area under a receiver operating characteristic (ROC) curve: parametric and nonparametric ways [J]. Signal Processing, 2013, 93(11): 3111-3123.
[40] RAGHAVAN R, ASHOUR F S, BAILEY R. A review of cutoffs for nutritional biomarkers [J]. Advances in Nutrition, 2016, 7(1): 112-120.
[41] ALSING S. The evaluation of competing classification[D]. Ohio, USA: Air Force Institute of Technology, 2002.
[42] PERKINS N J, SCHISTERMAN E F. The inconsistency of “optimal” cutpoints obtained using two criteria based on the receiver operating characteristic curve [J]. American Journal of Epidemiology, 2006, 163(7): 670-675.
[43] 周萌, 梁涛. 应用ROC曲线探讨心脏术后获得性吞咽障碍相关因素的临界值[J]. 慢性病学杂志, 2018(7): 857-860.
ZhOU M, LIANG T. The ROC curve was used to investigate the critical value of the fac tors associated with postoperative cardiac acquired swallowing disorders [J]. Chronic Pathematology Journal, 2018(7): 857-860.
[44] MACSKASSY S, PROVOST F. Confidence bands for ROC curves: Methods and an empirical study[R]. [2004-08-22]. https://www.researchgate.net/publication/221055307
[45] MACSKASSY S A, PROVOST F, ROSSET S. ROC confidence bands: An empirical evaluation[C]//Proceedings of the 22nd International Conference on Machine Learning. Bonn, Germany: ACM, 2005: 537-544.
[46] FLACH P A. The geometry of ROC space: understanding machine learning metrics through ROC isometrics[C]//Proceedings of the 20th International Conference on Machine Learning (ICML-03). Washington DC, USA: AAAI Press, 2003: 194-201.
No related articles found!
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2017 《广东工业大学学报》编辑部
地址：广州市东风东路729号广东工业大学 邮编：510090
电话：020-37626653,020-37626139,020-37626396
邮箱：xbzrb@gdut.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发