A Comparison of Error Correcting Output Coding Methods for Multiclass Classification by Using Support Vector Machine: The Prediction of Self-Monitoring of Blood Sugar
DOI:
https://doi.org/10.6000/1929-6029.2013.02.02.06Keywords:
Support vector machines, Multiclass classification, Error correcting codes, Data mining, Kernel function, Bood sugar monitoringAbstract
SVMs were initially developed to perform binary classification. However, in many real-world problems, particularly pattern recocnation studies, aimed to determine the distinctive features of large number of class or group. For this reason, a number of methods to generate multiclass SVMs from binary SVMs have been proposed by researchers and this is still a continuing research topic. In this study we aimed to compare classification accuracy and computational cost of four multiclass approaches using a original and simulated data sets.
Error Corrected Output Coding (ECOC) based multiclass approaches that is used in this study creates many binary classifiers and combines their results to determine the class label of a test pixel.
As a result of the comparisons for all conditions examined in this study, it’s found that the classification accuracy and computational cost of One vs. One multiclass approach is better than the other multiclass approaches.
In classification or pattern recognization problems, some of supervised machine learning methods or algorithms can be easily extended to multiclass problems. However, some other powerful and popular classifiers, such as AdaBoost and Support Vector Machines, do not extend to multiclass easily. In those situations, the usual way to proceed is to reduce the complexity of the multiclass problem into multiple simpler binary classification problems.
References
Burges C. A tutorial on support vector machines for pattern recognition. Data Min Knowldisc 1998; 2: 121-67. http://dx.doi.org/10.1023/A:1009715923555 DOI: https://doi.org/10.1023/A:1009715923555
Hsu CW, Lin CJ. A comparison of methods for multiclass. IEEE T Neural Networ 2002; 13(2): 415-25. http://dx.doi.org/10.1109/72.991427 DOI: https://doi.org/10.1109/72.991427
Erastö P. Support vector machines - Academic Dissertation for the Degree of Licentiate of Philosophy, Helsinki: Rolf Nevanlinna Institute 2001; pp.1-70.
Platt JC. Probabilistic outputs for support vector machines and comparison to regularize likelihood methods. Advances in Large Margin Classifiers 2000; 61-74.
Madzarov G, Gjorgjevikj D, Chorbev I. A multi-class SVM classifier utilizing binary decision tree. Informatica-Lithuan 2009; 33(2): 225-33.
Pujol O, Radeva P, Vitria J. Discriminant ECOC: A heuristic method for application dependent design of error correcting output codes. IEEE T Pattern Anal 2006; 28(6): 1007-12. http://dx.doi.org/10.1109/TPAMI.2006.116 DOI: https://doi.org/10.1109/TPAMI.2006.116
Demirkesen C, Cherifi H. A Comparison of Multiclass SVM Methods for Real World Natural Scenes. In: Blanc-Talon J,
et al. Eds. Advanced Concepts for Intelligent Vision Systems, 10th International Conference; 2008; France: Juan-les-Pins 2008; pp. 752-63.
Pal M. Multiclass approaches for support vector machine based land cover classification. CoRR 2008; abs/0802.
Duan K-B, Keerthi SS. Which is the best multiclass SVM method? An empirical study; 2005: Proceedings of the Sixth International Workshop on Multiple Classifier Systems 2005: June 13-15; Seaside, CA, US, Springer-Verlag. pp. 278-85. DOI: https://doi.org/10.1007/11494683_28
Kikuchi T, Abe S. Comparison between error correcting output codes and fuzzy support vector machines. Pattern Recogn Lett 2005; 26(12): 1937-45. http://dx.doi.org/10.1016/j.patrec.2005.03.014 DOI: https://doi.org/10.1016/j.patrec.2005.03.014
Wang Z, Xu W, Hu J, Guo J. A multiclass SVM method via probabilistic error-correcting output codes. 2010 International Conference on Internet Technology and Applications 2010; pp. 1-4. DOI: https://doi.org/10.1109/ITAPP.2010.5566126
Dietterich TG, Bakiri G. Error-correcting output codes: A general method for improving multiclass inductive learning programs: 1991: Proceedings of the Ninth National Conference on Artificial Intelligence (AAAI-91); 1991: July 14-19; Anaheim, California, pp. 572-77.
Crammer K, Singer Y. On the Learnability and Design of Output Codes for Multiclass Problems: 2000: Proceedings of the Thirteenth Annual Conference on Computational Learning Theory; 2000: June 28-July 1; San Francisco, CA, USA; pp. 35-46.
Dietterich TG, Bakiri G. Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 1995; 2: 263-86. DOI: https://doi.org/10.1613/jair.105
Xiao-Feng L, Xue-Ying Z, Ji-Kang D, Eds. Speech recognition based on support vector machine and error correcting output codes 2010: First International Conference on Pervasive Computing Signal Processing and Applications (PCSPA); 2010: Sep 17-19; Harbin Institute of Technology, Harbin, China, pp. 336-9. http://dx.doi.org/10.1109/PCSPA.2010.88 DOI: https://doi.org/10.1109/PCSPA.2010.88
Angulo C, Parra X, Català A. K-SVCR. A support vector machine for multi-class classification. Neurocomputing 2003; 55(1-2): 57-77. http://dx.doi.org/10.1016/S0925-2312(03)00435-1 DOI: https://doi.org/10.1016/S0925-2312(03)00435-1
Jiang Z-Q, Fu H-G, Li L-J. Support vector machine for mechanical faults classification. J Zhejiang Univ Sci 2005; 6(5): 433-9. http://dx.doi.org/10.1631/jzus.2005.A0433 DOI: https://doi.org/10.1631/jzus.2005.A0433
Chang C-C, Lin C-J. LIBSVM: A Library for Support Vector Machines. Taipei, Taiwan. 2001, Available from: http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf and LIBSVM FAQ: http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq. html
Bilgisayarkavramlari [homepage on the Internet]. Seker S.E: Çok sınıflı DVM (Multiclass SVM). 2008, Available from: http://www.bilgisayarkavramlari.com/2008/12/01/cok-sinifli-dvm-multiclass-svm/ 05.10.2012.
Bose RC, Ray-Chaudhuri DK. On a class of error correcting binary group codes. Inf Technol Control 1960; 3(1): 68-79. http://dx.doi.org/10.1016/S0019-9958(60)90287-4 DOI: https://doi.org/10.1016/S0019-9958(60)90287-4
Peterson WW, Weldon EJ. Error-correcting codes. 2nd ed. MIT Press. Cambridge, Mass 1972.
Übeyli DE. ECG beats classification using multiclass support vector machines with error correcting output codes. Digit Signal Process 2007; 17(3): 675-84. http://dx.doi.org/10.1016/j.dsp.2006.11.009 DOI: https://doi.org/10.1016/j.dsp.2006.11.009
Allwein E, Shapire R, Singer Y. Reducing multiclass to binary: A unifying approach for margin classifiers. J Mach Learn Res 2000; 1: 113-41.
Baltaci D, Kutlucan A, Ozturk S, Saritas A, Celer A, Celbek G, Ankarali H. Effectiveness for self-monitoring of blood sugar on blood glucose control in Turkish patients with type 2 diabetes mellitus. Med Glas 2012; 9(2): 213-19.
Arrindell WA, Ende JV. An empirical test of the utility of the observations-to-variables ratio in factor and components analysis. Appl Psych Meas 1985; 9(2): 165-78. http://dx.doi.org/10.1177/014662168500900205 DOI: https://doi.org/10.1177/014662168500900205
Masulli F, Valentini G. Effectiveness of error-correcting output coding methods in ensemble and monolithic learning machines. Pattern Anal Appl 2003; 6(4): 285-300. http://dx.doi.org/10.1007/s10044-003-195-9 DOI: https://doi.org/10.1007/s10044-003-195-9
Liu Y. Using SVM and error-correcting codes for multiclass dialog act classification in meeting corpus. INTERSPEECH – ICSLP; 2006: Sep 17-21; Pittsburgh, Pennsylvania: ISCA. pp. 1938-41. DOI: https://doi.org/10.21437/Interspeech.2006-532
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2013 Özge Aksehirli, Handan Ankarali, Duygu Aydin, Davut Baltaci
This work is licensed under a Creative Commons Attribution 4.0 International License.
Policy for Journals/Articles with Open Access
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are permitted and encouraged to post links to their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work
Policy for Journals / Manuscript with Paid Access
Authors who publish with this journal agree to the following terms:
- Publisher retain copyright .
- Authors are permitted and encouraged to post links to their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work .
