Improved Ridge Regression Estimators for Binary Choice Models: An Empirical Study

Authors

  • Kristofer Månsson Department of Economics and Statistics, Jönköping University, Sweden
  • B.M. Golam Kibria Department of Mathematics and Statistics, Florida International University, Miami, Florida, USA
  • Ghazi Shukur Centre for Labour Market Policy (CAFO), Department of Economics and Statistics, Linnaeus University, Sweden

DOI:

https://doi.org/10.6000/1929-6029.2014.03.03.5

Keywords:

Binary Choice Models, Estimation, MSE, Multicollinearity, Ridge Regression, Simulation

Abstract

This paper suggests some new estimators of the ridge parameter for binary choice models that may be applied in the presence of a multicollinearity problem. These new ridge parameters are functions of other estimators of the ridge parameter that have shown to work well in the previous research. Using a simulation study we investigate the mean square error (MSE) properties of these new ridge parameters and compare them with the best performing estimators from the previous research. The results indicate that we may improve the MSE properties of the ridge regression estimator by applying the proposed estimators in this paper, especially when there is a high multicollinearity between the explanatory variables and when many explanatory variables are included in the regression model. The benefit of this paper is then shown by a health related data where the effect of some risk factors on the probability of receiving diabetes is investigated.

Author Biographies

B.M. Golam Kibria, Department of Mathematics and Statistics, Florida International University, Miami, Florida, USA

Department of Mathematics and Statistics

Ghazi Shukur, Centre for Labour Market Policy (CAFO), Department of Economics and Statistics, Linnaeus University, Sweden

Centre for Labour Market Policy (CAFO), Department of Economics and Statistics

References

Van Den Noortgate W, De Boeck P, Meulders M. Cross-classification multilevel logistic models in psychometrics. J Educat Behav Statist 2003; 28: 369-86. http://dx.doi.org/10.3102/10769986028004369 DOI: https://doi.org/10.3102/10769986028004369

Cameron AC, Trivedi PK. Microeconometrics: Methods and Applications. Cambridge University Press 2005. http://dx.doi.org/10.1017/CBO9780511811241 DOI: https://doi.org/10.1017/CBO9780511811241

Schaefer RL, Roi LD, Wolfe RA. A ridge logistic estimator. Commun Statist Theory Methods 1984; 13: 99-113. http://dx.doi.org/10.1080/03610928408828664 DOI: https://doi.org/10.1080/03610928408828664

Månsson K, Shukur G. On ridge parameters in logistic regression. Commun Statist Theory Methods 2011a; 40: 3366-81. http://dx.doi.org/10.1080/03610926.2010.500111 DOI: https://doi.org/10.1080/03610926.2010.500111

Hoerl AE, Kennard RW. Ridge regression: biased estimation for non-orthogonal problems. Technometrics 1970a; 12: 55-67. http://dx.doi.org/10.1080/00401706.1970.10488634 DOI: https://doi.org/10.1080/00401706.1970.10488634

Hoerl AE, Kennard RW. Ridge regression: application to non-orthogonal problems. Technometrics 1970b; 12: 69-82. http://dx.doi.org/10.1080/00401706.1970.10488635 DOI: https://doi.org/10.1080/00401706.1970.10488635

Hoerl AE, Kennard RW, Baldwin KF. Ridge regression: some simulation. Commun Statist Theory Methods 1975; 4: 105-23. http://dx.doi.org/10.1080/03610927508827232 DOI: https://doi.org/10.1080/03610917508548342

McDonald GC, Galarneau DI. A Monte Carlo evaluation of some ridge-type estimators. J Am Statist Assoc 1975; 70: 407-16. http://dx.doi.org/10.1080/01621459.1975.10479882 DOI: https://doi.org/10.1080/01621459.1975.10479882

Lawless JF, Wang P. A simulation study of ridge and other regression estimators. Commun Statist Theory Methods 1976; 5: 307-23. http://dx.doi.org/10.1080/03610927608827353 DOI: https://doi.org/10.1080/03610927608827353

Gibbons DG. A simulation study of some ridge estimators. J Am Statist Assoc 1981; 76: 131-39. http://dx.doi.org/10.1080/01621459.1981.10477619 DOI: https://doi.org/10.1080/01621459.1981.10477619

Kibria BMG. Performance of some new ridge regression estimators. Commun Statist Theory Methods 2003; 32: 419-35. DOI: https://doi.org/10.1081/SAC-120017499

Khalaf G, Shukur G. Choosing ridge parameters for regression problems. Commun Statist Theory Methods 2005; 34: 1177-82. http://dx.doi.org/10.1081/STA-200056836 DOI: https://doi.org/10.1081/STA-200056836

Alkhamisi MA, Khalaf G, Shukur G. Some modifications for choosing ridge parameter. Commun Statist Theory Methods 2006; 35: 1-16. http://dx.doi.org/10.1080/03610920600762905 DOI: https://doi.org/10.1080/03610920600762905

Alkhamisi M, Shukur G. Developing ridge parameters for SUR model. Commun Statist Theory Methods 2008; 37: 544-64. http://dx.doi.org/10.1080/03610920701469152 DOI: https://doi.org/10.1080/03610920701469152

Muniz G, Kibria BMG. On some ridge regression estimators: An empirical comparisons. Commun Statist Simulat Comput 2009; 38: 621-30. http://dx.doi.org/10.1080/03610910802592838 DOI: https://doi.org/10.1080/03610910802592838

Arashi M, Valizadeh T. Performance of Kibria’s methods in partial linear ridge regression model. To Appear in Statistical Papers 2014. DOI: https://doi.org/10.1007/s00362-014-0578-6

Aslam M. Performance of Kibria's method for the heteroscedastic ridge regression model: Some Monte Carlo evidence. Commun Statist Simulat Comput 2014; 43: 673-86. http://dx.doi.org/10.1080/03610918.2012.712185 DOI: https://doi.org/10.1080/03610918.2012.712185

Hefnawya AE, Faraga A. A combined nonlinear programming model and Kibria method for choosing ridge parameter regression. Commun Statist Simulat Comput 2014; 43: 1442-70. http://dx.doi.org/10.1080/03610918.2012.735317 DOI: https://doi.org/10.1080/03610918.2012.735317

Kibria BMG, Månsson K, Shukur G. Performance of some logistic ridge regression estimators. Comput Econom 2011; 40: 401-14. http://dx.doi.org/10.1007/s10614-011-9275-x DOI: https://doi.org/10.1007/s10614-011-9275-x

Smith JW, Everhart JE, Dickson WC, Knowler WC, Johannes RS. Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In Proceedings of the Symposium on Computer Applications in Medical Care. In Proceedings of the Symposium on Computer Applications in Medical Care Computer Society Press 1988.

Downloads

Published

2014-08-05

How to Cite

Månsson, K., Kibria, B. G., & Shukur, G. (2014). Improved Ridge Regression Estimators for Binary Choice Models: An Empirical Study. International Journal of Statistics in Medical Research, 3(3), 257–265. https://doi.org/10.6000/1929-6029.2014.03.03.5

Issue

Vol. 3 No. 3 (2014)

Section

General Articles

License

Copyright (c) 2014 Kristofer Månsson, B.M. Golam Kibria, Ghazi Shukur

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Policy for Journals/Articles with Open Access

Authors who publish with this journal agree to the following terms:

  • Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

  • Authors are permitted and encouraged to post links to their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work

Policy for Journals / Manuscript with Paid Access

Authors who publish with this journal agree to the following terms:

  • Publisher retain copyright .

  • Authors are permitted and encouraged to post links to their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work .
Crossref
Scopus
Google Scholar
Europe PMC