Comprehensive Evaluation of Reference Values of Parametric and Non-Parametric Effect Size Methods for Two Independent Groups
DOI:
https://doi.org/10.6000/1929-6029.2022.11.11Keywords:
Effect Size, Parametric Effect Size, Non-Parametric Effect Size, Two Independent GroupsAbstract
In the field of health and other sciences, effect size (ES) provides a scientific approach to the effectiveness of treatment or intervention. The p-value indicates whether the statistical difference depends on chance, while ES gives information about the effectiveness of the treatment or intervention, even if the difference is not significant. For this reason, ES has become a very popular measure in recent years. It depends on which ES will be used based on the distribution of data and the number of groups. In this study, parametric and non-parametric ES were evaluated for two independent groups.
When the literature was examined, there were no studies aimed at evaluating the reference values of the parametric and non-parametric ES methods used for two independent groups. In this study, the reference values of parametric and non-parametric ES methods for two independent groups were re-evaluated by a simulation study. As a result, the very small reference value of parametric ES methods was determined differently from the literature. It has been seen that the reference values of non-parametric ES methods are valid in cases where the skewness is low, and new reference values have been proposed at the varying skewness level.
References
Kelley K, Preacher KJ. On effect size. Psychological Methods 2012; 17: 137. https://doi.org/10.1037/a0028086 DOI: https://doi.org/10.1037/a0028086
Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd edn. Hillsdale, NJ: Lawrence Erlbaum Associates 1988.
Valentine JC, Cooper H. A systematic and transparent approach for assessing the methodological quality of intervention effectiveness research: The Study Design and Implementation Assessment Device (Study DIAD). Psychological Methods 2008; 13: 130. https://doi.org/10.1037/1082-989X.13.2.130 DOI: https://doi.org/10.1037/1082-989X.13.2.130
Sawilowsky SS. New effect size rules of thumb. Journal of Modern Applied Statistical Methods 2009; 8: 26. https://doi.org/10.22237/jmasm/1257035100 DOI: https://doi.org/10.22237/jmasm/1257035100
Cohen J. The statistical power of abnormal-social psychological research: a review. The Journal of Abnormal and Social Psychology 1962; 65: 145. https://doi.org/10.1037/h0045186 DOI: https://doi.org/10.1037/h0045186
Peng C-YJ, Chen L-T. Beyond Cohen's d: Alternative effect size measures for between-subject designs. The Journal of Experimental Education 2014; 82: 22-50. https://doi.org/10.1080/00220973.2012.745471 DOI: https://doi.org/10.1080/00220973.2012.745471
Borenstein M, et al. Converting among effect sizes. Introduction to Meta-Analysis 2009; pp. 45-49.
Fleishman AI. A method for simulating non-normal distributions. Psychometrika 1978; 43: 521-532. https://doi.org/10.1007/BF02293811 DOI: https://doi.org/10.1007/BF02293811
Luo H. Generation of Non-normal Data: A Study of Fleishman’s Power Method, ed, Department of Statistics, Uppsala University, 2011.
Caliński T, Harabasz J. A dendrite method for cluster analysis. Communications in Statistics-theory and Methods 1974; 3: 1-27. https://doi.org/10.1080/03610927408827101 DOI: https://doi.org/10.1080/03610927408827101
Rousseeuw PJ. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis Journal of Computational and Applied Mathematics 1987; 20: 53-65. https://doi.org/10.1016/0377-0427(87)90125-7 DOI: https://doi.org/10.1016/0377-0427(87)90125-7
Ben-Shachar MS, et al. Package ‘effectsize’, 2021.
Cebeci Z, et al. Package ‘pplust’. CRAN, 2019.
Fialkowski A. SimMultiCorrData: Simulation of Correlated Data with Multiple Variable Types. 2018, URL https://CRAN. R-project. org/package= SimMultiCorrData. R package version 0.2 1, pp. p251.
Kassambara A, Mundt F. Package ‘factoextra’. Extract and Visualize the Results of Multivariate Data Analyses 2017; 76.
Mangiafico S, Mangiafico MS. Package ‘rcompanion’. Cran Repos 2017; 20: 1-71.
Rogmann JJ, Rogmann MJJ. Package ‘orddom’, 2013.
Torchiano M, Torchiano MM. Package ‘effsize’. Package “Effsize”[(accessed on 9 March 2021)] 2020.
Wickham H, et al. Package ‘readxl’ 2019.
Yarberry W. DPLYR, in CRAN Recipes, Springer, 2021; pp. 1-58. https://doi.org/10.1007/978-1-4842-6876-6_1 DOI: https://doi.org/10.1007/978-1-4842-6876-6_1
MacQueen J. Some methods for classification and analysis of multivariate observations, in Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, Oakland, CA, USA, 1967; pp. 281-297.
Ellis P. Thresholds for interpreting effect sizes. Retrieved January 13, 2009; pp. 2014.
Sullivan GM, Feinn R. Using effect size—or why the P value is not enough; Journal of graduate medical education 2012; 4: 279-282. https://doi.org/10.4300/JGME-D-12-00156.1 DOI: https://doi.org/10.4300/JGME-D-12-00156.1
---, Primary, secondary, and meta-analysis of research. Educational Researcher 1976; 5: 3-8. https://doi.org/10.3102/0013189X005010003 DOI: https://doi.org/10.3102/0013189X005010003
Hedges LV. Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics 1981; 6: 107-128. https://doi.org/10.3102/10769986006002107 DOI: https://doi.org/10.3102/10769986006002107
Cliff N. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin 1993; 114: 494. https://doi.org/10.1037/0033-2909.114.3.494 DOI: https://doi.org/10.1037/0033-2909.114.3.494
Glass GV. A ranking variable analogue of biserial correlation: Implications for short-cut item analysis. Journal of Educational Measurement 1965; 2: 91-95. https://doi.org/10.1111/j.1745-3984.1965.tb00396.x DOI: https://doi.org/10.1111/j.1745-3984.1965.tb00396.x
Vargha A, Delaney HD. A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics 2000; 25: 101-132. https://doi.org/10.3102/10769986025002101 DOI: https://doi.org/10.3102/10769986025002101
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Policy for Journals/Articles with Open Access
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are permitted and encouraged to post links to their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work
Policy for Journals / Manuscript with Paid Access
Authors who publish with this journal agree to the following terms:
- Publisher retain copyright .
- Authors are permitted and encouraged to post links to their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work .
