Comparison of Some Methods of Testing Statistical Hypotheses: (Part I. Parallel Methods)
DOI:
https://doi.org/10.6000/1929-6029.2014.03.02.11Keywords:
Hypotheses testing, -value, likelihood ratio, frequentist approaches, Bayesian approach, constrained Bayesian method, decision regionsAbstract
The article focuses on the discussion of basic approaches to hypotheses testing, which are Fisher, Jeffreys, Neyman, Berger approaches and a new one proposed by the author of this paper and called the constrained Bayesian method (CBM). Wald and Berger sequential tests and the test based on CBM are presented also. The positive and negative aspects of these approaches are considered on the basis of computed examples. Namely, it is shown that CBM has all positive characteristics of the above-listed methods. It is a data-dependent measure like Fisher’s test for making a decision, uses a posteriori probabilities like the Jeffreys test and computes error probabilities Type I and Type II like the Neyman-Pearson’s approach does. Combination of these properties assigns new properties to the decision regions of the offered method. In CBM the observation space contains regions for making the decision and regions for no-making the decision. The regions for no-making the decision are separated into the regions of impossibility of making a decision and the regions of impossibility of making a unique decision. These properties bring the statistical hypotheses testing rule in CBM much closer to the everyday decision-making rule when, at shortage of necessary information, the acceptance of one of made suppositions is not compulsory. Computed practical examples clearly demonstrate high quality and reliability of CBM. In critical situations, when other tests give opposite decisions, it gives the most logical decision. Moreover, for any information on the basis of which the decision is made, the set of error probabilities is defined for which the decision with given reliability is possible.
References
Bauer P. et al. Multiple Hypothesenprüfung, (Multiple Hypotheses Testing), Berlin: Springer-Verlag (In German and English) 1988. DOI: https://doi.org/10.1007/978-3-642-52307-6
Berger JO. Could Fisher, Jeffreys and Neyman have Agreed on Testing? Statist Sci 2003; 18: 1-32. http://dx.doi.org/10.1214/ss/1056397485 DOI: https://doi.org/10.1214/ss/1056397485
Berger JO, Boukai B, Wang Y. Unified Frequentist and Bayesian Testing of a Precise Hypothesis. Statist Sci 1997; 12(3): 133-160. DOI: https://doi.org/10.1214/ss/1030037904
Berger JO, Boukai B, Wang Y. Simultaneous Bayesian-frequentist sequential testing of nested hypotheses. Biometrika 1999; 86: 79-92. http://dx.doi.org/10.1093/biomet/86.1.79 DOI: https://doi.org/10.1093/biomet/86.1.79
Berger JO, Brown LD, Wolpert RL. A Unified Conditional Frequentist and Bayesian Test for Fixed and Sequential Simple Hypothesis Testing. Ann Statist 1994; 22(4): 1787-1807. http://dx.doi.org/10.1214/aos/1176325757 DOI: https://doi.org/10.1214/aos/1176325757
Berger JO, Delampady M. Testing precise hypothesis (with discussion). Statist Sci 1987; 2: 317-352. http://dx.doi.org/10.1214/ss/1177013238 DOI: https://doi.org/10.1214/ss/1177013238
Berger JO, Sellke T. Testing a point null hypothesis. The irreconcilability of p-values and evidence (with discussion). J Am Statist Assoc 1987; 82: 112-139. DOI: https://doi.org/10.2307/2289131
Berger JO, Wolpert RL. The Likelihood Principle. IMS, Hayward: CA 1984.
Berger JO, Wolpert RL. The Likelihood Principle, 2nd ed. (with discussion). IMS, Hayward: CA 1988. DOI: https://doi.org/10.1007/978-1-4613-8768-8_15
Berk RH, Limiting behavior of posterior distributions when the model is incorrect. Ann Math Statist 1966; 37: 51-58. http://dx.doi.org/10.1214/aoms/1177699597 DOI: https://doi.org/10.1214/aoms/1177699597
Bernardo JM, A Bayesian analysis of classical hypothesis testing. Universidad de Valencia 1980; 605-617. DOI: https://doi.org/10.1007/BF02888370
Bernardo JM, Rueda R. Bayesian Hypothesis Testing: A Reference Approach. Int Statist Rev 2002; 1-22. DOI: https://doi.org/10.2307/1403862
Braun HI, The Collected Works of John W. Tukey. Vol. VIII: Multiple Comparisons: 1948-1983. New York: Chapman & Hall 1994.
Brownie C, Keifer J. The Ideas of Conditional Confidence in the simplest setting. Comm. Statist. Theory Methods 1977; 6: 691-751. http://dx.doi.org/10.1080/03610927708827526 DOI: https://doi.org/10.1080/03610927708827526
Cassela G, Berger RL, Reconciling evidence in the one-sided testing problem (with discussion). J Am Statist Assoc 1987; 82: 106-112. http://dx.doi.org/10.1080/01621459.1987.10478396 DOI: https://doi.org/10.1080/01621459.1987.10478396
Casella G, Wells MT. Comparing p-values to Neyman-Pearson tests. Technical Report BU-1073-M, Biometrics Unit and Statistics Cent., Cornell Univ 1990.
Christensen R. Testing Fisher, Neyman, Pearson, and Bayes. Am Statist 2005; 59(2): 121-126. http://dx.doi.org/10.1198/000313005X20871 DOI: https://doi.org/10.1198/000313005X20871
Dass SC, Berger JO. Unified Conditional Frequentist and Bayesian Testing of Composite Hypotheses. Scand J Statist 2003; 30(1): 193-210. http://dx.doi.org/10.1111/1467-9469.00326 DOI: https://doi.org/10.1111/1467-9469.00326
Delampady M, Berger JO. Lower bounds on Bayes factors for the multinomial distribution, with application to chi-squared tests of fit. Ann Statist 1990; 18: 1295-1316. http://dx.doi.org/10.1214/aos/1176347750 DOI: https://doi.org/10.1214/aos/1176347750
Edwards W, Lindman H, Savage LJ. Bayesian statistical inference for psychological research. Psychol Rev 1963; 70: 193-242. http://dx.doi.org/10.1037/h0044139 DOI: https://doi.org/10.1037/h0044139
Efron B. Large-Scale Simultaneous Hypothesis Testing. J Am Statist Assoc 2004; 99(465): 96-104. http://dx.doi.org/10.1198/016214504000000089 DOI: https://doi.org/10.1198/016214504000000089
Fisher RA. Statistical Methods for Research Workers, London: Oliver and Boyd 1925.
Gómez-Villegas MA, González-Pérez B. A Bayesian Analysis for the Homogeneity Testing Problem Using -Contaminated Priors. Commun Statist - Theory Methods 2011; 40(6): 1049-1062. http://dx.doi.org/10.1080/03610920903520547 DOI: https://doi.org/10.1080/03610920903520547
Gómez-Villegas MA, Maín P, Sanz L. A Bayesian analysis for the multivariate point null testing problem. Statistics 2009; 43(4): 379-391. http://dx.doi.org/10.1080/02331880802505173 DOI: https://doi.org/10.1080/02331880802505173
Good IJ. The Bayesian/Non-Bayesian compromise: a brief review. J Am Statist Assoc 1992; 87: 597-606. http://dx.doi.org/10.1080/01621459.1992.10475256 DOI: https://doi.org/10.1080/01621459.1992.10475256
Hochberg Y, Tamhance AC. Multiple Comparison Procedures, New York: Wiley 1987. DOI: https://doi.org/10.1002/9780470316672
Hoppe FM. Multiple Comparisons. Selection and Applications in Biometry, New York: Dekker 1993.
Hsu JC. Multiple Comparisons: Theory and methods, New York: Chapman & Hall 1996. http://dx.doi.org/10.1007/978-1-4899-7180-7 DOI: https://doi.org/10.1201/b15074
Hubbard R, Bayarri MJ. Confusion over Measures of Evidence (p’s) Versus Errors (α’s) in Classical Statistical Testing. Am Statist 2003; 57: 171-177. http://dx.doi.org/10.1198/0003130031856 DOI: https://doi.org/10.1198/0003130031856
Hwang JT, Casella G, Robert Ch, Wells MT, Farrell RH. Estimation of Accuracy in Testing. Ann Statist 1992 20(1): 490-509. http://dx.doi.org/10.1214/aos/1176348534 DOI: https://doi.org/10.1214/aos/1176348534
Jeffreys H. Theory of Probability, 1st ed. Oxford: The Clarendon Press 1939.
Kachiashvili KJ. Bayesian algorithms of many hypothesis testing, Tbilisi: Ganatleba 1989.
Kachiashvili KJ. Generalization of Bayesian Rule of Many Simple Hypotheses Testing. Int J Informat Technol Decision Making 2003; 2(1): 41-70. http://dx.doi.org/10.1142/S0219622003000525 DOI: https://doi.org/10.1142/S0219622003000525
Kachiashvili KJ. Investigation and Computation of Unconditional and Conditional Bayesian Problems of Hypothesis Testing. ARPN J Syst Softw 2011; 1(2): 47-59.
Kachiashvili KJ. The Methods of Sequential Analysis of Bayesian Type for the Multiple Testing Problem. Sequential Anal 2014; 33: 23-38. http://dx.doi.org/10.1080/07474946.2013.843318 DOI: https://doi.org/10.1080/07474946.2013.843318
Kachiashvili GK, Kachiashvili KJ, Mueed A. Specific Features of Regions of Acceptance of Hypotheses in Conditional Bayesian Problems of Statistical Hypotheses Testing. Sankhya: Ind J Statist 2012; 74(1): 112-125. DOI: https://doi.org/10.1007/s13171-012-0014-8
Kachiashvili KJ, Hashmi MA. About Using Sequential Analysis Approach for Testing Many Hypotheses. Bulletin of the Georgian Acad Sci 2010; 4(2): 20-25.
Kachiashvili KJ, Hashmi MA, Mueed A. Sensitivity Analysis of Classical and Conditional Bayesian Problems of Many Hypotheses Testing. Communin Statist—Theory Methods 2012; 41(4): 591-605. http://dx.doi.org/10.1080/03610926.2010.510255 DOI: https://doi.org/10.1080/03610926.2010.510255
Kachiashvili KJ, Mueed A. Conditional Bayesian Task of Testing Many Hypotheses. Statistics 2011; 47(2): 274-293. http://dx.doi.org/10.1080/02331888.2011.602681 DOI: https://doi.org/10.1080/02331888.2011.602681
Klockars AJ, Sax G. Multiple Comparison. Newbury Park, CA: Sage 1986. DOI: https://doi.org/10.4135/9781412985185
Kiefer J. Conditional confidence statement and confidence estimations (with discussion). J Am Statist Assoc 1977; 72(360): 789-808. http://dx.doi.org/10.2307/2286460 DOI: https://doi.org/10.1080/01621459.1977.10479956
Lehmann EL. The Fisher, Neyman-Pearson Theories of Testing Hypotheses: One Theory or Two? Am Statist Assoc J Theory Methods 1993; 88(424): 1242-1249. DOI: https://doi.org/10.1080/01621459.1993.10476404
Lehmann E L. Testing Statistical Hypotheses, 2nd ed. New York: Springer 1997. DOI: https://doi.org/10.1214/ss/1029963261
Miller RG. Simultaneous Statistical Inference, New York: Wiley 1966.
Moreno E, Giron FJ. On the Frequentist and Bayesian Approaches to Hypothesis Testing, SORT 30(1) January-June 2006; 3-28.
Neyman J, Pearson E. On the Use and Interpretation of Certain Test Criteria for Purposes of Statistical Inference. Part I, Biometrica 1928 20A: 175-240. DOI: https://doi.org/10.1093/biomet/20A.1-2.175
Neyman J, Pearson E. On the Problem of the Most Efficient Tests of Statistical Hypotheses, Philos. Trans Roy Soc Ser A 1933; 231: 289-337. DOI: https://doi.org/10.1098/rsta.1933.0009
Rao CR. Linear Statistical Inference and Its Application, 2nd ed. New York: Wiley 2006.
Rosenthal R, Rubin DB. Psychological Bulletin 1983; 94(3): 540-541. http://dx.doi.org/10.1037/0033-2909.94.3.540 DOI: https://doi.org/10.1037/0033-2909.94.3.540
Sage AP, Melse JL. Estimation Theory with Application to Communication and Control. New York: McGraw-Hill 1972.
Schaarfsma W, Tobloom J, Van der Menlen B. Discussing truth or falsity by computing a q-value. Statistical Data Analysis and Inference, Ed. Y. Dodge, pp. 85-100. Amsterdam: North-Holland 1989. DOI: https://doi.org/10.1016/B978-0-444-88029-1.50013-7
Shaffer JP. Modified Sequentially Rejective Multiple Procedures. J Am Statist Assoc 1986; 81(395): 826-831. http://dx.doi.org/10.1080/01621459.1986.10478341 DOI: https://doi.org/10.1080/01621459.1986.10478341
Shaffer JP. Multiple hypothesis testing. Annu Rev Psychol 1995; 46: 561-84. http://dx.doi.org/10.1146/annurev.ps.46.020195.003021 DOI: https://doi.org/10.1146/annurev.ps.46.020195.003021
Toothaker LE. Multiple Comparisons for Researchers, NewBury Park, CA: Sage 1991.
Wald A. Sequential analysis. New-York: Wiley 1947.
Wald A. Foundations of a General Theory of Sequential Decision Functions. Econometrica 1947; 15: 279-313. http://dx.doi.org/10.2307/1905331 DOI: https://doi.org/10.2307/1905331
Westfall PH, Johnson WO, Utts JM. A Bayesian Perspective on the Bonferroni Adjustment. Biometrica 1997; 84(2): 419-427. http://dx.doi.org/10.1093/biomet/84.2.419 DOI: https://doi.org/10.1093/biomet/84.2.419
Westfall PH, Young SS. Resampling-based Multiple Testing, New York: Wiley 1993.
Wolpert RL. Testing simple hypotheses. Data Analysis and Information Systems. Ed. H.H. Bock & W. Polasek, 7, Heidelberg: Springer 1996; pp. 289-297. DOI: https://doi.org/10.1007/978-3-642-80098-6_24
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2014 K.J. Kachiashvili
This work is licensed under a Creative Commons Attribution 4.0 International License.
Policy for Journals/Articles with Open Access
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are permitted and encouraged to post links to their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work
Policy for Journals / Manuscript with Paid Access
Authors who publish with this journal agree to the following terms:
- Publisher retain copyright .
- Authors are permitted and encouraged to post links to their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work .
