This article explicates some conceptual and methodological problems involved in the tendency of overreliance on statistical testing logic, logic of disproof. Although statistical hypothesis testing is only a subset of the whole process of empirical testing, a number of researchers tend to misconceive that statistical testing logic is universally applicable to the whole process of empirical theory testing. The authors show that this overreliance on logic of disproof leads to many problems: (1) conceptual problems such as confusion between the research hypothesis and the statistical hypothesis, and oversight of the necessity of logic of proof for accepting the “substantive” null hypothesis containing the researcher’s assertion per se, and (2) methodological problems such as a non-rigorous way of conducting empirical research (i.e., the breakdown of the boundary and the proper sequence between the deductive route and the inductive route in empirical theory testing), an incorrect interpretation of test results associated with the testing of the “substantive” null hypothesis, and confusion between logic of disproof (for testing the null hypothesis) and falsificationism (for testing theory).
To remove such conceptual and methodological points of confusion caused by overreliance on logic of disproof, the authors have proposed a seven-step model of empirical theory testing: Step 1 (Theory) → Step 2 (Setting up the research hypothesis: RHEF [i.e., the research hypothesis in existential form]/RHNF [i.e., the research hypothesis in non-existential form]) → Step 3 (Setting up the statistical hypothesis: Translating RHEF or RHNF into H0 and H1) → Step 4 (Testing the statistical hypothesis by logic of disproof/proof) → Step 5 (Testing the research hypothesis: RHEF or RHNF is supported/not supported) → Step 6 (Testing theory empirically: The theory is empirically supported/not supported) → Step 7 (Interpretation of this empirical test result from the researcher’s scientific standpoint, such as falsificationism or logical empiricism) (for better understanding, see Figure 1 in the conclusion section). If the research hypothesis is inductively derived from observations rather than theory, then Step 1, Step 6, and Step 7 become irrelevant.
With regard to the application of the seven step model, the authors have noted the following four points. Firstly, researchers should not intend to apply this seven step model to cases where statistical hypothesis testing is not valid (e.g., “H0: The Sun revolves around the Earth” and “H1: The Earth revolves around the Sun” [Johnson, 1999]; “H0: The defendant is innocent” and “Ha: The defendant is guilty” [Anderson et al., 1999]). Secondly, a theory (T), the research hypothesis (RH) derived from the theory, and the statistical hypothesis (H0 and H1) translated from the research hypothesis should be distinguished from each other clearly. Thirdly, even in cases of empirically testable theories or models, two competing theories or models should not be regarded as being the direct objects of statistical testing such as “H0: model A vs. H1: model B” (e.g., “H0: the hypothesized or simpler model” and “H1: the more general model” [e.g., Arora 1982]; “H0: the random-walk model” and “H1: the Markov switching model” (and vice versa) [e.g., Cheung and Erlandsson, 2005]). To test two competing theories or models, they usually have to use two separate seven-step or six-step models. Finally, the former Steps 1-3 correspond to the deductive route ruled by theory and the later Steps 4-6 stand for the inductive route ruled by empirical sample data. Thus, the boundary and the proper sequence between the deductive route and the inductive route must be observed in that order. If researchers observe these four points faithfully, unnecessary confusion can be avoided.
The authors believe that the seven step model of empirical theory testing can play a role in accelerating the sound development of scientific knowledge with regard to the empirical testing of theories and hypotheses.