Power+Analysis+and+Design+Sensitivity


 * ** EDU7006-8 ** ||  ||
 * ** Quantitative Research Design ** || ** 7 Samples, Power Analysis, and Design Sensitivity ** ||
 * Steve: Thank you for your work on this paper! Your responses to the essay questions and power analysis are thorough, thoughtful, and well supported. Based on what I currently know about your research, you are right on track with your power analysis, and the statistical tests you are selecting seem appropriate for your proposed research. **
 * While you will work with your dissertation chair and committee on your research questions and hypotheses, I’ve given you some feedback on them in this paper. I think you have a really interesting study topic, but I recommend clarifying and streamlining the language and variables in the research question and hypotheses. Please see my comments throughout your document for details and let me know if you have any questions. Thanks! **
 * While you will work with your dissertation chair and committee on your research questions and hypotheses, I’ve given you some feedback on them in this paper. I think you have a really interesting study topic, but I recommend clarifying and streamlining the language and variables in the research question and hypotheses. Please see my comments throughout your document for details and let me know if you have any questions. Thanks! **

=Samples, Power Analysis, and Design Sensitivity =

Part I
**1.** Validity focuses on the truth or accuracy of findings ( Cosby & Bates, 2012 ), are foundational principles used to determine the quality of research ( Trochim & Donnelly, 2008 ), and are operative when working with questions dealing with cause-and-effect ( Cosby & Bates, 2012 ). “ Internal validity is synonymous with control ” ( Salkind, 2009, p. 231 ). If a study has no confounds, the results of the outcome variable are exclusively attributable to the manipulation of the independent variable ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012; Yu & Ohlund, 2010 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), and the structural integrity of the research design allows no other plausible explanations for the results ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Oncu & Kakir, 2011 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), the study is said to have good internal validity. External validity, on the other hand, “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">is synonymous with generalizability <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Salkind, 2009, p. 231 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), and consists in the degree that the findings of a study can be generalized ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) to other populations, places ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Crosby & Bates, 2012; Oncu & Kakir, 2011; Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">treatment variables, and measurement instruments <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Dimitrov & Rumrill, 2003, p. 159 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">A study does not either have or not have internal or external validity; these constructs are not binary. Further, as the internal validity of a study increases, making it more determinant, the external validity tends to decrease making it less extendable. Conversely, the higher the generalizability the less likely a study will engender a causal relationship. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Research questions for which external validity would be the primary concern are those focused on being able to generalize the findings of the study to a larger population. For example, Ferguson and DeFelice (2010) conducted a study in which graduate students participating in a specific class that was offered in two formats based on length of course were surveyed to determine if there were differences in the students’ satisfaction with the communication in the class, whether students’ were more likely to take another online class, whether students' perceived learning differed, or whether students’ academic performance differed because of the two formats, when all else was kept reasonably constant. While it was instructive that there were differences between the students who attended the shorter summer classes as opposed to students who attended the regular length classes, the authors were more interested in generalizing the findings so that they could make inferences regarding ways to improve online classes, and suggested that the study “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">may have implications for an institution’s policies for determining the length of online courses <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 81 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Research questions for which internal validity is a primary concern are those that are specifically looking for cause-and-effect. For example, Chyung and Vachon (2005) investigated the research question, “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">what factors of an e-learning system do e-learners express as satisfying factors (i.e., motivation factors) and what factors do they express as dissatisfying factors (i.e., hygiene factors) <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 103 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">)? In this study, the authors specifically wanted to determine what causes satisfaction and dissatisfaction in students, and whether the study determined that satisfaction is not the inverse of dissatisfaction and vice verse. Joo, Park, Park, Kim, and Kim (2009) demonstrated their interest in internal validity with two research questions, “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">does learners’ satisfaction predict academic achievement in the corporate cyber education <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 3931 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) and “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">does learners’ satisfaction predict learning transfer in the corporate cyber education <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">(p. 3931 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Another way of phrasing internal validity is whether the findings of a study have predictive validity. It is this ability to predict that Joo et al. (2009) were hoping to find as a result of their study. They found that learner’s satisfaction is both predictive of academic achievement and transfer of learning. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">In order to make a strong claim regarding the applicability of a finding to a target population a study must have strong external validity. The most important strategy for researchers who desire strong external validity is to ensure that subjects are randomly selected ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012; Oncu & Kakir, 2011; Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Once a sample has been randomly selected it is important to minimize dropout ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), ensure that the researchers have been properly instructed on how to interact with subjects ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Salkind, 2009 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), and “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">be careful in interpreting the results,. . . because any over-interpretation would also be a threat to external validity <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Oncu & Kakir, 2011, p. 1105 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Another strategy for strengthening external validity is through replication of results in various settings, with different samples, and at different times ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Cozby & Bates, 2012; Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**2.** Random selection is a manner of choosing a sample from a population for a study so that each member of the population has an equal chance of being in the sample ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). A sample that is chosen randomly from an appropriate population and is of sufficient size is said to be representative of the population ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). When a sample is representative it enhances the external validity of a study. “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">When the sample is representative of the population, we can be fairly confident that the results we find based on the sample also hold for the population <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012, p. 100 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Random selection may also be used to identify different start times in a multiple baseline design ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Koehler & Levin, 1998 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Without random selection of subjects, determining the population study results can be generalized to is much more difficult, or impossible. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Once a sample is selected, random assignment is the manner of relegating the subjects in the sample to appropriate groups by chance, such that each subject has an equal chance of being in any specific group ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). True experiments are defined by random assignment of subjects to groups because “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">randomization ensures that the individual characteristic composition of the two groups will be virtually identical in every way <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Crosby & Bates, 2012, p. 82 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Statistically equivalent groups enhance internal validity within a study ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Without random assignment of subjects to control and experimental groups, outcomes must be interpreted with caution because “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">we can never conclude that the independent variable definitely caused any of the observed changes in the dependent variable <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012, p. 350 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) because of potential selection bias ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Oncu & Kakir, 2011 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Random selection and random assignment are not mutually exclusive. A study can have neither random selection nor random assignment, either random selection or random assignment, or both random selection and random assignment. Random selection increases the external validity of a study provided that the selection is done from the appropriate population that the researcher is hoping to generalize to. Random assignment, on the other hand, increases the internal validity of a study by providing probabilistic equivalency. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**3.** One aspect that is crucial to any experimental design is sample size. A sample with too few participants runs the risk of (a) poorly reflecting the underlying population, (b) not finding a significant result when the null hypothesis is false, and (c) nonreplicable results. A sample with too many participants, however, can be much more costly and slower to conduct ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Acheson, 2010 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). As the number of subjects from a single population increase, variability around the mean decreases; while as the number of subjects decrease, variability around the mean is likely to increase ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Statistical power analysis rests upon four variables; the sample size (//N//), the significance criterion (α), the population effect size (//ES//), and power (β). Each of these variables is directly related to the other three ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Cohen, 1992; Faul, Erdfelder, Land, & Buchneer, 2007 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">In a simple example, in order to reject the null hypothesis, a comparison is made between the mean of the control group, and the mean of the experimental group. If the mean of the experimental group when converted to a z-score, surpasses a critical value, the null hypothesis is rejected. The z-score is calculated using the following equation: <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;"> [Formula in Word document]* <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">To have a better possibility of rejecting the null hypothesis, z must be as large as possible. The outcome variable directly manipulates the numerator. To increase the probability that the outcome variable is significant I must manipulate the denominator; the smaller the denominator, the larger the value of z. The formula for the denominator is: <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;"> [Formula in Word document] <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">By increasing the size of //N//, the standard error of the mean gets smaller, increasing the size of z. This is the <span style="color: #ff0000; font-family: 'Times New Roman',Times,serif; font-size: 120%;">effect <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">* that I am looking for. Therefore, as the size of //N// increases, the value of z increases; increasing the likelihood of a statistically significant result. Conversely, as the size of //N// decreases, the value of z decreases; decreasing the likelihood of a statistically significant result. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**4.** Probability sampling utilizes random selection. For random selection, a procedure or process is implemented to ensure that each unit of the population has a fair and equal chance of being selected ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">Because the determination of who will end up in the sample is determined by nonsystematic and random rules, the chance that the sample will truly represent the population is great <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Salkind, 2009, p. 90 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Probability sampling has the following advantages; (a) allows estimating confidence intervals ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), (b) <span style="color: #ff0000; font-family: 'Times New Roman',Times,serif; font-size: 120%;">it is the <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">easiest way to get a representative sample ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Salkind, 2009; Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), (c) is “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">more accurate and rigorous than non-probabilistic samples <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008, p. 48 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), and (d) allows the computing of sampling variances ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">McCready, 2006 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="color: #ff0000; font-family: 'Times New Roman',Times,serif; font-size: 120%; text-decoration: line-through;">The use of <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Probability sampling, however, can also be “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">time consuming and tedious <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Salkind, 2009, p. 97 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">There are two types of non-probability sampling. The first type of non-probability sampling is purposive, in which subjects with specific criteria are sought for inclusion in a study ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012; Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). The second type of non-probability sampling may be called accidental ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), haphazard ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), person-on-the-street, or convenience sampling ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012; Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Convenience sampling is often used in educational research, where “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">the characteristics of a specific group of individuals match the attributes of the phenomenon being studied <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Rhode, 2009, p. 4; See also Ali & Ahmad, 2011; Boling, Hough, Krinsky, Saleem, & Stevens, 2011; Tallent-Runnels, Thomas, Lan, Cooper, Ahern, Shaw, & Liu, 2006 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Non-probability sampling is often used because it is (a) convenient, (b) inexpensive, (c) can be used “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">when it is not feasible, practical, or theoretically sensible to use random sampling <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012, p. 101 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), and (d) in situations where withholding of treatment may be unethical. The problem with non-probability sampling is that the ability to generalize is weakened ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Salkind, 2009; Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Random selection and random assignment are not mutually exclusive; it is possible for a study to have neither, one, or both randomizations.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Part II
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**1a.**The sample size must have at least 620 subjects to meet the given factors; 310 subjects for each group when //ES// = .2, α = .05, β = .2 on a one-tailed //t//-test with two independent groups of equal size (see Figure 1a below). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**1b.** For a sample size of 310 (155 in each group) using the compromise function, the resulting alpha and beta is α = .09, β = .35, p = 0.05 (see Figure 1b below). “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">Determining a meaningful effect size is a judgment call. . . . Selecting a desirable power is achieved by balancing the need to detect an outcome with the difficulty in obtaining large sample sizes <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Houser, 2007, pp. 2-3 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Due to the law of large numbers the benefit of a larger sample diminishes, while the cost and time involved increases ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">McCready, 2006 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Using a t-test for independent groups we compare the means of the samples to determine how likely the groups are from the same population, or whether they are different enough to conclude they are from differing populations. In this case, “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">if an outcome is detected, then power is not an issue, the sample was obviously large enough to detect it <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Houser, 2007, p. 1 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). The diminished statistical power because of the sample size does not mean that an effect cannot be found, it just means that the chance of finding a significant result is reduced. While a significance criterion of .09 and power of .65 is traditionally unacceptable ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Salkind, 2009 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), if the importance of the information to be attained is high, or preliminary, it may be worthwhile doing the study anyway. //Figure 1a// || //Figure 1b.// || <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**2a.** The sample size must have at least 969 subjects for a one-way analysis of variance (ANOVA) with three groups and //ES// = .1, α = .05, and β = .2 (see Figure 2a below). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**2b.** For a sample of 486, which is approximately half the size of 2a the resulting alpha and beta is α = .10, β = .39, //p// = 0.05 (see Figure 2b below). An analysis of variance (ANOVA) allows comparing the means of three or more groups to determine if “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">at least one group mean differs from the others by more than would be expected based on chance <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012, p. 287 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). It is traditional to utilize a beta/alpha ratio of 4:1 ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Cohen, 1992 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), but some authors argue that “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">the benefit of balanced Type I and Type II error risks often offsets the costs of violating significance level conventions <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">(Faul et al., 2007, p. 177 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). I chose a traditional ratio for this assignment. If an appropriate sample size is not attainable, a convenience sample size for this study still has a 90% chance of not committing a Type I error, and a 61% chance of not committing a Type II error. If the study is being conducted regarding a new or developing theory, and there are few previous reports or pilot studies, a study with a convenience sample may still be worthwhile conducting even with a diminished capacity to find a significant result. //Figure 2a.// || //Figure 2b.// || <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**3.** The following research question and corresponding hypotheses will be investigated relative to two experimental designs: <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Q1.** How does satisfaction of adult learners, as measured by the Learner satisfaction subsection of the LSTQ ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Gunawardena, Linder-VanBerschot, LaPoint, & Rao, 2010 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), vary, if at all, in an online live virtual classroom (LVC) environment between when learners continuously see the instructor through visual technology (webcam) versus when learners cannot? <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**H0.** Measures of learner satisfaction are statistically equivalent when the visual (webcam) element is used continuously as opposed to when it is not in online LVC instruction of adult technical professional development courses ( μLVC = μwc ). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Ha**. Measures of learner satisfaction are statistically different when the visual (webcam) element is used continuously as opposed to when it is not in online LVC instruction of adult technical professional development courses ( μLVC ≠ μwc ). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**3a.** In the first design, data will be collected from the online classes of ten (minimum) instructors who teach various technologies. Each instructor will teach two instances of two different online classes of five consecutive days or less duration. These classes will be paired, such that one instance of the class will be taught according to that instructor’s normal delivery (the control) and one instance will be taught in the normal style with the addition of a webcam transmitting the instructor’s image to the class during interactive periods of the class (the experiment). Whether the control class or experimental class will be taught first will be randomized. Each student will be encouraged at the end of the class to fill out the Learner Satisfaction and Transfer-of-learning Questionnaire (LSTQ) developed and validated by Gunawardena et al. (2010), in addition to the regular course evaluation. Incomplete or surveys that have the same value for all sixteen questions will be discarded. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">The independent variable in this first design is the visual element and has two attributes; full use of the webcam (1) and minimal use of the webcam (0). The dependent variable in this first design is learner satisfaction, which is a construct that will be derived from the Learner satisfaction subscale of the LSTQ; consisting of five 5-point Likert scale questions. The learner satisfaction construct is an ordinal variable varying from strongly agree = 5 to strongly disagree = 1. This design will use two groups of equal size, one representing each attribute of the independent variable. It is not known whether the distribution of scores from the Learner satisfaction subscale of the LSTQ will be normal, so the Wilcoxon rank-sum test will be used to determine whether one sample has significantly larger values than the other. According to the hypotheses, it is not known whether use of the webcam will increase or decrease scores, so a two-tailed test will be used. It is expected that the scores from the subscale will be leptokurtic, with negative skew; therefore the parent distribution of Laplace will be selected. As the standard deviation of the data is unknown, the effect size will be set to d = 0.3; slightly larger than a small effect size, but not a medium effect size. Traditional values of α = 0.05 and β = 0.2 are generally acceptable for most research in the social sciences (Salkind, 2009) and have been selected in this case. Based on the preceding factors, and using an //a priori// analysis, a minimum sample size of //N// = 234 is required to have an optimal chance of rejecting the null hypothesis, if it is false (see Figure 3a below). //Figure 3a.//
 * [[image:WattsSEDU7006-8-7.II.1a.png width="429" height="510"]]
 * [[image:WattsSEDU7006-8-7.II.2a.png width="429" height="512"]]

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**3b.** Utilizing the same collection procedure, in design two I add a second independent variable, technology, which can have five different values, making a 2 x 5 factorial design. In this design I can determine if there is a significant main effect regarding whether the webcam has a significant effect on learner satisfaction across all groups, or whether learner satisfaction is significantly affected by the technology of the class. The factorial design also allows for determining whether interaction effects exist between using the webcam and different technologies. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">The independent variables in this design are the visual element that has two attributes, and the type of technology, which has five attributes. According to Heine (n.d.) this is an ANOVA: Fixed effects, special, main effects and interactions statistical test. I have chosen an effect size to match the previous discussion; one slightly larger than a small effect size, but not a medium effect size, so d = 0.15. I have also chosen traditionally acceptable values for alpha and beta, where α = 0.05 and β = 0.2. Since this is a two factor test, the degrees of freedom are the number of possible values for the first factor less one, multiplied by the number of possible values for the second factor less one. In this case //df// = ( 2 – 1 ) ( 5 – 1 ) = 1*4 = 4. The numbers of groups in a factorial design are determined by multiplying the number of possible values of all factors together; in this case, 2 * 5 = 10. Based on the preceding factors, and using an //a priori// analysis, a minimum sample size of //N// = 536 is required to have an optimal chance of finding significant main and interactive effects if they exist (see Figure 3b below). //Figure 3b.//

 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[TT1] It can sometimes be difficult in MS Word to write and format formulas. This doesn’t have to do with this assignment, but it occurred to me that you might be interested in LaTeX ([|http://www.latex-project.org]) if you like writing, editing, and so on. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[TT2] If you mean “result” then it should be “effect.” We should use “affect” as a verb when we mean “to change”—For example: the study treatment affects the test scores. But we should use “effect” when we mean the “result of something”— For example: improved test scores are the effect of the study treatment. In this sentence, we clearly should use effect. Check out [] for a great tool! Or, try this for more information: [|http://public.wsu.edu/~brians/errors/affect.html] <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Or, you can look at the Oxford Dictionary (I just looked it up in the concise 10th edition). Or, for general grammar and punctuation issues, I always use Swan’s Practical English Usage. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[TT3] Some words are missing in this series. See my edit for an example and check the remaining series items. <span style="color: #ff0000; font-family: 'Times New Roman',Times,serif; font-size: 120%;">[In this case I'd have to disagree, and I'd have to disagree with the use of the colon, since a semi-colon is used to separate elements of a series that already contains commas (APA, 6th ed. para. 4.04 Semicolon).] <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[TT4] I am definitely not the last say on these research questions or hypotheses. Your dissertation chair and committee members are the faculty who will, of course, work with you on this in the near future. But, I think you might want to streamline the language in these questions to make it clear that your intention is to compare groups in terms of learner satisfaction scores (as opposed to relating variables). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[TT5] I see below that you indicate “minimal use of the webcam.” I don’t know much about your study at this point, but you need to clarify and consider quantifying what that means. I definitely recommend discussing it with your dissertation chair.


 * = References ||
 * * Acheson, A. (2010). Sample size. In N. J. Salkind (Ed.), //Encyclopedia of Research Design// (pp. 1300-1302). doi:10.4135/9781412961288
 * Ali, A., & Ahmad, I. (2011). Key factors for determining students’ satisfaction in distance learning courses: A study of Allama Iqbal Open University. //Contemporary Educational Technology, 2//(2), 118-134. Retrieved from http://cedtech.net/
 * Boling, E. C., Hough, M., Krinsky, H., Saleem, H., & Stevens, M. (2011). Cutting the distance in distance education: Perspectives on what promotes positive, online learning experiences. //Internet and Higher Education//. doi:10.1016/j.iheduc.2011.11.006
 * Chyung, S. Y., & Vachon, M. (2005). An investigation of the profiles of satisfying and dissatisfying factors in e-learning. //Performance Improvement Quarterly, 59//(3), 227-245. doi:10.1177/0741713609331546
 * Cohen, J. (1992). A power primer. //Psychological Bulletin, 112//(1), 155-159. doi:10.1037/0033-2909.112.1.155
 * Cozby, P. C., & Bates, S. C. (2012). //Methods in behavioral research// (11th ed.). Boston, MA: McGraw Hill Higher Education.
 * Dimitrov, D. M., & Rumrill, P. D. Jr. (2003). Pretest-posttest designs and measurement of change. //Work, 20//(2), 159-165. Retrieved from http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCoQFjAA&url=http%3A%2F%2Fwww.phys.lsu.edu%2Ffaculty%2Fbrowne%2FMNS_Seminar%2FJournalArticles%2FPretest-posttest_design.pdf&ei=Z0V1T4rLJ8Wi2gXum-mUDQ&usg=AFQjCNF6-c4j-7JJKm9ohkXr6jXJ1T-7pA&sig2=BW7hMizEWzQBfgUIIQ30rA
 * Faul, F., Erdfelder, E., Lang, A.-G., & Buchneer, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, biomedical sciences. //Behavior Research Methods, 39//(2), 175-191. Retrieved from http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/download-and-register/Dokumente/GPower3-BRM-Paper.pdf
 * Ferguson, J. M., & DeFelice, A. E. (2010). Length of online course and student satisfaction, perceived learning, and academic performance. //International Review of Research in Open and Distance Learning, 11//(2), 73-84. Retrieved from http://www.irrodl.org/index.php/irrodl
 * Gunawardena, C. N., Linder-VanBerschot, J. A., LaPointe, D. K., & Rao, L. (2010). Predictors of learner satisfaction and transfer of learning in a corporate online education program. //The American Journal of Distance Education, 24//(1), 207-226. doi:10.1080/08923647.2010.522919
 * Heine, H. (n.d.). //ANOVA: Fixed effects, special, main effects and interaction//. Unpublished manuscript, Department of Experimental Psychology, University of Dusseldorf, Dusseldorf, Germany. Retrieved from http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/user-guide-by-distribution/f/anova_fixed_effects_special
 * Houser, J. (2007). How many are enough? Statistical power analysis and sample size estimation in clinical research. //Journal of Clinical Research Best Practices, 3//(3), 1-5. Retrieved from http://firstclinical.com/journal/2007/0703_Power.pdf
 * Jackson, S. L. (2012). //Research methods and statistics: A critical thinking approach// (4th ed.). Belmont, CA: Wadsworth Cengage Learning.
 * Joo, Y.J., Park, S.H., Park, S.Y., Kim, S.M., Kim, E.K. & Kim, J.Y. (2009). Relationships among Learners’ Satisfaction, Academic Achievement and Learning Transfer in the Corporate Cyber Education. In G. Siemens & C. Fulford (Eds.), // Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2009 // (pp. 3930-3933). Chesapeake, VA: AACE.
 * Koehler, M. J., & Levin J. R. (1998). Regulated randomization: A potentially sharper analytical tool for the multiple-baseline design. //Psychologial Methods, 3//(2), 206-217. doi:10.1037//1082-989X.3.2.206//
 * //McCready, W. C. (2006). Applying sampling procedures. In F. T. L. Leong, & J. T. Austin (Eds.),// The Psychology handbook: A guide for graduate students and research assistants //(2nd ed., pp. 147-161). doi:10.4135/9781412976626.n10//
 * //Oncu, S., & Cakir, H. (2011). Research in online learning environments: Priorities and methodologies.// Computers & Education, 57//, 1098-1108. doi:10.1016/j.compedu.2010.12.009//
 * //Rhode, J. F. (2009). Interaction equivalency in self-paced online learning environments: An exploration of learner preferences.// The International Review of Research in Open and Distance Learning, 10//(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/603/1178//
 * //Salkind, N. J. (2009).// Exploring research //(7th ed.). Upper Saddle River, NJ: Pearson/Prentice Hall.//
 * //Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M., & Liu, X. (2006). Teaching courses online: A review of the research.// Review of Educational Research, 76//(1), 93-135. doi:10.3102/00346543076001093//
 * //Trochim, W. M. K., & Donnelly, J. P. (2008).// The research methods knowledge base //(3rd ed.). Mason, OH: Cengage Learning.//
 * //Yu, C.-H., Ohlund, B. (2010).// Threats to validity of research design//. Retrieved from http://www.creative-wisdom.com/teaching/WBI/threat.shtml ||