Exploring+the+Logic+of+Experimental+Design


 * ** EDU7006-8 ** ||  ||
 * ** Quantitative Research Design ** || ** 1 Exploring the Logic of Experimental Design ** ||
 * Stephen: Thank you for your work on this paper! You have done a nice job on this first assignment. Please see my comments throughout your paper for some additional feedback. FYI: I use the NCU rubric for all my grading. You can find the rubric in our syllabus or the Writing Center. Thanks! **
 * Stephen: Thank you for your work on this paper! You have done a nice job on this first assignment. Please see my comments throughout your paper for some additional feedback. FYI: I use the NCU rubric for all my grading. You can find the rubric in our syllabus or the Writing Center. Thanks! **

=Exploring the Logic of Experimental Design =

Jackson Chapter Exercises
**#2.** In the health magazine depression study there are two confounds that might have impacted the study and the observed findings. Due to the length of the study, history can be an intervening factor since myriad events outside of the study treatment could effect* each given subject. Mortality could also be a huge confounder in this study. Half of the original subjects showed improvement, and a great proportion of the other half dropped out of the study. If most of the subjects that dropped out were the most depressed, it is impossible to determine whether the program had the effect the researchers claimed. There was also no control group! **#4.** A major motivator in research is to answer questions regarding what causes what to happen; we want to better predict (and control) what happens in our world. When evaluating any claim that an outcome is caused by some input, the root of the analysis should be based in internal validity ( Trochim & Donnelly, 2008 ). “ Internal validity refers to the accuracy of conclusions about cause and effect ” ( Cozby & Bates, 2012, p. 69 ). A naïve but tempting motive to conduct research is to find universal truth. External validity determines how well and the extent to which findings of a study can be generalized beyond the bounds of the study, or to “ other persons in other places and at other times ” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008, p. 34 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Universal truth would have the greatest generalizability and supreme external validity, but the effects of chance, imperfect conceptualizations, and deficient understanding of associations makes determining it extremely unlikely, if not impossible. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**#6.** The biggest similarity between a within-subjects design and a matched-subjects design is that they are both statistically powerful, because of minimizing the variability between participants. In the within-subjects design this is due to the same participants acting as both control and experimental groups. For the matched-subjects design variability is minimized because subjects are matched based on relevant measures important to the study. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">There are several differences between the designs. In the within-subjects design half as many subjects are required because each participates in all conditions, where with the matched-subjects design, participants are randomly assigned to control and experimental groups. This randomization and separation minimizes participants realization of what the experiment is about and changing their response, or demand characteristics, in the matched-subjects design; a possible confound in the within-subjects design. Another potential confound with the within-subjects design is the carryover or testing effect of subjects being measured multiple times; something that does not occur in a matched-subjects design.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Answers to Assignment Questions
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**What is the purpose of conducting an experiment? How does an experimental design accomplish its purpose?** The primary purpose of conducting an experiment is to allow the “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">researcher to establish a cause-and-effect relationship <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012, p. 19 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) by manipulating a variable, controlling the environment, and observing the results. By controlling the environment, experiments allow the researcher to see what happens depending on the state of the independent variable. The purpose of the experimental design is to ensure that the environment, and the control and experimental groups are as similar as possible, to determine if when X happens then Y happens; when X does not happen, neither does Y ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**What are the advantages and disadvantages of an experimental design in an educational study?** A true experiment has strong internal validity. Strong internal validity allows the researcher to determine cause-and-effect. All that is required for strong internal validity is control over the environment of the study, statistically equivalent control and experimental groups, and a large enough sample. Unfortunately, the stronger the internal validity, the more it limits the ability of the researcher to generalize the results outside of the context in which it is measured. A true experiment may have limited or weak external validity for this reason. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Experiments are about control ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">); controlling variables through direct manipulation ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Cozby & Bates, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Cause-and-effect is easier to ascertain since extraneous variables in the experiment are kept constant ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Creswell, 2009 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). This allows the researcher to determine whether a treatment is the cause of an outcome and not some other factor. There are times or situations, however, where an experiment is not advised or ethical. Experiments are not conducted in real-world settings because the real-world is messy, and controlling for extraneous variables would be too difficult. Experiments have limited external validity because they measure conditions in an artificial environment that may not be generalizable to the real-world. Under some circumstances conducting an experiment is unethical. If the researcher is testing a drug that can ease or eliminate the pain and suffering of subjects, or cure a malady, but only administers it to the experimental group, while withholding it from the control group it is unjust, and therefore unethical. <span style="color: #ff0000; font-family: 'Times New Roman',Times,serif; font-size: 120%;">What about random selection and assignment, as well as control groups? Would those be easy or difficult to setup for an educational study? <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**What is more important in an experimental study, designing the study in order to make strong internal validity claims or strong external validity claims? Why?** Valid research consists of the interplay between internal validity and external validity. Internal and external validity exist on a continuum and have a negative correlation. A study does not either have or not have internal or external validity; these constructs are not binary. Further, as the internal validity of a study increases, making it more determinant, the external validity tends to decrease making it less extendable. Conversely, the higher the generalizability the less likely a study will engender a causal relationship. If a researcher is choosing to do an experiment, he or she is also choosing to emphasize internal validity because that is the nature of an experiment; to determine the effects of an independent variable on dependent variables. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**In an experiment, what is a control? What is the purpose of a control group? Of single or multiple comparison groups?** In an experiment it is essential to measure the outcome of what happens when treatment is given, while also measuring the outcome of what happens when treatment is not given. By comparing the results of the outcomes, a determination is made regarding whether the treatment produced the desirable outcome or not. The group that does not receive the treatment, or the situation of not receiving the treatment, is considered the control. The control group in some experiments receives an alternate treatment. In either case, the control acts as the baseline from which differences in outcome are measured. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">A single comparison uses a control and experimental group in order to determine if a treatment produces a different outcome than does non-treatment. A multiple comparison group allows multiple treatments to be compared with non-treatment and with each other. A well -known multi-group experiment is the control-placebo-experimental design. By using the placebo group, participation expectation effects are controlled, while a true measure of the treatment effect can be gained between the placebo and experimental groups (<span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**What are confounds? Give an example of a design that has three confounds. Describe three ways to alter the design to address these confounds and explain the advantages and disadvantages of each.** Confounding occurs when findings are ambiguous regarding which variable caused an effect to happen. Confounding variables are those that may not have been measured or figured in the design of the experiment. A confound is a situation that was not controlled in the conducting of the experiment. Confounds have the effect of lowering the ability of the researcher to determine cause-and-effect due to the extraneous variables that were not controlled for in the experiments design. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">A research design that has at least three confounds is the one-shot experimental case study ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Leedy & Ormrod, 2010 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). In this design the treatment is introduced and the measurement is taken to determine the effects of the treatment. Numerous extraneous variables could have influenced the subjects’ measure as well as the independent variable, including; (a) history effect, (b) maturation effect, (c) diffusion of treatment, (d) experimenter effect, or (e) subject effect. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">By altering the experiment into a two-group, posttest-only, randomized experiment it is possible to eliminate some possible confounds from the earlier design. In the earlier design it was impossible to know if the treatment was responsible for the outcome measured. In this design it is possible to determine that there is a difference because of application of the treatment; randomization allows the researcher to assume probabilistic equivalency between the groups. This design enhances internal validity because it is stronger than any single group design, and is proof against multiple-group threats as well except mortality. This design does not protect again social threats to internal validity ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Another possibility for modifying the design is with a within-subjects design. By utilizing the same subjects to determine both the control and experiment, the confounds associated with possible non-equivalency of the groups are eliminated and the number of required subjects is halved. Using this design, however, increases the possibility of demand characteristics and testing effect. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Another design consideration is a matched-subjects design. The matched subjects design requires that participants in the control and experimental groups be matched on variables that are important to the results of the study. By matching participants in this way, groups can be made not only probabilistically equivalent, but specifically equivalent in relevant areas, increasing the power of any statistical analyses performed. Having two groups minimizes demand characteristics, testing effect, or non-equivalent group confounds. To conduct this kind of study, however, requires many more subjects and may be more difficult to match relevant subjects, making it more susceptible to mortality confounds as well. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**What does “cause” mean and why is it an important concept in research? How are correlation and causation related?** In research, cause identifies that a certain event, treatment, or variable is the exclusive reason for a specific outcome. Determining the cause of a behavior or what brings about a specific outcome is one of the primary reasons for conducting research ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Crosby & Bates, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Causation in research can only be determined when three factors are present, (a) the cause always precedes the effect, (b) the effect only occurs when preceded by the cause, and does not occur when the cause is not present, and (c) there can be no other explanation for the effect other than the cause ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Crosby & Bates, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">The relationship between correlation and causation is unidirectional. If X causes Y, X is also perfectly correlated with Y. When X happens, so does Y. When X does not happen, neither does Y. The inverse of this relationship is not necessarily true. Just because there is a correlation between two variables does not mean that one causes the other. While there can be no causation without correlation, correlation does not imply causation.* <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Between-subjects design.** A between-subjects design involves using two randomly selected groups of subjects from the population; in this case, human beings who can smile. Half of the subjects are randomly selected for the control group and will use an evaluation of mood instrument at the culmination of the experiment, but would otherwise receive no other instructions regarding the relevant purpose of the study. The other half would be assigned to the experimental group and instructed to smile throughout the experiment. The same mood evaluation instrument would be administered at the end of the experiment. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Due to the random assignment of subjects it is assumed that the two groups are probabilistically equivalent, although there is the understanding that some degree of error is associated with that assumption. The two groups allows for comparison of results, so that the researcher can determine if there is a positive effect on affect between the experimental group and the control group. Two groups with no pretest minimizes the likelihood that the confounds of testing effects, or demand characteristics will effect the results. The primary disadvantages of the between-subjects design is the need to have sufficient participants to fully populate two (or more) groups, making it more time consuming. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Within-subjects design.** A within-subjects design involves using a single group of subjects selected from the population. In this design all subjects would be instructed that for the first half of the experiment they are to either smile or not, and will be selected for which to do first by random assignment, and then they would take the mood assessment. After the first assessment subjects would receive the opposite instructions as before the first assessment, and at the end of the experiment would again take the mood assessment. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Because the same subjects are measured in the two conditions the possibility of differences between groups is completely eliminated; making statistical analyses much more powerful. The within-subjects design also maximizes the number of usable subjects in the study since there is no need for a control group, minimizing the time for conducting the experiment. The primary disadvantage of this design is that the assessment must be administered twice, so it is more likely that the result could be confounded due to testing effects, or demand characteristics. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Matched-subjects design.** A matched-subjects design involves using two randomly selected groups of subjects from the population, but the participants are also matched on control variables that are deemed relevant to the subject; in this case, subjects may be matched based on gender, cultural-identity, and age range. Other than matching subjects in these relevant variables, subjects will be assigned to groups and the experiment conducted as in the between-subjects design. The matched-subjects design has all of the advantages of the between-subjects design, while further strengthening that design because of the matching. The matching allows for a stronger assumption of equity between the control and experimental groups. The primary disadvantage of this design is that because subjects must be matched in each group according to relevant variables, more subjects are required and some volunteers may not be able to <span style="color: #ff0000; font-family: 'Times New Roman',Times,serif; font-size: 120%;">participate <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">, and the loss of one subject in either group eliminates the matched subject as well, so this design is more susceptible to the mortality confound.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">You are a researcher interested in addressing the question: does smiling cause mood to rise (i.e., become more positive)? Sketch between-participants, within-participants, and matched-participants designs that address this question and discuss the advantages and disadvantages of each to yielding data that help you answer the question. Describe and discuss each design in 4-5 sentences. **

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[TT1] <span style="color: #ff0000; font-family: 'Times New Roman',Times,serif; font-size: 120%;">Check grammar. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;"> [TT2] Right!


 * = References ||
 * * Cozby, P. C., & Bates, S. C. (2012). //Methods in behavioral research// (11th ed.). Boston, MA: McGraw Hill Higher Education.
 * Creswell, J. W. (2009). //Research design: Qualitative, quantitative, and mixed methods approaches// (3rd ed.). Thousand Oaks, CA: SAGE.
 * Jackson, S. L. (2012). //Research methods and statistics: A critical thinking approach// (4th ed.). Belmont, CA: Wadsworth Cengage Learning.
 * Leedy, P. D., & Ormrod, J. E. (2010). //Practical research: Planning and design//. Upper Saddle River, NJ: Merrill.
 * Trochim, W. M. K., & Donnelly, J. P. (2008). //The research methods knowledge base// (3rd ed.). Mason, OH: Cengage Learning. ||