Quasi-Experimental+Designs


 * ** EDU7006-8 ** ||  ||
 * ** Quantitative Research Design ** || ** 5 Quasi-Experimental Designs ** ||
 * Steve: Thanks for your work on this assignment! This was a very long assignment: thank you for your comprehensive answers that are supported with our text and additional sources. I made some comments/edits in your document for a few writing issues, but please note that I did not line edit the document. For example, there are several points where I noticed an issue with personification, but I only marked one. Please let me know if you have any questions. Thanks! **
 * Steve: Thanks for your work on this assignment! This was a very long assignment: thank you for your comprehensive answers that are supported with our text and additional sources. I made some comments/edits in your document for a few writing issues, but please note that I did not line edit the document. For example, there are several points where I noticed an issue with personification, but I only marked one. Please let me know if you have any questions. Thanks! **

=Quasi-experimental Design=

Jackson (2012) Chapter Exercises
**#2.** The psychology professor has two sections of students that were not randomly assigned. The treatment will be weekly quizzes. The desired outcome is improved student learning. I will assume that student learning is measured by the total score on all exams not including the weekly quizzes. This design can be diagrammed with the following notation: Based on these assumptions I would recommend a nonequivalent control group posttest only design for this  quasi  experiment. If student learning is measured by scores on major exams through the course, however, the notation changes, as would the recommendation. The following notation represents an alternative possibility for this design: Based on these assumptions, I would recommend a nonequivalent control group time-series design for this experiment. Overall, this later design is a stronger design because of the multiple observations and points of comparison between the two sections. **#4.** Usually the nonequivalent control group design uses intact groups. Since these groups are not randomly selected, the major confound for this design is selection bias. Selection bias arises when the control and experimental groups are not comparable before the study and gives an alternative explanation for any differences in the posttest. In this design there are a couple of social interaction threats that are possible. Students in the experimental section may find out that students in the control section are not getting quizzes and react negatively, causing resentful demoralization, or react competitively, causing compensatory rivalry. Either case would tend “ to equalize the outcomes between groups, minimizing the chance of seeing a program effect even if there is one ” ( Trochim & Donnelly, 2008, p. 171 ). A single-group design’s worst confound is that there is no comparison group, therefore, if student scores are high or low, it is impossible to tell if they are that way because of the treatment, or because of some alternative explanation. With only a single group to measure, all of the single group threats to internal validity are possible. In this particular study the most likely would be a history threat, where an event or set of events could affect the outcome more than the treatment. **#6.** Three reasons a researcher might choose a single-case design include, (a) when a single person or condition is of interest, (b) when replication of results is essential, and (c) situations in which error variance needs to be eliminated ( Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Clinical trials often are interested* in the effect of a treatment on a single individual. One study was interested in mitigating or eliminating auditory hallucinations and delusions as a result of schizophrenia, and evaluated the use of “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">an innovative rational-emotive cognitive treatment <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Qumtin, Belanger, & Lamontagne, 2012, p. 114 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). From this treatment there was a noticeable and immediate reduction in depression and anxiety and an increase in the patient’s quality of life that extended through the 12-month follow up. A discussion regarding replication of empirical results resulting from single-case designs in psychology and education was written by Kratochwill and Levin (2010) along with suggestions for improving the credibility of these designs by using randomization. Error variance results from differences between participants in a group. In a single-case design there is no group, hence no error variance; making the determination of outcomes resulting from the independent variable much less complicated. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**#8.** Single case designs can be implemented in a number of ways. In a reversal design the focus is on a single participant, a single behavior, or a single situation, and involves the independent variable being applied and removed one or more times to assess its impact. By evaluating the behavior without the treatment applied and then comparing the behavior after treatment, a determination can be made regarding the treatments effect. Succeeding periods of non-treatment are called the reversal and can demonstrate what happens to the behavior with a return to the baseline.* A reversal design is very similar to a within-subjects group design because each subject experiences both the control and experimental condition. Greenhoot (2003) summarized that the reversal of an effect because of the removal of a treatment provided “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">strong evidence for a causal link between the independent and dependent variable <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 98 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">In a multiple-baseline design multiple participants, behaviors, or situations may be involved. In these single case studies the measures of interest are individualized and descriptive statistics are not used to aggregate them. The multiple-baseline design measures the effect of the application of a treatment at differing times for multiple subjects, or can be used with a single subject by applying the treatment in conjunction with different behaviors or in different scenarios.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">N X O **
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">N O **
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">N X O X O **
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">N O O **

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Part I Assignment Question Answers
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Describe the advantages and disadvantages of quasi-experiments? What is the fundamental weakness of a quasi-experimental design? Why is it a weakness? Does its weakness always matter?** A quasi-experimental design has certain advantages, including; (a) the ability to “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">draw slightly stronger conclusions than. . . with correlational research <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012, p. 342 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), (b) contributing original research to the body of knowledge in a field ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Ellis & Leavy, 2011 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), (c) allowing real-world research in the field as opposed to more controlled conditions ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), (d) may involve an independent variable that is nonmanipulated ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Hoadley, 2004 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), and (e) can be used with intact groups ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">de Anda, 2007 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). The disadvantages of using a quasi-experimental design are that (a) it “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">limits internal validity in a study <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012, p. 342 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), (b) does not establish a causal relationship between variables ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), while (c) probabilistic equivalence cannot be assumed ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Dimitrov & Rumrill, 2003 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). The fundamental weakness of a quasi-experimental design, and what makes it “quasi”, is that it does not involve random assignment of participants to experimental conditions ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Greenhoot, 2003 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Nonrandom assignment weakens internal validity, diminishes the ability to establish cause-and-effect by eliminating the expectation that groups are equivalent. This weakness does not exclude quasi-experiments from a researchers arsenal because of the advantages mentioned above. These advantages make quasi-experiments the most used form of quantitative research ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Ellis & Leavy, 2011 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). There may be times when subjects cannot be randomly assigned to groups, there is only one subject of interest, or the topic of research either exists or it does not; in all of these cases a quasi-experiment is a viable option for conducting the research while an experiment is not ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Greenhoot, 2003 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">).* <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">If you randomly assign participants to groups, can you assume the groups are equivalent at the beginning of the study? At the end? Why or why not? If you cannot assume equivalence at either end, what can you do? Please explain. If participants are randomly assigned to groups from the same population of interest to the study, the groups can be said to be probabilistically equivalent at the beginning of a study ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Random assignment does not ensure groups are exactly the same, but that differences between the groups can be statistically determined since “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">groups can differ only due to chance assignment <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 189 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). If, however, participants are chosen from a pool that does not represent the study population, or from multiple pools with differing characteristics, they will neither be equivalent or probabilistically equivalent. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Assigning subjects to control and experimental groups randomly is the best means of ensuring probabilistic equivalence at the beginning of a study, as long as the subjects are chosen from the appropriate population. Even though the groups will not be exactly the same, the probability that they are different can be calculated and minimized. There are also designs that allow for the use of the same participants in both control and experimental conditions, ensuring equivalence between conditions. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Whether groups are probabilistically equivalent at the end of a study is the purpose of research. In a perfect world, where the treatment has an effect, the experimental group will consist of a different population from the control group at the end of a study, and this difference is the treatment effect. Unfortunately, there are confounds other than the application of the treatment that may exacerbate or minimize differences between the groups, consisting of the threats to internal validity. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">The best way to ensure that the only conditions that change between the groups is by designing the experiment to minimize threats to internal validity. By demonstrating that an independent variable precedes the dependent variables, and that they covary together, while showing that there are no other plausible explanations for the relationship, internal validity is high and can explain causality. With a properly designed study, the first two criterion* are easy to control so it is critical to focus on the third criteria, and minimize alternative explanations. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Explain and give examples of how the particular outcomes of a study can suggest if a particular threat is likely to have been present.** Trochim and Donnelly (2008) identified five possible outcomes for a research study and the potential internal validity threats associated with each outcome. They noted that the crossover pattern has the “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">clearest pattern of evidence for the effectiveness of the program of all five of the hypothetical outcomes <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 215 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) and that there are no plausible threats to internal validity. In this particular pattern the control group does not appear to change from pretest to posttest, but the pretest of the experimental group has a lower score than the control, while the posttest score of the experimental group is higher than that of the control group. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Two patterns identified by Trochim and Donnelly (2008) have the control and experimental groups close in score on the pretest, which I will label as diverging patterns. In the first diverging outcome pattern, the control group maintains a consistent pretest-posttest score, while the treatment groups score increases from pretest to posttest. The most likely internal validity threat presented in this situation is a selection-history threat where an event occurs to which the treatment group reacts and the control group does not. The second diverging outcome pattern reflects improvement in scores for both groups between pretest and posttest, but the treatment group improves more. Because of the improvement in scores by both groups almost all of the multiple-group threats to internal validity are possible, with the exception of selection-regression. Improvements by both groups can indicate selection-maturation, selection-history, selection-testing, and selection-instrumentation as possible threats that affected the scores. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Two patterns identified by Trochim and Donnelly (2008) have the control and experimental groups close in score on the posttest, which I will label as converging patterns. In the first converging outcome pattern the two groups begin far apart on pretest scores, with the experimental group being much higher than the control group, and end with both groups’ posttest scores close together. The second converging outcome pattern is similar except in this case the pretest scores are much lower for the experimental group, and in the posttest the control and experimental scores are close together. In both cases, the control group remains consistent between pretest and posttest scores, while the experimental groups vary greatly and approach the scores of the control group. The most likely threat to internal validity in the converging scenarios is selection-regression. Participants may have been selected because of high or low scores on a measure, and even though the treatment may or may not have had an effect, the scores are closer to the mean. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;"> **a. Non-equivalent control group posttest only**. The nonequivalent control group posttest only design has at least two nonrandomly selected groups. One of the groups is the experimental group and will be administered the new treatment, while the other group is the control group and will have no treatment, or the standard treatment. If the two groups can be shown to be relatively equivalent the results of this design are very similar to a true experiment, and the comparison between the groups on the posttest should reflect the effect of the treatment ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). The problem with this design is that it allows very little within the design to determine if the groups are roughly equivalent in the first place, meaning that “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">you can never be sure the groups are comparable <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008, p. 211 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). This means that the design is especially susceptible to selection bias since the groups are not randomly selected, and the groups may not be equivalent. It is possible using this design that the two groups could have different history or maturation during the duration of the program that could impact the posttest, but none of the other multiple-group threats to internal validity are applicable for this design. If participants of each group are not isolated from each other there is also the possibility of the social interaction threats to internal validity; diffusion or imitation of treatment, compensatory rivalry, or resentful demoralization ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;"> **b. Non-equivalent control group pretest/posttest.** The nonequivalent control group pretest/posttest design has at least two nonrandomly selected groups. One group comprises the experimental group, and receives treatment, while the other group is the control group, receiving no treatment or the standard treatment. Each group receives a measure regarding the dependent variables at the beginning of the study, and then another equivalent measure regarding the dependent variables at the end of the study. The addition of the pretest allows the researcher to compare the groups with respect to the dependent variables to determine whether the groups are roughly equivalent. If the groups are equivalent the internal validity of the posttest comparison is increased, and if they are not the posttest scores can be statistically adjusted based on the pretest scores. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;"> **c. Cross-sectional.** A cross-sectional design studies multiple strata of a population at the same time. For example, a cross-sectional design focused on writing development may sample from 7th, 9th, and 11th grade students. The logic behind a cross-sectional study is to collect as much data regarding the stratum as quickly as possible. This study often does not adequately address selection bias, since the subjects are by definition stratified around some criteria that distinguishes them from the other strata. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;"> **d. Regression-Discontinuity.** The regression-discontinuity design begins by administering a pretest to all participants, and then relegates them to control or experimental groups based on a specific cutoff score. Those with the greatest need for treatment are administered the treatment, while the other group serves as a control. Unlike randomized experiments this design makes no attempt to equalize the groups. While in other designs nonequivalence is damaging to internal validity, in a regression-discontinuity design it serves to strengthen the internal validity. The reason for this is that the groups are intentionally designed to not be equivalent, but instead have a linear relationship with each other. If during the posttest there is still a linear relationship between the groups, the treatment had no effect. However, if there is a discontinuity between the groups at the point of cutoff, a gap between each group’s linear representations of scores, this indicates the effect of the treatment. This design addresses all threats to internal validity from multiple groups, but can still be affected by social interaction threats that will tend to diminish the effect size. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Why are quasi-experimental designs used more often than experimental designs?** Quasi-experimental designs are often used when it is not feasible or ethical to conduct a randomized controlled study ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). For example, in education research the general desire is to know how some treatment affects students in the classroom. In most cases if an experiment is going to be conducted in a classroom setting, students cannot be randomly selected by teachers for inclusion into an experimental and control group. Thus, for the majority of educational studies it is not feasible to randomize. Further, if a treatment is known to be efficacious, it is unethical to withhold that treatment from certain individuals while providing it to others. Most medical research chooses quasi-experimental designs for this reason ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Harris et al., 2006 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**One conclusion you might reach (hint) after completing the readings for this assignment is that there are no bad designs, only bad design choices (and implementations). State a research question for which a single-group post-test only design can yield relatively unambiguous findings.** Many training classes use this very design. At the culmination of the training an evaluation is presented to the students and they are requested to fill it out, depicting their satisfaction with various factors regarding the course. Students could not very well be satisfied with or express how much they learned from the class at the beginning of the class, so in this situation a pretest would make little sense.
 * <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Describe each of the following types of designs, explain its logic, and why the design does or does not address the selection threats discussed in Chapter 7 of Trochim and Donnelly (2006): **

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Part II Assignment Question Answers
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**What research question(s) does the study address?** Research questions are designed to determine relationships between variables. In this study there were two independent variables, identified as income level of parents, and exposure time to children’s TV commercials. One research question for this study could have been: Among pre-pubescent children, how effective are commercials directed toward them at product recognition? Another research question could have been: What effect does parent’s income level have on the amount of time children view TV? <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**What is Goldberg’s rationale for the study? Was the study designed to contribute to theory? Do the results of the study contribute to theory? For both questions: If so, how? If not, why not?** Goldberg’s (1990) rationale for conducting the quasi-experiment was three-fold. First, Goldberg identified that “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">research using an experimental paradigm has tended to support the view that the influence of commercials targeted at children is considerable <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 445, emphasis in original <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) and gave a number of examples. Second, Goldberg demonstrated that “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">much of the research stream that has utilized a survey research/correlational paradigm to study the effects of advertising on children tends to indicate that advertising has a fairly minor role in influencing measures such as children’s preferences <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 446, emphasis in original <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Third, Goldberg identified that others had “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">noted the need to combine the two approaches <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 446 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) and proposed a quasi-experiment as a way to have “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">some of the advantages of both the experimental and survey methods <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 446 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) to determine the effect of advertising directed toward children. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">The study was designed to contribute to theory because Goldberg (1990) continued to test whether advertising directed towards children is efficacious. To this point, the literature was inconclusive regarding the effect of advertising on children due to a divergence between outcomes gathered through experimental versus case studies. If advertising geared toward children is ineffectual, companies need to determine a better and more effective use of their advertising dollars. The applicable theory in this study is advertising theory; the assumptions and principles that encourage people to buy products. This framework is subdivided in this study, focusing on children and how commercials directed towards them impacts their recognition of and desire to obtain certain products. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">The results of Goldberg’s (1990) study did contribute to theory. The outcomes indicated that children with minimal to no exposure to commercials targeting them were much less likely to recognize advertised products, or have them in their homes. On the other hand, children with more exposure to advertising were shown to be more likely to recognize certain products and to have them in their homes. This study corroborates much of the experimental research regarding the effectiveness of advertising on children and contributes to advertising theory. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**What constructs does the study address? How are they operationalized?** The main construct in Goldberg (1990) is advertising. Is advertising to children effective? Advertising as a construct was operationalized in the study through statistics regarding what is advertised to children. Since, “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">the most prevalent product categories advertised to children are toys. . . and foods <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 448 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) these were chosen as a way to determine the effectiveness of the advertisements. Further, one-quarter of the foods marketed toward children are breakfast cereals, so “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">children’s toys and cereals were selected as the focus of the study <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 448 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). A moderating construct was created in the study based on income level. Since some of the children’s parents’ income level was unavailable, this construct was operationalized by collecting the children’s mothers and fathers occupation. Another construct that was utilized in the study was exposure time to French-based and English-based TV stations by using a survey to determine which shows were watched when. By calculating how often shows were watched, the researchers were able to “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">estimate the total number of [American children’s commercial TV] programs each child watched during the year <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 448 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">What are the independent and dependent variables in the study? The independent variables in Goldberg’s (1990) study were “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">cultural affiliation as defined by the language spoken <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 448 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">the total number of ACTV programs each child watched during the year <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 448 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), and family income as differentiated by parental occupations. Goldberg’s dependent variables consisted of recognition of toys and breakfast cereals that were marketed on American TV. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Name the type of design the researchers used.** Goldberg (1990) used a nonequivalent control group posttest only quasi-experimental design for the main focus of this study. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**What internal and external validity threats did the researchers address in their design? How did they address them? Are there threats they did not address? If so how does the failure to address the threats affect the researchers’ interpretations of their findings? Are Goldberg’s conclusions convincing? Why or why not?** Conducting each groups survey at one time, and gathering surveys from different socio-economic groups in disparate locations, the threats to internal validity through social interaction were eliminated or minimized. The study consisted of multiple groups, eliminating the single group threats to internal validity. With multiple groups there is only a single threat to internal validity; that of selection bias ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Jackson, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). By taking the surveys at one time for each group, and not having a pre-test the selection threats of testing, instrumentation, mortality, and regression were eliminated. With all surveys for all groups taken within a two-week period the selection threat of maturation was minimized or eliminated. The only remaining threat to internal invalidity is history; there could have been some event that occurred between the surveys taken in the schools and the surveys taken in the camps that may have affected the later collection. Goldberg (1990) addressed the threat of history by using a t-test to determine if there were any significant differences between the groups with regards to the dependent measures, and there were not. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">External validity addresses the generalizability of a study. For this study the major threat to external validity was that the groups were not randomly selected into the control and experimental groups. Goldberg (1990) raised the issue that an alternative explanation for the results may be “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">due to cultural differences between the two groups <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 453 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Goldberg attempted to address this issue by comparing the dependent variables with the independent variables within-groups, finding that “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">at comparable levels of ACTV viewing, English- and French-speaking children purchased equivalent numbers of children’s cereals <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 453 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). He admits, however, that in regards to the recognition of toys “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">other factors. . . may have contributed to the observed difference in toy awareness levels between English- and French-speaking children <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 453 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Goldberg’s (1990) conclusions are reasonably convincing, but there are limitations. The study shows that there is a difference between the language groups, and his rationale regarding the children’s access to commercials geared toward them is appealing. A major weakness to the overall conclusion is that he only compares children within the single province of Quebec, and admits that differences in recognition could be the result of cultural factors, and he does not proceed to the next logical step of showing whether recognition of toys is the same as buying them. So, I am willing to accept that advertising directed toward children is effective at garnering brand recognition; I cannot find support that the legislation “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">appears to have reduced consumption of those cereals <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 453 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Nowhere in his study did Goldberg have provisions for comparing current consumption with past consumption. Further, his conclusion that reduced exposure to toy commercials “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">would leave children unaware of the toys and thus less able to pressure their parents to buy them <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 453 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) is not warranted by his study, since he tested recognition only, but failed to determine if this recognition translated into “pressure” on parents to buy more toys.*

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[TT1] <span style="color: #ff0000; font-family: 'Times New Roman',Times,serif; font-size: 120%;">Watch out for personification in your writing. It isn’t the clinical trial that is interested in the treatment! <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[TT2] <span style="color: #ff0000; font-family: 'Times New Roman',Times,serif; font-size: 120%;">This sentence needs a bit of work. Try writing it in more of an active voice instead of a passive voice. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[TT3] In education you see more quasi experiments than experiments. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[TT4] <span style="color: #ff0000; font-family: 'Times New Roman',Times,serif; font-size: 120%;">The plural form of criterion is criteria. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[TT5] A good critique!


 * = References ||
 * * de Anda, D. (2007). Intervention research and program evaluation in the school setting: Issues and alternative research designs. //Children & Schools, 29//(2), 87-94. Retrieved from ERIC Database. (EJ762838)
 * Dimitrov, D. M., & Rumrill, P. D. Jr. (2003). Pretest-posttest designs and measurement of change. //Work, 20//(2), 159-165. Retrieved from http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCoQFjAA&url=http%3A%2F%2Fwww.phys.lsu.edu%2Ffaculty%2Fbrowne%2FMNS_Seminar%2FJournalArticles%2FPretest-posttest_design.pdf&ei=Z0V1T4rLJ8Wi2gXum-mUDQ&usg=AFQjCNF6-c4j-7JJKm9ohkXr6jXJ1T-7pA&sig2=BW7hMizEWzQBfgUIIQ30rA
 * Ellis, T. J., & Levy, Y. (2011). Framework of problem-based research: A guide for novice researchers on the development of a research-worthy problem. //Informing Science: the International Journal of an Emerging Transdiscipline, 11//(1), 17-33. Retrieved from http://inform.nu/Articles/Vol11/ISJv11p017-033Ellis486.pdf
 * Goldberg, M. E. (1990). A quasi-experiment assessing the effectiveness of TV advertising directed to children. //Journal of Marketing Research, 27//(4), 445-454. Retrieved from http://www.jstor.org/stable/3172629
 * Greenhoot, A. F. (2003). Desgin and analysis of experimental and quasi-experimental investigations. In M.C. Roberts & S. S. Ilardi (Eds.), //Handbook of Research Methods in Clinical Psychology// (p.92-114). doi:10.1002/9780470756980.ch6
 * Harris, A., McGregor, J. C., Perencevich, E. N., Furuno, J. P., Zhu, J., Peterson, D. E., & Finkelstein, J. (2006). The use and interpretation of quasi-experimental studies in medical informatics. //Journal of American Medical Informatics Association, 13//(1), 16-23. doi:10.1197/jamia.M1749
 * Hoadley, C. (2007). Learning sciences theories and methods for e-learning researchers. In R. Andrews, & C. Haythornthwaite (eds.), //The SAGE handbook of e-learning research// (pp. 139-156). Los Angeles, CA: SAGE.
 * Jackson, S. L. (2012). //Research methods and statistics: A critical thinking approach// (4th ed.). Belmont, CA: Wadsworth Cengage Learning.
 * Kratochwill, T. R., & Levin, J. R. (2010). Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. //Psychological Methods//, //15//(2), 124-144. doi:10.1037/a0017736
 * Qumtin, E., Bélanger, C., & Lamontagne, V. (2012). A single-case experiment for an innovative cognitive behavioral treatment of auditory hallucinations and delusions in schizophrenia. //International Journal Of Psychological Studies//, //4//(1), 114-121. doi:10.5539/ijps.v4nlp114
 * Trochim, W. M. K., & Donnelly, J. P. (2008). //The research methods knowledge base// (3rd ed.). Mason, OH: Cengage Learning. ||