Hypothetical+Research+Designs


 * ** EDU7702-8 ** ||  ||
 * ** Research Design ** || ** 7 Hypothetical Research Designs ** ||
 * Stephen **
 * You offered a complete and well organized presentation of both types of hypothetical designs – nice work. I am curious as to which you would choose in this case. It seems your intent is to measure satisfaction, engagement and learning in both cases – so which do you think is the better design, and why? You will need to be prepared to defend your choice at some point. I mention this because you appear to make convincing cases for both – yet one should be the choice based on which answers your research questions best. **
 * Keep working hard! **
 * Keep working hard! **

=Hypothetical Research Designs= There are a number of ways to conduct research. Research can be used to describe, predict, understand, or identify the causes of a phenomenon or behavior ( Cozby & Bates, 2012 ). Case studies are used to observe and describe the behavior of an individual or context ( Trochim & Donnelly, 2008 ), generally involve the collection of large amounts of data ( Creswell, 2009; Leedy & Ormrod, 2010 ), and are an excellent way to formulate postulates and theories regarding a phenomenon ( Harlow, 2009 ). In the proposed qualitative design introduced below, a two-phase, case study design is used to analyze historical data and conduct semi-structured interviews to identify patterns and themes regarding student satisfaction or dissatisfaction with online professional development classes. Quasi-experimental designs are used in situations where true experimental designs are impractical or not possible ( Cozby & Bates, 2012 ). The lacking feature of a quasi-experimental design is random assignment of subjects to control and experimental groups. Nonequivalent group, quasi-experimental designs is the most frequent research design used in the social sciences ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). In the proposed quantitative design introduced below, a nonequivalent group, quasi-experimental design is used to determine the effect of utilizing a continuous visual element upon the satisfaction, participation, and perceived learning of students in an online environment. =<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Proposed Qualitative Design =

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Research Design
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">This study will use a qualitative, case study design. In a case study, research is centered on “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">gaining an in-depth understanding of particular phenomena in real-world settings <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Blichfeldt & Andersen, 2006, “Similarities”, para. 2 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). This case study will study an online instructor-led professional development environment in two phases. In the first phase, ten years of historical data from student evaluations is available and will be categorized by technology and delivery method; face-to-face (F2F) or live virtual class (LVC), to determine if there are differences in satisfaction and perceived learning based on delivery method. In the second phase, randomly selected learners from current LVC will be interviewed by phone to validate and corroborate the factors that increase learner satisfaction, in class participation, and perceived learning.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Data Collection and Analysis
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Unlike experiments or surveys, case studies do not have a single data collection method or method for analysis. The purpose of this case study is to determine factors of satisfaction and dissatisfaction in online classes and to classify into themes student comments related to their experience. In Phase one, anonymous data from student evaluations regarding professional development classes in my organization will be analyzed. In Phase two, telephone interviews with randomly selected LVC students will be conducted. Factors determined in Phase one to be determinants of student satisfaction or dissatisfaction will be explored in the interviews and may involve the impact of the instructor, courseware, or technical environment.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Measurement
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Using an open-coding system ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Strauss & Corbin, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">), categories of satisfaction and dissatisfaction will be developed and tentative hypotheses will be proposed in phase one of the study. Specifically, overall satisfaction and perceived learning by students will be evaluated from a question that uses a 5-point Likert scale, “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">Overall this course met my expectations and requirements <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">”. These ordinal scores will be correlated by technology and delivery method. In addition, the information from two open-ended questions on the evaluation, “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">Would you recommend [this] training to others? Why or why not? <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” and “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">Suggest how we could improve your satisfaction with the course <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” will be analyzed and categorized to cluster factors of satisfaction and dissatisfaction. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Telephone interviews in Phase two will be recorded so that they can be transcribed and analyzed for themes or patterns. In these semi-structured, one-on-one interviews, students will be asked questions to gain additional insights into their feelings regarding expectations of, satisfaction with, and factors that contributed to either their satisfaction or dissatisfaction in the online classroom environment. Questions regarding each student’s expectations when he or she enrolled for the course, and whether those expectations were met will be open ended to encourage a full exploration of the factors that contributed to satisfaction or dissatisfaction. =<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Proposed Quantitative Design =

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Research Design
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">This study will use a quantitative, nonequivalent control group, quasi-experimental design. Classes at my organization are scheduled at the discretion of management and based upon various factors, such as popularity and instructor availability. Classes can be conducted or cancelled and students rescheduled depending on the number of enrollments in each class. Whether a student purchases a technological course that is conducted in a traditional face-to-face environment or in a live virtual classroom (LVC) may not be completely at the discretion of the learner, providing institution, or presenting instructor. Without randomization there can be no true experimental design; without an experiment it is more difficult to determine cause-and-effect. I chose a quasi-experimental design because it is not possible to randomly place learners into separate control and test groups since learners purchase the appropriate class for their professional development needs and such other motivators personal to each student. Even without randomization, a sufficient sample size and control group allows for statistical manipulation to roughly approximate randomization ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Edgington, 1966; Wright, 2006 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">).

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Data Collection and Analysis
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Data will be collected from a minimum of ten instructors who teach various technologies. Each instructor will teach two instances of two different online classes of five consecutive days or less duration. These classes will be paired, such that one instance of the class will be taught according to that instructor’s normal delivery (the control) and one instance will be taught in the normal style with the addition of a webcam transmitting the instructor’s image to the class during interactive periods of the class (the experiment). Whether the control class or experimental class will be taught first will be randomized. Each student will be encouraged at the end of the class to fill out the Learner Satisfaction and Transfer-of-learning Questionnaire (LSTQ) developed and validated by Gunawardena, Linder-VanBerschot, LaPointe, and Rao (2010), in addition to the regular course evaluation. Incomplete or surveys that have the same value for all sixteen questions will be discarded. Using an a priori power analysis the total number of respondents needs to exceed 620 to have sufficient power to obtain a significant result (δ = 0.2, α = 0.05, β = 0.20, and approximately even sample sizes between groups). The data will then be separated into pertinent constructs for this study; satisfaction, engagement, and perceived learning. ANOVA and t tests using the IBM SPSS statistics software will be conducted to determine if differences exist between the means of control and experimental groups in the appropriate measures.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Operational Definition of Variables
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">The independent variable for this study is whether the visual element (webcam) of the instructor is continuously transmitting to adult learners in a LVC as in the experimental classes, or not as in the control classes. All instructors engaged in the research are to conduct and facilitate their classes as they normally would with the sole exception of the independent variable. Measures of three dependent variables will be collected; these dependent variables are learner satisfaction, learner participation, and perceived learning. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Independent variable - visual element.**The webcam in this research allows the transmitting of limited facial expressions and body language of the instructor to the student. It is a nominal variable as minimal or no visual transmissions will occur in the control classes. In the experimental classes there will be continuous visual transmissions during lecture or participation cycles between instructor and students. The visual element variable has two attributes; full use of webcam (1) and minimal use of webcam (0). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Dependent variable - learner satisfaction.** Learner satisfaction has been chosen as a dependent variable. Studies indicate that as student satisfaction increases so does participation ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Gunarwardena et al., 2010 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">) and learning outcomes ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Gunawardena et al., 2010; Kozub, 2010; Martinez-Caro, 2009; McGlone, 2011 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Learner satisfaction is a construct that will be derived from the Learner satisfaction subscale of the LSTQ; consisting of five 5-point Likert scale questions. The learner satisfaction construct is an ordinal variable varying from strongly agree = 5 to strongly disagree = 1. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Dependent variable - learner engagement.** Learner engagement has been chosen as a dependent variable. Studies indicate that as students are interactive with the instructor, other students, and the content they learn more effectively ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Abrami, Bernard, Bures, Borokhovski, & Tamim, 2010; Bradley, 2009 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Learner engagement is a construct that will be derived from both the learner-learner interaction and learner-instructor interaction subscales of the LSTQ and consists of six 5-point Likert scale questions.* The learner engagement construct is an ordinal variable varying from strongly agree = 5 to strongly disagree = 1. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Dependent variable - learner perceived learning.** The objective of adult professional development is to enhance the knowledge and skills of adult workers so that they are more productive and effective in their working environment. Generally, students in adult professional development courses do not participate in evaluated activities or receive grades. Multiple studies have identified that a student’s self-perception of learning is as valuable an indicator of learning as any external measure ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Gunawardena et al., 2010; Kenner & Weinerman, 2011 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Perceived learning is a construct that will be derived from the ability to transfer subscale of the LSTQ and consists of five 5-point Likert scale questions. The perceived learning construct is an ordinal variable varying from strongly agree = 5 to strongly disagree = 1.*

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Measurement
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;"> Collection of data for this research will be done at the culmination of each of the LVC through an online survey. The 16 questions from the LSTQ will be presented in no particular order with a 5-point Likert scale. The LSTQ has been previously validated from similar research regarding student satisfaction and transfer of learning. The learner satisfaction subscale of the LSTQ has a Cronbach alpha of .83 making it extremely reliable. The reliability of the learner-learner interaction subscale of the LSTQ has a Cronbach alpha of .69 for good reliability and the learner-instructor interaction subscale of the LSTQ has fair reliability with a Cronbach alpha of .52. The ability to transfer subscale of the LSTQ has fair to good reliability with a Cronbach alpha of .62.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[AM1] Who is filling out these questionnaires – the learner or the instructor – or both? <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">[AM2] Is there no other way to measure their learning that you could incorporate such as a competency based assessment?


 * = References ||
 * * Abrami, P. C., Bernard, R. M., Bures, E. M., Borokhovski, E., & Tamim, R. (2010, July). Interaction in distance education and online learning: Using evidence and theory to improve practice. //The Evolution from Distance Education to Distributed Learning//. Symposium conducted at Memorial Union Biddle Hotel, Bloomington, IN. doi:10.1007/s12528-011-9043-x
 * Blichfeldt, B. S., & Andersen, J. R. (2006). Creating a wider audience for action research: Learning from case-study research. //Journal of Research Practice, 2(1)//. Article D2. Retrieved from http://jrp.icaap.org/index.php/jrp/article/view/23/43
 * Bradley, J. (2009). Promoting and supporting authentic online conversations – which comes first – the tools of instructional design? //International Journal of Pedagogies and learning, 5//(3), 20-31. doi:10.5172/ijpl.5.3.20
 * Cozby, P. C., & Bates, S. C. (2012). //Methods in behavioral research// (11th ed.). Boston, MA: McGraw Hill Higher Education.
 * Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: SAGE.
 * Edgington, E. S. (1966). Statistical inference and nonrandom samples. //Psychological Bulletin, 66//(6), 485-487. doi:10.1037/h0023916
 * Gunawardena, C. N., Linder-VanBerschot, J. A., LaPointe, D. K., & Rao, L. (2010). Predictors of learner satisfaction and transfer of learning in a corporate online education program. //The American Journal of Distance Education, 24//(1), 207-226. doi:10.1080/08923647.2010.522919
 * Harlow, E. (2009). Contribution, theoretical. In A. J. Mills, G. Durepos, & E. Wiebe (Eds.), //Encyclopedia of Case Study Research// (pp. 237-239). Thousand Oaks, CA: SAGE.
 * Kenner, C., & Weinerman, J. (2011). Adult learning theory: Applications to nontraditional college students. //Journal of College Reading and Learning, 41//(2), 87-96. Retrieved from http://www.crla.net/journal.htm
 * Leedy, P. D., & Ormrod, J. E. (2010). //Practical research: Planning and design//. Upper Saddle River, NJ: Merrill.
 * Strauss, A. L., & Corbin, J. (2008). //Basics of qualitative research: Techniques and procedures for developing grounded theory// (3rd ed.). Los Angeles, CA: SAGE.
 * Trochim, W. M. K., & Donnelly, J. P. (2008). //The research methods knowledge base// (3rd ed.). Mason, OH: Cengage Learning.
 * Wright, D. B. (2006). Comparing groups in a before-after design: When //t// test and ANCOVA produce different results. //British Journal of Educational Psychology, 76//, 663-675. doi:10.1348/000709905X52210 ||