External,+Internal,+and+Construct+Validity


 * ** EDU7702-8 ** ||  ||
 * ** Research Design ** || ** 2 Validity, External, internal, construct ** ||
 * Stephen **
 * Excellent submission overall – thorough, well written and great use of examples. **
 * I left a few comments in the paper for you to consider, but it is clear that you researched, understand and I am confident you can apply these concepts to your own research. Remember to try to keep a consistent writing style throughout – at times you use rather ‘high end’ academic type language, yet at others you use more common/folksy language. Good submission and I look forward to reading more. **
 * I left a few comments in the paper for you to consider, but it is clear that you researched, understand and I am confident you can apply these concepts to your own research. Remember to try to keep a consistent writing style throughout – at times you use rather ‘high end’ academic type language, yet at others you use more common/folksy language. Good submission and I look forward to reading more. **

=External, Internal, and Construct Validity in E-learning = Even though Knowles (1970) introduced andragogy in the U.S. over 40 years ago there are few empirical studies investigating the truth of its assumptions ( Merriam, Cafferella, & Baumgartner, 2007; Taylor & Kroth, 2009 ). Only through research can the acceptance or rejection of theory be justified.* Only through research can theory be developed, tested, winnowed, and advanced. Research informs application of theory and implementation of theoretical principals ( Tolutiene & Domarkiene, 2010 ). Research furthers, broadens, and strengthens, but only if it is based on verity. The foundation of verity in research is validity ( Trochim & Donnelly, 2008 ). Objective reality, the absolute truth, or how things truly are in any discipline is ephemeral and the purpose of research is asymptotic ( Creswell, 2009 ).* Validity in research has three interworking and counterbalancing concepts; (a) does an identified variable accurately portray the concept or truth that is being sought, (b) how strongly can a causal relationship be established between two variables, and (c) how well do the results convey universal truth that is fully generalizable ( Cozby & Bates, 2012 )? The concepts are labeled by individual authors differently, but the underlying models and conceptions are very similar.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Variables, Verity, and Validity
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">The hardest to identify, most time consuming, and important type of validity is construct validity ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Salkind, 2009 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Leedy and Ormrod (2010) stated that construct validity is focused on measurement and the “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">extent to which an instrument measures a characteristic that cannot be directly observed but is assumed to exist based on patterns in people’s behavior <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 92 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Cozby and Bates (2012) indicated that construct validity is focused on the variables being tested and “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">the adequacy [that] operational definition of variables. . . actually reflect the true theoretical meaning of the variable <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 71 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Salkind (2009) declared that the focus is on the results and the extent that those results relate “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">to an underlying psychological construct <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">p. 120 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). The fundamental model of each of these authors for construct validity is how closely does the research test what is being tested. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Threats to construct validity.** If construct validity is missing in research nothing else matters. If a researcher believes that he or she is measuring student satisfaction but is instead measuring student interest or motivation, the research lacks validity because it is not evaluating the construct that it purports to analyze.* A bicyclist may pedal continuously, but if the direction of travel is contrary to the expected destination he or she will never arrive. The greatest threat to construct validity is inexperience; in implementing research or in subject matter. To counteract this threat, it is essential to get the input of others who have experience in both research and in the specific field of study. If the area of study is new, a Delphi approach that includes a dozen or more experts in the field can be conducted with several iterations. This approach focuses on areas of agreement and disagreement and attempts to attain a high level of consensus between the experts ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">So & Bonk, 2010 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Use of experts is essential to formulating strong construct validity ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Salkind, 2009 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Impact of construct validity and threats on envisioned research.** I envision in my research to investigate whether the addition of a visual element (webcam) to online synchronous classes will foster increased learner participation, satisfaction, and perceived learning ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Watts, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). The independent variable of using or not using the webcam is easily operationally defined and thus has excellent construct validity. The concepts of participation, satisfaction, and perceived learning, however, are not so easily quantified since they cannot be heard, tasted, smelled, or seen. To measure differences in student participation, satisfaction, or perceived learning it is essential to have an instrument that will accurately represent and depict differences in and between these concepts. The Learner Satisfaction and Transfer-of-learning Questionnaire (LSTQ) was created and validated using Cronbach’s alpha ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Gunawardena, Linder-VanBerschot, LaPointe, & Rao, 2010 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">).* The LSTQ learner satisfaction subscale consisting of five 5-point Likert scale questions, has an extremely reliable Cronbach alpha of .83, and measures the learner satisfaction construct. The LSTQ learner-learner interaction subscale has good reliability with a Cronbach alpha of .69 and the learner-instructor interaction subscale has fair reliability with a Cronbach alpha of .52. Both subscales consist of six 5-point Likert scale questions to measure the construct of participation. The concept of perceived learning by students will be derived from the LSTQ ability to transfer subscale that consists of five 5-point Likert scale items and a Cronbach alpha of .62 representing fair to good reliability in measuring the concept.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Causality and Validity
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">A major motivator in research is to answer questions regarding what causes what to happen; we want to better predict (and control) what happens in our world.* When evaluating any claim that an outcome is caused by some input, the root of the analysis should be based in internal validity ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">Internal validity refers to the accuracy of conclusions about cause and effect <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Cozby & Bates, 2012, p. 69 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Three elements contribute to strong internal validity; temporal precedence, covariance, and elimination of confounding variables ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Cozby & Bates, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">).* <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Threats to internal validity.** The biggest threat to internal validity is in not eliminating other possible reasons for the results that were obtained in a study ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Leedy & Ormrod, 2010 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). If five possible explanations exist for a specific result, but only one is tested, even if the correlation is perfect, no causal relationship can be claimed because of weak internal validity. The fewer alternative reasons for a phenomenon, the stronger inferences can be made about cause-and-effect ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Cozby & Bates, 2012 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Experimental designs are especially vulnerable to internal validity since their purpose is to discover causal relationships ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Leedy & Ormrod, 2010 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Other threats to internal validity include; (a) external events that influence the results, (b) changes in an experiments subjects not related to the experiment, (c) regression toward the mean, (d) voluntary selection, (e) participant drop out, (f) interaction of control and experimental groups, (g) jealousy between groups, (h) envy between groups, (i) testing recall by participants, and (j) instrument drift ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Creswell, 2009 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Each of these threats introduces an alternate explanation for an experiments result not wholly dependent on the independent variable. Without eliminating or minimizing these threats internal validity is weakened and may acutely impact what can be inferred from the results. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Impact of internal validity and threats on envisioned research.** The subjects that are available for my research cannot be randomly assigned to control and experimental groups.* Random assignment of subjects to groups affords the strongest possibility of establishing a causal relationship, while a nonrandomized, nonexperiment provides the weakest internal validity ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). In my research I expect to use a quasi-experimental design since prospective subjects purchase the appropriate class for their professional development needs and such other motivators personal to each student, and assigned to available classes outside of my control or that of the instructor. Similarly, the classroom mode of the prospective subjects may not be completely at the discretion of the learner, the presenting instructor, or the providing institution. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">By using a group of instructors to deliver course pairs, one control and one experimental, in a variety of subjects, reasonable internal validity can be expected.* Internal validity may be increased by using the control group results for statistical manipulation to roughly approximate randomization effects ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Edgington, 1966; Wright, 2006 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Though students are not randomly assigned to control or experimental groups, there is no reason to expect that the students assigned through the enrollment process to courses that are then randomly assigned to either control or experimental would be different.

<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Generalizability and Validity
<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">A naïve but tempting motive to conduct research is to find universal truth. External validity determines how well and the extent to which findings of a study can be generalized beyond the bounds of the study, or to “ <span style="color: #0000ff; font-family: 'Times New Roman',Times,serif; font-size: 120%;">other persons in other places and at other times <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">” ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008, p. 34 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Universal truth would have the greatest generalizability and supreme external validity, but the effects of chance, imperfect conceptualizations, and deficient understanding of associations makes determining it extremely unlikely, if not impossible. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Threats to external validity.** Any incorrect inference regarding applying the findings of a study to an inappropriate group, time, or place diminishes external validity ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Trochim & Donnelly, 2008 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). Any findings that overstep the subject traits, distinctiveness of place, or timeliness of the study weakens the validity of the findings ( <span style="font-family: 'Times New Roman',Times,serif; font-size: 90%;">Creswell, 2009 <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">). If all of the subjects of a study were male it is inappropriate to make inferences regarding females.* If a relationship study is conducted exclusively at a research facility it is unsuitable to make inferences regarding relationships outside of a controlled environment. If a study focuses on 21st century youth between the ages of 10 and 14 it is incongruous to make inferences about their grandparents at the same age or youth in the 18th century. <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">**Impact of external validity and threats on envisioned research.** The prospective subjects for my research are adults of mixed gender, college educated, employed, and have a technical background. Because of these characteristics I would probably not be able to generalize my findings to young, uneducated, unemployed, or non-technical individuals taking online courses. Subjects are only taking classes from a single U.S.-based Technology Company.* Extending the findings of my study to other adult online technical training, and perhaps adult online training seems feasible. Another consideration regarding external validity in my envisioned study is since data collection will be conducted in a short period of time, it is possible that the students who enroll during that period may have characteristics or traits that are different from students who enroll at other times of the year. For example, if data collection proceeds during the holiday season, student characteristics such as working for company’s who are slower at the end of the year, or being better workers who are enrolled as a reward for good work, may differentiate these students from others who attend at other periods. =<span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Conclusion = <span style="font-family: 'Times New Roman',Times,serif; font-size: 120%;">Valid research consists of the interplay between three dimensions of truth; construct validity, internal validity, and external validity. Construct validity is absolutely essential. The closer variables or constructs reflect the reality they are designed to represent, the more likely results will reflect and approach truth. The other types of validity, however, exist on a continuum, and have a negative correlation. A study does not either have or not have internal or external validity; these constructs are not binary. Further, as the internal validity of a study increases, making it more determinant, the external validity tends to decrease making it less extendable. Conversely, the higher the generalizability the less likely a study will engender a causal relationship. All are necessary and must be considered while designing a study because research only furthers, broadens, and strengthens theory and application to the extent it is based on truth.

<span style="font-family: 'Times New Roman','serif'; font-size: 16px;">- Well, clearly we hope so :-) <span style="font-family: 'Times New Roman','serif'; font-size: 16px;">- Are these your words or a direct quote –just curious. [Yes, these are definitely my words - I'm not sure that outside of a computer class I've ever heard the word asymptotic used. Well, true, I originally heard the word in a math class years ago.] <span style="font-family: 'Times New Roman','serif'; font-size: 16px;">- Good use of a relatable example here. <span style="font-family: 'Times New Roman','serif'; font-size: 16px;">- Developed by these researchers or used by them? [Both. Gunawardena et al. cover validating the tool, and using it.] <span style="font-family: 'Times New Roman','serif'; font-size: 16px;">- Your opinion, or do you have a source? [This is my opinion, but I have no doubt that I could find a source for it as well! It's not like I'm really going to come up with anything unique that no one has thought before.] <span style="font-family: 'Times New Roman','serif'; font-size: 16px;">- It would be appropriate to briefly define each of these. <span style="font-family: 'Times New Roman','serif'; font-size: 16px;">- Because???? [OK, I'll give her this one. She has no way of knowing what my research will be doing or not. Should have explained this better.] <span style="font-family: 'Times New Roman','serif'; font-size: 16px;">- ??? [OK, this one I can't give her. She is a research Mentor after all. Nonexperimental research is neither experimental or quasi-experimental. What else would you call research that isn't based on an experiment?] <span style="font-family: 'Times New Roman','serif'; font-size: 16px;">- Why/how? [Again, late night; not thinking my explanation and audience all the way through.] <span style="font-family: 'Times New Roman','serif'; font-size: 16px;">- Again, good use of an example. <span style="font-family: 'Times New Roman','serif'; font-size: 16px;"> - No need to capitalize unless you use the actual name of the company.


 * = References ||
 * * Cozby, P. C., & Bates, S. C. (2012). //Methods in behavioral research// (11th ed.). Boston, MA: McGraw Hill Higher Education.
 * Creswell, J. W. (2009). //Research design: Qualitative, quantitative, and mixed methods approaches// (3rd ed.). Thousand Oaks, CA: SAGE.
 * Edgington, E. S. (1966). Statistical inference and nonrandom samples. //Psychological Bulletin, 66//(6), 485-487. doi:10.1037/h0023916
 * Gunawardena, C. N., Linder-VanBerschot, J. A., LaPointe, D. K., & Rao, L. (2010). Predictors of learner satisfaction and transfer of learning in a corporate online education program. //The American Journal of Distance Education, 24//(1), 207-226. doi:10.1080/08923647.2010.522919
 * Knowles, M. S. (1970). //The modern practice of adult education: Andragogy vs. pedagogy//. New York, NY: Cambridge Book.
 * Leedy, P. D., & Ormrod, J. E. (2010). //Practical research: Planning and design//. Upper Saddle River, NJ: Merrill.
 * Merriam, S. B., Caffarella, R. S., & Baumgartner, L. (2007). //Learning in adulthood: A comprehensive guide// (3rd ed). San Francisco, CA: Jossey-Bass.
 * Salkind, N. J. (2009). //Exploring research// (7th ed.). Upper Saddle River, NJ: Pearson/Prentice Hall.
 * So, H.-J., & Bonk, C. J. (2010). Examining the roles of blended learning approaches in computer-supported collaborative learning (CSCL) environments: A Delphi study. //Educational Technology & Society//, //13//(3), 189–200. Retrieved from ERIC Database. (EJ899878)
 * Taylor, B., & Kroth, M. (2009). Andragogy's transition into the future: Mata-analysis of andragogy and its search for a measurable instrument. //Journal of Adult Education, 38//(1), 1-11. Retrieved from ERIC Database. (EJ891073)
 * Tolutiene, G. & Domarkiene, J. (2010). Learning needs and the possibilities of their satisfaction: The case of prospective andragogues. //Tiltai, 50//(1), 147-158. Retrieved from http://www.ku.lt/leidykla/tiltai.php
 * Trochim, W. M. K., & Donnelly, J. P. (2008). //The research methods knowledge base// (3rd ed.). Mason, OH: Cengage Learning.
 * Watts, S. W. (2012). //Technological tools impact on learning in online professional development courses//. Unpublished manuscript, Department of Education, Northcentral University, Prescott Valley, AZ. Retrieved from https://stevesncujourney.wikispaces.com/file/view/WattsSEDU7707-8-8Graded.docx
 * Wright, D. B. (2006). Comparing groups in a before-after design: When //t// test and ANCOVA produce different results. //British Journal of Educational Psychology, 76//, 663-675. doi:10.1348/000709905X52210 ||