Internal and external validity are two fundamental measures in the methodology used to assess the quality and applicability of the results in research. As validity is important when it comes to the representativeness of a study, the following article will compare internal vs external validity, its threats, improvements and provide examples to further illustrate the topic.
Internal vs. External Validity – In a Nutshell
Internal validity refers to the reliability of the cause-and-effect relationship established between variables.
External validity refers to the generalizability and applicability of the results to the natural environment in our world.
Definition: Internal vs. external validity
When examining cause-and-effect interactions, validity can be divided into internal vs. external validity.
Internal validity relates to the robustness of the research design and the extent to which it can provide evidence of the link between an independent and a dependent variable, unaffected by other factors than the ones investigated.
External validity is the extent to which the findings of a study may be extended to other contexts, groups, or events. It concerns the generalizability of the study results outside the laboratory setting.
Similarities and differences
External validity is frequently sacrificed for greater internal validity, and vice versa. The sort of study you select reflects the research’s priorities.
Discussed below is a trade-off between internal vs. external validity:
Example
A causal relationship can be studied in a lab or “real world.” In a lab, external effects can be eliminated, boosting internal validity. External validity is weakened because a lab environment is different from “real world”. This trade-off can be resolved by researching in a controlled environment to verify a causal relationship, followed by a field experiment to test the conclusions in the real world.
- Similarities
Internal and external validity are both concepts that influence the meaning of a study. They determine, whether research is useful and representative or not. Furthermore, they do not exclude each other, which is why both have to be considered during study design.
- Differences
Internal validity refers to the variables that are monitored during a study, investigating their sources and consequences. External validity, on the other hand, focuses on the applicability to the real world. This way, a completely internally valid study can be useless in the natural setting, while a externally valid study can be useless without knowing which variables caused the results.
Internal validity
Example
The management of firm X wants to determine whether flexible work hours will increase employee job satisfaction. They designed an experiment involving two groups:
- Staff with fixed work hours as a control
- Test group with employees who have flexible work hours.
The duration of the experiment is six months. All employees complete a job satisfaction survey before (pre-test) and after(post-test) the experiment.
Threats
Many factors can compromise your research’s internal validity, which are explained in the following.
Unanticipated occurrences can alter the study’s conditions and affect its outcome unexpectedly.
Example
During the study, a new manager is hired, which boosts employee satisfaction.
The dependent variable is influenced by time.
Example
Throughout the six-month experiment, people gained experience and became better at their professions. Consequently, job satisfaction could increase.
The pre-test influences the post-test findings.
Example
Employees feel compelled to maintain consistency between their pre- and post-test responses.
On a second measurement, extreme results approach the mean.
Example
Employees who scored extremely low in the initial job satisfaction survey are likely to have a higher increase in job satisfaction than those who scored average.
During the study, the dependent variable is measured differently.
Example
The post-test questionnaire comprises additional questions compared to the pre-test questionnaire.
Interactions between individuals from distinct groups affect the outcome.
Example
The group with fixed working hours resents the one with flexible working hours, and as a result, their job happiness declines.
Participants with similar characteristics drop out of the study.
Example
Some members of the fixed hours group drop out because they do not like their work times.
A different variable, that has not been considered, influences the outcome of the study.
Example
Some members of the experimental group experience trouble at home, leading to lower wellbeing, though unrelated to the work shifts.
The researcher behaves differently with each group, affecting the results.
Example
When the researcher expects low satisfaction with the control group, his own emotions might influence the mood of the individuals negatively.
Improvements
There are also factors that can improve the internal validity of a study, explained in the following.
The participants, an in a double-blind study the researcher too, do not know about the treatment they are receiving.
Example
Instead of just making two groups, you apply the different work hour trials to two separate companies and the researcher will not know which company tries which concept.
Actively manipulating a variable instead of just observing a connection.
Example
Divide the participants into two groups instead of observing those individuals that already apply the different types of working schedule.
Selecting participants randomly so they are more likely to represent the entire population.
Example
Select participants from different departments or even companies instead of dividing just one department into two groups.
Make a plan for the study and stick to it to prevent the introduction of unintended effects.
Example
Set the questions for the pre- and post-trial surveys before the start of the study and also consider all steps in between to minimize the risk of bias.
External validity
Example
A researcher plans a two-month trial to see if regular mindfulness meditation helps persons with mental illness. They look for depressed people between 20 and 29 who reside nearby.
Participants are given a pre and post-test to measure the frequency with which they experienced anxiety throughout the past week. Each participant receives tailored mindfulness instruction and must practice every morning for 15 minutes.
The researcher concludes that mindfulness can assist all clinical populations because anxiety levels dropped between the pre and post-test results.
Threats
To design a robust study, it is essential to identify and mitigate threats to external validity.
The sample does not adequately represent the population.
Example
The sample comprises depressed individuals. They possess traits that distinguish them from other clinical populations, such as those with personality disorders or schizophrenia.
An unconnected occurrence alters the results.
Example
A natural disaster strikes a neighboring state just before the pre-test.
Consequently, pre-test anxiety scores are more significant than they would be otherwise.
Unintentionally, the qualities or actions of the experimenter influence the results.
Example
The mindfulness training instructor unintentionally emphasized the significance of this study for the research department’s financing.
As a result, participants exert additional effort to minimize their anxiety throughout the study.
Participants may alter their behavior when they know they are being monitored.
Example
Participants actively avoid anxiety-provoking circumstances throughout the study since they know they are participating in research.
Pre- or post-test administration influences the outcomes.
Example
Participants have less anxiety during the post-test because they are more familiar with the format and questions from the pre-test.
The generalizability of the findings is hindered by variables such as the setting, time of day, geography, or researchers’ characteristics.
Example
The study is replicated with one modification: the individuals now practice mindfulness in the evening.
This time, no progress is evident in the results.
The individuals in the same group align in their mentality and differ from the participants in the other one.
Example
The participants in the meditation group do not believe in its helpfulness or do not want to spend their time meditating.
Improvements
Besides threats, there are also ways to improve the external validity before, during or after a study.
The study is conducted outside the lab in a natural setting.
Example
Instead of assembling the participants to meditate in an observed environment, let them meditate at home or where they would do it themselves.
Define the characteristics you want the participants to have to gain a specific image of the population you are studying.
Example
The study includes only depressed people from the age of 20 to 29, which means the results are also only applicable to this part of the population.
Deceive participants by inventing a cover story about the research goal to minimize the risk of unblinding.
Example
You could for example tell the participants that the study is about how meditation reduces blood pressure to keep them from thinking about their anxiety levels.
Repeat the study with different samples and see if you get the same results.
Example
Repeat the study with samples of other age for example and see if the results stay the same.
Statistical adjustments to the results to make up for possible validity issues.
Example
If your sample includes more older participants, you might have to overweight the answers of the younger ones to gain equality.
Ecological validity
Ecological validity is a judgement of whether the results of a study can be applied to natural real-life situations. It is a subtype to external validity. However, it does not only focus on the applicability of the findings to people with other prepositions than the participants, but to the complete setting and environment outside the clinical scenery. Ecological validity can thus not be measured statistically but has to be judged by the researchers or analysts themselves.
Internal vs. external validity: Examples
Below are internal vs. external validity examples:
Example
Internal validity:
- You want to test if coffee boosts memory.
- College-aged participants are randomized to morning and evening lab sessions. The morning session is the treatment group and the evening session is the control group.
- The treatment group received coffee upon arriving at the lab, while the control group received water.
- You also give memory tests to both groups.
- After evaluating the results, it’s decided that the treatment group did better on the memory test than the control group.
Example
External validity:
- Memory test external validity is affected by participant inclusion criteria and lab conditions.
- If you examine college-aged people, your results may only apply to them.
- This improves internal validity and reduces external validity.
FAQs
Internal vs. external validity measures how accurately the independent variable affects the dependent variable. External validity generalizes research.
Internal vs. external validity focuses on extraneous variable control, while external validity emphasizes practical applicability.
Using a cognitive map, researchers may methodically address internal vs. external validity to reflect treatment effects and generalize findings appropriately.
