INTRODUCTION

Also the author of this website has made an effort to ensure the accuracy and completeness of the presented information, the author assumes no responsibility for any errors, missing information, inaccuracies, or inconsistencies.

Validity & Reliability

It is important to think about validity and reliability when designing your study. This will impact how they will be able to be generalized to the population.

Validity-How likely the measure you are using actually measures your dependent variable.

There are different types of validity:

1. Internal-How well the observed relationship accurately reflect the relationship between the intended variables (Heiman, 2001).

2. External-Are the results able to be generalized to population beyond the participants studied.

3. Construct-How well a test reflects the construct it is meant to test.

4. Content-How well a test measures ONLY the construct in question (Heiman, 2001).

5. Criterion-How well a test is able to tell participants apart on abilities. An example could be the Mini Mental States exam. It has been designed to distinguish between normal memory and the beginnings of dementia.

6. Concurrent-How well a test measures a participant's current abilities.

7. Predictive-How well a test predicts a participant's future abilities.

8. Convergent-How well a test correlates to another well accepted test (Heiman, 2001).

9. Discriminant-How well a test DOES NOT correlate to another accepted test.

10. Face-How well a test "appears" to measure the construct in question ("face value").

11. Ecological-Generalizations of what participants can do in a study to what they can do in real life.

12. Temporal-Generalizations to other time frames (i.e. 5 and 30 minute word recalls for memory).

Reliability

Reliability-How likely the measure you are using measures the dependent variable consistently (does it measure it each time) (Heiman, 2001).

There are different types of validity:

1. Inter-rater-How consistent results are between raters. An example could be the results of three different judges on a snowboarder's performance.

2. Test-retest-How consistent the results from a specific test are over time (Heiman, 2001). An example could be Grooved Pegboard scores for hand-eye coordination that are measured twice at six month intervals.

3. Split-half-There should be strong correlations between participant's scores on one half of a test compared to the other (Heiman, 2001). An example could be hand-eye coordination scores, where the participant's scores are divided in half (odd vs even trials). The summary scores of the odd numbered trials should have strong correlations to the even numbered trials.

DESIGNS

The type of research project you are doing will depict the design you will use:

1. Within subjects-Used when the researcher is interested in determining whether or not one condition is better than another, and is dependent upon having the same participants in both conditions. An example could be determining whether website A is more efficient than website B.

2. Between subjects-Used when the researcher is interested in determining whether or not one condition is better than another, but participants are placed in only one condition. An example could be determining whether using PowerPoint presentations assist students get better grades compared to overhead projections (Heiman, 2001).

Counterbalancing-The practice of randomizing the order of condition for participants so balance the number of participants that receive condition A and condition B (see Tables 1 and 2). This allows for an equal number of participants in each condition (Heiman, 2001).

Table 1. Counterbalancing for within subjects design

Participant #	First Condition	Second Condition
1	A	B
2	B	A
3	A	B
4	B	A

Table 2. Counterbalancing for between subjects design

Participant #	Condition
1	A
2	B
3	A
4	B

REFERENCE

Heiman, G. (2001). Research Methods in Psychology (3rd ed.). Cengage Learning, pgs. 71-73, 76-77, 114-115, 165, 173, 282-283.