Reliability and Validity MCQs: 25 Selection Questions With Answers

This set is for students revising personnel selection and psychological testing: the “testing” chapters of a BS or MSc I/O course, the psychometrics part of a board review, or a selection unit in the UK or US. It has a short topic summary, a table of reliability and validity types, 25 practice questions with explanations, and a plan for long answers. The questions are practice questions written to the published syllabus, not past-paper or board items.

Several questions need a calculator: Spearman-Brown, the standard error of measurement, the content validity ratio and the four-fifths rule. Each explanation shows the working, and each formula has a free calculator on this site so you can check your own numbers.

Reliability, validity, utility and fairness in one page

Reliability means consistency: would the same person get roughly the same score again? It is estimated in four main ways. Test-retest gives the test twice and correlates the scores (stability over time). Parallel forms correlates two equivalent versions. Internal consistency asks whether the items hang together; Cronbach (1951) gave the coefficient alpha used in almost every thesis, and split-half reliability is the older version corrected with the Spearman-Brown formula. Inter-rater reliability asks whether two judges, such as two interviewers, agree. From reliability you get the standard error of measurement (SEM = SD x the square root of (1 – reliability)), which tells you how far an observed score may sit from the true score.

Validity is about the inferences drawn from scores, not the test itself. The Standards for Educational and Psychological Testing (AERA, APA and NCME, 2014) treat it as one idea supported by different kinds of evidence, but exams still use three labels. Content validity: do the items sample the job domain? Experts judge this, and Lawshe’s (1975) content validity ratio puts a number on their agreement. Criterion-related validity: do scores correlate with a criterion such as supervisor ratings? In a predictive design you test applicants and measure performance later; in a concurrent design you test current staff and measure performance at the same time. Construct validity: does the test measure the trait it claims to? Cronbach and Meehl (1955) set out the idea, and Campbell and Fiske (1959) added convergent and discriminant evidence. Face validity, whether a test looks job-related to candidates, affects their reactions but is not evidence that scores predict anything.

Utility asks whether a valid test is worth using. Taylor and Russell (1939) showed that the gain in the share of successful hires depends on three things: the validity coefficient, the selection ratio (hired divided by applicants) and the base rate (the share who would succeed anyway). Later utility models put the gain in money using SDy, the spread of job performance valued in rupees or dollars.

Adverse impact is when a neutral-looking procedure selects one group at a much lower rate. Under the US Uniform Guidelines (29 CFR 1607.4(D)), a group’s selection rate below four-fifths (80%) of the highest group’s rate is generally treated as evidence of adverse impact. After Griggs v. Duke Power Co. (1971), such a procedure must be shown to be job-related. Read more in types of validity in employee selection and reliability in psychological testing.

Summary table: the terms these questions test

Practice MCQs: selection, reliability and validity

1. Which reliability estimate is most affected by memory and real change in people between the two testings?

   1. Test-retest reliability
   2. Internal consistency from one sitting
   3. Inter-rater reliability
   4. Split-half reliability
   Show answer

   A. Test-retest needs two sittings, so people may remember items or genuinely change in between, which is why the gap must be chosen with care. Internal consistency and split-half come from one sitting, and inter-rater reliability is about judges, not time.

2. A knowledge test is scored right or wrong on every item. Which internal consistency coefficient, a special case of alpha for such items, would you report?

   1. KR-20 (Kuder-Richardson formula 20)
   2. Cohen’s kappa
   3. Lawshe’s content validity ratio
   4. The test-retest correlation
   Show answer

   A. Kuder and Richardson (1937) gave KR-20 for dichotomous items, and Cronbach (1951) showed alpha is the general form. Kappa measures agreement between raters on categories, CVR measures expert agreement on content, and test-retest is stability over time. The item analysis calculator gives KR-20.

3. A testing body builds Form A and Form B of an aptitude test with equivalent items, so that candidates who resit do not see the same questions. The same group takes both forms and the scores are correlated. This estimates:

   1. Inter-rater reliability
   2. Predictive validity
   3. Parallel (alternate) forms reliability
   4. The selection ratio
   Show answer

   C. Correlating two equivalent versions taken by the same people is parallel forms reliability. Inter-rater reliability needs two judges, predictive validity needs later performance data, and the selection ratio is hires divided by applicants.

4. A 40-item clerical test has a reliability of .90. The company wants to cut it to 20 items to save time. Using the Spearman-Brown formula, what is the predicted reliability of the shorter test?

   1. .45
   2. .95
   3. .90
   4. .82
   Show answer

   D. With half the length (n = 0.5): (0.5 x .90) / (1 + (0.5 – 1) x .90) = .45 / .55 = .82. .45 halves the reliability, which overstates the loss; .90 assumes length does not matter; .95 is what you would expect from making the test longer, not shorter. Check it with the Spearman-Brown calculator.

5. A test’s standard error of measurement is 3. A candidate scores 70. Using one SEM either side, her true score probably lies between:

   1. 67 and 73
   2. 69 and 71
   3. 64 and 76
   4. 70 and 73
   Show answer

   A. One SEM either side of the observed score (70 plus or minus 3) gives a band of 67 to 73, which holds the true score about 68% of the time. 69 to 71 uses 1 point, 64 to 76 uses two SEMs (about 95%), and 70 to 73 looks only upward. See SEM and banding.

6. Which of these is NOT a method of estimating reliability?

   1. Split-half
   2. Parallel forms
   3. Test-retest
   4. Correlating test scores with supervisors’ ratings six months later
   Show answer

   D. Correlating scores with a later performance measure is a criterion-related (predictive) validity study. Split-half, parallel forms and test-retest all estimate how consistent the scores are.

7. Which statement about reliability and validity is correct?

   1. A test can be valid without being reliable
   2. Reliability is necessary for validity, but a reliable test is not automatically valid
   3. If a test is reliable, it is valid for every purpose
   4. Validity sets the upper limit on reliability
   Show answer

   B. Scores that bounce around cannot predict anything well, so reliability limits validity. But a bathroom scale that is always 5 kg off is reliable and still wrong. Option D reverses the relationship: reliability sets the ceiling on validity.

8. Adding a conscientiousness test to a cognitive ability test raises the multiple correlation with job performance from .50 to .58. This gain shows the personality test’s:

   1. Incremental validity
   2. Face validity
   3. Test-retest reliability
   4. Adverse impact
   Show answer

   A. Incremental validity is the extra prediction a new measure adds over what you already use. Face validity is how the test looks, reliability is consistency, and adverse impact is about group selection rates.

9. Lawshe’s content validity ratio for an item comes out at exactly 0. What does that mean?

   1. Every expert rated the item essential
   2. No expert rated the item essential
   3. Exactly half the experts rated the item essential
   4. The item is perfectly reliable
   Show answer

   C. CVR = (ne – N/2) / (N/2), so it is 0 when ne equals N/2. All experts gives +1 and none gives -1, and CVR says nothing about reliability. Try the CVR calculator.

10. A telecom company gives a new sales test to all applicants, hires on its usual criteria without looking at the scores, and correlates the scores with sales figures a year later. This design is called:

    1. Concurrent validation
    2. Test-retest reliability
    3. Predictive validation
    4. Content validation
    Show answer

    C. Testing applicants and measuring the criterion later is a predictive design. A concurrent design tests current employees and collects performance at the same time. Test-retest is about consistency, and content validation uses expert judgement, not performance data.

11. A new integrity test correlates .05 with a numerical reasoning test, as expected, but .80 with a scale that measures the wish to look good (social desirability). What is the main concern?

    1. Its test-retest reliability is too low
    2. Its discriminant validity is weak: it may be measuring faking good rather than integrity
    3. Its content validity is proven
    4. Its selection ratio is too high
    Show answer

    B. A test should correlate weakly with traits it is not meant to measure. A very high correlation with social desirability suggests the scores reflect impression management. Campbell and Fiske (1959) called this pairing of expected and unexpected correlations convergent and discriminant evidence. Nothing here speaks to retest stability, expert review or hiring ratios.

12. Applicants for a nursing post say a scenario test “really looks like the job”. This tells you about the test’s:

    1. Criterion-related validity
    2. Face validity, which shapes reactions but does not show that scores predict performance
    3. Construct validity
    4. Internal consistency
    Show answer

    B. Face validity is how job-related a test appears to the people taking it. It matters for acceptance and fairness perceptions, but it is not one of the sources of validity evidence in the testing Standards. Predictive, construct and reliability evidence all need data.

13. A test correlates .30 with job performance. A manager says it “explains 30% of performance”. What is the correct figure?

    1. 30%
    2. 9%
    3. 3%
    4. 70%
    Show answer

    B. Variance explained is r squared: .30 x .30 = .09, or 9%. 30% confuses r with r squared, 3% drops a decimal place, and 70% is 1 – r. Even a modest r can be worth a lot when many people are hired.

14. Sackett, Zhang, Berry and Lievens (2022) argued that many earlier validity estimates for selection methods were too high. What was the main reason?

    1. The original studies used fake data
    2. Corrections for range restriction had been applied too heavily
    3. Job performance cannot be measured
    4. Only cognitive tests had been studied
    Show answer

    B. Their paper’s subtitle is about systematic overcorrection for restriction of range; with more cautious corrections, mean validities fell, though structured interviews still ranked first. The other options misstate the issue.

15. In Schmidt and Hunter’s (1998) review, which of these methods showed essentially no validity for predicting job performance?

    1. Graphology (handwriting analysis)
    2. Work sample tests
    3. General mental ability tests
    4. Structured interviews
    Show answer

    A. Graphology, like applicant age, had validity close to zero. Work samples, general mental ability tests and structured interviews were among the strongest predictors in the same review.

16. A company validates a test only on its current top performers, so the range of scores is narrow. What usually happens to the observed validity coefficient?

    1. It is inflated above the true value
    2. It is unaffected, because correlation ignores range
    3. It is lowered below the value in the full applicant pool
    4. It becomes negative
    Show answer

    C. Range restriction shrinks the correlation, because there is less spread to predict. Correlations do depend on range, inflation is the wrong direction, and a sign change would need a different problem. The range restriction calculator shows the correction.

17. Which of these is NOT one of the three inputs to the Taylor-Russell tables?

    1. The validity coefficient
    2. The selection ratio
    3. The base rate of success
    4. The cost of testing each applicant
    Show answer

    D. Taylor and Russell (1939) used validity, selection ratio and base rate to show the expected share of successful hires. Costs enter later, money-based utility models, not the Taylor-Russell tables. Try the Taylor-Russell calculator.

18. Which selection ratio leaves a valid test the most room to improve the quality of hires?

    1. 0.05
    2. 0.50
    3. 0.90
    4. 1.00
    Show answer

    A. A low ratio (5 hires per 100 applicants) lets the employer pick from the top of the score range, so a valid test makes a big difference. At 0.90 or 1.00 almost everyone is hired and the test cannot change much. Check scenarios with the Taylor-Russell calculator.

19. According to the Taylor-Russell tables, a valid test adds the most to the share of successful hires when:

    1. The selection ratio is high and the base rate is close to 1.0
    2. The selection ratio is 1.0, so everyone is hired
    3. The selection ratio is low and the base rate is moderate, near .50
    4. Validity is zero but many people apply
    Show answer

    C. With few posts and many applicants, the test can pick the best; with a moderate base rate there is room to improve. If almost everyone succeeds anyway (base rate near 1.0) or everyone is hired (ratio 1.0), a test adds little, and a test with zero validity adds nothing.

20. Other things equal, a valid selection test produces the largest money gain for jobs where:

    1. Everyone performs at almost the same level
    2. Differences between good and average performers are worth a lot of money
    3. Only one person is hired every ten years
    4. The test has zero validity
    Show answer

    B. Utility models multiply validity by SDy, the money value of differences in performance, and by the number hired and their tenure. If performance hardly varies, or few people are hired, or validity is zero, the gain is small. See the utility analysis calculator.

21. Of 100 male applicants, 60 are hired; of 50 female applicants, 20 are hired. Using the four-fifths rule, what do you conclude?

    1. No adverse impact, because 20 women were hired
    2. The impact ratio is 0.67, below 0.80, so this is evidence of adverse impact
    3. The impact ratio is 1.5, so men are disadvantaged
    4. The rule cannot be used when group sizes differ
    Show answer

    B. Selection rates are 60% for men and 40% for women; 40 / 60 = 0.67, which is under four-fifths. Counting hires ignores the rates, 1.5 divides the wrong way round, and the rule works on rates, so group sizes can differ. Check with the adverse impact calculator.

22. Which statement about the four-fifths rule is NOT correct?

    1. It compares each group’s selection rate with the highest group’s rate
    2. A ratio below 80% is generally regarded as evidence of adverse impact
    3. Smaller differences can still count as adverse impact if they are statistically and practically significant
    4. A ratio below 80% proves that the employer discriminated illegally
    Show answer

    D. The Uniform Guidelines (29 CFR 1607.4(D)) treat the 80% rule as a rule of thumb that flags evidence of adverse impact. The employer can still show the procedure is job-related. Options A to C describe the rule correctly.

23. In Griggs v. Duke Power Co. (1971), the US Supreme Court held that:

    1. Aptitude tests are always illegal in hiring
    2. Only intentional discrimination breaks Title VII
    3. Requirements that exclude a protected group at a higher rate must be shown to be related to job performance
    4. Employers may use any diploma requirement they choose
    Show answer

    C. The Court said the diploma and test requirements had not been shown to relate to job success, and that practices fair in form but discriminatory in operation are covered. Tests were not banned, intent was not required, and diploma rules must also be job-related.

24. A call centre’s typing-speed cut-off is applied to everyone in the same way, but it screens out older applicants at a much higher rate. This is best described as:

    1. Disparate (intentional) treatment
    2. Range restriction
    3. Low inter-rater reliability
    4. Adverse impact
    Show answer

    D. Adverse impact (also called disparate impact) is a neutral rule with unequal results. Disparate treatment is treating people differently on purpose. Range restriction and rater reliability are measurement issues, not fairness outcomes.

25. Why do structured interviews usually predict job performance better than unstructured ones?

    1. They are shorter
    2. The interviewer can follow up on personal interests
    3. Every candidate gets the same job-related questions, scored on anchored rating scales
    4. They avoid asking about past behaviour
    Show answer

    C. Standard questions drawn from a job analysis and scored against fixed anchors raise both reliability and validity. Length is not the reason, free-flowing chat is what makes unstructured interviews weaker, and many structured interviews ask about past behaviour on purpose. More in structured vs unstructured interviews.

The four-fifths rule, worked through

The adverse impact question (number 21) is the one students get wrong most easily, because it is tempting to compare head counts. Work with rates. Divide hires by applicants for each group, then divide the lower rate by the highest rate. If the result is under 0.80, the procedure needs a closer look. It is not a verdict: the employer may show that the test is valid and job-related, and should also check whether an equally valid procedure with less impact exists.

How to write a selection and validity answer in your exam

Long questions on this topic look like “What is validity? Discuss its types with reference to employee selection” or “Differentiate between reliability and validity”. Examiners want definitions, the types, an example from a real hiring setting and a link between the two ideas.

Questions students ask

What is the difference between reliability and validity?

Reliability is consistency: the same person gets similar scores each time. Validity is whether the scores support the decision you make with them, such as predicting job performance. A test can be reliable and still invalid for a job, but it cannot be valid if it is unreliable.

What is the difference between predictive and concurrent validity?

Both correlate test scores with a criterion. Predictive validity tests applicants and measures performance later. Concurrent validity tests current employees and measures performance at the same time. Concurrent studies are quicker but suffer from range restriction, because current staff are already selected.

What is a good Cronbach’s alpha?

Many textbooks use .70 as a rough minimum for research scales and higher values for tests used to make decisions about individuals. Treat these as conventions, not laws, and remember that a very high alpha can mean the items are repetitive.

What is the four-fifths rule in simple words?

Work out the hiring rate for each group. If one group’s rate is less than 80% of the best group’s rate, that is generally treated as evidence of adverse impact under the US Uniform Guidelines. The employer then needs to show the procedure is job-related.

Is face validity a real type of validity?

Not in the technical sense. It describes whether a test looks relevant to the people taking it. That affects motivation and fairness perceptions, but it says nothing about whether the scores predict performance.

Which selection method is the most valid?

Older textbooks, using Schmidt and Hunter (1998), put general mental ability tests at the top. A 2022 re-analysis by Sackett and colleagues found structured interviews had the highest mean validity, with cognitive ability lower than previously thought. Learn both, and say which source you are using.

References

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. https://www.testingstandards.net/
2. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105. https://doi.org/10.1037/h0046016
3. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. https://doi.org/10.1007/BF02310555
4. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302. https://doi.org/10.1037/h0040957
5. Griggs v. Duke Power Co., 401 U.S. 424 (1971). https://www.law.cornell.edu/supremecourt/text/401/424
6. Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2(3), 151-160. https://doi.org/10.1007/BF02288391
7. Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563-575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
8. Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040-2068. https://doi.org/10.1037/apl0000994
9. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262-274. https://doi.org/10.1037/0033-2909.124.2.262
10. Taylor, H. C., & Russell, J. T. (1939). The relationship of validity coefficients to the practical effectiveness of tests in selection: Discussion and tables. Journal of Applied Psychology, 23(5), 565-578. https://doi.org/10.1037/h0057079
11. Uniform Guidelines on Employee Selection Procedures, 29 C.F.R. § 1607.4(D) (1978). https://www.law.cornell.edu/cfr/text/29/1607.4

Next: return to the I/O psychology MCQs hub, read the personnel psychology notes, or go on to the performance appraisal MCQs and the recruitment vs selection guide. For real hiring settings, see assessment centres and personality tests at work.