Kirkpatrick’s Four Levels of Training Evaluation (With Examples)

This page is for I/O psychology, HRM and education students who must explain training evaluation in an exam, and for trainers who want a clear, honest version of the model. It covers each level with a workplace example, what to measure and when, Phillips’s ROI calculation with a worked example, the research criticism (Alliger and Janak, 1989; Holton, 1996), a 10-mark answer plan and practice MCQs.

Training evaluation is the last step of the training cycle that starts with training needs analysis. Both sit in the training unit of the personnel psychology notes.

Where did Kirkpatrick’s four levels come from?

Donald L. Kirkpatrick published a four-part series called “Techniques for evaluating training programs” in the Journal of the American Society of Training Directors, starting in November 1959 and continuing into 1960. Each article covered one step: reaction, learning, behaviour and results. The journal later became Training and Development. The framework spread quickly because it gave trainers a simple language for a hard question: did the training work?

Kirkpatrick later expanded the ideas in his book Evaluating Training Programs: The Four Levels, with a third edition written with James Kirkpatrick in 2006. The levels are now taught in almost every HRM and I/O course. A note on wording: “Kirkpatrick” is also used as a brand for training and certification programmes today. This page describes the published academic model only and has no link to any such programme.

The four levels explained, with examples

The running example: a bank in Lahore trains 120 branch staff in a one-day course on spotting suspicious transactions under its anti-money-laundering rules.

Level 1: Reaction

Question: How did the trainees feel about the training? Did they find it useful, relevant and well delivered?

How to measure: a short questionnaire at the end of the session (often called a “happy sheet”), with rating items and open comments. In the bank example: “The case studies were relevant to my branch”, “I can use what I learned this week”.

Why it matters and its limit: reactions are cheap to collect and warn you early if a course is confusing or irrelevant. But liking a course is not the same as learning from it. Alliger, Tannenbaum, Bennett, Traver and Shotland (1997) split reactions into affective reactions (did you enjoy it?) and utility reactions (was it useful for your job?). In their meta-analysis of 34 studies, utility reactions were more strongly related to learning and to on-the-job performance than affective reactions were. So if you only ask one kind of reaction question, ask about usefulness, not enjoyment.

Level 2: Learning

Question: What knowledge, skills or attitudes did trainees gain?

How to measure: tests before and after the training, skill demonstrations, role plays or case analyses. In the bank example: a 20-item test of suspicious-transaction scenarios given before and after the course, so the improvement can be seen. A control group of staff who have not yet attended makes the result stronger.

Its limit: passing a test at the end of the day does not guarantee the skill is used at the counter next month.

Level 3: Behaviour

Question: Do trainees use what they learned on the job? This is often called transfer of training.

How to measure: observation, supervisor ratings, work records, or 360-degree ratings, usually some weeks or months after training. In the bank example: compliance officers check how many suspicious transactions each branch reports correctly in the three months after the course, compared with the three months before.

Its limit: behaviour change depends on more than the course. If branch managers discourage reporting because it slows the queue, trained staff may not apply what they learned. That is why the work environment is part of any serious evaluation.

Level 4: Results

Question: What changed for the organisation because of the training?

How to measure: business outcomes such as quality, cost, safety incidents, customer complaints, sales or regulatory findings. In the bank example: fewer regulatory findings about missed suspicious transactions at the next audit.

Its limit: results are affected by many things at once (a new IT system, a change of manager, the economy), so it is hard to show that the training caused the change. Comparison groups and before-after designs help.

Level 5: Return on investment (Phillips)

Jack Phillips added a fifth level, return on investment (ROI), in his 1983 Handbook of Training Evaluation and Measurement Methods. Level 5 converts level 4 results into money and compares them with the full cost of the programme. The usual formulas are:

• Benefit-cost ratio (BCR) = programme benefits ÷ programme costs
• ROI (%) = (programme benefits − programme costs) ÷ programme costs × 100

Worked example: a hospital in Karachi spends PKR 2,000,000 on infection-control training for ward staff (trainer fees, materials and the cost of staff time away from the ward). Over the following year it estimates PKR 3,000,000 saved from fewer hospital-acquired infections, after isolating the effect of training from other changes. BCR = 3,000,000 ÷ 2,000,000 = 1.5. ROI = (3,000,000 − 2,000,000) ÷ 2,000,000 × 100 = 50%. For every rupee spent, the hospital got its rupee back plus 50 paisa.

The hard parts are counting all the costs, including participants’ time, and isolating the training’s effect from everything else that changed. The site’s utility analysis calculator uses a related I/O approach to estimate the money value of better selection and training decisions.

Kirkpatrick’s four levels: summary table

Criticism of Kirkpatrick’s model

Exam answers that only describe the levels get average marks. Add the research criticism to get the top band.

Alliger and Janak (1989): three problematic assumptions

Thirty years after the first articles, Alliger and Janak reviewed how the model was used and identified three assumptions that people commonly make about it:

1. The levels are arranged in ascending order of information, so each higher level is more informative than the one below.
2. The levels are causally linked: good reactions cause learning, learning causes behaviour change, and behaviour change causes results.
3. The levels are positively intercorrelated.

They examined the correlations reported in published studies and found weak support for these assumptions, especially for the link between reactions and the other levels. The practical lesson: a high score at level 1 does not tell you that levels 2, 3 or 4 will follow, so each level must be measured in its own right.

Holton (1996): a taxonomy, not a model

Holton argued that the four levels are really a taxonomy of outcomes (a list of kinds of result) rather than a full evaluation model, because they do not explain why training works or fails. He proposed a model that includes intervening factors such as trainees’ motivation to learn, trainability, job attitudes and the conditions for transfer back at work.

Other common criticisms

• Organisations measure mostly level 1 because it is cheap, and rarely level 3 or 4, so the model is often used in its weakest form.
• The model says little about the work environment (manager support, chances to practise), which strongly affects transfer.
• Level 4 results are hard to attribute to training without a comparison group.

Does training work at all?

Yes, on average. Arthur, Bennett, Edens and Bell (2003) meta-analysed organisational training studies and found medium to large effects across all four kinds of criteria: d = 0.60 for reaction, 0.63 for learning, 0.62 for behaviour and 0.62 for results. They also found that the training method, the skill trained and the choice of evaluation criteria were related to how effective training appeared. Salas, Tannenbaum, Kraiger and Smith-Jentsch (2012) reached the same practical conclusion in their review: training works, and how it is designed, delivered and followed up matters.

Practice MCQs: Kirkpatrick’s four levels

Practice questions written to the published syllabus, not past-paper items.

1. Which is the correct order of Kirkpatrick’s four levels?

   1. Learning, reaction, results, behaviour
   2. Reaction, learning, behaviour, results
   3. Reaction, behaviour, learning, results
   4. Results, behaviour, learning, reaction
   Show answer

   B. Reaction, learning, behaviour, results. The other orders mix up the sequence from immediate feelings to organisational outcomes.

2. Supervisors check whether trained nurses follow the new hand-hygiene procedure on the ward two months later. This is level:

   1. 1
   2. 2
   3. 3
   4. 4
   Show answer

   C. Use of the skill on the job is behaviour (transfer). Level 2 would be a test right after training; level 4 would be infection rates.

3. A fall in customer complaints after a service course is evidence at which level?

   1. Reaction
   2. Learning
   3. Behaviour
   4. Results
   Show answer

   D. An organisational outcome is a level 4 result. Complaints are not feelings, test scores or individual behaviour.

4. Who added return on investment as a fifth level?

   1. Holton
   2. Phillips
   3. Alliger
   4. McGehee
   Show answer

   B. Jack Phillips (1983). Holton and Alliger criticised the model; McGehee and Thayer wrote on training needs analysis.

5. A programme costs 400,000 and produces benefits worth 600,000. Its ROI is:

   1. 150%
   2. 50%
   3. 1.5
   4. 200%
   Show answer

   B. (600,000 − 400,000) ÷ 400,000 × 100 = 50%. 1.5 is the benefit-cost ratio; 150% confuses BCR with ROI.

6. Which is NOT one of the three assumptions Alliger and Janak (1989) identified?

   1. The levels are causally linked
   2. The levels are positively intercorrelated
   3. Higher levels give more information
   4. Reaction data should be collected by external evaluators
   Show answer

   D. Their three assumptions were ascending information, causal links and positive intercorrelation. Who collects reaction data was not one of them.

7. Holton (1996) argued that the four-level system is:

   1. a complete causal model
   2. a taxonomy of outcomes rather than a full evaluation model
   3. only suitable for technical training
   4. the same as Phillips’s ROI model
   Show answer

   B. Holton said it lists outcomes but does not explain why training works, and proposed adding intervening variables such as motivation to learn.

8. According to Alliger et al. (1997), which kind of reaction measure relates more strongly to learning and job performance?

   1. Affective reactions (enjoyment)
   2. Utility reactions (usefulness)
   3. Trainer popularity ratings
   4. Venue ratings
   Show answer

   B. Utility reactions were stronger correlates of learning and transfer than affective reactions. Trainer and venue ratings are affective-type items.

9. A trainer gives a test before and after a workshop. This mainly measures level:

   1. 1
   2. 2
   3. 3
   4. 4
   Show answer

   B. Pre- and post-tests measure learning. They do not show feelings (level 1), on-the-job use (level 3) or organisational outcomes (level 4).

10. Why is level 4 the hardest to evaluate?

    1. Trainees forget to fill in forms
    2. Organisational results are affected by many factors besides training
    3. It needs a written test
    4. It must be measured on the last day of training
    Show answer

    B. Many things change results at once, so attributing a change to training needs comparison groups or careful designs. Level 4 is measured months later, not on the last day.

More practice: the training and development MCQs set.

Questions students ask

What are Kirkpatrick’s 4 levels of training evaluation?

Level 1 reaction (how trainees felt about the training), level 2 learning (what knowledge and skills they gained), level 3 behaviour (whether they use it on the job) and level 4 results (what changed for the organisation). Kirkpatrick introduced them in 1959.

What is level 5 of training evaluation?

Level 5 is return on investment, added by Jack Phillips in 1983. It converts level 4 results into money and compares them with the full cost of the training, using the benefit-cost ratio and ROI percentage.

What are examples of each Kirkpatrick level?

Reaction: an end-of-course feedback form. Learning: a pre- and post-test. Behaviour: a supervisor checking whether trainees use the new procedure two months later. Results: fewer complaints, errors or accidents, or higher sales after the training.

What is the main criticism of Kirkpatrick’s model?

Alliger and Janak (1989) showed the levels are not reliably linked: good reactions do not guarantee learning, and learning does not guarantee behaviour change or results. Holton (1996) added that the model lists outcomes without explaining why training succeeds or fails.

Which Kirkpatrick level is most commonly measured?

Level 1, because reaction forms are quick and cheap. Higher levels need more time, comparison data and cooperation from managers, so they are measured far less often, even though they matter more.

How does training needs analysis connect to evaluation?

The needs analysis defines what the training should change, which gives the evaluation its targets. If the needs analysis says branch staff miss suspicious transactions, the level 3 and 4 measures follow directly from that gap.

References

1. Alliger, G. M., & Janak, E. A. (1989). Kirkpatrick’s levels of training criteria: Thirty years later. Personnel Psychology, 42(2), 331-342. https://doi.org/10.1111/j.1744-6570.1989.tb00661.x
2. Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(2), 341-358. https://doi.org/10.1111/j.1744-6570.1997.tb00911.x
3. Arthur, W., Jr., Bennett, W., Jr., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88(2), 234-245. https://doi.org/10.1037/0021-9010.88.2.234
4. Holton, E. F., III. (1996). The flawed four-level evaluation model. Human Resource Development Quarterly, 7(1), 5-21. https://doi.org/10.1002/hrdq.3920070103
5. Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of the American Society of Training Directors (four-part series, November 1959 to February 1960).
6. Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). Berrett-Koehler.
7. Phillips, J. J. (1983). Handbook of training evaluation and measurement methods. Gulf Publishing.
8. Salas, E., Tannenbaum, S. I., Kraiger, K., & Smith-Jentsch, K. A. (2012). The science of training and development in organizations: What matters in practice. Psychological Science in the Public Interest, 13(2), 74-101. https://doi.org/10.1177/1529100612436661

Back to the hub: Personnel psychology notes. Related: training needs analysis, performance appraisal errors and employee onboarding: the first 90 days.