Level 1: Reaction and Planned Action. Reaction is defined as what the target population thinks of the program or project. Reaction is often a critical factor in adjusting or refining the performance solution. Planned actions add another dimension that shows how the population will implement a new requirement, program or process, or how they will utilize the capability of new technology. Also, the process of developing planned actions enhances the transfer of the business solution to the work setting and the internalization of change.
Level 2: Learning. A favorable reaction does not necessarily mean that learning or change will occur. A Level 2 evaluation is concerned with measuring the extent to which principles, facts, processes, procedures, techniques, and skills that are presented have been learned. It is more difficult to measure than reaction. The measures must be objective with quantifiable indicators of how new requirements are understood and absorbed.
- Most money spent
- Participants engaged
- Medium doesnt matter
- Learning issues
Level 3: Job Application/Implementation. Superior learning achievement during a program introduction or implementation does not always result in improved performance or changed behavior on the job. A Level 3 evaluation measures on-the-job behavior change after the business solution or program has been implemented.
- On the job
- Execution issues
Level 4: Business Impact. On-the-job behavior change may not produce a business impact. A business solution is usually initiated because one or more business measures are below expectation or because external factors threaten an organization’s ability to perform and meet goals. A Level 4 evaluation determines the business solution’s success in improving business performance. Typical objective measures include costs savings, output increases, timesavings or quality improvements. Typical subjective measures (equally important as objective measures) include customer satisfaction, employee satisfaction, customer retention, response time to customers, etc. Level 4 evaluation always involves linking the outcomes of the business solution to the appropriate business measures. It involves collecting data before and after the implementation of the business solution and analyzing the business performance improvement.
- If they execute it, what is the outcome?
Level 5: Return on Investment. An improvement in a business impact measure from a business solution may not produce a positive ROI. The ultimate level of evaluation is a comparison of the monetary benefits of the business solution with the cost of the business solution. This shows the true value of the program in terms of its contribution to an organization’s objectives. It is presented as an ROI value or cost benefit ratio.