How to measure eLearning ROI

Calculating eLearning ROI isn’t as simple as it sounds, but there is a lot at stake if it is ignored. American companies spent $92.3 billion on workplace training in 2021, and according to a Capterra study of HR leaders – 49% of organizations said they’d be increasing L&D spend this year. 

Measuring eLearning ROI can help learning and development teams provide evidence of its effectiveness for stakeholders, to justify the investment. The Society for Human Resource Management states that measuring eLearning ROI also helps:

• Quantify the effectiveness of eLearning and identify areas for improvement
• Determine specific outcomes in terms of changes in costs, time, and behaviors
• Build trust and credibility for the L&D function
• Determine the overall L&D budget

However, a 2021 survey found that only 8% of L&D teams calculate ROI for their programs. If measuring eLearning ROI is so important, why aren’t more organizations doing it?

Measuring eLearning ROI

There is widespread agreement in the L&D community that one of the greatest challenges in measuring eLearning ROI is the lack of standard metrics. The math involved in measuring ROI is simple, but it requires the identification and quantification of outcomes in monetary terms, which can be far from simple. There may be disagreement within the L&D team as to what benefits and costs should be included and how they should be calculated. 

Historically, the emphasis in calculating ROI has been on tangibles, such as time, salary, cash outlay, sales, and other factors that can be quantified and converted into dollar figures. But how can L&D teams assign monetary value to intangibles such as employee engagement, attitudinal changes, or the impact of eLearning on an organization’s culture, mission, people, and brand?

Isolating the impact of eLearning from all of the other factors that drive an organization’s performance and results can also be challenging – current economic and market conditions to name a couple.  

This uncertainty surrounding the quantification of costs and benefits can call ROI calculations into question, particularly when L&D teams lack experience in measuring eLearning ROI. Fortunately, L&D teams can find guidance for determining eLearning ROI in the framework below.

The Phillips ROI methodology

The best known model for measuring the effectiveness of training programs is Donald Kirkpatrick’s model, which has four levels: reaction, learning, behavior, and results. However, Kirkpatrick’s model provides little guidance for evaluating the extent to which a training program has met the expectations of stakeholders. It acknowledges that training evaluations often “falter at [level 4] because devising appropriate quantifiable measures of improvement can be difficult.”

Jack Phillips, PhD – director of the ROI Institute – considered this a limitation that could be overcome. He built on Kirkpatrick’s 4-level model to create his 5-level methodology. In truth, Phillips created a 6-level model by adding level 0 (inputs) and level 5 (ROI). He also developed some strategies for isolating the effects of learning from the effects of other factors like employee bonus programs, the competitive environment, and seasonality that leads to a temporary improvement in business performance. 

Such strategies include the use of control groups (comparing the performance of one group trained in a soft launch to that of another group that has not been trained), trend line analysis, forecasting, and estimates of the program’s impact from participants, supervisors, customers, and subject matter experts.

Level 0 of the Phillips model involves identifying direct and indirect cost elements that ultimately will enter into determining elearning ROI, such as the organization’s investment in the learning program and the number of learners who have completed it. 

Level 5 is divided into several stages for determining ROI: planning, data collection, analysis, and reporting. Those stages collectively comprise nine steps, outlined below.

Stage 1: Evaluation planning 

• Step 1. Develop project objectives. Identifying the objectives of the ROI evaluation is key to asking the right questions during data collection
• Step 2. Develop evaluation plans. This refers to plans for data collection, data analysis, and ROI calculation. 

Stage 2: Data collection

• Step 3. Collect data during and after project implementation. Use tools such as reaction surveys, polls, interviews, and focus groups in accordance with the data collection plan.
• Step 4. Isolate the training effect. This step helps determine the amount of improvement resulting from the program and ensure the credibility of the ROI calculation.

Stage 3: Data analysis

• Step 5. Convert training results to monetary value. Hard data is readily converted, and intangibles are identified below.
• Step 6. Capture training costs. Refer to the inputs identified at Level 0.
• Step 7. Identify intangibles. Convert to monetary value if feasible, or report as unquantified intangible benefits.
• Step 8. Calculate ROI. More on this step later in the article.  

Stage 4: Reporting

• Step 9. Develop a report for each audience and communicate results.

Because this ROI calculation is preceded by the four levels – evaluating reaction, learning, behavior, and results – L&D teams should be able to demonstrate that the business impact (positive or negative) was the result of the eLearning under evaluation. If the ROI does not meet expectations or is negative, the previous levels make it possible to determine the reason why. 

Calculating eLearning ROI

ROI typically is expressed as a benefits/cost ratio, or as a percentage. The formulas are similar. Calculating the benefits/cost ratio (BCR) is a simple matter of dividing the total program benefits by the total program costs. 

For example, assume that the benefits derived from an eLearning program developed internally and completed by 260 employees totaled $1.2 million in the first year. And that the total cost of the program in year one was $420,000. 

The BCR would be 2.86 to 1. For every dollar spent, the training program returned a whopping $2.86 in benefits to the organization. 

ROI, however, backs total costs out of the total benefits. To find eLearning ROI, divide the net benefits (total program benefits minus total program costs) by the total program cost. Then multiply the result by 100 to express it as a percentage.

Using the same numbers in the previous example for total benefits and costs, the ROI calculation looks like this:

With an ROI of 186%, the net gain was $780,000 out of the total return of $1.2 million. The return on each dollar spent was 186%.

When should you use BCR vs. ROI? BCR is often used to predict the benefits anticipated from a particular investment, while ROI is applied after the fact, once actual benefits have been realized and costs have been tallied. 

Calculating both eLearning ROI and BCR and comparing the two has the additional benefit of providing insight on the accuracy of the estimation process. One thing to be aware of is the possibility of optimism bias, which stems from the fact that people tend to overstate benefits and understate costs.

The benefits and costs of eLearning

Organizations are typically able to quantify the monetary benefits of eLearning in terms of cost savings when compared to other training methods, such as instructor-led classroom training. For example, they can calculate the monetary value of the reduction in instructional time by factoring in learner “seat time,” instructor preparation, platform, and follow-up time. The average reduction in instructional time and the overall cost reduction compared to instructor-led training are reported to be about 60% and 50% respectively. 

The benefits of learning gains can also be quantified through measurement of KPIs. But some benefits may be impossible to measure. For example, surveying an organization’s workforce may reveal a quantifiable increase in morale and employee engagement, but attributing such improvements exclusively to eLearning is another matter altogether.  

Phillips suggested that some benefits may simply need to be reported as intangible and not included in an ROI calculation, though they may serve as food for thought in evaluating eLearning initiatives. Some suggest using learner satisfaction ratings from post-training evaluations as a proxy for such intangible benefits. 

Try Big Think+ for your business

Engaging content on the skills that matter, taught by world-class experts.

Request a Demo

The costs associated with eLearning are typically easier to anticipate and quantify than its benefits. The costs related to technology include:

• Developing or acquiring digital content 
• External or internal content hosting 
• Distribution when hosting is external
• LMS costs 
• Hardware and software purchase and maintenance 

There are also eLearning costs related to L&D staff, such as the costs associated with content development, administration, consultancy, and providing support. These are readily quantified in terms of the number of hours of L&D staff involvement. The same is true of the paid time employees devote to eLearning.

Some of the information needed to assign monetary values can be obtained from systems maintained by Human Resources or Finance/Accounting. Organizations can also obtain data that is helpful for measuring eLearning ROI from their LMS, for example:

• Time spent on individual courses by learners
• Level of learner engagement
• Results of assessments 
• Progress and course completion by individual learners
• Course popularity
• Student feedback and survey results

There are a number of tools online that make it easier to calculate eLearning ROI. The ROI Institute offers free calculators for monetary benefits, program cost, ROI and BCR, and turnover cost. Another useful tool is a standard ROI calculator, like this one. 

Final note

Measuring eLearning ROI is the best way to prove its value. To make a strong case for eLearning to stakeholders, the measures used should be those that are most important to the organization. This might be the amount of time it takes learners to complete an eLearning course, or the amount of time it takes L&D to turn a storyboard into a live module. 

Figure out which impact from eLearning is most closely related to stakeholder expectations, and make that the focus of efforts to measure ROI.