لطفا منتظر باشید ...
What Can Be Tracked in e-Learning?
When considering what to measure in e-learning, you might find that you have more questions initially than you have answers. It is important to gain clarity on the answers you will require later to identify the types of questions you need to ask. These questions will provide the information required for the creation of a variety of reports. While this chapter is on tracking and reporting of e-learning, it is strongly recommended to approach tracking with a long term goal of measuring business impact and return on investment (ROI).
e-Learning Evaluation Questions at The Hartford *
The course was informative. The course met my expectations. The pace was appropriate. The training was appropriate for someone in my position. I will apply what I learned at my work place. I would recommend an online course to my friends/colleagues/acquaintances. The course content and activities were appropriately engaging. The course was easy to move through. The objectives of the course were achieved. Scale utilized: 1.Strongly Disagree, 2.Disagree, 3.Neutral, 4.Agree, 5.Strongly Agree.
*See Chapter 12:The Hartford Company
Chapter 2.
Figure 6-1: e-Learning Measurement Framework
Level 0, e-Learning Participation, has evolved as a very important measurement level for e-learning. As a comparison, measurement of participation in face-to-face classroom events was never difficult. If a classroom course was developed, it was just a matter of scheduling the course and signing people up. Education professionals calculated the number of attendees and classroom days, and handed out paper-based course evaluation sheets ( Level 1, Satisfaction) at the end of the class. The world of e-learning is very different. By solely making e-learning courseware available, there is no guarantee that people will take and/or complete the courses. Therefore, an understanding of the level of participation in e-learning courseware is the foundation metric before anything else can be measured. Compared with classroom, there are a number of different ways to measure e-learning participation. This can include the number of hits, downloads, live plays, orders, unique users, live e-learning attendance, or overall usage. These definitions will be reviewed in the next section.
Is it necessary to track all levels all of the time? No. In the early stages of defining an e-learning initiative, it is important to think through your measurement strategy. In some initiatives, you may decide to only focus on Levels 0 through 2. In other initiatives, you may focus on Levels 4 and 5. Table 6-1 provides guidelines for the recommended percentage of overall e-learning initiatives measured at each level.
Measurement Level | Percentage of e-Learning Initiatives |
|---|---|
Level 0 | 100% |
Level 1 | 100% |
Level 2 | 70 to 100% |
Level 3 | 30 to 70% |
Level 4 | 20 to 30% |
Level 5 | 5 to 20% |
It is recommended that for every e-learning initiative you capture data at a minimum for participation and satisfaction (Levels 0 to 1); whereas you may only measure Level 4 results for 20 to 30 percent of e-learning, and Level 5 ROI for 5 to 20 percent of your e-learning initiatives. These decisions are based on the following questions: How much is being invested in this initiative? What is the shelf life of this initiative? What is the targeted number of expected participants in this initiative? How critical is this initiative? Is it directly aligned with business goals? What is the key stakeholder's perspective on the initiative? Most organizations evaluate Levels 1 to 4 for a selected number of e-learning initiatives. However, many e-learning success stories are still focused on the results for Level 4 evaluation, primarily focused on the cost savings realized by utilizing e-learning in the place of physical classroom events. Despite energetic dialogue about ROI, most learning organizations have limited experience with ROI calculations. However, many involved in deploying e-learning initiatives intend to increase ROI measurements in the future to acquire or retain investment dollars for e-learning solutions. For more information on calculating the ROI of e-learning investments, see Chapter 2.
As with traditional classroom learning, completions are still an important metric to capture. However, there is one word of caution where e-learning differs significantly from classroom learning. Many people think that tracking completions is the only metric to truly measure success for a given program or class. When this approach is used, one of the most significant values of an e-learning strategy is missing, namely just-in-time learning, where the learner might only complete the aspect of the e-learning course that directly applies to their current work-related requirements.
e-Learning Measurement at Cingular Wireless*
“Cost effectiveness was a big one. For example, to deliver the nine compliance courses to the entire organization costs about $1.45 per employee. That's about $0.16 per course per employee to deliver, which represents a 90 percent cost reduction in content delivery if we had to deliver it in an instructor-led classroom format.” – Rob Lauber *See Chapter 13: Cingular Wireless
With this in mind, there are many other points to consider. Much of what is tracked is dependent on an organization's approach to e-learning. Whether the overall philosophy is to track specific courses and/or certification programs, provide access to a library of courses for just-in-time or developmental learning, or a mixture of the above, numerous metrics must be captured in order to validate the learning strategy and inform the business case.
