Training Metrics and Return on Investment (ROI)
Show that your training investment has produced tangible results and a positive ROI
The Transportation Learning Center can provide technical assistance to measure your training metrics and ROI:
- Collaborate with operations, training departments and joint committees to assess impact and value of programs
- Identify sources of measurement data
- Collect and analyze pre and post training data
- Disseminate results in interactive presentations or formal reports
You never know how well you’re doing until you measure training outcomes. The Transportation Learning Center has capitalized on this insight through a series of in-depth research reports chronicling work by the Keystone Transit Career Ladder Partnership and Project Empire Transit Career Ladder Partnership and is now offering this service to transit agencies nationwide.
What to measure and how to measure it are critical questions when trying to determine if an investment in training is paying off for an agency. The Center’s expert research staff knows how to examine these questions to design the best measurement tools and find the quantitative and qualitative data that will show leadership, ridership, and funding sources that financial expenditures by the agency are paying off. Metrics established by the Center have a proven track record in assessing how well training programs meet their objectives, demonstrating program performance and outcomes to investors and partners, and supporting continuous improvement of the training program.
Read MoreThese metrics address the five levels of evaluation recognized in training literature, combining the four basic levels of learning measurement introduced by Donald Kirkpatrick in 1994 along with a fifth level added by Jack Phillips that addresses return on investment (ROI). These five levels of what has become known as the Kirkpatrick-Phillips Learning Measurement Model consist of:
Level 1 - Reaction: measures learners’ satisfaction with the program & captures planned actions.
Level 2 - Learning: measures changes in knowledge, skills and attitudes (competency).
Level 3 - Application: measures changes in on-the-job behavior.
Level 4 - Business Impact: measures changes in business impact variables.
Level 5 - Costs and Benefits Assessment: compares program benefits to the costs.
The specific measurements used in the transit industry with this model include:
• Supervisor and worker training satisfaction surveys.
• Promotion, retention and wage increases resulted from training.
• Pre-test and post test results and practical examination pass rates before and after training.
• Transit maintenance efficiency and quality improvements following training, including:
a. Mean distance between failures/road calls
b. Late pullout/Bus days lost/Runs cancelled
c. Maintenance cost per mile/vehicle operated/vehicle available (labor and parts)
d. Repeat and chronic mechanical failure/Part rework
e. Percentage of unnecessary part replacements
f. Percentage of preventive maintenance jobs performed on schedule
g. Ratio of scheduled versus unscheduled maintenance.
• Vehicle operator improvements resulting from training including enhanced customer service and satisfaction, more effective pre-trip vehicle inspections, driving practices that save fuel, improved vehicle handing in adverse conditions, etc.
• An assessment of the financial benefits and costs of training program.
In Pennsylvania, the Keystone Development Project and the Center helped introduce a unique partnership-based, data-driven training program that has produced very positive results for the state of Pennsylvania. A combined investment of $2.6 million in this training through state government funding and employer match has produced an estimated cost savings of between $10 million and $22 million in bus maintenance and fleet procurement over a four year period. The resulting four-year ROI is estimated between 293 and 745 percent. The annual ROI rate has climbed to a stable 5 to 12 times the investment in subsequent years.
As stated by Doug Greenwood, Director of Maintenance Cambria County Transit Authority, Johnstown:
“Everyone we sent [to Keystone training] has come back home showing results. Our electrical problems are fewer with faster repairs because they now know what to look for. Our A/Cs have become more reliable. All in all, this program is a plus.”
An ROI study can show your agency through a numerical analysis why a partnership-based training program is worth the time and expense. THE BOTTOM LINE in today’s economy: designed formulas to achieve top return.
