CEO Briefs - Evaluation of training; are you getting value for money?

Are you getting value for money from your investment in training?

If you’re spending money on staff training, it is reasonably expected you may be concerned about what you’re getting in return.

The problem 1

Training is almost always a welcome experience for most employees.  It is an opportunity to get  away from work, take some rest, engage in fun learning activities and in some cases, even travel to new places, eat sumptuous meals and meet new people.  This is why employees will seldom ever rate, an external training course especially, poor or unsatisfactory.

Sometimes, the persons that attend a particular training course do so for all the wrong reasons, including  “there was a training budget to be spent”; “my boss said I should attend”; “that is the training that was paid for by the funder on the project”; “everyone was required to attend”; “I requested for the training in my appraisal two years ago”; “it was what we could afford” and other imperfect reasons.

These reasons are certainly the reasons that a responsible Training Manager should be sending people on training for.

Impact on the quality of training

Notwithstanding, these reasons are responsible for a great amount of very poor training out there.  Because the trainers also knows that there are no accountabilities to measure up to, most do not feel obliged to put in the necessary work into their training; rather, they know that participants want “to have fun”, and so pack the training event with lots of fun activities; they know that participants want to travel and meet new people, so they host the training events in socially attractive and prestigious destinations; they know that participants don’t want to do any heavy lifting on the course, or after, and so they use simple methodologies that create comfort for all and leave everyone happy.

While learning is supposed to be fun, and yes, travelling, meeting new people and taking time off for new experiences elsewhere all indeed do have a place in personal, career and professional development, these are seldom the objectives that the Training Manager is paying for in the regular training programme.

Important considerations

Staff training and development is intended to bridge the gap between an employee’s current level of performance and a desired level of performance.  This understanding presumes that there is a prescribed (documented) level of performance that that the employee’s (documented) current performance is not measuring up to.  This gap is what gives shape to the learning objectives, content and methodology.

From a andrological point of view, the gap in question may be an absence or deficiency in cognitive awareness, an absence or deficiency in motor or intellectual skills or a misalignment of personal attitudes with the desired performance.  This is what needs fixing through management training programmes to achieve the desired levels of performance.

The problem 2

Because of a variety of reasons, however, particularly the quality of the organisation’s performance management process, quite often, both the current and desired levels of an employee’s performance in a particular area are only notionally understood, without a detailed assessment of the type, source and impact of performance concerns on the one hand and a detailed prescription of performance expectations on the other.

In these circumstances, matching performance concerns to training interventions can be difficult; just as expecting performance improvement after training interventions can be elusive.

This is why the Training Manager is obliged to comprehensive analyse performance concerns to determine if they are attributable to “an absence or deficiency in cognitive awareness; an absence or deficiency in motor or intellectual skills or a misalignment of personal attitudes with the desired performance”.  These are the only circumstances that render the performance concern “a training problem”.  There are a myriad other reasons why employees may not be performing at the expected level that are not training related, and which cannot, therefore, be addressed with training solutions.

What then, are the expectations that the Training Manager should set for training programmes

There are FIVE levels that the Training Manager should consider evaluating the training at.

Evaluation	Considerations	Implications of this level of evaluation
Reaction	This level of evaluation in interested in how the                      participants felt about the training, and specifically the    following, among other considerations: Did the they enjoy the training Did they consider the training relevant Was it a good use of their time Did they like the venue, hospitality and scheduling What did they feel about the level of participation Is it an experience the participants or others back at work would look forward to	It is best done immediately, at the end of the training It is easy and not expensive to collect and analyze It important to confirm whether the training was a pleasant experience from a customer satisfaction perspective
Learning	This level of evaluation is interested in the increase in         knowledge or intellectual capability obtained from the     training experience, and it focuses on the following,             among other considerations: Did the trainees learn what intended to be taught Did the trainees experience what it was intended they experience What is the specific change in the following targeted areas of impact:  Cognitive awareness Motor or intellectual skills Alignment of personal attitudes with the desired performance	This is best done after a month or so of the training It easy to administer through end-of-training tests and examination It could also be administered at the end of the training event It introduces measurement and evaluation of knowledge, skills and attitude change Is especially suitable for technical skills training It is not ideal for higher learning domains such as attitude development It is more elaborate and costlier to administer
Behaviour	This level of is interested in the extent to which the               trainees applied the learning they achieved in the                 intended areas and focuses on the following, among           other considerations: What was the demonstrable change in the trainees behaviour as a result of the training Is the trainee aware of their change in knowledge, skill, attitude or behaviour Was the change in behaviour and new level of performance sustained Did the trainees put their learning into effect when they got back to work What was the specific knowledge, skill or attitude applied in the targeted areas of improvement Was there a measurable change in the trainee’s performance in the targeted areas of improvement Would the trainee be able to transfer their learning to another person	This is best done after a reasonable  period of observation back at wor It gives meaning to the reaction and learning levels of evaluation of training It presupposes that after the training the trainees actively work in areas where the training acquired will be regularly demanded of It cannot be administered in the same way as the Reaction and Learning levels of evaluation It requires detailed measuring and evaluation instruments Its effectiveness depends on the skills and active cooperation of line managers It takes up management time to observe, measure and evaluate the employee's performance It takes longer to get feedback from It requires well thought-through post-training performance measurement and evaluation instruments
Results	This level of evaluation in interested in the immediate       impact the training had on the business, based on the         employees’ improved performance, and focuses on             only two things: In which key performance indicators (KPIs) of the business has there been recorded improvement as a result of the trainees’ improved performance What is the measurable degree of performance improvement in these key performance indicators (KPIs)	It is relatively easy to administer for simple (not complex) work environments with linear performance relaionships It requires the development of complex measurement and evaluation instruments that will reliably link the  performance of specific individuals to the organisation’s overall performance – even in specific KPIs Is not easy to determine in the context of multiple roles, multiple training programmes and multiple performance improvements Is easily disrupted by the scale and frequency of changing structures, roles and responsibilities within the organization, all of which make it difficult to validate accountability relationships for improved performance Both positive and negative organisational performance is also attributable to many internal and external factors, which makes it difficult to isolate and determine the real contribution of any one performance variable
Return on       investment	This level of evaluation seeks to determine whether             the investment in training was worthwhile and it                   focuses on the following: What were the total direct, indirect and opportunity costs of the training What are the total direct and indirect financial benefits the business has obtained as a result of the trainees’ improved performance How does the total cost of the training compare with the total financial benefits the business has obtained	This is ideally the ultimate test; are you getting value for your money? In the absence of clarity at the Results level of evaluation, however, it may not be possible to obtain any greater clarity at this level of evaluation.

 This conversation is based on Prof. Donald Kirkpatrick's Learning Evaluation Model, 1959

These perspectives of evaluation of training should ideally be discussed with the employee in advance of the training as well as with the trainer, in order that the necessary follow up expectations and supporting infrastructure is established well in advance of the intended training.

 

Where do we go from here

Though it is desired, it may not be possible to always do an end-to-end evaluation of every training programme.  What is important is for the Training Manager to have clear expectations on the outcomes of the intended training, to fully understand what it will take to achieve each, and most importantly, to develop the necessary capabilities to evaluate the training’s outcome at whatever level the Training Manager will have determined is satisfactory for them.

 We know that no one organisation is like another.  Our CEO Briefs are intended to give CEOs and other C-suite                  Executives short, informative perspectives and insights into what we know really matters to them, to hopefully both       provoke and inspire them to take necessary action in ways that best suite their circumstances.

ARTEMIS Transition Partners

Please contact us to discuss specific requirements

OR schedule an initial, no obligation, online meeting