Home Page spacer
Outcomes

Initial Plan

An important element of the DAWN Project was for there to be verifiable outcomes regarding the efficacy of DAWN sponsored training.
The PLAR process was designed to assess the training needs of the workplace, as well as the current skill set of the employee.  This process was to consist of:

The initial plan regarding post assessment was that the same tools would be used to measure the skill set of participating employees at the conclusion of the DAWN sponsored training.


Challenges/Learnings

The first group of workers were assessed by DAWN staff, using CLB tools, and TOWES self-assessment tools that are available online.  As the project progressed, an agreement was reached with one of the DAWN partners, Pembina Valley Language and Education for Adults, for their certified CLB testers to conduct assessments on DAWN participants.  When it came time for post assessments to be done, it was made clear by the CLB testers that the CLB tests were intended to be used only as a pre-assessment tool.  The certified testers were instructed by CLB not to conduct post-tests.  Similar difficulties were identified with the use of the TOWES tests as post-assessment tools.
DAWN staff met with the DAWN Essential Skills trainers to discuss the use of these tools.  The ES trainers found that the TOWES testing documents did not test what had been learned.
In light of these findings it was decided that the CLB and TOWES testing would not be used as post assessment tools.
During these meetings, there was discussion regarding what exactly we wanted to measure.  We started by looking at the individual training plans that had been developed for each worker.  Operational goals were stated in terms of Essential Skills.  Each training plan identified specific expected outcomes in some of the nine Essential Skills. 

It was agreed that what we wanted to know is;
To what degree did the worker meet the stated goals of the training plan?  The question arose as to how to assess specific skill acquisition in general terms.  A scale was developed whereby the trainers would assess the degree of success in meeting the outcomes, which were stated in terms of the 9 Essential Skills:

It was also determined, as the project progressed, that the most useful tool in terms of assessing employee needs was the individual interview.  These interviews, conducted by the Skills Developer, were conducted in a collaborative manner with the workers.  The most useful question asked was, “What are the challenges you have in your current job?  During the interview process, the Skills Developer was able to build a rapport with the worker where it was not uncommon for the worker to express challenges they were aware of, but which there was a reluctance to share with their employer. 


Best Practices

Based on what was learned, it was decided that CLB and TOWES testing would be used only as pre-assessment tools, to assist in the development of the individual training plans.  The CLB testing was also used as an indicator for eligibility for DAWN sponsored training.  It had been agreed that trainees should be functioning at a minimum level of CLB 3.  If trainees were assessed at a lower level, they were referred to further EAL training. 
The individual interviews conducted by the Skills Developer were considered to be the most effective pre-assessment tool.  The DAWN Outcomes “before” and “after” employer assessments of the worker’s value to the company were the primary evaluation tool in assessing the effectiveness of training.  The Individual Outcomes Report assessment tool developed by DAWN staff and trainers provided an effective analysis of the extent to which training outcomes were achieved.

(These tools are listed in the Project Implementation section)

The analysis of data collected and recorded in the database will be covered in the “Database” section of this report.

PROJECT RESULTS

Expected

Delivered


Expected

Delivered


Expected

Delivered


Expected

Delivered


PARTICIPANT TESTIMONIALS

Documents

Cornie Klassen

Daniel Kauenhofen

Geoffrey William

DAWN testimonials

Videos (Click to view)

Daniel Kauenhofen (Lode King)

Viktor Friesen (Convey-All Industries)

Cornie Klassen (Conquest Mfg.)

Geoffrey William (Jodale-Perry)

Gloria Lee (Employer's perspective)

ANALYSIS OF OUTCOMES

Vector Cost Analysis

Vector Cost Analysis

The bulk of training costs (60%) was allocated to Converging: i.e. Costs associated with worker time spent in training directed at acquiring new skills. This included worker wages, and trainer expenses, including preparation for training.   
Bridging (28%) was the next largest identified expense: i.e. Costs associated with acquiring new skills and new workplace cultures, including technical writing skills, and the technology associated with these skills.
Costs associated with assessments of workers, and compensation for management and supervisors in the organizing of training and mentorship of trainees was allocated to Diverging.  This represented 16% of the overall cost.
Integrating (5%) represented the smallest portion of training expenses.  This included costs associated with the usual, routine, and ongoing training needed by all workers on health and safety, including safe work procedures.

Time Cost Analysis

Time Cost

An analysis of costs based on the factors identified by the employers as being the most important considerations when planning for training showed a fairly even split between Worker Time and Training Costs.    The costs related to the delivery of training comprised 45% of the overall costs.  This included Essential Skills Instructors’ preparation, and instructional time, as well as internal trainer wages, and outsourced training costs.  The cost of training materials was also included in this category.
The time the trainee spent in training initiatives that took them away from their regular duties, including pre and post assessment time, accounted for 41% of the overall costs. 
Management Time Costs, for hours spent by management in the planning and monitoring of the training process represented a much smaller portion of costs, at 6.5%. 
Job Mentoring Time was not a significant portion of allocated training time.  Not all trainees were allocated this training expense, and those who were, usually consisted of only 1 hour/week for supervisors for debriefing their workers.

Before/After Outcomes Analysis

EMPLOYER BENEFITS

Employers were asked to assess their employees regarding:

  1. Workplace Knowledge;  i.e. the worker’s knowledge of their duties in the workplace
  2. Production Value;  i.e. the worker’s production levels
  3. Quality Value;  i.e. the quality of the worker’s work

Pre and post training assessments indicated whether the employee did not meet standards, met standards, or exceeded standards.

Employers were categorized into sectors:
S1:  Metal Sector                              S2:  Wood Sector                              S3:  General Sector

Employee Performance

Prior to training, the overall average of workers’ workplace knowledge, production value, and quality value indicated that on the whole, they were below company standard.
After training, employer assessment of workers indicated that overall, the workers were above standard.
This clearly shows that the training interventions had a positive effect on worker capacity to meet company standards.

Mutual Benefits

Employers were also asked to assess workers regarding;

  1. Skill Sets/Duties;  i.e. the relative fit between the skill set and job duties
  2. Stability & Growth Opportunities with Employer; i.e. the estimated level of opportunity for the employee within the company.

Pre and post assessments indicated whether the relative “fit” between the employee’s skill set and the required duties, and the stability and growth opportunities with the company were poor, acceptable, or good.

Mutual Benefits

Prior to training, the relative “fit” hovered just above or below the acceptable level.
Post training assessments indicated an overall increase well within the acceptable level.

Overall, there was an improved perception by employers of the capacity of workers to meet the demands of their jobs.

Employee Benefits

Employers were asked to indicate the rate of pay and the hours of work per week before and after training, and to indicate whether or not there was a position change.
Data was recorded on a yes/no basis; i.e.

  1. Was there a change in pay?
  2. Was there an increase in the hours worked?
  3. Was there a position change?

Employee Benefits

Although a slight increase can be seen in the “after” data, it should be noted that the second question, regarding the number of hours worked, did not provide meaningful data.  Generally speaking, workers involved in DAWN training were working a full 40 hour week before training, and continued to do so after training.
Wage increases that were indicated after training, could not always be attributed to the training.  Employers indicated that when a wage increase occurred, it was more often than not attributed to experiential factors.

The most significant data in this section was the question regarding position change.  The increases visible in the graph were mostly due to position changes/promotions that came about as the result of DAWN training.

Essential Skills Training Analysis

After the decision was made not to use CLB and TOWES as post assessment tools (*see Outcomes Analysis section), a tool was developed to measure the degree to which the worker met the stated goals of the training plan. 
A scale was developed whereby the trainers assessed the degree of success in meeting the outcomes, which were stated in terms of the 9 Essential Skills:

Essential Skills Training Analysis

This chart clearly demonstrates that across the board, workers involved in DAWN training met the outcomes as stated on their individual training plans.

ANECDOTAL RESULTS

In addition to the information recorded on the database regarding outcomes analysis, information was gathered through the monitoring process and through exit interviews with employers.
Following is a compilation of comments made by employers/employees regarding DAWN sponsored training.

EMPLOYEE COMMENTS:

Increased opportunities for immigrant employees:  Many of the immigrant employees who were selected for training, were not previously identified as having potential

Increase in pay and/or positions;  many of the employees have received pay increases after successful DAWN training, and some have earned promotions – some employees who had been considered to be “at risk” have become valued employees, with increased job security

Commitment to lifelong learning; DAWN participants appreciated the opportunity for advancement, and expressed interest in further learning opportunities

More safety awareness

EMPLOYER COMMENTS:

SUMMARY

When DAWN was first conceptualized in 2007, small and medium manufacturing enterprises in the Pembina Valley were thriving.  There was, however, a chronic shortage of skilled workers in the manufacturing sector, and this area therefore became the first in Manitoba to actively recruit immigrant workers through the Provincial Nominee Program.  As the immigrant workforce in the area increased, challenges were identified. 
It was the intent of DAWN to develop a common framework for skills assessment and training.  Through the process of determining the base skill set of immigrant workers, and identifying gaps between this skill set and the workplace needs, DAWN would assist employers with addressing these needs through appropriate training interventions.  The goal was to optimize the current and future contribution of immigrants in the workplace.
During the 2 year time frame of the DAWN Project, 19 manufacturing employers participated in DAWN assessment and training initiatives.  Feedback from these employers noted an increase in employee efficiency and productivity.  Contrary to initial concerns that they would lose the employees they trained, employers recognized that they gained more capable and effective employees who have remained loyal to their company.   Comments made by employers indicated that what they learned through DAWN participation about Essential Skills, NOC codes and job profiles has helped them improve how they write job descriptions, post job ads, and how they track employee skill acquisition.
The number of immigrant employees impacted by DAWN vastly exceeded the initial expectations.  The total number of trainees was 160 – double the expected 80.   Feedback provided by these participants highlighted the importance of DAWN sponsored training activities.  Participants felt valued by their employer and expressed an increased confidence in their abilities.  Not only did training result in improved Essential Skills in the workplace, but at home and in the community as well.  DAWN participants appreciated the opportunity for advancement, and expressed interest in further learning opportunities.
The DAWN Project has helped Pembina Valley manufacturers to experience firsthand the benefits of training in the workplace.  They have seen that training results in more capable and efficient employees, which results in increased productivity.
It is hoped that the legacy of the DAWN Project will be that Pembina Valley employers will continue to provide training opportunities to develop a skilled labour force as they strive for corporate success.

Valid XHTML 1.0 Transitional Valid CSS!

185 main street
winkler, manitoba
r6w 1b4
ph. 204.325.7261
fax. 204.325.8290