Business improvement through project management training

PHOTO: © NOLA VIGLIETTI / ADOBE STOCK.

Success in consulting engineering depends in part on the effective management of projects for (or on behalf of) clients. Most young engineers entering the field, however, receive little to no project management training during their academic schooling.

Such training is available through the Project Management Institute (PMI) and its book, A Guide to the Project Management Body of Knowledge. This process can lead to Project Management Professional (PMP) certification.

In consulting engineering workplaces, on the other hand, formal project management training is often absent. Instead, the relevant skills are acquired ‘on the job,’ under the guidance of an experienced project manager.

Such guidance, however, is not always sufficient. In response to an undesirable level of project write-offs, claims and client pressure, an international consulting engineering firm introduced a more formal, three-day project management training course.

The intent was to provide the knowledge and tools to help the firm’s staff become better project managers. In parallel with the course, a program of project management audits was also introduced.

These audits were undertaken in the firm’s offices every three months. They involved checking a random selection of projects against a 12-point list of essential project management documentation—e.g. signed agreement with conditions, authorization to proceed, approved change orders and review of final work product. This list was introduced as part of the course as a tool to guide the firm’s project managers.

While the firm’s stated goal for the course and the list was to improve project delivery, it never explicitly defined the desired outcomes. The implicitly desired and discussed outcomes were to reduce the project write-offs and claims and to increase profitability and client satisfaction. The success of the course was internally evaluated, using the participants’ course satisfaction ‘scores.’

After the course had been in place for about four years, five courses had been presented in one of the company’s major offices. Independent of the company, this author undertook an evaluation of the benefits of the course to that office.

This evaluation focused on the extent to which the course had resulted in:

• Participant satisfaction.
• Implementation of the requirements of the list.
• A decrease in write-offs and claims.
• A positive change in profitability and client satisfaction.

The process was undertaken using the Kirkpatrick Model’s four levels of evaluation for the course:

• Reaction (Level 1) – Did they like it?
• Learning (Level 2) – Did they learn it?
• Behaviour (Level 3) – Did they use it?
• Results (Level 4) – Did it make a difference?

Relevant skills are often acquired ‘on the job.’
PHOTO: © MOJO_CP / ADOBE STOCK.

Did they like it?

The firm’s target was an average participants’ satisfaction score above 7.0 out of 9.0 (i.e. 78%). Following the evaluation, the firm considered the course to have been a great success, based on the consistently high scores achieved and due to very positive feedback from the course participants.

Reactions were evaluated using the participants’ course satisfaction feedback sheets, which were made available in the subject office for all of the presented courses. Administered at the end of each course, they represented the limit of evaluation undertaken by the firm itself.

The sheets solicited feedback from the participants on the course content, presentation, format and venue and asked whether the course fell below, met or exceeded their expectations.

As mentioned, there was a high degree of participant satisfaction. The average scores for course content, presentation and format were 87%, 91% and 85%, respectively and the participants’ expectations were consistently met (54%) or exceeded (46%).

Project management effectiveness can be measured.

Did they learn it?

To evaluate the increase of the participants’ knowledge, a self-assessment questionnaire was designed.

It asked them to measure the extent to which their knowledge had increased in 26 specific areas, related to nine aspects of project management, using the following system:

• 1 indicates no (0%) increase.
• 3 indicates a moderate (50%) increase.
• 5 indicates a significant (100%) increase.

The knowledge questionnaire was sent to 71 individuals who had participated in the course in the subject office. Some 31 of these were completed and returned, for a response rate of 44%.

Five responses were from experienced and 26 from inexperienced project managers. In an attempt to ensure honest and unbiased reporting, the questionnaires were completed anonymously and returned to an impartial co-ordinator.

The inexperienced project managers reported an overall increase in knowledge of 60%. The two greatest areas of knowledge increase (78%) were understanding the importance of clearly defining the project scope and the value of regular communications with the client.

The experienced project managers reported an overall increase in knowledge of 28%. The greatest area of knowledge increase (56%) was in understanding the skills and attributes of a successful project manager. Other indicated areas of enhanced knowledge included satisfying the client, understanding the interface with the client, issues related to schedule achievability and the importance of implementing the list.

Did they use it?

A similar self-assessment questionnaire was designed to measure the degree of change in 15 specific project management behaviours of the participants, related to nine aspects of project management, again as per the following system:

• 1 indicates ‘never,’ no (0%) change.
• 3 indicates ‘sometimes,’ moderate (50%) change.
• 5 indicates ‘always,’ dramatic (100%) change.

The behaviour questionnaire was sent with the knowledge questionnaire to the same 71 individuals and saw the same response rate. The inexperienced project managers reported a 53% positive change in behaviours. The greatest change, at 72%, was in practising their delegation skills; while a 70% increase was reported for satisfying the client and paying more attention to exercising leadership skills.

The experienced project managers, meanwhile, reported a 38% positive change in behaviours. The two greatest changes in behaviour, both at 60%, were in better understanding the client’s organizational structure and improving communications with the client. A 52% increase was noted for referring to and implementing the list.

Post-training surveys provided valuable insight.
PHOTO: © DREAMSTUDIOS / ADOBE STOCK

Did it make a difference?

The most difficult part of the evaluation process was quantification of the results. There was a desire from management to prove the value—i.e. return on investment (ROI)—of the training in monetary terms.

The results were evaluated by considering the findings of the quarterly project management audits conducted in the subject office over a period coincident with the presentation of the courses. The office’s project write-offs and 12-month running average percentage of profitability over the same time period were also reviewed.

Unfortunately, work in progress and receivables were not included in the evaluation. In retrospect, these two elements would perhaps have been the best direct measures of project management effectiveness.

Over the seven quarters coincident with the presentation of the courses, project management audit scores increased from 74% and 69% in the first two quarters, respectively, to 94%, 79% and 89% in the last three quarters, respectively. Project write-offs were mixed at 0.88% and 1.19% (as a percentage of revenues) in the first two quarters, respectively, and at 0.76%, 2.18% and 1.76% for the last three quarters, respectively.

There was a 5% increase in the 12-month running average profitability of the office over the same period. There was also limited, anecdotal evidence of improved client satisfaction, based on client interviews.

Making a direct causal link between the course and changes in office profitability is difficult. Office profitability is primarily controlled by the cyclic nature of office revenues and the chargeability (i.e. billability) of staff. Another important consideration was the number of project managers who had participated in the course in the subject office.

Unfortunately, the benefits of behavioural changes due to the course are not immediate. Sufficient time must elapse between the course and the evaluation of its effectiveness, value and benefits, so as to allow the participants’ increase in knowledge and positive changes in behaviours, while working on new projects, to take effect and make a difference.

Indeed, it was difficult to provide proof of a positive ROI for the course. Rather, it was only possible to provide a weight of evidence that it had been a success.

Participants enjoyed the course and expressed satisfaction with it. Their knowledge increased and their behaviours underwent positive change.

It was particularly interesting to note the greatest increases in knowledge and behavioural changes, for both inexperienced and experienced project managers, were in ‘soft skills’ like leadership, delegation, communication, understanding clients and satisfying their needs. Experienced project managers significantly increased (52%) their knowledge of the importance of the list and changed their behaviour (52%) by increasing their reference to and implementation of it when managing projects. These findings confirmed the benefit and value of introducing the list as an effective tool for project managers.

Bryan Leach is a retired Calgary-based engineer who has been designated P.Eng. in Alberta and C.Eng. in the U.K. and now focuses on helping organizations learn. He can be reached at bryleach@telusplanet.net.