1. Introduction

As a result of a Fraud and Waste Hotline report, the Office of the Auditor General (OAG) conducted a review of the City of Ottawa’s (or “the City”) job evaluation processes.

2. Background and context

Job evaluation (or “JE”) is the systematic process of establishing the relative value or worth of jobs within an organization. The value of a job is based on an objective, consistent and rational measurement of job content and requirements. Job evaluation focuses on what is required to perform the job responsibilities and duties.  The JE process does not include a review of the volume of work or skills, abilities or performance of the individual employee who occupies the job. JE is used to provide the basis for an equitable and defensible pay structure, particularly in determining equal pay for work of equal or comparable value.

It is important to note that job evaluation is not a mechanism to adjust the salary of a position (or for an individual in that position) or for performance evaluation. Rather, it is the job that is being evaluated, not the person who is performing in the role. Other corporate mechanisms are designed to address, manage, recognize and reward an individual’s performance and achievements outside of the JE process.

The fundamental principles of JE are to ensure objectivity, independence and neutrality in the assessment of all jobs. JE, as a function, should: ensure all jobs are valued correctly, support internal equity and parity, and build the foundation for other employee-based programs and mechanisms in the organization (e.g., compensation, succession planning). 

Overview of the Job Evaluation Process

The JE function is governed by the Ontario Pay Equity Act; it allows the City, as an employer, to determine and manage fair and competitive compensation through a system that ensures internal relativity (within the City) and external relativity (in comparison with the external labour market). Job evaluations are conducted in the following scenarios:

• Creation of a job – a new position created for work that is not currently defined within the organization.
• Re-classification of a job - substantial changes to the duties and responsibilities defined within an existing job which potentially impact the scope and complexity of the work and requires a re-evaluation of the job.

A JE is triggered by the client department, when either the incumbent or supervisor identifies the need for a job review. The departmental representatives are required to work with the City’s JE staff to design jobs, assign duties and responsibilities, and prepare all job descriptions. The JE staff analyze job duties and responsibilities (specifically what is new or has changed), comparable jobs across the City, and organizational design principles to then assess and score the job against pre-defined factors. A rating is then determined which in turn corresponds to pay levels or grades.

Roles and Responsibilities within JE

The JE function is embedded within Human Resources (HR) and is overseen by the Director of HR. As part of a reorganization in 2019, the HR Branch implemented a decentralized organizational structure for the JE function whereby Organizational Design (OD)/JE staff were split.

OD/JE Consultants (“Consultants”) are housed within the HR Hubs (a combination of all HR functions to act as a one stop shop for client groups and are led by a Program Manager). Consultants are responsible for performing the JE analysis, working with the client department to develop the documents required to support the creation or re-classification of a job, and presenting their assessment to the Rating Committee.

OD/JE Specialists (“Specialists”) are under the Total Rewards Unit, a separate unit within HR (led by a Program Manager and overseen by a Manager). They are responsible for the governance of the JE function and for developing the JE Plans. They are also responsible for providing guidance and training to Consultants, dealing with complex files, and performing quality assurance (QA) activities (e.g., conducting reviews, playing a challenge function, performing consistency checks to ensure fairness).

Once materials are prepared by the client department and Consultant, they are provided to the Rating Committee which is responsible for reviewing and rating jobs (by consensus). The City has two types of Rating Committees: a Management and Professional Exempt (MPE) Rating Committee made up of City personnel, and a joint Rating Committee (made up of two City personnel and two representatives from the particular bargaining or affiliation unit) for unionized jobs. The Rating Committees are responsible for ensuring internal equity, corporate consistency and that the principles of the Pay Equity Act are maintained within the City.

3. Review objective and scope

The objective of this review was to assess the City’s job evaluation processes, for both the creation of new positions and re-classifications. More specifically, the review assessed whether:

• Job evaluation activities and processes have been established to support objective, fair, and transparent decision making.
• Job evaluations are conducted in accordance with job evaluation principles, City of Ottawa policies, and procedures.

Jobs within the City are assigned to bargaining or exempt employee groups. The scope of this review was limited to the MPE jobs evaluated by JE within the time period of January 1, 2019 – December 31, 2021.

The scope of this review did not include an assessment or verification of whether job duties in jobs descriptions were valid, or whether the incumbent was performing them as stated.

Readers are cautioned about the important distinction between a review and an audit.  Audits are designed to provide a high level of assurance over its findings and will typically feature rigorous testing and analysis.  While this review was conducted in a systematic and professional manner, the extent of activities undertaken by the OAG was relatively narrow compared to an audit.

Please see Appendix 1 for detailed review criteria.

4. Conclusion

The City has responsibilities and obligations as an employer. JE is a critical function in ensuring the City responsibly upholds equity and transparency, complies with applicable pay legislation and maintains strong relationships with employees. The City must guard against the JE process being used as a mechanism for increasing the salary of a position to reward the individual in the position for performance or to attract/retain individuals. This is the constant challenge for any JE function.

Based on the work conducted, we found that there is a need to strengthen the governance and control mechanisms within the JE function to ensure it supports objectivity and independence, while adhering to JE principles and best practices. This includes formalizing a QA function to mitigate any inherent conflicts of interest that could exist within the current structure. Further, the City should consider developing a JE policy and framework that governs JE, establishing the importance and reinforcing the purpose of the function.  

We noted that the process related to, and composition of, the Rating Committee should be reviewed to ensure it is supporting a timely, objective, and independent process. Additionally, the review found that a key control is missed when a job is not presented to the Rating Committee.

5. Findings and recommendations

5.i        GOVERNANCE, ROLES AND RESPONSIBILITIES AND AUTHORITIES

5.i.1    The client service delivery model can create inherent conflicts of interest for job evaluation

An effective JE function is objective and independent and structured in a way that can critically evaluate job descriptions, challenge client departments, and provide feedback that may not be popular to ensure JE principles are consistently adhered to.

In the current HR client service delivery model, Consultants are in place to support their client departments with their JE needs. This involves them:

• Assisting the client in drafting/revising job descriptions and related materials;
• Completing a thorough analysis of the job duties and relativity within the organization;
• Challenging the client as required;
• Assigning a rating to the job; and
• Presenting the job to the Rating Committee.

This model can create the inherent challenge or conflict of supporting the expectations of the client while ensuring the application of JE principles and undertaking sufficient due diligence. It was indicated that Consultants are not consistently playing the challenge function to their clients in instances where they may be feeling pressure. Without this challenge, materials submitted for review may not align with the established JE principles.

5.i.2   The job evaluation QA/challenge activity has not been formalized

An effective JE function is governed through a policy or framework that establishes its importance (while ensuring it is well understood by the organization) and defines the responsibilities and accountabilities of those performing the function in support of fair, transparent and prudent decision making.  

The review found that there is no policy or framework in place that governs the JE function and related processes. While the Manual of Maintenance Procedures and JE Plans for each bargaining unit provide high level roles and responsibilities of stakeholders involved in the evaluation, there is no overarching policy or framework that governs the function as a whole.  Additionally, it was indicated that while roles and responsibilities of JE staff have been documented, there is an opportunity to revisit these roles to ensure they are clearly defined and understood, are relevant and reflect operational requirements, and are embedded at the right level.

We understand that recently a QA process was introduced whereby Consultants are required to submit their Rating Committee packages to a Specialist for review and feedback in advance of presentation to Rating Committee.  The Specialist review is intended to support an independent QA and challenge function to:

• Ensure consistency, relativity, and objectivity;
• Be a mechanism to question and push back when they believe the ratings are not appropriate, while taking into consideration the overall City-wide context; and
• Identify where the client or Consultant have not sufficiently documented or substantiated the rationale or changes in the job duties.

During our file review, we noted instances where Specialists provided a challenge function but there was no evidence indicating that their feedback was considered. In these examples, feedback from Specialists included, but was not limited to, the following:

• the rating/paygrade was not adequately substantiated;
• the job duties and associated rating were not consistent with comparable jobs across the organization;
• the job did not meet organizational design guidelines; and/or
• the job duties did not demonstrate sufficient change to warrant a higher rating and paygrade.

In these instances, we did not see supporting documentation to resolve the questions and comments raised by the QA undertaken by the Specialist. The QA responsibilities of the Specialists have not been formally established within existing process/roles and responsibilities documentation related to JE. Without the role formalized, there is no current requirement for the Consultant to respond to the feedback provided from the QA nor is there a requirement for the Specialist to conduct the QA challenge on all files.

Conclusion

The objectivity and independence of Consultants could be impacted if there is a disproportionate focus on client service, resulting in potential conflicts of interest. This is further heightened in a model whereby the Specialists’ feedback is not consistently considered. Formalizing the QA function will allow the Consultants to provide strong client service while ensuring independence, objectivity and expertise is embedded within the process.

Recommendation 1 - CREATION OF A JE POLICY OR FRAMEWORK

The Director of HR should create and implement a standalone JE policy and framework that clearly outlines:

• Roles, responsibilities, and expectations for JE as a function, Specialists, Consultants, HR Director and Rating Committee;
• Roles, responsibilities, and expectations for client departments; and

Responsibilities and authorities to support an independent QA function for JE activities.

Management Response 1

Management agrees with the recommendation.

A standalone policy and framework will be developed to provide additional guidance and governance of the job evaluation function including:

• Roles, responsibilities, and expectations for JE as a function, Specialists, Consultants, HR Director and Rating Committee;
• Roles, responsibilities, and expectations for client departments; and
• Responsibilities and authorities to support an independent QA function for JE activities.

This work is expected to be completed in Q2 2023.

Recommendation  2 - FORMALIZE THE QUALITY ASSURANCE PROCESS

The Director of HR should formalize the QA process and define how the Specialists’ feedback will be taken into consideration. This process should include documenting the resolution of the QA review results.

Management Response 2

Management agrees with the recommendation.

As part of the standalone policy and framework to be developed in response to Recommendation 1, Human Resources will formalize the QA process and define how the Specialists’ feedback will be taken into consideration.

This work is expected to be completed in Q4 2022.

5.i.3   There is an opportunity to review the MPE Rating Committee review process and composition to strengthen its due diligence and transparent decision making

To support JE principles, the City has established a process whereby job ratings are required to be presented and approved by an independent Rating Committee. The decision of the Rating Committee is final and binding.

The MPE Rating Committee is made up of 3-4 senior executives from across the City who are responsible for reviewing and evaluating all MPE jobs according to the MPE Job Evaluation Plan/Manual. The Committee is chaired by a representative from HR, who also rates each job. The Rating Committee is provided a package that includes the job description, job evaluation questionnaire, organizational chart, a proposed rating sheet and comparable data (if applicable). The proposed rating sheet documents the JE analysis (i.e., scoring of each factor and rationale) and proposed rating they have assessed.

The review found that, during COVID, the MPE Rating Committee review process was not as robust as it is designed to be. The Rating Committee process is designed to have each member individually review the materials provided and determine their agreement for the scoring by factor, leading up to a rating. The members would be then expected to meet as a committee to review each member’s scores and arrive at a consensus. During COVID, meetings were not consistently held, and results were, at times, provided by email by each individual member with no formal discussion.

It was indicated during our review that there can be, at times, reliance from the Rating Committee on the proposed ratings provided by JE representatives based on their expertise and previous history of good work. Rating Committee members trust they are getting the right information and as such, can at times, sign off on ratings without any formal discussion.

While the current members of the MPE Rating Committee bring significant understanding of the organization to the table, they are senior executives and have been members for a long period of time. We understand that in the past the MPE Committee has had gender representation, but at the time of the review, it was an all-male committee. There is an opportunity to consider additional membership to support the Rating Committee responsibilities and process. 

Lastly, we noted that the feedback from Specialists is not provided with the Consultant’s proposed rating. In instances where there are disagreements on the rating, the Rating Committee would need that information to inform their decision making. Currently, the process has the Consultant presenting the job at Rating Committee and representing the client department. While the Specialists do attend the meeting, they are there to guide Rating Committee members through the evaluation process and tools. Existing process documentation does not establish the role for Specialists to provide their feedback to the Rating Committee; as such, there is a risk that Specialists may not feel comfortable doing so.

Conclusion

The MPE Rating Committee is intended to be an independent and objective review and approval mechanism for all job evaluations and is the key control within the process. More recently, as a result of significant pressures during the COVID emergency response as indicated by management, the Rating Committee has not consistently been meeting to arrive at a final rating and as a result, the intended benefits of a fulsome discussion to support consensus-based decision making may not be realized. Additionally, without consistent feedback from the Specialists’ review, Rating Committee may not have all the insights required to support their decision making.

Recommendation 3 - REVIEW OF THE RATING COMMITTEE PROCESS

The Director of HR should review the Rating Committee process to ensure it supports the Committee’s role and consensus-based decision-making process. This should include, at a minimum:

• Including the feedback resulting from the Specialist’s QA review in the Rating Committee package;
• Ensuring each job is discussed and rated by the Rating Committee; and

Ensuring the Rating Committee review is conducted via in person/virtual meetings to facilitate a fulsome discussion and gain consensus and that this discussion and consensus is documented and supported.

Management Response 3

Management agrees with the recommendation.

Human Resources will review the composition of the job evaluation package provided to Rating Committee members to ensure that it is a complete submission including relevant feedback from Consultants and Specialists. This work is expected to be completed by Q4 2022.

The Rating Committee has resumed meeting in-person or virtually as the City recovers from the COVID emergency response and re-establishes previously held practices.

Recommendation 4 - UPDATE OF THE RATING COMMITTEE COMPOSITION

The Director of HR should consider, and update as required, the composition of Rating Committee. This could include:

• A roster of members that JE staff can select from depending on the nature of job, the department, political sensitivity, etc., that are independent from the job being evaluated; and

Appropriate gender representation on the Committee. 

Management Response 4

Management agrees with the recommendation.

Human Resources recently began the process to renew the membership of the MPE Rating Committee following the retirements of several members and will continue to solicit members to ensure a gender-balanced roster of members from across the organization, including Director level employees. Training for new members will be provided. This will allow for a rotational roster of members to mitigate potential conflicts of interest, ensure quorum and bring additional perspectives to Committee. This work is expected to be completed by the end of Q4 2022.

5.ii.      PROCESS AND TOOLS

5.ii.1    Not all required MPE jobs are going to Rating Committee to be formally reviewed and rated

As described above, the City has established a process whereby jobs are required to be reviewed/rated and approved by the appropriate Rating Committee to support objective, transparent and consistent decision making.

There are instances where jobs can be interim rated (i.e., assigned a rating based on the Consultant or Specialist’s initial review) to support client departments when they have urgent operational needs/time constraints. An interim rating is assigned with the expectation that the job will be reviewed and approved by the Rating Committee as soon as possible.

We noted that some of the interim rated jobs have yet to be presented to Rating Committee almost three years later. There are currently no established timelines for presentation to the Rating Committee for interim rated jobs, nor is there a process in-place to monitor and ensure that these jobs are reviewed and approved in a timely manner.

Conclusion

As mentioned earlier, the MPE Rating Committee is intended to be an independent and objective review and approval mechanism for all job evaluations. It is the key control in the JE process.  Without all jobs being provided to the Rating Committee for evaluation in a timely manner, a critical step in the JE process is being missed.

Recommendation 5 – ENSURE ALL INTERIM RATED JOBS ARE REVIEWED AND APPROVED BY THE RATING COMMITTEE IN A TIMELY MANNER

For jobs that have been interim rated due to pressing needs and/or created on a temporary/time limited basis, the Director of HR should establish a standard to ensure that every job is presented to the Rating Committee within a specified time period. Additionally, the list of interim jobs should be provided to the Rating Committee and the General Manager (GM), Innovative Client Services Department (ICSD) on a regular basis. This will allow the members of the Rating Committee appropriate oversight of the jobs pending review and approval.

Management Response 5

Management agrees with the recommendation.

Human Resources will establish a service standard related to the completion of the job evaluation package for any jobs where an interim rating is required. An interim rating template will be developed to document the details of the interim rating, advise the client of the requirement and date for presentation to Rating Committee and ensure client acknowledgment of any additional work required to complete the JE file. This work is expected to be completed by the end of Q4 2022.

A report will be developed to provide the Rating Committee and the GM, ICSD with visibility on any outstanding interim ratings and dates that they are scheduled to be reviewed by Rating Committee. This work is expected to be completed by the end of Q4 2022.

5.ii.2     A process/mechanism to address exceptions within the standard JE process has not been formalized

In our file review, we noted instances where the determination of a rating for a job was “directed” despite the supporting documentation not necessarily being available to support that rating. While there may be a rationale for the direction, our file review did not identify evidence to support the decision made. In specific instances, the directed jobs have not been presented to the Rating Committee. In a limited number of cases, an exception form (signed off by HR Management) was on file.

For some directed cases, the job was treated as an exception and flagged as a non-comparator. A non-comparator is a job that has been deemed not appropriate to be used to substantiate a similar type of job given it has been determined to be an exception. There are only a limited number of situations where a job should be deemed an exception or a non-comparator. There are no established criteria to determine what is an appropriate exception and who has the authority to approve these.

A subset of the directed files we reviewed were jobs within HR. Our review found that there is no process or mechanism in place for additional oversight or challenge for internal HR jobs.

Conclusion

Without clearly defined criteria and process for the treatment of exceptions, there is a risk that decisions are made without the appropriate level of transparency. 

Recommendation 6 – IMPLEMENT A PROCESS AND CRITERIA FOR EXCEPTIONS

As part of the development of the JE policy/framework, the Director of HR should establish a process and criteria for the treatment of exceptions (i.e., jobs that do not get submitted to the Rating Committee, directed jobs). The process should define who has the authority to approve exceptions and under what circumstances.

Management Response 6

Management agrees with the recommendation.

As part of the standalone policy and framework to be developed in response to Recommendation 1, Human Resources will establish a process and criteria for the treatment of exceptions and the approval thereof.

This work is expected to be completed in Q2 2023.

Recommendation 7 – DEVELOP A PROCESS/MECHANISM FOR THE CREATION AND RE-CLASSIFICATIONS FOR INTERNAL HUMAN RESOURCES JOBS

The GM, ICSD, in consultation with the Director of HR, should develop and implement a process/mechanism for the creation and re-classifications of internal HR jobs that facilitates an independent review and approval process. Consideration should be given that the Rating Committee review and approve all HR job creations, prior to posting, and re-classifications prior to implementation of the revised job/pay grade.

Management Response 7

Management agrees with the recommendation.

The GM, ICSD and Human Resources will work collaboratively to develop a process to ensure that HR jobs are reviewed objectively and independently including consideration for an expedited Rating Committee review, if required. This work is expected to be completed by the end of Q3 2022.

5.ii.3   MPE Job Evaluation Plan should be reviewed

The City is required to have a JE Plan to support the review and rating of a job. The JE Plan is a point factor job evaluation system; factors that are valued by the organization are defined and scored using a points system. The points awarded under each factor are added together to establish the total point value, which determines the pay band for the job. For example, one of the seven factors defined for MPE jobs is “Accountability and Impact of Decisions”. It measures the extent to which the job is accountable for project/program management and/or the achievement of organizational objectives, and the impact of typical decisions (internal and external) made within the authorities of the job.

It was indicated that there have been challenges with the current MPE JE Plan when using it as a tool to complete job evaluations. The City has established categories of jobs (e.g., Program Manager, Manager and Director) that have an associated MPE rating level (e.g., Program Managers within the City typically are assigned a rating level of MPE 3 or 4). They do so to ensure consistency and minimize the number of one-off job descriptions.  However, this results in generic job descriptions that are not reflective of the actual job duties to be performed.  It is our understanding, through the conduct of our interviews and file review, that the MPE JE Plan can sometimes be used to lead to a desired outcome (i.e., rate the factors in a way to get the MPE 3 or 4 rating based on the title “Program Manager”) rather than using the factors to evaluate the job duties, responsibilities and skills required for the job.

It was noted that some factors do not reflect the current nature of work (e.g., factors more heavily weighted on the dollars and resources an individual is responsible for as compared to the outcomes they are responsible for). Additionally, some factors are inherently subjective and could potentially lead to varying interpretations and/or manipulations. As such, there is the potential for ratings to be inflated if one or two of these subjective factors are artificially rated higher.

Conclusion

The JE plans are the foundation for evaluating jobs. If they do not reflect the current realities of the work environment or are not used appropriately and/or consistently in conducting job evaluations, the intended outcomes from the JE process might not be realized.

Recommendation 8 – REVIEW THE MPE JE PLAN TO IDENTIFY OPPORTUNITIES FOR IMPROVEMENT AND SUPPORT CONSISTENT APPLICATION 

The Director of HR should contract a third party to conduct an independent review of JE Plan to:

• Identify any opportunities to strengthen factors to reflect the current nature of work at the City and to support consistent application of the JE plan in job evaluations; and
• Identify any risks related to pay equity noncompliance.
  
  

Management Response 8

Management agrees with the recommendation.

Subject to the allocation of sufficient funds as part of the 2023 budget approval, Human Resources will proceed with a competitive procurement process to engage a third party to undertake an independent review of the MPE JE Plan. This work is expected to be completed by the end of Q2 2023.

5.ii.4   There is no process to periodically review jobs

Other levels of government and large-scale organizations have a process by which all jobs are periodically reviewed to ensure job duties are accurately reflected and pay levels are appropriate based on job duties being performed.

The review noted that the City does not have a process to periodically review MPE jobs. A job can only be evaluated if a client initiates the process for either the creation or re-classification of a job. JE, as a function, does not currently have the authority to call in a job for a review, even if there are known issues or changes within the job that could impact the rating (and subsequently the pay). While we understand this can be a significant undertaking, it is a key mechanism and best practice to ensure pay equity across the organization.

Conclusion

Having a process to periodically review all jobs ensures that the City is doing its due diligence on an ongoing basis and demonstrates its efforts in being prudent with taxpayer dollars.

Recommendation 9 – PERIODICALLY REVIEW MPE JOBS

The Director of HR should develop and implement a risk-based process whereby all MPE jobs are independently and periodically reviewed every five to ten years. This will help ensure job duties and job descriptions are accurate and aligned to ensure proper spending of taxpayer dollars.

Management Response 9

Management agrees with this recommendation.

The Director of HR will develop and implement a risk-based process whereby all MPE jobs will be independently and periodically reviewed every five to ten years. This work will be completed by the end of Q4 2022.

6.  Appendix 1 – ABOUT THE REVIEW

Audit objectives and criteria

The objective of this review was to assess the City’s job evaluation processes, for both the creation of new positions and re-classifications.

The following table outlines our review criteria.

Review approach and methodology

The review methodology included the following activities:  

• Interviews and walkthroughs with select staff within job evaluation.
• Review of documentation (e.g., guiding documents, tools, etc.).
• Review and testing of a sample of job evaluation files.