Final Report

Prepared for : NSERC-SSHRC Evaluation Division

Prepared by: Goss Gilroy Inc.
Management Consultants
Suite 900, 150 Metcalfe Street
Ottawa, ON K2P 1P1
Tel: (613) 230-5577
Fax: (613) 235-9592
E-mail: ggi@ggi.ca

Date: June 14, 2016

PDF Version

---

Executive Summary

Introduction

This report on the evaluation of the Canada Research Chairs Program (CRCP) assesses the relevance and performance of the CRCP, covering the five core issues set out by the Treasury Board Directive on the Evaluation Function, over fiscal years 2010-11 to 2014-15. The evaluation targeted two areas of primary focus: 1) The relevance of the current objectives of the CRCP and possible emerging/new objectives; and 2) The extent to which the CRCP is contributing to institutional strengthening of strategic research areas.

The CRCP is a tri-agency program launched in 2000 that was designed to enable Canadian universities and affiliated research institutes and hospitals to foster research excellence and to enhance their role as centres of research excellence in the global, knowledge-based economy. A total of 2,000 Chairs are allocated to eligible institutions, based on the research grant funding received by institutions from the tri-agencies in the three years prior to the year of the allocation. Canada Research Chairs are divided into two tiers:

• Tier 1 Chair awards are worth $200,000 annually, tenable for seven years and renewable indefinitely; and
• Tier 2 Chair awards are worth $100,000 annually, tenable for five years and renewable once.

Evaluation Methodology

The evaluation employed seven lines of evidence: namely, surveys of institutions and chairholders, a bibliometric analysis, case studies, key informant interviews, document review, administrative data review and cost-efficiency analysis. The evaluation methods were carried out by a hybrid team composed of internal evaluators at the Natural Sciences and Engineering Research Council’s and Social Sciences and Humanities Research Council’s Evaluation Division and two external consulting firms.

A few challenges and limitations were encountered during the evaluation. The review of external documentation was not exhaustive, and in some areas was unable to uncover significant amounts of more recent documentation (from 2010 onward). For example, there appears to be a lack of more recent research on migration of highly skilled researchers into and out of Canada. CRCP funding data was incomplete for 2014-15, and consequently was excluded from analysis. For interviews and case studies, knowledge of the program differed by respondent type; respondents were therefore asked to comment only on aspects with which they were most familiar.

Findings and Recommendations

The CRCP is cost-efficient and, in most areas examined, effective and relevant. The program is selecting the right nominees for the award, and is contributing towards research production, research centres, collaborations, and high quality training for students, postdoctoral researchers and others. The program is aligned with federal and tri-agency priorities and the objectives of the program remain relevant.

Despite this, the chairholder package—the benefits provided to chairholders by the institution—is diminishing. This trend is largely driven by the fact that the award value has not changed since 2000 and is therefore diminishing in real terms. CRCP funds and institutional funds are both increasingly being used to cover chairholder salaries, resulting in less funding available for other components of the chairholders’ package, such as research funding. This may present a risk to relevance, and may be linked to some observed challenges to program effectiveness, namely challenges in recruitment and turnover among chairholders from abroad, and an increase in Chair vacancies.

Strategic Research Plans (SRPs), which are created by institutions to outline their research priorities, may no longer be a useful tool to ensure CRCP funds are being used strategically. Despite this, the evaluation found evidence that the CRCP does in fact contribute to institutional investments in strategic areas of research.

While most components of program design and delivery were found to be appropriate, a few areas were noted to require attention. More work is needed to address barriers of access to CRCP for designated groups, as the majority of institutions do not currently meet targets. Secondly, the Chair re-allocation process can cause challenges for institutions when it results in the loss of an occupied Chair. Finally, the possibility of indefinite renewals for Tier 1 chairholders may be creating an increasingly large age gap between Tier 1 and Tier 2, reducing the ability of the CRCP to attract Tier 1s earlier in their career.

The evaluation offers the following seven recommendations, resulting from the evidence described above.

Recommendation 1: Management should investigate the feasibility of increasing the award value and/or indexing it to the inflation rate.

Since program inception, the CRCP award value has diminished by a third in real terms. This, in turn, contributes to a diminishing chairholder package, which threatens the achievement of all four CRCP objectives.

Recommendation 2: Management should examine options to ensure more robust chairholder packages are offered by institutions, in order to support the continued achievement of the program’s objectives in the future.

Although there is an expectation that a package should be provided to chairholders, this package has been diminishing in many institutions, driven in part by the declining value of the award. The diminishing chairholder package may threaten the achievement of all four of CRCP’s objectives:

• it may reduce the award’s competitiveness, affecting its ability to attract and retain (Objective 1);
• it may inhibit training of HQP (Objective 2) and/or research (Objective 3); and
• if the perceived difference between a Chair position and a regular faculty position diminishes, it may affect the extent to which the awards can be used strategically (Objective 4).

The options examined for this recommendation will be partly informed by the outcome of Recommendation 1 (see above) and management will need to consider the role of institutions in responding to this recommendation.

Recommendation 3: Management should set targets for the attraction of researchers from abroad, and, if necessary, develop a plan to better support this group in order to ensure that the program’s objective of attracting top researchers from outside Canada is realized.

In the past five years, there has been a large reduction in the amount of researchers from abroad being awarded a Chair. At the same time, chairholders from abroad are more likely than others to resign their award. Attraction of top researchers from outside Canada is one of the expected outcomes of the program and was a key impetus for the creation of the program in 2000. Management should examine the extent to which the evaluation findings on researchers from abroad pose a risk to the program’s objectives, and set targets for international attraction. If current levels are lower than the targets set, a plan should be developed to encourage greater attraction and retention of researchers from abroad.

Recommendation 4: Management should examine the extent to which the “fit with SRP” review criterion remains relevant, and whether or not it should be replaced with a criterion assessing alignment to demonstrated priority areas.

At some institutions, the extent to which Strategic Research Plans (SRP) are used to inform strategic action is in question. An institution’s strategy for allocating Chairs may not be explicitly reflected in its SRP, which may be kept broad to ensure maximum flexibility of funding use. If the review criterion is not changed, the program’s peer review process may be less effective in supporting CRCP’s Objective 4 (best possible use of research resources through strategic planning), and if the SRP does not represent the best indicator of an institution’s priority research areas.

Recommendation 5: Management should require institutions to adopt greater transparency in their processes for allocation of Chair positions and selection and renewal of chairholders, in order to ensure institutions have greater accountability in terms of meeting their equity targets.

Despite work done to date by the Secretariat to promote equitable practices, the majority of institutions do not meet targets set for women, visible minorities and persons with disabilities. Program management should investigate options to require institutions to adopt transparent processes and criteria, including a requirement that such processes be made publicly available.

Recommendation 6: Management should investigate ways to minimize the impact of loss of occupied Chairs in the re-allocation process.

Loss of occupied Chair positions presents a challenge for all institutions, especially for small institutions. Program management should find ways to diminish the need to take away occupied Chair positions. Potential solutions to be explored may include:

• carrying out an annual recalculation to provide advance notice of impending gains and losses;
• removing occupied, active Chair positions from the re-allocation formula; and
• changing the rolling average calculation to smooth out the effect of sudden, large changes.

Recommendation 7: Management should consider imposing a limit of one renewal for Tier 1 Chairs.

There is some evidence of an age gap between Tier 1 and Tier 2, which is due to the renewal of Tier 1 chairholders, and which is expected to increase in the near future. This introduces a risk that a growing number of researchers may fall into a gap between these two groups. Furthermore, the fact that the majority of Tier 1 Chair awards are now held by renewed chairholders means that institutions have fewer opportunities to attract new researchers through the Tier 1 award.

Acronyms

CAUT	Canadian Association of University Teachers
CERC	Canada Excellence Research Chairs
CFI	Canada Foundation for Innovation
CFREF	Canada First Research Excellence Fund
CIHR	Canadian Institutes of Health Research
CRCP	Canada Research Chairs Program
HQP	Highly qualified personnel
IAC	Interdisciplinary Adjudication Committee
NSE	Natural sciences and engineering
NSERC	Natural Sciences and Engineering Research Council
PMS	Performance Measurement Strategy
SRP	Strategic Research Plan
SSH	Social sciences and humanities
SSHRC	Social Sciences and Humanities Research Council
TIPS	Tri-agency Institutional Programs Secretariat

---

1.0 Introduction

1.1 Purpose

The fifteenth-year evaluation of the Canada Research Chairs Program (CRCP) is required as per the evaluation coverage requirements stipulated in the Treasury Board Policy on Evaluation and with respect to section 42.1 of the Financial Administration Act. This evaluation will assess various aspects of the relevance and performance of the CRCP, covering the five core issues set out in the Treasury Board Directive on the Evaluation Function (2009). The five-year evaluation coverage time frame has been set from fiscal year 2010-11 to fiscal year 2014-15.

The CRCP has been evaluated or reviewed three times since its inception in 2000. The current evaluation is the 15th year evaluation of the program. During the design of the evaluation, the following two key areas of focus emerged:

• Exploring the relevance of the current objectives of the CRCP and exploring possible emerging/new objectives that could lead to changes in the program logic and design.
• Exploring the extent to which the CRCP is contributing to institutions strengthening their performance in strategic areas of research, and how it can work better with other programs to achieve this outcome.

To the extent possible, evaluation evidence was used to address these key areas. Because the previous (2010) evaluation examined questions on performance in great depth, these data were used as a baseline for longitudinal comparison for the current evaluation.

1.2 Overview of the Program

The CRCP is a permanent program launched in 2000 that was designed to strengthen Canada’s research capacity and offset “brain-drain” pressures by helping Canadian universities and their research affiliates retain talented Canadian researchers and attract the best international researchers to Canada. The program’s budget was $300 million per year until 2010-11, after which it was decreased to $265 million.

The program is a tri-agency initiative of the Canadian Institutes of Health Research (CIHR), the Natural Sciences and Engineering Research Council of Canada (NSERC), and the Social Sciences and Humanities Research Council of Canada (SSHRC); the Canada Foundation for Innovation (CFI) also provides accompanying funds to support infrastructure. Housed within SSHRC, the Tri-agency Institutional Programs Secretariat (TIPS—formerly the Chairs Secretariat) is responsible for the day-to-day administration of the CRCP.

The main objective of the CRCP is to enable Canadian universities, together with their affiliated research institutes and hospitals, to foster research excellence and to enhance their role as centres of research excellence in the global, knowledge-based economy. More specifically, the program intends to achieve the following:

• Increase Canada’s research capacity. The program will increase Canada’s research capacity by attracting and retaining 2,000 top researchers from within Canada and abroad.
• Improve training of highly qualified personnel (HQP). The program will increase the number of HQP trained through research in Canadian universities. It is also expected that the quality of training will be improved.
• Improve universities’ capacity to generate and apply new knowledge. By attracting and retaining top researchers, Canadian universities will be able to increase research outputs as well as the dissemination, transfer and use of knowledge.
• Best possible use of research resources through strategic planning. Strategic planning will help universities to focus their efforts on their research strengths as well as new areas in which they want to develop strengths.

The direct beneficiaries of the program are Canadian universities and their affiliates, chairholders, and chairholders’ students and trainees (the HQP).

A total of 2,000 Chairs are allocated to institutions that have received an average of $100,000 or more annually from the three federal research funding agencies in the three years prior to the year of the allocation. Canada Research Chairs are divided into two tiers:

• Tier 1 Chairs are tenable for seven years and renewable indefinitely. These are for outstanding researchers acknowledged by their peers as being world leaders in their fields. Tier 1 nominees must be full professors or associate professors who are expected to be promoted to the full professor level within one or two years of the nomination. Should they come from outside of academia, nominees must possess the necessary qualifications to be appointed at these levels. For each Tier 1 Chair, universities receive $200,000 annually for seven years.
• Tier 2 Chairs are tenable for five years and renewable once. They are for exceptional emerging researchers, acknowledged by their peers as potential leaders in their field. Nominees for Tier 2 positions must be assistant or associate professors or possess the necessary qualifications to be appointed at these levels. Universities must justify why (e.g., clinical training, years in industry, breaks in career) a Tier 2 nominee is more than ten years away from the highest degree at the time of nomination. For each Tier 2 Chair, universities receive $100,000 annually for five years.

A more detailed description of the program is presented in Appendix A.

1.3 Evaluation Issues and Questions

The evaluation addressed 10 evaluation questions in three issue areas.

Relevance

1. Is there a continued need for the CRCP given the evolving research funding context since 2009-10?
2. Is the CRCP aligned with Government of Canada’s and the tri-agencies’ mandates/priorities?
3. Is the provision of CRCP awards an appropriate role for the federal government?

Performance—Effectiveness, Efficiency and Economy

4. Have the CRCP awards resulted in the attraction and/or retention of excellent Canadian and foreign researchers?
5. Have the CRCP awards resulted in the creation of dynamic research teams and projects in support of the Chairs?
6. Have the CRCP awards resulted in the creation, dissemination and application of new knowledge in strategic areas of research?
7. Have the CRCP awards resulted in a comparative advantage for universities in strategic areas of research?
8. Has the CRCP resulted in any unintended impacts (either negative or positive)?
9. Has the CRCP been delivered in a cost-efficient manner?

Design and Delivery

10. To what extent is the implementation of the design and delivery features of the CRCP appropriate for meeting its current (and potential future) objectives?

1.4 Organization of the Report

This report is organized into two additional sections and two appendices. The next section (2.0) outlines the methodology and the associated limitations. The following section (3.0) presents the findings for the evaluation. The findings are presented by main theme, which include: overall success and relevance; institutional packages for chairholders; link with strategic areas of research; and program design and delivery. In general, findings presented in this report show results for the CRCP program overall, although areas of significant differences across agencies are also reported. The recommendations stemming from the evaluation are embedded within the appropriate sections of the report. Appendix A presents the logic model for the program and Appendix B presents a crosswalk between the evaluation questions and the sections of the report.

---

2.0 Methodology

2.1 Overview

Seven methods were implemented to carry out the 15th year evaluation of the CRCP.

• document review;
• administrative data and cost-efficiency analysis;
• key informant interviews, including 15 interviews with tri-agency, Innovation, Science and Economic Development Canada, Canada Foundation for Innovation and Health Canada representatives, members of the Tri-agency Institutional Programs Secretariat, members of the Interdisciplinary Adjudication Committee (IAC), and unsuccessful applicants;
• survey of institutions, with 53 completed questionnaires out of 75 who were sent the survey, for a response rate of 70.7%;
• survey of chairholders, with 1,191 completed questionnaires out of 2,402 valid contacts who were sent the survey, for a response rate of 49.6%;
• bibliometric analysis; and
• case studies, including 10 cases representing a mix of region, institution size, the proportion of Tier 1 and Tier 2 Chairs, the proportion of CFI funding, and so on.

The evaluation methods were carried out by a hybrid team composed of internal evaluators at the NSERC-SSHRC Evaluation Division and an external consulting firm, Goss Gilroy Inc. (GGI). The Evaluation Division conducted the document review, parts of the administrative data review, the survey of institutions, the cost-efficiency analysis and four case studies. They also oversaw the bibliometric analysis, conducted by Science Metrix. GGI conducted key informant interviews, the administrative data review, the survey of chairholders and 6 case studies. Each of these methods is described in detail, below.

2.2 Limitations of the Approach

Some data challenges and limitations were encountered during the document review. In particular, the webscan conducted to identify relevant documents was not exhaustive, but rather specifically targeted key evaluation questions, collecting information on key indicators. For certain evaluation questions, there seemed to be a general lack of recent documentation (from 2010 and onward) available online. This may be due simply to a reduction in the attention paid to these areas: for example, research on “brain drain” in Canada appears to have diminished over time, which is probably indicative of a lower level of national concern about this issue, compared to previous decades.

Regarding the administrative data review, CRCP funding data available for 2014-15 do not cover the full fiscal year. Consequently, data for 2014-15 was excluded when calculating the breakdown of funding support by source. In a few instances qualitative data from the administrative data does not fully address the evaluation question. Data from other lines of evidence (such as the survey and case studies) were used to paint a more complete picture.

Knowledge of the program and its various aspects differed by respondent type for the interviews and case studies. For example, senior management, unsuccessful applicant, collaborator and student respondents were less familiar with the details of the design and delivery of the program. Naturally, respondents were only asked to comment on aspects of the program with which they were most familiar. Another implication is that, for some aspects of the design and delivery of the program, only a small number of respondents offered an opinion. This limitation was mitigated with weighting of qualitative evidence towards those with more knowledge of the program, as well as through triangulation of this evidence with other sources, such as surveys and case studies.

---

3.0 Findings

Overall, the evaluation found that the program is performing well in most areas. However, the quality of the chairholder package is diminishing and the evaluation found that this is starting to affect some areas of performance and relevance. As well, the Strategic Research Plan (SRP) may no longer be a useful tool to promote institutional strategic planning, although there is evidence that the CRCP contributes to institutional strategic planning. Finally, the evaluation found that some aspects of program design and delivery need some attention.

3.1 Program Success and Relevance

3.1.1 Effectiveness

The right nominees are being selected

According to the bibliometric analysis, CRCP’s successful nominees almost systematically outperform unsuccessful nominees in terms of scientific impact, scientific quality and output (i.e., number of published papers) prior to the nomination process, indicating a potential validation of the program’s peer review process. This was also the finding from the 2010 evaluation.

The fact that chairholders are overrepresented among Canada’s highly cited researchers also supports the finding that excellent researchers are being selected as Chairs. In particular, although current and former Chairs constitute about 4% of academic researchers in Canada, this group makes up about 20% of the most highly cited researchers in Canada.^{Footnote 1}

CRCP positively contributes to research output

Not only are the more productive nominees selected for the award, but those who are awarded increase their research output after receiving the award, further widening the gap with unsuccessful nominees. The bibliometric analysis found that, at the tri-agency level, and for each of Tier 1 and Tier 2 chairholders, the CRCP has had a statistically positive effect on the scientific output of CRCP chairholders. Note that these results are similar to those from the 2010 evaluation.

Scientific quality and impact remain high after award

The bibliometric analysis confirmed that scientific quality and impact remain high after the award at the tri-agency level, for both tiers, alone and aggregated.^{Footnote 2}

Research is applied in a number of ways

The review of annual reports found that 86% of chairholders reported sharing their research outside of academia in the previous year. As indicated in Table 1, 43% of chairholders indicated that they frequently engaged in activities to share the insights gained through conducting their research with people outside of the academic world; a similar proportion revealed they undertake this sort of knowledge translation occasionally (43%). The table also illustrates that the frequency of knowledge sharing outside academia appears to be increasing, moving from 41% in 2010-11 to 45% in 2014-15.

Table 1: Knowledge Sharing Outside Academia: Frequency by Year

	2010-11	2011-12	2012-13	2013-14	2014-15	Total
Never	14.8%	14.6%	12.7%	12.7%	13.3%	13.6%
Occasionally	44.2%	44.0%	43.8%	43.4%	41.5%	43.4%
Frequently	40.9%	41.4%	43.5%	43.9%	45.2%	42.9%
Total	100.0%	100.0%	100.0%	100.0%	100.0%	100.0%

These findings were confirmed in the survey of chairholders where more than half report applications/impacts outside of academia. For instance, 81% had regularly been invited to present their research results to external organizations; 40% had their research results referred to in reports, studies and strategic plans prepared by interested parties; 39% shared their results with the media in support of the development of a news story; and 24% had been called upon to participate in a public or private sector advisory body, commission or consultative group to examine and/or report on an issue of social or economic importance. Many of those results were more likely to be achieved by Tier 1 chairholders.

At the same time, the review of chairholder annual reports also revealed that knowledge translation activities were more common among chairholders in small universities. Chairholders in small universities were most likely to indicate sharing knowledge with people outside the academic world frequently (55.0%), compared with chairholders in medium (45.9%) and large (40.3%) universities.

CRCP is contributing to research centres, clusters and collaborations

All lines of evidence confirmed that research teams, centres and clusters have been established and/or supported as a result of the CRCP. According to institutional annual reports, 93% of institutions report that the CRCP (with the CFI) is important to support the reinforcement of existing research teams, clusters and/or centres. In total, 83% said it is important in the creation of new research teams, clusters and/or centres.

Case studies confirmed the important role played by chairholders and the CFI in building capacity in a research area. Most respondents for the large majority of case studies of all institution sizes felt that the program has contributed to the development of research teams, centres or clusters. In some cases, existing centres/clusters were reinforced or bolstered and in other cases new centres/clusters were established. The CRCP funding not only contributes to research teams’ salaries to work at the centres/clusters, but also serves to attract (and occasionally fund) students, and enables chairholders time to develop collaborations and partnerships. Also, many institutions follow the practice that once the Canada Research Chair has been awarded, the researcher can apply for CFI funding to ensure the necessary infrastructure and equipment for these centres/clusters are in place.

In total, 58% of chairholders surveyed said their institution had a research centre related to their area of research (similar to 53% reported in 2010). Of those, 47% of chairholders said the centre was established because of them (similar to 51% reported in 2010).

Collaborations
Annual reports from chairholders confirm collaboration is occurring within their institution, within Canada and outside Canada (but less so with government, private sector and not-for-profit sector). The bibliometric study also found more international collaborations after the award, stronger than in the 2010 evaluation^{Footnote 3}. However, no significant differences were noted in terms of national collaborations.

Highly qualified personnel training experience high

According to the review of chairholder annual reports, CRCP chairholders supervised an average of 15,751 HQP per year over the five-year period from 2010-11 to 2014-15. CRCP chairholders are directly supervising 9.63 HQP on average (see Table 2). The number of HQP directly supervised was markedly higher in 2014-15 (10.76) than in previous years covered by the evaluation. Doctoral students were the most commonly supervised HQP over the five-year period under study (2.85), followed by master’s students (2.11) and undergraduate students (1.98). Chairholders at larger institutions tend to supervise more postdoctoral scholars and other HQP than medium and small institutions do. Medium-sized institutions, on the other hand, tend to supervise more master’s students than the large and small institutions do. There appears to have been a reduction in the number of HQP trained between the 2010 evaluation and the findings of the current evaluation, although this difference may be due to methodological differences.

Table 2: Average Number of HQP under Direct Supervision of Chairholders by University Size

	University Type			Total
	Large	Medium	Small
Undergraduate students	2.02	1.94	1.77	1.98
Master’s students	1.98	2.46	2.30	2.11
Doctoral students	2.99	3.02	1.74	2.85
Postdoctoral scholars	1.61	1.18	.72	1.43
Other HQP	1.35	1.11	.94	1.26
Total	9.96	9.70	7.46	9.63

Source: CRCP Chairholder Annual Reports 2010-11 to 2014-15

HQP appear to experience high quality training, according to chairholders and students themselves. When asked to what extent their Chair position, including the associated CFI funding, had enhanced the training they could otherwise provide to their HQP, most chairholders (77.1%) submitting an annual report agreed that the CRCP had a significant impact. Chairholders in small (84.7%) and medium (82.5%) institutions were more likely to indicate that the CRCP had a significant impact on the training of HQP than chairholders in large universities. The survey of chairholders also revealed that most chairholders feel that students and research staff working with Chairs had more frequent opportunities than before they received the Chair award. These opportunities include collaborations with researchers within and outside of Canada, write papers, present results at national and international conferences, and lead research projects.

While the large majority of the students interviewed for all case studies said that they had not chosen the chairholder as their supervisor because they were a Chair, almost all students interviewed were positive about their experience working with the chairholder. Students were able to provide a number of examples of benefits of training with the chairholder (such as access to equipment, more possibilities for networking and collaboration, and the added prestige of working with a chairholder), many of which were echoed by chairholders themselves. As well, most of the students interviewed felt that their training and working experience with the chairholder had influenced their career plans or plans for further education. Most students from most case studies indicated that their career prospects had improved because they had worked with the chairholder, due to the research and publication experience gained, their supervisor’s professional connections, and general prestige of the chairholder.

Unintended outcomes are generally positive

In addition to contributing towards expected outcomes, the CRCP can also benefit chairholders’ careers, other faculty and Canada’s reputation.

The impact of the program on chairholders’ careers was primarily explored through the case studies; the large majority of chairholders who commented on this matter felt that the program had an impact in this regard. This was driven by: having more freedom to pursue their own areas of research and/or bigger projects; more time to focus on their research; the ability to build and maintain networks; and the prestige associated with the award.

In terms of impacts on other faculty, the survey of institutions explored this issue in a couple of ways. Most directly, survey respondents were asked whether they agreed that the CRCP has been beneficial for a variety of faculty/researchers in their university, not just the chairholders themselves. In all, 87% of respondents agreed. Also when asked whether they agreed that the CRCP has had a negative impact on non-Chair faculty due to greater concentration of university resources among chairholders, 70% disagreed and only 8% agreed. Both of these results are similar to those from the 2010 evaluation.

With respect to Canada’s reputation, institutional representatives reported that the CRCP had a large impact on their institution’s reputation in Canada (96% reporting a positive or very positive impact) and abroad (92%). As well, 88% agreed that the program has contributed to Canada’s reputation as a world leader in research. This was also supported by case study evidence.

3.1.2 Efficiency

The program is operating in a cost-efficient manner

The cost-efficiency analysis included grants and administrative expenditures from 2010-15. Overall, it was determined that the CRCP was delivered in an efficient manner. CRCP has lower cost-efficiency ratios compared to those of a similar program, the Canada Excellence Research Chairs (CERC) Program.^{Footnote 4} Administering the CRCP costs approximately $1.20 per $100 of CRCP funds granted whereas administering the CERC costs $3.42 per $100. It is also important to note that the CRCP ratio has been decreasing over time.

Although not a measure of cost-efficiency, the evaluation also compared CRCP to CERC in terms of total chairholder funding versus administration costs incurred by both TIPS and the grantee institutions. In this respect the CRCP again looks favourable in comparison to CERC.

The value of volunteer time for the CRCP (i.e., the involvement of external reviewers and IAC members) is estimated to be over $500,000 per year, representing a savings on administrative expenses for the program.

3.1.3 Relevance

CRCP is aligned with research mandate of federal government and with tri-agency priorities

The program is directly aligned with the government of Canada’s knowledge pillar, which aims to achieve research excellence in priority areas while making research that is funded by the federal government more open and transparent.^{Footnote 5} Two of the CRCP’s objectives directly align with the knowledge pillar: the program’s objective to “improve universities’ capacity for generating and applying new knowledge” and “to optimize the use of research resources through strategic planning.”^{Footnote 6} The program’s two other objectives, “to attract and retain excellent researchers in Canadian universities” and “to strengthen the training of highly qualified personnel”^{Footnote 7} are clearly aligned with the Government of Canada’s people pillar, which states that the government will “develop, attract and retain highly qualified and skilled individuals, as well as top experts and leaders needed for Canada to thrive in the global knowledge economy.”^{Footnote 8}

The program objectives are aligned with the three federal granting agencies’ research capacity building priorities. The objectives align with CIHR’s Strategic Direction 1, which focuses on “promoting excellence, creativity and breadth in health research and knowledge translation”^{Footnote 9}, with NSERC’s priority that aims to “attract, retain and develop highly qualified people in natural sciences and engineering”^{Footnote 10} and SSHRC’s priority to ”make Canada a world leader in social sciences and humanities research and research training.”^{Footnote 11}

Programs such as CERC and CFREF do not negatively affect CRCP relevance

Overall, the emergence of new international chair programs and the implementation of new Canadian programs—such as CERC and the Canada First Research Excellence Fund (CFREF)—were not seen by most interview, case study and institutional survey respondents to have a particularly large impact on the CRCP or its ongoing relevance. The CRCP is considered a broad-based program, different from what the other programs are trying to achieve.

In fact, of those institutional survey respondents who had applied to CERC and/or CFREF, 96% either agreed or strongly agreed that their applications were to fund areas where their institution currently had one or more Canada Research Chairs. The survey also explored whether the advent of these programs (CERC and CFREF) has changed the way their institution uses the CRCP. In response, 65% said there had been little or no change. Among those who said they had changed the use of CRCP and provided an explanation of that change, some said they adjusted the allocation of Chairs and some said they made more strategic use of funding.

The original objectives are still relevant

The CRCP intends to achieve the following objectives:

• Increase Canada’s research capacity. The program will increase Canada’s research capacity by attracting and retaining 2,000 top researchers from within Canada and abroad.
• Improve training of highly qualified personnel. The program will increase the number of highly qualified personnel trained through research in Canadian universities. It is also expected that the quality of training will be improved.
• Improve universities’ capacity to generate and apply new knowledge. By attracting and retaining top researchers, Canadian universities will be able to increase research outputs as well as the dissemination, transfer and use of knowledge.
• Best possible use of research resources through strategic planning. Strategic planning will help universities to focus their efforts on their research strengths as well as new areas in which they want to develop strengths.

All lines of evidence confirmed that the four original objectives of the program are still relevant. While national concern over ‘brain-drain’ (one of the drivers for the program’s introduction in 2000) has since diminished, there is some evidence that the subsequent ‘brain gain’ enjoyed by Canada is now beginning to subside amid increasing international competition. Therefore, a program to help Canadian academic institutions attract and retain excellent researchers remains needed and relevant. The last CRCP evaluation determined that, without the CRCP program, Canada “would likely lose ground in competing for leading researchers on an international scale.”^{Footnote 12}

3.2 Institutional Packages for Chairholders

In the context of the CRCP, the chairholder package is comprised of the CRCP award, plus the institutional funding support provided (including support for a CFI application and/or access to the CFI funding associated with the CRCP), plus other, non-financial benefits. The financial component of the package can cover, in whole or in part, the chairholder’s salary, research team salaries, student salaries, research funding and so on. The non-financial component can include: protected research time (also referred to as teaching release time), fewer requirements to participate on committees and so on.

Institutions are expected (but not required) to provide a package to Canada Research Chairs and not just use the CRCP funds to pay for the Chair’s salary. Institutional commitment (i.e., the support offered by the institution to a Chair nominee) is one of the selection criteria in the program’s peer review process. Institutions must demonstrate that they will provide chairholders with the support they need to ensure the success of their work. However, peer reviewers sometimes face challenges in balancing the excellence of a nominee with the support package being offered for them. Furthermore, monitoring of institutions’ compliance in providing the support promised is limited to certain situations (e.g., nomination for renewal).

3.2.1 CRCP Award Value

Amount has not changed since program inception in 2000

The award value has not changed since the CRCP was introduced in 2000, declining by one third in real terms (using Bank of Canada rates of inflation). Figure 1 presents these results. In order to offer an award that would have the same monetary value in 2015 as it did in 2000, the annual value of the CRCP awards would have to be $266,387.43 for the Tier 1 Research Chairs (see Figure 1) and $133,193.72 for Tier 2 Research Chairs (see Figure 2).

Figure 1: Award Value in Constant (2000) Dollars, by Year

Description of figure

Figure 1. Award Value in Constant (2000) Dollars, by Year

This line graph shows the award value in constant dollars, by year, benchmarked from the year 2000. The y-axis shows the award amount ranging from $0 to $200,000 in increments of $50,000. The x-axis shows each year, starting from 2000 and ending at 2015.

Two lines on the graph represent Tier 1 and Tier 2. Tier 1 begins at $200,000 in 2000 and ends at $132,240 in 2015, while Tier 2 begins at $100,000 and ends at $67,120 in 2015. Each line shows a steady downward progression from the initial award amounts.

Concerns in terms of the award value were noted by most interview respondents. Overall, most felt that the amount of the award should be increased to improve its value and prestige. However, some criteria to dictate the proportion of the award to support research should be established. Some (unsuccessful candidates) suggested that the funds need to take inflation into account.

The review of annual reports corroborates this point. When asked for suggestions to improve the program, the value of the award was the most frequently cited issue by both institutional representatives (30%) and chairholders (32%).

3.2.2 Institutions’ Use of CRCP Funds

The financial package is used mostly for the chairholder’s salary

Overall, CRCP funds accounted for 8% of gross federal government expenditures on research and development in the natural sciences and engineering (NSE) and the health sciences, and 10%-11% in the social sciences and humanities (SSH).

According to administrative data, institutional expenditures of CRCP funds and institutional funding support amounted to between $429 million and $441 million per year between 2010-11 and 2013-14. Institutional funding support accounted for 40% of these expenditures over the period. The proportion of total support paid by the institution has increased slightly over time, from 39% in 2010-11 to 41% in 2013-14.

CRCP funds make up a larger proportion of the total financial package for medium institutions, SSH Chairs and Tier 1 Chairs.

Most of the CRCP and institutional funds are spent on chairholders’ salaries, followed by other salaries (for HQP, for example) (see Table 3).

Table 3: Breakdown of CRCP and Institutional Expenditures as Per cent of Total

	CRCP Funds	Institutional Funds
Administrative	6%	5%
Equipment, supplies, travel, and other	6%	21%
Salaries to CRCs	73%	55%
Salaries, other	14%	19%
Total	100%	100%

Sources: Chairholder funding packages (Finance Tab) and CRCP Institutional Funding Support

According to evidence from the case studies, often the CRCP award no longer covers the entire salary for the chairholder, whereas this used to be the case, according to these respondents (for example, the 2010 evaluation found that the award amount just covered the salary of many Tier 2s). As shown in the figure below, chairholders’ salaries as a per cent of CRCP funds rose between 2002-03 and 2007-08, and this has held steady since. Institutions tend to spend a larger portion of their share of funding on chairholders’ salaries when they are small institutions, for Chairs in the SSH, Tier 2 Chairs and when located in Quebec.

Figure 2: Breakdown of CRCP Funds at Different Points in Time

Description of figure

Figure 2. Breakdown of CRCP Funds at Different Points in Time

This stacked bar graph breaks down the CRCP funds at different points in time, as a percentage of total funds. The y-axis shows percentages, from zero to 100% in increments of 25%. The x-axis shows the fiscal years 2002-03, 2007-08 and 2014-15.

Three bars on the graph show the breakdown of how funds were spent in a fiscal year. They are split into coloured sections that represent four categories: chairholder salaries (blue), other salaries (red), equipment, supplies, travel and other (green), and administrative (purple).

 

• In 2002-03, 53% of CRCP funds were spent on chairholder salaries, 29% on other salaries, 10% on equipment, supplies, travel and other, and 8% on administrative.
• In 2007-08, 74% of CRCP funds were spent on chairholder salaries, 12% on other salaries, 7% on equipment, supplies, travel and other, and 7% on administrative.
• In 2014-15, 73% of CRCP funds were spent on chairholder salaries, 14% on other salaries, 6% on equipment, supplies, travel and other, and 6% on administrative.

Interview respondents and chairholders felt that institutional packages, particularly funding for research, were not sufficiently strong. Most interviewees of all types (except unsuccessful applicants) expressed concern about institutions’ use of CRCP funds for primarily salary support rather than direct support for research. In many cases, this results in chairholders spending more time on grant writing and having less time for research. Along the same vein, a few interviewees also felt that institutions should provide more support to chairholders, although interviewees did not specify the nature of the support that is required. Most chairholders consulted for the case studies also felt strongly that more funding for research should be part of the package. According to the case studies, because most institutions that participated in the case studies use the funds primarily for salary support, institutions are able to offer less in the way of a package. Most packages do include teaching release time (although not all and less so for large institutions) and a few include additional funding for research (although small amounts—between $10K and $15K) and students. However, most chairholders consulted for the case studies felt strongly that more funding for research should be part of the package.

Similarly, the majority of current chairholders that participated in the survey felt the level of funding available for the costs of research was inadequate in their field and respondents reported that funding was much less or somewhat less than what is needed for: salaries or stipends for research team (77% felt funding was inadequate in this area); direct research costs (60%); salaries for researchers (45%); and indirect research costs (38%).

CFI funding plays a role in rounding out the chairholder package

Although comprehensive data on CFI funding provided to chairholders is not available, by matching up datasets from the CFI and TIPs, a sample of 441 chairholder nominations spanning competition years 2004 to 2013 was created where data was available on CFI funding, CRCP funding, and institutional support.^{Footnote 13} Institutions in this sample received an average of $152,576 in CFI funding per chairholder, $710,037 in CRCP funds per chairholder, and provided an average of $580,381 from their own funds to support the chairholder, as shown in the table below. On an annual basis, the three fund sources together represented an average of $308,523 per year for Tier 1 chairholders and $202,073 per year for Tier 2 chairholders.

Table 4: Average Funding by Source and Tier

	CFI	CRCP	Institution	Total
Tier 1	$181,583	$1,199,000	$779,080	$2,159,663
Tier 2	$135,067	$414,862	$460,438	$1,010,367
Total	$152,576	$710,037	$580,381	$1,442,994

Significant differences in the amount of CFI funding provided per chairholder were observed by university size (with large institutions receiving a greater amount than medium or small), and by region (with Ontario universities receiving greater amounts than those in Western or Eastern Canada). These differences were significant for Tier 2 only, however.

The case studies also found many instances where CFI funding played an important role in not only attracting Chairs and HQP, but also to the achievement of outcomes (as described above).

3.2.3 Importance of the Chairholder Package

The package is the most important deciding factor for nominees when deciding whether to accept the award

According to the results from the chairholder survey, the chairholder package is the most important factor for nominees to accept a Chair award. The top three factors that influence a chairholders’ decision to accept the award include: capacity to support students and staff (90%); availability of research funding (87%); and CRCP chairholder status/prestige (84%).

The chairholder package also appears to be one of the reasons why chairholders resign. The administrative data confirmed that approximately 46 chairholders resign per year. Reasons are usually unrelated to the program, but 16% said it was because of poor benefits (e.g., salary, teaching load, research funding) of the Chair position.

The evaluation also found that institutions recognize that the chairholder package is important to attract and retain top researchers. When asked to discuss barriers to attraction and retention, the main barrier identified by institutions and interview respondents was the availability of funding to support the chairholder with research funds, competitive salary and access to infrastructure (such as a new lab or research centre). The other commonly mentioned barriers are institution-specific (weather, regional isolation, proximity to urban centre, jobs for spouses, cost of housing/living) and thus outside the control of the program.

Despite the evidence for the value of the CRCP for retention, a study conducted by Courty & Sim (2014) which was restricted to British Columbia and Ontario found that a chairholder was just as likely to change jobs after having been nominated for a Chair as a non-chairholder. They also confirmed that despite the CRCP’s value as a temporary retention tool in the beginning of the tenure, the fact that a researcher is a chairholder seemed to have no impact on whether that researcher would stay in that institution or not.^{Footnote 14} The present evaluation did not compare chairholder turnover to that of non-chairholders, but merely confirmed the perceived importance of the program for retention and determined that the level of turnover is at an acceptable level.

3.2.4 Contextual Trends

Contextual trends exacerbating challenges with chairholder package

The evaluation identified a number of changes in the program’s context that would reinforce the importance of the chairholder package. The trend that was most often mentioned in interviews and confirmed through the document review was the overall decrease in the availability of research grant funding. This includes decreased funding in some areas, decreasing success rates in some areas, and more restrictions on the funding that is available (e.g., targeted funding, greater accountability requirements). Coupled with the increase in the number of researchers (growing by 9.5% between 2007-08 and 2013-14^{Footnote 15}), research funding is becoming more difficult to access.

Specifically, federal government expenditures in the higher education sector fell in 2009-10 and have since remained lower than the expenditure amount in 2008-09.^{Footnote 16} Despite the country’s need for “basic and applied research,” research funding has been cut drastically. The decrease in federal R&D expenditures has greatly affected research grant programs.^{Footnote 17} As well, the Canadian Association of University Teachers study also found that increases in federal funding for university-based research has been declining since 2005: while it increased 30% between 2000 and 2005, the increase between 2006 and 2011 was only 6.6%, and only 1.4% between 2011 and 2012.^{Footnote 18} Funding for the tri-agencies has also decreased by 7.5% between 2007-08 and 2013-14.

As well, the document review found that provincial funding for institutions has been decreasing, which results in more reliance on other sources of funding and less “money in the pot” to contribute to chairholder packages.

3.2.5 Possible Impacts on Relevance and Effectiveness

Consistency with federal role to support research at risk

In terms of consistency with the federal government’s roles and responsibilities, the evaluation evidence confirmed that the CRCP is consistent with the federal role since very little has changed in the provincial-federal division of roles in the education sector since the last evaluation. The boundaries between federal and provincial roles lack in clarity because while the provinces are responsible for education, the federal government is responsible for “national interest, equality of treatment and opportunity, economic development, and Indians and lands reserved for the Indians.”^{Footnote 19} The federal government’s support of university-based research, through their granting councils and intermediary bodies such as the CFI, is justified by the government’s responsibility for economic development.

The last CRCP evaluation concluded that “none of the program level’s interviewees disputed the federal government’s role in these matters or raised evidence to suggest that federal support to enhance the Canadian research environment (i.e., universities) should be reduced.”^{Footnote 20} It was found that the federal government was the only one who could play this role, thereby justifying the provision of the CRCP awards as being an appropriate role.

However, trends in the use of CRCP funds may present a risk for the program’s continued alignment to the federal role. In the case studies, many institutions reported a greater reliance on CRCP funds over time. This trend, in combination with the reduced funding package, may put into question the role of the program in supporting research rather than providing operational dollars for institutions. This finding was also flagged by some key informants during interviews.

Reduced attraction from abroad and turnover among researchers from abroad

There is evidence that researchers from abroad (including both foreign-born and expatriate Canadian researchers) are being recruited less often for Chair positions than previously, and when recruited may be subject to higher attrition than Canadian chairholders.

The administrative data reveals that most CRCP chairholders (71.9%) are being nominated within Canada, a slight increase from the findings of the tenth-year evaluation of the program (68%). Most CRCP awardees are nominated within their university (57.5%, versus 52% in the 2010 evaluation), 28.1% of awardees originated from outside Canada (compared to 32% in the 2010 evaluation) and 14.4% came from outside the institution but within Canada (compared to 15% in the 2010 evaluation).

Further, foreign nominees accounted, on average, for 13% and 15% respectively of new Tier 1 and Tier 2 nominees over the five-year period 2010-14. This represents a large decrease in the number of foreign nominees from the previous five-year period; the proportion of new nominees from abroad has more than halved since 2005-09. See Table 5 for these results.

Table 5: Percentage of New Nominees from Outside Canada

	Tier 1	Tier 2
2010	16%	17%
2011	24%	12%
2012	7%	13%
2013	9%	14%
2014	8%	17%
Total (2010-14)	13%	15%
2005-09 Average	32%	31%

Source: CRCP PMS 201 foreign recruitment Tier1 vs Tier2

Figure 3: CRCP Awards Starting and Ending Within the Review Period, by Nominee Origin

Description of figure

Figure 3. CRCP Awards Starting and Ending within the Review Period, by Nominee Origin

This stacked bar graph presents the CRCP awards, by nominee origin, starting and ending in the review period of 2010-15. The y-axis shows the percentage, from zero to 100% in increments of 25%.

The first bar shows the percent of nominees by origin for awards starting in 2010-15. The bar is broken into three coloured sections, which represent 13% of nominees from outside Canada (dark blue), 10% within Canada (blue) and 77% within the nominating university (light blue).

The second bar shows the per cent of nominees by origin for awards ending in 2010-15. The bar is broken into three coloured sections, which represent 31% of nominees from outside Canada (dark blue), 17% within Canada (blue) and 52% within the nominating university (light blue).

Increase in Chair vacancies

Data on active Chairs by year were analyzed to estimate the per cent of vacant Chair positions over time. Table 6 below shows the estimated percentage of vacancies for all Chairs by year, broken down by allocation type (special vs. regular Chair awards). Regular Chair awards are further broken down by tier and agency.^{Footnote 22}

While the per cent of vacancies in special Chairs has remained relatively stable over time, the percentage of vacancies in regular Chair positions has increased steadily. This increase in vacancies has happened in both tiers and across all three agencies, but is more apparent in Tier 2 than Tier 1, and in SSHRC than in the other two agencies.

Table 6: Per Cent of Chair Positions that are Vacant, by Year, Tier, Agency and Allocation Type

Year	Regular Chairs						Special Chairs	All Chairs
	By Tier		By Agency			All Regular
	Tier 1	Tier 2	NSERC	SSHRC	CIHR
2012-13	17%	6%	13%	9%	12%	12%	17%	12%
2013-14	18%	13%	15%	15%	15%	15%	15%	15%
2014-15	19%	18%	18%	21%	17%	18%	15%	18%

No conclusive evidence was found to explain the cause of these increases. While they may be related to the decline in chairholder packages discussed above, a number of other explanations are possible as well. Firstly, there has been a decrease in the success rates for CRCP awards in recent years. Secondly, given that the program is 15 years old, the timing could be right for the program to be experiencing retirement for some early-appointed Tier 1s combined with the end of second terms for early-appointed Tier 2s, leading to more awards coming available. Thirdly, institutions explained that there are barriers to recruitment (particularly for international recruitment) and that recruitment is time consuming. Fourthly, for SSHRC specifically, vacancies could have been affected by the guidelines released in 2009 regarding eligibility for SSHRC funding for health-related research. Finally, it is possible that some institutions are not filling Chair vacancies for fear that they may be lost in the next round of re-allocation.

 

 

3.3 Link with Strategic Areas of Research

The CRCP contributes to institutional strategic planning, but the SRP may not be the optimal tool to assess this

Consistent with CRCP’s Objective 4 (“best possible use of research resources through strategic planning”), institutions are expected to allocate their Chairs into research areas that are a priority to the institution. To encourage this, participating institutions are required to develop and publish Strategic Research Plans (SRPs), which outline the institution’s research priorities. When a nomination is made for a Chair award, the peer review includes an assessment of whether or not the proposed Chair fits with the university’s SRP.

Overall, there is a difference of opinion with respect to the implementation of SRPs and the extent to which they are useful documents for ensuring alignment of nominated chairholders to institutional strategic planning.

Most institutional representatives reported greater investment in strategic areas than other areas (including CRCP, CERC and CFREF). Furthermore, according to the data from annual reports, institutions indicated that the Chairs program, including the associated CFI funding, had supported the development of research themes defined in their Strategic Research Plan. Specifically, institutions identified newly built or updated infrastructure (26%), an increase in HQP (22%), and the development of new academic programs, courses, and research programs (22%) as the top impacts of the program.

These findings are consistent with the results from the survey of institutional representatives. Overall, the SRPs seemed to be in the process of implementation at most universities: 88% of respondents indicated that infrastructure had been built or improved in strategic areas of research than in other areas of research and 78% indicated that a greater proportion of their institution’s total funding had been invested and/or leveraged towards strategic areas of research than towards other areas of research. Further, 92% of respondents indicated that decisions on the allocation of CRCP awards were made according to their institution’s Strategic Research Plan.

On the other hand, there is evidence that at some institutions SRPs may be set deliberately broad, and strategic planning may happen independently of SRPs. For example, all interview respondents agreed that institutions have not effectively identified strategic areas of research and thus have not focused CRCP funding strategically. According to most interviewees, most Strategic Research Plans (SRPs) are very broad, high level, and lack specific details, especially at larger institutions.

3.4 Program Design and Delivery

3.4.1 Access for Designated Groups

More work is needed to address barriers to access for designated groups

As part of the programmatic changes resulting from the settlement agreement of a 2006 case at the Canadian Human Rights Tribunal, all participating institutions are expected to establish a fair and transparent recruitment and nomination process for its Chair awards, removing barriers to access for designated equity groups (women, visible minorities, people with disabilities and Aboriginal Peoples). The Secretariat has established guidelines articulating the program’s expectations with respect to what such a process would entail. The Secretariat also has a set of measures in place to encourage equitable practices, including monitoring of recruitment practices, establishing equity targets and an exemplary equity practices recognition process. Despite these measures, the majority of institutions do not meet equity targets for many designated groups.

For each institution, equity targets are calculated as a function of the number of active Chairs at the institution, as follows:

Group	Target formula
Women	18% of active NSERC Chairs + 30% of active CIHR Chairs + 46% of active SSHRC Chairs
Visible minorities	15% of active Chairs
Persons with disabilities	4% of active Chairs
Aboriginal Peoples	1% of active Chairs

According to a review of the administrative data, the majority of institutions do not meet their targets for women (54%), visible minorities (75%) and people with disabilities (59%). Large institutions are less likely to meet targets than medium or small institutions. See Table 7 for these results.

Table 7: Per cent of Institutions not Meeting Equity Targets, by Institution Size

	Women	V. Minorities	P. Disabilities	Aboriginal
Small	45%	73%	30%	5%
Medium	56%	63%	93%	13%
Large	73%	93%	100%	57%
Total	54%	75%	59%	18%

Source: All three Target-Setting Cohorts collected from 2012 to 2014—Gaps for Institutions

The administrative data also includes evidence regarding the size of the gaps between the target for designated groups and the actual number who have been selected as chairholders: Table 8 shows this gap as a per cent of target size. Obviously, significant gaps remain, particularly for visible minorities and persons with disabilities.

Table 8: Average Gap in Meeting Equity Targets by Institution Size

	Women	V. Minorities	P. Disabilities	Aboriginal
Small	24.3%	62.6%	29.7%	5.4%
Medium	20.1%	36.3%	87.8%	13.3%
Large	12.8%	43.9%	83.2%	55.4%
Total	20.8%	52.4%	54.3%	17.8%

Source: All three Target-Setting Cohorts collected from 2012 to 2014—Gaps for Institutions

Although the work done to date was widely recognized by most interview respondents, 24% of institutional representatives and 20% of chairholders felt that systemic barriers remain for the CRCP and that more work is needed in this regard (interestingly, almost twice as many chairholders agreed there were systemic barriers than in the 2010 evaluation where 11% agreed with this statement). In the surveys, institutional representatives and chairholders identified Aboriginal Peoples and women as experiencing these barriers most often.

When interview respondents and institutional representatives were asked for their views on the reasons why barriers remain, three types of answers were given: 1) due to continuing institutional and/or cultural biases that disadvantage the designated groups (e.g., selection that favours more “traditional” career paths and disciplines); 2)because it is difficult to find members of designated groups who have the prerequisites for a Chair award, such as a PhD, research track record and high number of research outputs; and 3) because the selection pool is not deep in these areas, since there are not many individuals in the underrepresented groups who are active in many disciplines/fields (particularly science and engineering) and even in academia more broadly. The fact that the opinion in the last point is held may indicate a lack of understanding about the nature of the equity targets and how they are calculated.

3.4.2 Chair Re-allocation Process

The re-allocation process can cause challenges when it results in the loss of an occupied Chair

The number of Chairs allocated to each institution is calculated based on the portion of tri-agency funding received by that institution. Every two years, this allocation is recalculated based on the funding received over the previous three years; this process is often referred to as the “re-allocation” of the Canada Research Chairs. The re-allocation determines the number of CRCP awards institutions are allocated over the following two years.

There were mixed views regarding the effectiveness of the CRCP’s Chair allocation formula/process. While key interviewees generally agreed that the formula/process was appropriate, case study respondents noted some areas in need of improvement.

Case study respondents were particularly concerned with the implications of dealing with lost Chairs and the resultant disruption at the institution and frequency. However, while most did not want to see a change, a few did suggest that the calculation could be done annually to give institutions an idea of their upcoming allocation so that they could better manage lost Chairs, for example.

This was especially difficult for smaller institutions. In the event that the institution loses a Chair, the smaller institutions that participated for two case studies are typically not able to pay their salary and the researcher leaves the institution. This is disruptive and negatively impacts the institution’s reputation.

Mixed opinions were also found among institutional representatives. According to survey results, about half (52%) of institutional representatives feel the Chair allocation process has a positive or very positive impact, while 30% said it has a negative or very negative impact. These findings were similar to those reported in 2010.

3.4.3 Tier System

There is support for limiting renewals for Tier 1 Chairs

Overall, 57% of Tier 1 nominations are for renewals (compared with 33% for new nominations and 11% for advancements). However, the proportion of nominations that are renewals for Tier 1s has been decreasing over the five year period of the evaluation, from 65.5% of nominations in 2010-11 to 46.1% in 2014-15.

Most interviewees and those consulted for case studies agreed that indefinite renewals for Tier 1 Chairs are not appropriate because they are not consistent with the objective of supporting excellence and do not lead to innovation. A few respondents from case studies suggested that indefinite renewals of Tier 1 is part of the reasons why there is a gap between Tier 2 and Tier 1; that is, that there are insufficient Tier 1 positions available for Tier 2 Chairs who are ready to make the transition. More on the gaps between Tier 1 and Tier 2 is presented below.

According to a few respondents for the case studies, there has not been an issue with retaining Tier 2 Chairs after their second term as these institutions have transition strategies that include awarding institutionally-funded Chairs and gradual re-introduction to a regular teaching load. As well, the evaluation found that some institutions have introduced, or are planning to introduce, policies that limit the number of renewals for Tier 1 Chairs to one. In these (few) cases, the institutional representatives indicated that it has been working effectively so far and has not posed an additional barrier to recruitment.

There may be a gap between Tier 1 and Tier 2

The administrative data review also analyzed the age of chairholders (as a proxy for career stage) by tier and agency to assess whether there was a group of researchers excluded by the two-tier system. As shown in the chart below, overall (across funding agencies), there is significant overlap in the age distributions of the two tiers. However, there is some evidence of a bimodal (two-peaked) distribution, indicating a possible “gap” between Tier 1 and Tier 2 where potential mid-career nominees may be missed. From this chart it can be seen that Tier 1 renewals significantly outnumber new Tier 1 chairholders and alter the demographics of this group. This data cannot be considered conclusive in the absence of information on the population from which this sample comes, against which the sample could be compared. Examining the data by agency, SSHRC chairholders appear to be older than the population of SSHRC-funded researchers, while CIHR chairholders are younger than the population of CIHR-funded researchers, and NSERC chairholders match reasonably well the population of NSERC-funded researchers. The bimodal distribution, driven largely by Tier 1 renewals, is particularly evident in SSHRC and CIHR chairholders.

Figure 4: Age Distribution of Canada Research Chairholders, 2015

Description of figure

Figure 4. Age Distribution of Canada Research Chairholders, 2015

This stacked line graph represents the age distribution of Canada Research Chair holders in 2015. The x-axis shows the age ranges from 25 to 29, up to 70+, in increments of five-year periods. The y-axis shows the number of chairholders beginning at zero and going to 350, increasing in increments of 50. The four shaded areas each represent a category: new Tier 2 chairholders (solid blue), renewed Tier 2 chairholders (dark blue horizontal lines), new Tier 1 chairholders (solid light blue) and renewed Tier 1 chairholders (light blue horizontal lines).

The line graph shows a general peak for all Tier 1 and Tier 2 categories between ages 35 to 39, and 40 to 44. After this age range, there is a steep decline in all categories except Tier 1, renewal, which remains high until tapering down around the 65 to 69 age category.

Between 2010 and 2014, the average age was 54.2 years for Tier 1 chairholders and 39.5 years for Tier 2 chairholders. The average age overall was 44.7 years, decreasing slightly over the period. However, for Tier 1 renewals, the average age was higher, 56.1 years, and reaching over 60 years in 2014. The average age of renewed Tier 1 chairholders increased by 8.5% in the period, while the average age of all other groups declined.

The age difference between Tier 1 and Tier 2 increased by 25% from 2010 to 2014, due to the renewal of Tier 1 chairholders. As shown in the chart below, while the age difference between Tier 1 and Tier 2 has not changed significantly for new chairholders, there was a large jump in 2014 for renewed chairholders. Those awarded a Tier 1 Chair title in the first year of the program (2000) would largely be coming up for their second renewal in 2014, which likely explains this jump. If indefinite renewals continue, it is to be expected that the average age of Tier 1 renewals will continue to increase.

Figure 5: Average Age Difference between Tier 1 and Tier 2 Chairholders, 2010 to 2014

Description of figure

Figure 5. Average Age Difference between Tier 1 and Tier 2 Chairholders, 2010 to 2014

This line graph shows the average age difference between Tier 1 and Tier 2 chairholders from 2010 to 2014. A blue line represents new Chairs, and a red line represents renewals only. The x-axis shows each year from 2010 to 2014 while the y-axis shows the differences in age from zero to 20 years, increasing in increments of five years.

The two lines are matched at around 12.5 years for 2010 and 2011. The red line drops to 10 years at 2012 and steadily increases to 15 in 2013 and 17.5 in 2014. The blue line increases slightly to 15 years in 2013 before dropping back to 12.5 in 2014.

Some interviewees, especially members of the Chairs Secretariat, senior managers, and unsuccessful applications, shared concerns regarding mid-career researchers, and whether there is place for them in the CRCP. Overall, there were mixed views on whether it was both appropriate and effective to target this group.

 

 

---

4.0 Conclusion

The evaluation finds that the program is performing well in most areas and continues to be relevant, but has uncovered some issues requiring attention. Firstly, the quality of the chairholder package is diminishing, driven in part by the declining real value of the CRCP award, and may be starting to affect some areas of performance and relevance. Secondly, the Strategic Research Plan (SRP) may not be the optimal tool to assess the alignment of CRCP nominees with institutional strategy. Thirdly, the evaluation found that some aspects of program design and delivery need some attention.

In most areas examined, the CRCP is effective, cost-efficient and relevant. The program is selecting the right nominees for the award, and is contributing towards research outputs, research centres and collaborations. The program is aligned with federal and tri-agency priorities and the objectives of the program remain relevant.

Despite this, the chairholder package—the benefits provided to chairholders by the institution—is diminishing. This trend is largely driven by the fact that the award value has not changed since 2000 and is therefore diminishing in real terms. CRCP funds and institutional funds are both increasingly being used to cover chairholder salaries, resulting in less funding available for other components of the chairholders’ package, such as research funding. This may present a risk to relevance, and may be linked to some observed challenges to program effectiveness, namely challenges in recruitment and turnover among chairholders from abroad, and an increase in Chair vacancies.

Strategic Research Plans, which are created by institutions to outline their research priorities, may no longer be a useful tool to ensure CRCP funds are being used strategically. Despite this, the evaluation found evidence that the CRCP does in fact contribute to institutional investments in strategic areas of research.

While most components of program design and delivery were found to be appropriate, a few areas were noted to require attention. More work is needed to address barriers to access to CRCP for designated groups, as the majority of institutions do not currently meet targets. Secondly, the re-allocation process can cause challenges for institutions when it results in the loss of an occupied Chair. Finally, the indefinite renewals offered to Tier 1 chairholders may be creating an increasingly large age gap between Tier 1 and Tier 2, reducing the ability of the CRCP to attract younger Tier 1s.

Recommendations

The evaluation offers the following seven recommendations, resulting from the evidence described above.

Recommendation 1: Management should investigate the feasibility of increasing the award value and/or indexing it to the inflation rate.

The CRCP award value has remained unchanged in nominal terms over the program’s 15-year history, representing a one-third reduction in value in real terms. The diminishing real value of the CRCP award affects its use in practice by institutions, threatening the program’s relevance and effectiveness. The declining award may be indirectly linked to increased challenges around recruitment and retention of international researchers, and increasing Chair vacancies. It also presents a threat to the federal role of the program, as the Chair position becomes gradually less distinguishable from a regular faculty position, and as institutions increasingly view CRCP funds as part of their core operating budget.

The declining award value contributes to a diminishing chairholder package, which in turn threatens the achievement of all four CRCP objectives, as explained in the next recommendation.

Management should investigate the possibility of increasing the award value, and the relative merits of different options for financing this award increase (e.g., reducing the number of awards). In order to ensure the risks to the program’s objectives are addressed, an increase in award value should be carried out in concert with efforts to ensure more robust chairholder packages, as described in Recommendation 2.

Recommendation 2: Management should examine options to ensure more robust chairholder packages are offered by institutions, in order to support the continued achievement of the program’s objectives in the future.

Recent trends in institutional use of CRCP funds, partially the result of the decreasing award value, threaten to diminish the relevance/effectiveness of those funds. While program management should respect the freedom and flexibility of institutions’ use of funds, there is an expectation on the part of the program and chairholders that some type of package should be provided. Although there are guidelines, there are currently no requirements regarding chairholder packages.

The diminishing chairholder package may threaten the achievement of all four of CRCP’s objectives. If the package continues to diminish to the point of becoming uncompetitive, there will be a threat to Objective 1 (increasing Canada’s research capacity). Similarly, if this diminishing chairholder package reduces the amount of funding available for HQP, it would represent a threat to Objective 2 (training of HQP). If the amounts available in the chairholder package for research diminish, this would represent a threat to Objective 3 (generating and applying new knowledge). If the package diminishes to the point of becoming indistinguishable from a regular faculty position, this may represent a threat to Objective 4 (best possible use of research resources through strategic planning), as the chairholder appointments may no longer represent key investments in institutional strategy.

Management could ensure more robust packages by considering one or more of the following options: 1) requiring a minimum amount of protected research time for all chairholders; 2) targeting a portion of CRCP funds to research activities and/or HQP stipends or salaries; and 3) placing a restriction on use of CRCP funds for chairholder salaries. If the award amount is increased as per Recommendation 1, the amount of this increase could be tied to the above requirements (i.e., the additional funds could be targeted to research only). Management will need to consider the role of institutions in responding to this recommendation.

Recommendation 3: Management should set targets for the attraction and retention of researchers from abroad, and if necessary develop a plan to better support this group, in order to ensure that the program’s objective of attracting top researchers from outside Canada is realized.

In the past five years, there has been a large reduction in the amount of researchers from abroad awarded a Chair. At the same time, international chairholders are much more likely than others to resign their award.

Attraction of top researchers from outside Canada is one of the expected outcomes of the program. A key impetus for the creation of the program was the desire to reverse the outward migration of top researchers. If not addressed, this issue may threaten the achievement of Objective 1, which aims to “increase Canada’s research capacity by attracting and retaining 2,000 top researchers from within Canada and abroad”. The program will increasingly serve the function of retention as opposed to attraction, which arguably contributes less to the goal of increasing capacity.

The relative importance of the use of the program to attract researchers from abroad should be explored and confirmed, examining the extent to which the evaluation’s findings around reduced attraction and greater attrition pose a risk to the program’s objectives. Based on this, targets should be set for attraction from abroad. These targets should be captured within the program’s Performance Measurement Strategy. If current levels are lower than the targets set, a plan should be developed to encourage greater attraction and retention of researchers from abroad.

Recommendation 4: Management should examine the extent to which the “fit with SRP” review criterion remains relevant, and whether or not it should be replaced with a criterion assessing alignment to demonstrated priority areas

At some institutions, the extent to which Strategic Research Plans are used to inform strategic action is in question. An institution’s strategy for allocating CRCs may not be explicitly reflected in its SRP, which may be kept broad to ensure maximum flexibility of funding use. According to the action plan coming out of the 2015 Pan-Canadian Consultation, the CFI will remove alignment to SRPs as a merit review criterion, instead looking at whether the application “represents a tangible demonstration of institutional priority”, based on institutional track records.

If the review criterion is not changed, the program’s peer review process may be less effective in supporting CRCP’s Objective 4 (best possible use of research resources through strategic planning), if the SRP does not represent the best indicator of an institution’s priority research areas.

The CRCP may wish to carry out consultations with CFI on its merit review revisions, and investigate the extent to which similar revisions would be desirable for the CRCP.

Recommendation 5: Management should require institutions to adopt greater transparency in their processes for allocation of Chair positions and selection and renewal of chairholders, in order to ensure institutions have greater accountability in terms of meeting their equity targets.

Despite work done to date by the Secretariat to promote equitable practices, the majority of institutions do not meet targets set for women, visible minorities and persons with disabilities. Although guidelines exist for establishing fair and transparent processes, the extent to which these practices have been adopted is unknown.

While respecting the freedom of institutions, program management should require institutions to adopt transparent processes and criteria for nominee selection/review and chairholder renewal. This would include a requirement that such processes be made publicly available.

Recommendation 6: Management should investigate ways to minimize the impact of loss of occupied Chairs in the re-allocation process.

Loss of occupied Chair positions presents a challenge for all institutions, especially for small institutions. The loss of an occupied Chair at a smaller institution can mean the loss of the researcher, as smaller institutions are sometimes unable to absorb the unanticipated cost of the Chair award into their core operating budget. Furthermore, the prospect of the loss of Chairs may provide an incentive for institutions to keep some Chair positions vacant, to prevent future losses of occupied Chairs. In both cases, these represent threats to the program’s Objective 1 (increasing Canada’s research capacity).

Program management should find ways to diminish the need to take away occupied Chair positions. Potential solutions to be explored may include:

• carrying out an annual recalculation to provide advance notice of impending gains and losses;
• removing occupied, active Chair positions from the re-allocation formula (i.e., only vacant and term-ended Chair positions are subject to re-allocation); and
• changing the rolling average calculation to smooth out the effect of sudden, large changes.

Recommendation 7: Management should consider imposing a limit of one renewal for Tier 1 Chairs.

There is some evidence of an age gap between Tier 1 and Tier 2, which is due to the renewal of Tier 1 chairholders. This gap is expected to increase in the near future, as the earliest Tier 1 chairholders begin to become eligible for their second renewal. The fact that the majority of Tier 1 Chair awards are now held by renewed chairholders means that institutions have fewer opportunities to attract new researchers through the Tier 1 award. The evaluation did not uncover evidence that the limit on renewals for Tier 2 chairholders has had a negative impact on the ability of institutions to retain these researchers after the end of their award.

Therefore, this evaluation recommends that a limit on the number of renewals (i.e., one renewal) should be extended to all chairholders, Tier 1 and 2 alike.

---

Appendix A—Detailed Program Description

A.1 Policy Context and Rationale

The CRCP is a permanent program launched in 2000 that was designed to strengthen Canada’s research capacity and offset “brain-drain” pressures by helping Canadian universities and their research affiliates retain talented Canadian researchers and attract the best international researchers to Canada. Along with other programs (Vanier, Banting, Canada Excellence Research Chairs, etc.), the CRCP is expected to play a prominent role in the Government of Canada’s 2014 strategy, Seizing Canada's Moment: Moving Forward in Science, Technology and Innovation. As such, the CRCP strives to make Canada one of the world’s top countries for research and development in order to build an innovative and competitive economy, increase future job opportunities and, ultimately, improve Canadians’ quality of life.

The program is a tri-agency initiative of the Canadian Institutes of Health Research (CIHR), the Natural Sciences and Engineering Research Council of Canada (NSERC), and the Social Sciences and Humanities Research Council (SSHRC); the Canada Foundation for Innovation (CFI) also provides accompanying funds to support infrastructure.

A.2 Program Objectives

The main objective of the CRCP is to enable Canadian universities, together with their affiliated research institutes and hospitals, to foster research excellence and to enhance their role as centres of research excellence in the global, knowledge-based economy. More specifically, the program intends to achieve the following:

• Increase Canada’s research capacity. The program will increase Canada’s research capacity by attracting and retaining 2,000 top researchers from within Canada and abroad.
• Improve training of highly qualified personnel. The program will increase the number of highly qualified personnel trained through research in Canadian universities. It is also expected that the quality of training will be improved.
• Improve universities’ capacity to generate and apply new knowledge. By attracting and retaining top researchers, Canadian universities will be able to increase research outputs as well as the dissemination, transfer and use of knowledge.
• Best possible use of research resources through strategic planning. Strategic planning will help universities to focus their efforts on their research strengths as well as new areas in which they want to develop strengths.

A.3 Governance Structure and Responsibilities

Housed within SSHRC, the Tri-agency Institutional Programs Secretariat (TIPS) is responsible for the day-to-day administration of the CRCP. It reports to the Management Committee, which itself reports to the tri-agency programs Steering Committee. The main responsibilities of TIPS include:

• the calculation of allocations of Chairs;
• the review of nominations and organization of the peer review process;
• the provision of advice and guidance to institutions and chairholders;
• ongoing financial and operational monitoring of compliance of recipients with the terms and conditions of the program; and
• the collection of performance and financial data.

The Management Committee is a coordinating mechanism composed of a representative at the director general level from each of the three granting agencies—the Canada Foundation for Innovation, Health Canada and Innovation, Science and Economic Development Canada—as well as the executive director of the Chairs program. It is chaired by the SSHRC representative. The Management Committee’s mandate consists of:

• The operational oversight of the organization, implementation and continuing administration of the program, acting on authorities delegated from the Steering Committee; and
• Advising the Steering Committee by providing recommendations for its consideration on matters of program design, competitions, post-award monitoring, evaluation and audit, communications, research policy, reporting and national tri-agency policy.

In addition to playing an important role in the peer review process, the Steering Committee oversees the management of the program and provides strategic advice on the program’s general direction, more particularly on:

• the program design and delivery;
• performance management, post-award monitoring, evaluation;
• communications and reporting activities;
• accountabilities (governance structure);
• control activities; and
• high-order governance for the suite of tri-agency programs, including the foresight of longer-term developments that bear on the science and technology environment and may impact the demand for or contribution of tri-agency programs.

A.4 Stakeholders and Beneficiaries

The direct beneficiaries of the program are Canadian universities and their affiliates, chairholders, and chairholders’ students and trainees (the HQP). Canadian industry, governments, not-for-profit organizations and, ultimately, Canadian citizens may benefit from new technologies, products and treatments that increase their quality of life.

A.5 Program Delivery

Chair allocation process

A total of 2,000 Chairs are allocated to institutions that have received an average of $100,000 or more annually from the three federal agencies in the three years prior to the year of the allocation. Canada Research Chairs are divided into two tiers:

• Tier 1 Chairs are tenable for seven years and renewable indefinitely. These are for outstanding researchers acknowledged by their peers as being world leaders in their fields. Tier 1 nominees must be full professors or associate professors who are expected to be promoted to the full professor level within one or two years of the nomination. Should they come from outside of academia, nominees must possess the necessary qualifications to be appointed at these levels. For each Tier 1 Chair, universities receive $200,000 annually for seven years.
• Tier 2 Chairs are tenable for five years and renewable once. They are for exceptional emerging researchers, acknowledged by their peers as potential leaders in their field. Nominees for Tier 2 positions must be assistant or associate professors or possess the necessary qualifications to be appointed at these levels. Universities must justify why (e.g., clinical training, years in industry, breaks in career) a Tier 2 nominee is more than ten years away from the highest degree at the time of nomination. For each Tier 2 Chair, universities receive $100,000 annually for five years.

Chairs are distributed to institutions in an alternating order: a Tier 2 is awarded first, followed by a Tier 1, followed by a Tier 2, and so on. Every two years, the number of Chairs allocated to eligible universities is recalculated and revised based on the level of financial support received from the three granting councils (CIHR, NSERC and SSHRC) over the previous three years; this process is often referred to as the “re-allocation” of the Canada Research Chairs. Of the total 2,000 Chairs, 1,880 are regular allocations distributed by disciplinary area as follows: NSERC 45%, CIHR 35% and SSHRC 20%.

The program also sets aside a special allocation of 120 Chairs for universities that have received one per cent or less of the total funding paid out by the three federal granting agencies over the three years prior to the year of the allocation. This is known as the “1% threshold.” Unlike regular allocations, these Chairs are not allocated by area of research aligned with those of the granting agencies, and universities can choose the areas in which they would like to use the Chairs. Universities may also exchange one Tier 1 Chair for two Tier 2 Chairs, or two Tier 2 Chairs for one Tier 1 Chair. This is true for both special and regular Chairs. To be eligible for a special Chair allocation, institutions must meet all three of the following criteria:

• their total grant dollars from all three granting agencies must be less than the 1% threshold;
• they must have received more than an average of $100,000 of grant dollars over the three fiscal years; and
• they must have received less than 11 regular Chairs. The maximum number of special Chairs allocated per university is three Chairs.

Nomination process

Each institution that receives Chairs must first develop a Strategic Research Plan that demonstrates how they will use funding from the Chairs (including CFI funding) to attract and retain world-class experts in specific areas of research where they intend to develop or maintain excellence. Institutions are required to have an open, fair and transparent nomination process as outlined in the Guidelines for ensuring a fair and transparent recruitment and nomination process, available on the CRCP website. Then institutions must submit a nomination to TIPS for each Chair they were allocated. The nominee should be a researcher whose work complements the strategic research plan of the institution and who meets the program’s high standards. All nominations are assessed against the following criteria:

• the quality of the nominee and the quality of the proposed research program; and
• the quality of the institutional environment and commitment and fit of the proposed Chair with the university’s strategic research plan.

Universities may choose to include a request for infrastructure support from the CFI with their Chair nomination or request funding from the CFI for infrastructure shared by two or more Chairs. Note that the university is responsible for the administration of the funds for successful nominations. These are awarded directly to the institution (and not to the chairholders), who allocates the funds according to program guidelines on eligible expenses.

The program follows a peer review process governed by the College of Reviewers and the Interdisciplinary Adjudication Committee (IAC). All nominations, whether they are for a new Chair or the renewal of a current Chair, are assessed by the College of Reviewers. The College of Reviewers is composed of experts (including current chairholders) from a wide range of fields of research. For each Chair nomination, TIPS selects a minimum of three reviewers from the College of Reviewers to assess nominations and the accompanying CFI infrastructure requests. Based on these assessments, TIPS makes funding recommendations to the Steering Committee. If the reviewers concur and their assessment is favourable, TIPS makes a funding recommendation to the Steering Committee to support the Chair. In the case of nominations for researchers from abroad (foreign nationals or Canadian citizens) where peer review is unanimously favourable, the executive director of CRCP has the authority to approve the recommendations. If any of the three assessments is not favourable, the nomination goes to IAC, which then makes a recommendation to the Steering Committee on whether to support the nomination.

IAC is composed of 15 experts plus one Chair from the College of Reviewers. A superior record of research achievement, extensive experience, sound judgment, and proven ability to recognize excellence are the prime considerations in the selection of IAC members. As well, the membership of IAC balances, as far as possible, language (English/French), gender, region, economic sector, academic discipline and type of institution. IAC reviews all nominations in cases where an unfavourable review is received by the College of Reviewers. IAC also plays a major role in ensuring the consistency of standards across the program.

Equity

In the earlier years of the program, women were underrepresented among CRCP nominees, and there were concerns about access to the program by members of the other three designated groups (i.e., visible minorities, persons with disabilities and Aboriginal Peoples). CRCP committed to ensuring access and opportunities to all qualified candidates, while maintaining standards of excellence. Particular attention was paid to increasing women’s access to the program.

A.6 Resources

The federal government provided an initial budget of $900 million for the creation of CRCP in 2000, and $300 million per year until 2010-11. As of 2012, the amount allocated to the program reflects the Deficit Reduction Action Plan budget reductions. Following the Deficit Reduction Action Plan, the annual program budget is $265 million, down from $300 million after a structural surplus from unfilled Chairs was returned to the fiscal framework. The CFI initially provided $250 million for infrastructure to support chairholders’ research and continues to provide infrastructure funds via the John R. Evans Leaders Fund. Chairholders may also obtain funding through the CFI programs not tied to CRCP.

Table A1: CRCP Annual Budget, 2010–14

Period	Total budget	Administrative budget	Grants	Paid	Surplus
2010-11	$300,000,000	$3,976,000	$296,024,000	$260,511,774	$35,512,226
2011-12	$300,000,000	$3,976,000	$296,024,000	$261,689,386	$34,336,614
2012-13	$282,750,000	$3,976,000	$278,774,000	$258,597,345	$19,876,655
2013-14	$265,500,000	$3,976,000	$261,524,000	$254,853,515	$6,670,485

Source: Tri-agency Institutional Programs Secretariat

A.7 Logic Model

The program’s logic model is presented below.

Download Image

Source: CRCP (2014). Performance Measurement Strategy for the Canada Research Chairs Program (CRCP)

Description of Logic Model

This image presents the program’s logic model. The logic model consists of seven rows, each with four boxes. The first box of each row is labeled as follows:

1. Inputs
2. Activities
3. Outputs
4. Immediate Outcomes
5. Intermediate Outcomes
6. Long-term Outcomes
7. Strategic Outcomes

The first row of boxes describes the three inputs for the program:

• 1.1 NSERC/SSHRC/CIHR: $265 million per year
• 1.2 CFI: Initial investment of $250 million and New Leaders Opportunity Fund (now the John R. Evans Leaders Fund)
• 1.3 Universities: Resources to support Chairs (research funding, infrastructure, teaching relief, hiring of researchers, etc.);

The second row of boxes describes the three activities of the program:

• 2.1 CRC program: Calculation of allocations and communication of result to universities
• 2.2 College of reviewers and IAC: Peer review of Chairs nominations and infrastructure support
• 2.3 Universities: Development of Strategic Research Plans (SRPs), nomination of candidates, and submission of applications including SRPs and commitment to Chairs

The third row of boxes describes the three outputs of the program:

• 3.1 Tier 1 and Tier 2 Chairs are awarded to top researchers
• 3.2 CFI support awarded and invested in leading-edge infrastructure
• 3.3 Universities invest in focus efforts on strategic research areas (areas of Chairs’ research)

The fourth row of boxes describes the three immediate outcomes of the program:

• 4.1 Attraction and retention of excellent researchers in Canadian universities
• 4.2 Universities establish dynamic research teams in areas of Chairs’ research
• 4.3 Universities use resources effectively through strategic planning

The fifth row of boxes describes the three intermediate outcomes of the program:

• 5.1 Improved capacity to generate and apply new knowledge in Canada
• 5.2 Establishment of partnerships and leverage of partner contributions
• 5.3 Improve training of highly qualified personnel

The sixth row of boxes describes the three long-term outcomes of the program:

• 6.1 Increased intersectoral and interinstitutional collaboration
• 6.2 Larger pool of highly qualified personnel in Canada
• 6.3 Canadian universities develop a comparative advantage in their strategic areas of research

The seventh and final row of boxes describes the three strategic outcomes:

• 7.1 Canada is a world leader in social sciences and humanities research and research training (SSHRC)
• 7.2 A world-class health research enterprise that creates, disseminates and applies new knowledge across all areas of health research (CIHR)
• 7.3 Highly skilled science and engineering professionals in Canada (NSERC)

This logic model was sourced from the CRCP (2014) Performance Measurement Strategy for the Canada Research Chairs Program.

---

Appendix B—Evaluation Question Coverage

Evaluation Question	Section of Evaluation Report
1. Is there a continued need for the CRCP given the evolving research funding context since 2009-10?	3.1.3, 3.2.4
2. Is the CRCP aligned with the Government of Canada’s and the tri-agencies’ mandates/priorities?	3.1.3
3. Is the provision of CRCP awards an appropriate role for the federal government?	3.2.5
4. Have the CRCP awards resulted in the attraction and/or retention of excellent Canadian and foreign researchers?	3.1.1, 3.2.5
5. Have the CRCP awards resulted in the creation of dynamic research teams and projects in support of the Chairs?	3.1.1, 3.2.5
6. Have the CRCP awards resulted in the creation, dissemination and application of new knowledge in strategic areas of research?	3.1.1
7. Have the CRCP awards resulted in a comparative advantage for universities in strategic areas of research?	3.2.2, 3.3.1, 3.3.2
8. Has the CRCP resulted in any unintended impacts (either negative or positive)?	3.1.1
9. Has the CRCP been delivered in a cost-efficient manner?	3.1.2
10. To what extent is the implementation of the design and delivery features of the CRCP appropriate for meeting its current (and potential future) objectives?	3.2.1, 3.2.3, 3.4.1, 3.4.2, 3.4.3

---

Footnotes