2022 CES Conference / Congrès 2022 de la SCÉ: Full Schedule

Diversity, our interwoven experiences / La diversité, nos expériences qui s’entremêlent

10:00 CDT

(Workshop/Atelier) Diversité et complémentarité des approches d'évaluation des interventions publiques

Toutes les interventions publiques sont réalisées de façons interreliées et multi-dimensionnelles. Pour discerner leur diversité et complémentarité, nous aurons recours à une stratification de six niveaux d'intervention dont chacun oblige l'application de concepts évaluatifs spécifiques et appropriés aux objets d'évaluation. Le premier niveau est celui des activités administratives réalisées grâce aux procédures ayant des ciblages précis et le second niveau est celui des projets qui impliquent plusieurs parties prenantes interagissant selon un scénario déterminé. Le troisième niveau est celui des programmes multipartites réalisés en partenariat. Le quatrième niveau concerne l'efficacité institutionnelle des intervenants. Le cinquième niveau est celui des stratégies interactives d'intervention et le sixième, celui du systémique des impacts des politiques publiques. Durant l'atelier nous examinerons l'outillage nécessaire à chaque niveau et les liens inter-niveaux pour une meilleure compréhension de la connectivité des interventions publiques.

Speakers/Présentateur·trice·s

Jean Serge Quesnel

ENAP

Professeur associé à ENAP. Fut Directeur d'évaluation à l'ACDI, BID et UNICEF; Président du Groupe d'expert en évaluation de l'OCDE, Président du Groupe de coopération en évaluation des Institutions financières internationales, Président du Groupe des Normes d’évaluation... Read More →

Saturday June 11, 2022 10:00 - 13:30 CDT
Online

Online, Diversity within evaluation/Diversité au sein de l'évaluation

14:00 CDT

(Workshop/Atelier) État des lieux mondial de l'évaluation - Diversité et cohérence systémique

Depuis les vingt-cinq dernières années, l'évaluation s'est développée mondialement à un rythme exponentiel. Un examen de l'état des lieux permettra de discerner les acquis et les tendances lourdes. L'atelier examinera les divers niveaux de réseaux d'évaluation allant du global au local, et à chaque niveau les caractéristiques particulières. L'atelier permettra de réfléchir sur les nombreuses inter-influences entre ces niveaux, à la fois verticalement et horizontalement. L'atelier exposera les entités contribuant au développement de l'évaluation. Un examen de ces situations diverses permettra de dégager des méga tendances vers une harmonisation de l'institutionnalisation et la professionnalisation de l'évaluation, et ce, dans le respect de la diversité.

Speakers/Présentateur·trice·s

Jean Serge Quesnel

ENAP

Saturday June 11, 2022 14:00 - 17:30 CDT
Online

Online, Diversity of Evaluators/Diversité d'évaluateur·trice·s

14:00 CDT

(Workshop/Atelier) Impact Evaluations in the Canadian Context: Beyond "What Works" to What Works, for Whom, and Why

This workshop, developed in response to the Government of Canada's publication "Measuring Impact by Design", will provide background to core concepts behind impact evaluations. It should be considered an introduction to the topic. The first main goal is to build attendees' intuitions around methods of experimental and quasi-experimental evaluations. The second main goal is to equip attendees with concepts and language important for understanding how impact evaluations can be used to go beyond the question of "what works" to assess "what works for whom and under what conditions". We will address four main topics: (1) Why impact studies? Benefits and limitations (1hr) (2) Review of four main designs (randomized experiments, matching, difference-of-means and pre-post) (1hr) (3) Achieving flexibility in design and analysis to address variation in impacts based on criteria of diversity and for assessing equity in outcomes (1hr) (4) Looking to the future (1/2hr).

Speakers/Présentateur·trice·s

Andrew Jaciw

Dr., ImpactEval

Jaciw is an impact evaluator and teacher. He holds an MS in Math Education(OISE), an MSc in Epidemiology(Stanford) and a PhD in Education Research(Stanford). He has led 30 impact evaluations, most recently a cluster randomized trial involving 700 schools and 180,000 households. Interests... Read More →

Saturday June 11, 2022 14:00 - 17:30 CDT
Online

Online, Diversity within evaluation/Diversité au sein de l'évaluation

07:00 CDT

(Food/Alimentation) Breakfast/Déjeuner

Sunday June 12, 2022 07:00 - 09:00 CDT
Tamarak

Tamarak

08:00 CDT

(Workshop/Atelier) Analyzing Qualitative Data Using Qualitative and Quantitative Analysis Techniques

This intermediate level workshop presents qualitative and quantitative analysis techniques of data such as interview findings. At the end of the workshop, participants will be able to conduct these types of analyses in a clear and systematic fashion. The workshop makes a clear distinction between both techniques and will review key qualitative and quantitative analysis techniques using MS Excel. A high-level overview of computer-assisted qualitative analysis software will be provided. Key techniques will be presented, in a step-by-step process, using a live case. Participants will be invited to bring their laptops to apply the techniques directly. Participants without a computer can also attend and follow the process on the main screen. The pros and cons, and best practices, of quantifying the evidence using percentages and proportions will be discussed.

Speakers/Présentateur·trice·s

Pascale Latulippe

Senior Consultant, Goss Gilroy Inc.

Simon Roy

Dr, Goss Gilroy Associate

Simon Roy is a credentialed evaluator with over 25 years of experience. He conducted evaluations in multiple sectors, including human resources and social development, policy, Indigenous programs, natural resources, science and R&D, health, safety, and many others. He teaches program... Read More →

Sunday June 12, 2022 08:00 - 11:30 CDT
Jack Pine

Jack Pine, Diversity within evaluation/Diversité au sein de l'évaluation

08:00 CDT

(Workshop/Atelier) Making Reconciliation Real: Embedding key "94 Calls to Action" into evaluation practice

The goals of this workshop: 1) to increase knowledge of the 94 Calls to Action as the relate to providing an evaluation practice that integrates the UN Declaration on the Rights of Indigenous Peoples; 2) to increase skill in designing and implementing evaluations that contribute to achieving the relevant Calls to Action; and, 3) to develop evaluators' skills to work with clients on this integration. Main topics: 1) explore health, education and justice Calls to Action; 2) evaluators and the National Council for Reconciliation; 3) retrospective look at proposals and evaluation frameworks - where would reconciliation have fit? 4) Going forward - how do we embed reconciliation, tactically, in our evaluation work? Guiding questions: How does knowing more about the Indian Act affect your practice? In what ways can our work support social and political impacts of replacing the Indian Act and Doctrine of Discovery with UNDRIP.

Speakers/Présentateur·trice·s

Kate Woodman

Catalyst Research and Development Inc

Kate is a PHD and CE with over 25 years' experience in research and evaluations in Canada, Ukraine and Zambia. She applies a human rights based evaluation lens and brings these values into work with diverse and vulnerable populations, most especially in sectors of violence against... Read More →

Krista Brower

Director, Evaluation & QI, EOPCN

Krista is a CE with over 17 years’ experience in a variety of research and evaluation settings. She holds a PhD in Research, Evaluation and Enhancement as well as other graduate degrees in evaluation. Krista is passionate about building high performing teams and positive workplace... Read More →

Sunday June 12, 2022 08:00 - 11:30 CDT
Balsam

Manitoba Maple, Diversity within evaluation/Diversité au sein de l'évaluation

09:00 CDT

(Workshop/Atelier) Evaluating Community Collaboratives

This interactive, participative workshop will provide evaluators with information to help them: 1) understand collaborative dynamics; 2) apply models, frameworks and guiding principles for collaboratives into their evaluations; 3) identify key concepts and approaches to evaluating collaboratives; 4) gain working knowledge of available measures and tools for evaluation of collaboratives; 5) be prepared to overcome challenges; 6) apply a participative, community empowered approach; and, 7) know how to apply best practices to promote the evaluation use evaluation among collaboratives. Woven throughout each topic will be guidelines to incorporate cultural humility, equitable evaluation, and anti-racism. The presenters will use a combination of large and small group discussions and exercises, individual reflections, think-pair-share activities, and minimal lecture. The presenters received positive feedback for this workshop when they presented virtually at the CES conference in 2021.

Speakers/Présentateur·trice·s

Ann Price

President, Community Evaluation Solutions, Inc

Dr. Price is a community psychologist with over 20 years of clinical, prevention and evaluation experience. In her work with community coalitions and collaboratives that are working to promote systems-level change in communities, Ann helps community coalitions and collaboratives use... Read More →

Susan Wolfe

CEO/Community Consultant, Susan Wolfe and Associates

Dr. Susan Wolfe, a Community Psychologist, has over 36 years of evaluation and community experience, that includes evaluating collaboratives. She has facilitated workshops on this topic internationally. She is the co-editor of the 2020 New Directions for Evaluation issue titled “Evaluating... Read More →

Sunday June 12, 2022 09:00 - 17:00 CDT
Cedar

Cedar, Diversity of Evaluators/Diversité d'évaluateur·trice·s

09:00 CDT

(Workshop/Atelier) Systems Evaluation Theory: Bridging theory to practice

The demand for evaluators who can apply systems thinking to evaluate complex systems is growing (Patton, 2020). However, one barrier to evaluators pursuing systems thinking is the perception that systems evaluation is complex and abstract. This workshop will simplify systems thinking principles for participants by drawing on everyday examples, the COVID pandemic, and a series of published work surrounding Systems Evaluation Theory (SET) by the lead facilitator including the CJPE 2020 best practice article. The workshop goals are to help participants recognize the difference between a program and system evaluation, understand how system properties inform SET, and to learn how to systematically apply SET to navigate the evaluation of complex systems.

Speakers/Présentateur·trice·s

Ralph Renger

President, Just Evaluation Services (JESS)

Dr. Renger specializes in advancing theory and methods for program and system evaluation. He taught program and system evaluation for 23 years at the Universities of Arizona and North Dakota before starting his own consulting firm. Most recently he has garnered international recognition... Read More →

Sunday June 12, 2022 09:00 - 17:00 CDT
Prairie Grass

Prairie Grass, Diversity within evaluation/Diversité au sein de l'évaluation

09:00 CDT

(Workshop/Atelier) Opening Space for More Worldviews: A Workshop for Facilitators

As we seek to bring more perspectives around decision-making tables, evaluators need to build their capacity to navigate diverse ways of seeing the world while also guiding those who are used to a single way of being, amidst navigating timelines and deliverables. When people differ from ourselves in their epistemic (ways of knowing) and ontologies (sense of truth about the world), our bodies respond. For those who live privileged in most contexts, hearing diverse perspectives and lived experiences can feel unusually troubling. This session is for evaluators who facilitate in multi-stakeholder and team settings and seek to identify epistemic and power considerations, towards the aim of building more equitable spaces. Specifically, we will discuss how to create "stretching" spaces where new-to-them perspectives can be (more) valued and heard for intentional and respectful interweaving of perspectives.

Speakers/Présentateur·trice·s

Brian Hoessler

Founder, Strong Roots Consulting

Brian Hoessler often follows his curiosity, whether in exploring a city, learning a new skill, or building connections with organizations and communities. As the founder of Strong Roots Consulting, he calls on this superpower alongside those of strategic planning and program evaluation... Read More →

Carolyn Hoessler

Senior Specialist; Educational Program Designer, @ Higher Education and Beyond; University of Saskatchewan

Carolyn Hoessler, PhD, a Credentialed Evaluator, higher education change consultant, curriculum developer, research design & data educator, and facilitator of over 30 international and national workshops. Through Higher Education & Beyond, I facilitate meaningful and effective processes... Read More →

Sunday June 12, 2022 09:00 - 17:00 CDT
Sweet Grass

Sweet Grass, Diversity of Evaluators/Diversité d'évaluateur·trice·s

12:00 CDT

(Food/Alimentation) Lunch/Dîner

Sunday June 12, 2022 12:00 - 13:00 CDT
Tamarak

Tamarak

13:00 CDT

(Workshop/Atelier) Tailoring evaluation capacity development for diverse partners: A guide for facilitators

This workshop is meant for facilitators of capacity building and organizational learning. Those with a background in evaluation and in communication will find it most relevant. For the last decade, our team has provided evaluation and communication capacity building support through just-in-time mentoring. It is a form of facilitation that helps partners move at their own pace; it allows them to discover the value of program evaluation and communication planning through experiential learning. We constantly adjust the intent and tone of the conversation to the context, depending on the questions raised, and rely upon our own intuition -- our practical wisdom. We do this within a framework of utilization-focused evaluation (UFE) and communication capacity building principles. We will share the principles that guide our work, as well as the practical steps we follow during mentoring. Participants will have free access to a Guide summarizing our experience, to be published in 2022

Speakers/Présentateur·trice·s

Ricardo Ramirez

DECI Project, independent

Ricardo Ramirez has been registered as an independent researcher and consultant in Guelph, Ontario since 1997. He holds a PhD in Inter-disciplinary Rural Studies and is a Credentialed Evaluator with the Canadian Evaluation Society. His current work includes the design and implementation... Read More →

Sunday June 12, 2022 13:00 - 16:30 CDT
Jack Pine

Jack Pine, Diversity within evaluation/Diversité au sein de l'évaluation

13:00 CDT

(Workshop/Atelier) Sustainability-ready evaluation

The world is facing unprecedented environmental threats yet evaluation with its focus on human systems is paying scant attention to sustainability and climate and contributing little to efforts to forestall the climate and sustainability crises. The first half of the workshop will provide an overview of current core elements for evaluating sustainability and climate and working in small groups identify how they might implement these. Changing well entrenched practices is challenging when the intellectual infrastructure for evaluation focuses on human system matters. In the second half of the workshops presenters and participants will undertake several thought experiments during which presenters will provide some important tools and processes developed by Footprint Evaluation specifically for addressing human and natural systems together. An important ambition is to identify further strategies and actions that participants feel would assist them to become sustainability-ready evaluators.

Speakers/Présentateur·trice·s

François Dumaine

Partner, PRA Inc.

François Dumaine is a Partner at PRA Inc. As an evaluation practitioner, he has had a wide range of opportunities to bridge theory and practice, and explore new ways of ensuring that information needs of decision-makers are efficiently met. François served as President of CES and... Read More →

Debbie DeLancey

D.J. DeLancey Consulting

Debbie has worked with Dene communities coordinating research and advocacy on major resource development projects, facilitating community engagement, and managing an Indigenous owned consulting firm specializing in participatory research and traditional knowledge. With NWT Government... Read More →

Andy Rowe

Dr, Footprints Evaluation

Andy has a PhD from the London School of Economics, is a Fellow and former President of the CES and an early contributor to evaluating sustainability and mainstreaming sustainability in evaluation. He initiated the CES effort to mainstream sustainability and is a founding member of... Read More →

Sunday June 12, 2022 13:00 - 16:30 CDT
Balsam

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

18:00 CDT

(Social) Opening Reception & Silent Auction/Réception d'ouverture et encan silencieux

Sunday June 12, 2022 18:00 - 21:00 CDT
Tamarak

Tamarak

07:00 CDT

(Social) Morning Movement /Exercices matinaux

Monday June 13, 2022 07:00 - 08:00 CDT
Hospitality Desk/Bureau

Hospitality Desk/Bureau

07:30 CDT

(Food/Alimentation) Breakfast/Déjeuner

Monday June 13, 2022 07:30 - 08:30 CDT
Tamarak

Tamarak

08:30 CDT

(Keynote/Conférence principale) Opening / Ouverture

Co-Chair introduction / Introduction par les Co-présidents
Elder welcome / Mot de bienvenue de l'aîné
Welcome by the CES President / Bienvenue de la présidente de la SCÉ
Welcome by the Co-Chair / Bienvenue des Co-présidents

Speakers/Présentateur·trice·s

Doaa Saddek

CES President, CES

Doaa Saddek has 22 years of professional experience in program evaluation and policy and socioeconomic research through a broad involvement with the non-profit sector, international and governmental organizations in Canada and abroad. As an economist, she evaluated programs under... Read More →

Matthew Sanscartier

Business Intelligence Analyst, Downtown Community Safety Partnership

I have a PhD in sociology (specializing in organizational sociology and quantitative/qualitative research methods). I currently work for the Downtown Community Safety Partnership, a non-profit dedicated to making the heart of Winnipeg safe for everyone who lives, plays, and works... Read More →

Ryan Catte

Manager, Evaluation and Impact, Research Manitoba

Monday June 13, 2022 08:30 - 09:00 CDT
Tamarak

Tamarak

09:00 CDT

(Keynote/Conférence principale) Opening Keynote/Conférence d'ouverture

Speakers/Présentateur·trice·s

Isha Khan

Chief Executive Officer, Canadian Museum for Human Rights

Isha Khan (she/her) is a lawyer, educator and community leader dedicated to building a culture of human rights in Canada and beyond. She assumed her role as CEO of the Canadian Museum for Human Rights in August of 2020. Born in Winnipeg, she holds degrees from the University of Manitoba... Read More →

Sponsors/Commanditaires

Ference & Company Consulting Ltd.

Ference & Company is one of the most experienced leading authorities on program evaluation in Canada. Initiated in 1980, the firm has conducted over 700 assignment including mixed-method summative program evaluations, formative evaluations, ongoing evaluations, evaluation frameworks... Read More →

Monday June 13, 2022 09:00 - 10:00 CDT
Tamarak

Tamarak

10:00 CDT

(Food/Alimentation) Break/Pause

Monday June 13, 2022 10:00 - 10:30 CDT
Conference Foyer/Hall du congrès

Conference Foyer/Hall du congrès

10:30 CDT

Solo (Expert tutorial/Tutoriel d'expert·e) Queering evaluation: An LGBTQ2S+ primer

Gender and sexual minority individuals in Canada -- including lesbian, gay, bisexual, transgender, queer, Two-Spirit, and other gender and sexual minority (LGBTQ2S+) people -- continue to experience diverse forms of systemic and structural inequities. This presentation provides a practical introduction to gender identity and expression, sexual orientation, the LGBTQ2S+ community, and relevant issues for evaluators interested in ensuring their work is culturally responsive and accessible. It is designed as a guide for conducting evaluation that considers gender and sexual orientation (among other intersecting characteristics) in evaluation projects spanning diverse areas of policy. With information about key terms and concepts, relevant guidelines, theories and frameworks, this presentation serves as a resource for evaluators looking for inclusive and ethical approaches to measuring, collecting, analyzing, and reporting data with LGBTQ2S+ individuals.

Speakers/Présentateur·trice·s

Basia Pakula

Senior Research Associate, Social Research and Demonstration Corporation

Dr. Basia Pakula (she/her) is a queer, immigrant settler on the unceded territories of the Musqueam, Squamish and Tsleil-Waututh Nations. With over 15 years of experience in the planning, monitoring, and evaluation of social programs and initiatives, she has dedicated her career to... Read More →

Andrea Bobadilla

Social Research and Demonstration Corporation

Andrea Bobadilla is an Ottawa-based Research Associate with the Social Research and Demonstration Corporation (SRDC). She contributes to projects across social policy and population health contexts, including youth development, education, skills, and mental health. Her research background... Read More →

Chloe Halpenny

Social Research and Demonstration Corporation

Chloe Halpenny (she/her) is a bisexual Research Associate based in SRDC's Ottawa office, living and working on unceded, unsurrendered territory of the Algonquin Anishinaabe People. A multi-disciplinary researcher with a passion for social, gender, and economic justice, Ms. Halpenny... Read More →

Monday June 13, 2022 10:30 - 12:00 CDT
Manitoba Maple

Manitoba Maple, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

Solo (Debate/Débat) EvalAgenda 2030: Reinventing the wheel, or revolutionizing evaluation?

In 2015, EvalPartners and its five networks launched the Global Evaluation Agenda (EvalAgenda) -- a global first for the evaluation profession. In 2020, the global partnership developed a Roadmap towards EvalAgenda2030, marked by four key steps: i) Stocktaking, ii) Stakeholder engagement, iii) Co-creating of the EvalAgenda; and; iv) Dissemination and advocacy of the EvalAgenda2030. The stocktaking exercise used various data sources and processes to determine progress towards the four pillars of the Global Evaluation Agenda 2015-2020: 1. Strengthening an enabling environment for evaluation, 2. Strengthening institutional capacities, including those of VOPEs and civil society, 3. Strengthening individual capacities for evaluation, and 4. Strengthening the inter-linkages between each of these dimensions This session will present key findings of the stocktaking to ignite a debate about revolutionizing evaluation through a global collaboration.

Speakers/Présentateur·trice·s

Andrealisa Belzer, CE

Sr Evaluation Advisor, Indigenous Services Canada

Andrealisa Belzer is a Credentialed Evaluator and CES Award winner, employed with the Atlantic Region of Indigenous Services Canada. She serves as Past President of CES Nova Scotia Chapter, and on the National Board’s Standing Committee for DEI and Sustainability. She also participates... Read More →

Benoît Gauthier

Circum inc.

Benoît Gauthier is a credentialed evaluator and a Fellow of the Canadian Evaluation Society. He was president of the CES, vice-president and president of the Francophone Evaluation Network and vice-president of the International Organization for Cooperation in Evaluation as well... Read More →

Samandar Mahmodi

EvalPartners

Samandar Mahmodi is EvalPartners Coordinator. He is an evaluator with over 10 years of experience. He is also founder of Afghan Evaluation Society (AfES) and outgoing Vice President of Asia-Pacific Evaluation Association. Mr. Mahmodi is currently working with EvalPartners coordinating... Read More →

Monday June 13, 2022 10:30 - 12:00 CDT
Prairie Grass

Prairie Grass, Diversity of Evaluators/Diversité d'évaluateur·trice·s

10:30 CDT

G11.1 (Short Presentation/Présentation courte) Patient-Oriented Research in Evaluation: We are Already Behind!

Patient-Oriented Research (POR) is an important foundation of evidence-informed health care in Canada and globally. POR is research that involves, and is informed by, patients' needs and priorities at all stages of health research, from initial conception to the dissemination and implementation of knowledge. The Canadian Institute for Health Research (CIHR) recognized the importance of POR in 2011 with the launch of the national Strategy for Patient-Oriented Research (SPOR). As such, eleven provincial/territorial SPOR SUPPORT Units have been established across Canada. This presentation will focus on the general principles and three distinct case examples (two evaluations of SPOR SUPPORT Units and one evaluation of a separate health research program) of including patients in the evaluation of health research programs. This distinct approach to evaluating health research programs will show the value of including a diverse range of patients in health research program evaluations and how evaluation needs to catchup to this notion.

Speakers/Présentateur·trice·s

Brigitte Bouchard-Morris

Partner, PRA Inc.

Brigitte is a seasoned researcher who has managed numerous government program evaluations as well as led evaluations, research and strategic planning for not-for-profit organizations. She has interacted with hundreds of evaluators, academics, senior decision-makers, front-line service... Read More →

Paul Simpson

PRA Inc.

Paul Simpson is a senior research manager at PRA with over 15 years of evaluation experience. He has contributed to and managed a number of federal, provincial, and municipal-level projects that have highlighted his background in economics and extended his skill set. He has completed... Read More →

Monday June 13, 2022 10:30 - 12:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

G11.2 (Short Presentation/Présentation courte) Mentoring program as a tool for impacting youth professional development

According to the UN, today, there are 1.8 billion people between the ages of 10-24 and they are the largest generation of youth in history. Close to 90 per cent of them live in developing countries, where they make up a large proportion of the population. Their numbers are expected to grow between 2015 and 2030 alone, about 1.9 billion young people are projected to turn 15 years old. A 2017 featuring EvalYouth International Mentoring Program showed that 852 applications were received from 104 countries with 56% from Africa and 31years average ages. This article shares lessons learned from a reverse-mentorship and examines four interrelated questions: What is an effective mentoring program? Is rapid mentoring a mentoring program? What is the role of mentors and mentees? What are the key tips to a successful mentoring program?

Speakers/Présentateur·trice·s

Samuel Kouakou

President, Ivorian Initiative for Evaluation (2IEval)

Samuel KOUAKOU is a Credentialed Evaluator (CE) and International senior evaluation expert.He is the former Vice-President of the African Evaluation Association (AfrEA) and President of the Ivorian Initiative for Evaluation (2IEval).He is member of the Executive Committee Member of... Read More →

Monday June 13, 2022 10:30 - 12:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

G11.3 (Short Presentation/Présentation courte) Evaluation in the City: Navigating Urban Dynamics in our Practice

Evaluators are increasingly called to look beyond what is immediately visible to consider the contexts in which a program operates, including demographic, economic, social, cultural, political, and historical considerations. Despite the growing recognition of the need to understand these situational elements, including in the CES Competencies, most evaluators (and the programs they evaluate) pay little attention to the complex interconnected systems that 4 in 5 Canadians call home - better known as cities. In this presentation, I introduce a framework that situates evaluation within the urban context, identifies where our practice can be informed by a greater understanding of urban dynamics, and articulates how evaluation can contribute to re-imagining city spaces that, in the words of urban thinker Jane Jacobs, are created by everybody and thus provide for everybody.

Speakers/Présentateur·trice·s

Brian Hoessler

Founder, Strong Roots Consulting

Monday June 13, 2022 10:30 - 12:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

G2.1 (Short Presentation/Présentation courte) Evaluation methodology for Aboriginal Skills and Employment Training Strategy (ASETS), and Skills and Partnership Fund (SPF)

This presentation discusses the method used for, and the results of the incremental impacts and the cost-benefit associated with the Aboriginal Skills and Employment Training Strategy (the Strategy) and the Skills and Partnership Fund (the Fund). This analysis leveraged administrative data to estimate the incremental impacts for participants who started an intervention from 2011 to 2012. The analysis applied propensity score matching combined with difference-in-differences to estimate the incremental impacts on labour market outcomes. The evaluation results show that both the Strategy and the Fund are effective at improving labour market attachment of participating Indigenous peoples, which helps to achieve Government of Canada broad diversity inclusion objectives. Social benefits of participation exceeded the cost of investments for most interventions over time, yielding a positive social return on investment over the 10 years post-participation.

Speakers/Présentateur·trice·s

Charles Bouwer

Employment and social development Canada

Charles enjoys complex problem solving and his interests include machine learning, security, and labour market performance. Charles holds a Masters in Economics from the University of Ottawa and has been working in the Federal Government since 2016. Charles has been a part of the... Read More →

Leeroy Rikhi

Employment and Social Development Canada

Tristan Rikhi is an evaluator in the Evaluation Directorate at Employment and Social Development Canada. He has been involved in quantitative analysis and methodology of the evaluation projects such as Youth Employment Strategy and other labour market programs. He has several years... Read More →

Momath Wilane

Employment and social development Canada

Momath Wilane is a senior data analyst at Evaluation directorate within Employment and Social development Canada. Momath hold a Master in Economics from University Laval in Quebec City. His main tasks in evaluation directorate consist of providing methodological and data advice, as... Read More →

Monday June 13, 2022 10:30 - 12:00 CDT
Sweet Grass

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

G2.2 (Short Presentation/Présentation courte) Equity, Diversity, and Inclusion (EDI) Should Be The Main Attraction, Not The Side Show In Evaluation!

By intentionally using an EDI lens in evaluation, a new world emerges. A world filled with unique ideas, knowledges, and voices. We are challenged to consider whose voices are included or excluded. With EDI, the solutions we propose, gaps we discover, and recommendations we develop are human centered. We intentionally seek to consider the impact of inequity and exclusion in policies, budgets, laws, research, services, and programs from the perspectives of equity deserving groups. We design questions with the intent of understanding first-hand the impact of policies, budgets, and laws today on the quality of lives of children, youth, adults, and seniors tomorrow. EDI enables us to put the human being front and center. It intentionally creates spaces for each person to be recognized for the value they bring to the table, along with the unique knowledges they possess and most importantly, for their humanity. EDI is more than a conversation; it is a journey to find our better selves.

Speakers/Présentateur·trice·s

Zainul Virgi

Dr., Goss Gilroy Inc.

Dr. Virgi has 15+ years of experience working with equity deserving groups whose voices have been absent in health, education, social, economic, and political spheres. Zainul has worked nationally and internationally. She is an experienced thought leader committed to equity, diversity... Read More →

Monday June 13, 2022 10:30 - 12:00 CDT
Sweet Grass

Sweet Grass, Diversity of Evaluators/Diversité d'évaluateur·trice·s

10:30 CDT

G2.3 (Short Presentation/Présentation courte) Our sense of identity: A move from an individualistic to a collective mindset in decolonizing evaluation practice

The murder of George Floyd in 2020 spotlighted issues of diversity, equity, and inclusion, and the reality of systemic racism in our communities today. As a result, evaluators in the US are becoming more aware of the prevalence of systemic racism and oppression within the field, and have begun discussing their role in advocating for marginalized communities in their professional practice. These discussions have spurred evaluators to acknowledge the issues of systemic oppression in methodology. As foreign-born emerging evaluators, we have come to understand the importance of our identity in re-orienting ourselves in our approach to evaluation. We will reflect on identity and the influence it has on evaluative thinking, as it relates to our understanding, advocacy, and implementation of decolonizing methodologies. As we try to step away from systems of oppression in our profession, we hope to explore the impact that western individual approaches have on transforming the evaluation field.

Speakers/Présentateur·trice·s

Charity Odetola

UNC at Greensboro

Charity Odetola is a second-year graduate student at UNC Greensboro, pursuing a degree in Educational Research and Methodology with an emphasis in Program Evaluation. She has a background in social work and women's and gender studies. A Nigeria-born Senegal-raised budding evaluator... Read More →

Joy Alcantara Chuquiruna

Graduate Student, University of North Carolina at Greensboro

Joy Alcantara is a graduate student at UNC Greensboro studying Educational Research Methodology with an emphasis in Program Evaluation. She is originally from Lima, Peru, and moved to the US at 7 years old. Given her background, she is interested in researching decolonizing methodologies... Read More →

Monday June 13, 2022 10:30 - 12:00 CDT
Sweet Grass

Sweet Grass, Diversity of Evaluators/Diversité d'évaluateur·trice·s

12:00 CDT

(Food/Alimentation) Lunch/Dîner

Monday June 13, 2022 12:00 - 13:00 CDT
Tamarak

Tamarak

13:00 CDT

(Short Presentations/Présentations courtes) Student Case Competition presentations / Présentations dans le cadre du Concours d'étude de cas

Monday June 13, 2022 13:00 - 15:00 CDT
Balsam

Balsam

13:30 CDT

Solo (Expert tutorial/Tutoriel d'expert·e) What Has GBA+ Done for Me Lately?: A Post-Reflection Five Years After Its Introduction in the Policy on Results

In the five years since GBA+ was introduced as a priority in the Policy on Results (2016), there are still federal government organizations struggling to apply GBA+ in the context of their evaluation efforts. GBA+ has been comprehensively introduced as a requirement across the policy cycle and yet the evaluation function still lags behind other government functions (SWC, 2018). Using GBA+ as a vehicle to improve equity, diversity and inclusion (EDI) outcomes has advanced diversity but fails to consistently improve equity and inclusion. This session presents the more significant shortcomings of GBA+ with respect to EDI and provides suggestions to improve equity and diversity outcomes. This necessarily involves reflections on evaluation thinking, methods, reporting and accountability structures to consider whether GBA+'s contribution to diversity is sufficient.

Speakers/Présentateur·trice·s

Jane Whynot

Dr., Goss Gilroy Inc. (GGI)

Dr. Jane Whynot joined Goss Gilroy Inc. in 2021 after completing her doctoral studies at the University of Ottawa where her research explored how GBA+ is integrated in the federal government evaluation function. Jane is also a contract instructor at Carleton University in their graduate... Read More →

Monday June 13, 2022 13:30 - 15:00 CDT
Manitoba Maple

Manitoba Maple, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

Solo (Leading Edge Panel/Panel d'avant-garde) Putting ethics to work in evaluation: An interactive panel to share, fail, and learn

This panel blends themes of ethics, feminist practice, and storytelling to make ethical principles more tangible and actionable in evaluation practice. This non-traditional and collaborative panel will be co-facilitated by all panelists who will share their journeys of walking the talk of ethical evaluation. Rooted in both intersectional feminist theory and personal examples of wanting to work differently, stories of "Failing Forward" will be used to set the stage and encourage honest dialogue. Panelists will then lead a reflexive exercise for audience members to collectively share new insights and discuss lingering questions. The goal of this panel is to foster bravery, vulnerability, and imagination. By weaving together diverse experiences and perspectives, this interactive panel will contribute to important conversations around ethical evaluation while deepening our understanding of the importance of articulating and addressing the morally salient features of our work as evaluators.

Speakers/Présentateur·trice·s

Jennica Nichols

Director & Senior Consultant, AND implementation

I'm a credentialed evaluator and mixed methods researcher. I am passionate about arts-based methods, co-design, and meaningful measurement. I hold a MPH (epidemiology) from UofT and am moments away from finishing my PhD at UBC. I am the co-founder of the Vancouver-based consulting... Read More →

Maya Lefkowich

Research & Evaluation Senior Consultant, AND implementation inc.

Maya Lefkowich is from Toronto, Ontario and currently lives in Vancouver, BC. Passionate about equity-centred and imaginative approaches, she is a research methodologist, artist, and emerging evaluator. Maya holds a PhD in interdisciplinary studies focused on anti-oppressive approaches... Read More →

Janis Grychowski

Global Affairs Canada

Janis Grychowski is an acting Deputy Director with the Evaluations Division at Global Affairs Canada. She has 6+ years experience evaluating the Government of Canada's international assistance programs. Previously, Janis spent almost a decade working on development and humanitarian... Read More →

Isabelle Mercier

Global Affairs Canada

Isabelle Mercier is the Director of Evaluation at Global Affairs Canada. Isabelle's recent focus is in implementing feminist research principles in her work. This has included focussing on participatory methods, leveraging local expertise and knowledge, ethics, and integrating the... Read More →

Monday June 13, 2022 13:30 - 15:00 CDT
Prairie Grass

Prairie Grass, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G10.1 (Short Presentation/Présentation courte) Reflections on Conducting Participatory Evaluation of SmartICE Projects in Inuit Nunangat

Participatory evaluation centres on meaningful co-construction of knowledge. This is an intricate process, especially in unique cultural and contextual settings. In this presentation, a program evaluator and a program stakeholder & rightsholder reflect on their experiences with participatory evaluation in Inuit Nunangat. Jennifer Rae is an Evaluator from SRDC, a non-profit research organization, based in Ottawa. Shawna Dicker is a Project Support Lead with SmartICE, a social enterprise, hailing from Nain, Nunatsiavut. They are each part of broader teams collaborating to develop, implement, and evaluate SmartICE programs in Inuit partner communities. The aim of this session is to discuss strategies related to participatory evaluation: the complex collaboration of northern and southern team members; successes and challenges in community engagement; ways of incorporating Inuit knowledge and societal values in evaluation; and how evaluators, funders, and others can advance such efforts.

Speakers/Présentateur·trice·s

Jennifer Rae

Dr, SRDC

Jennifer Rae is based in Ottawa as a Research Associate at SRDC, a non-profit research organization. She contributes to community-based, participatory evaluation projects in the areas of training and employment, mental health, homelessness, and youth-focused interventions. She also... Read More →

Monday June 13, 2022 13:30 - 15:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G10.2 (Short Presentation/Présentation courte) Estimating the Net Impact of the Opportunities Fund for Persons with Disabilities Program using Administrative Data

This presentation discusses the method used for, and the results of the evaluation of the Opportunities Fund for Persons with Disabilities. This is a Government of Canada program helping to fulfill broad diversity inclusion objectives. We explain how we used a rich set of socio-demographic and labour market characteristics of the participants from the Labour Market Program Data Platform of Employment and Social Development Canada. This analysis applies propensity score matching combined with the difference-in-differences methodology. This method estimates the net impact of the Program on participants over five-year post-participation up to 2017. The evaluation showed that the Program is effective at improving labour market attachment of persons with disabilities by reducing the barriers they face. The Program yields a positive social return on investment over a ten-year post-participation period.

Speakers/Présentateur·trice·s

Jamil Sayeed

Employment and Social Development Canada

Jamil Sayeed is an evaluator in the Evaluation Directorate at Employment and Social Development Canada. He has been involved in quantitative analysis and methodology of the evaluation projects such as Opportunities Fund for person with Disabilities and other labour market programs... Read More →

Yves Gingras

Consultant, DG Innovation

Yves was Head of Evaluation (Director General) for Employment and Social Development Canada (ESDC). Yves has received recognition for his work in the field of evaluation and data development. Driven by value for money approaches, he contributed to the development of the Government... Read More →

Monday June 13, 2022 13:30 - 15:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G10.3 (Short Presentation/Présentation courte) Estimating the Impact of Active Labour Market Programs in Canada

This presentation discusses the methods used and the results found for an evaluation of the impacts associated with participation in labour market programs delivered under the Labour Market Development Agreements in Canada. These agreements help a broad spectrum of Canadians with diverse backgrounds to find jobs. This study uses the data rich Labour Market Program Data Platform to estimate the incremental impacts for participation in these labour market programs and services. This analysis applies propensity score matching combined with a difference-in-differences methodology. The targeted participants are from the 2010 to 2012 cohort, and were tracked for up to seven years during and post-program. The overall findings suggest that participation in Skills Development, Targeted Wage Subsidies and Job Creation Partnerships programs improved participants' labour market attachment and reduced their dependence on government income supports compared to similar non-participants.

Speakers/Présentateur·trice·s

Essolaba Aouli

Employment and Social Development Canada

Essolaba Aouli is currently a Manager in the Evaluation Directorate at Employment and Social Development Canada. He is responsible for conducting quantitative analysis and methodology of evaluation projects. He has many years of experience and expertise in leading evaluation projects... Read More →

Lynne Lapointe

Employment and Social Development Canada

Lynne Lapointe is a Senior Evaluator in the Evaluation Directorate of Employment and Social Development Canada. She is currently responsible for the Evaluations of the Labour Market Development Agreements. Lynne is a CES Credentialed Evaluator with over 12 years of experience in leading... Read More →

Leeroy Rikhi

Employment and Social Development Canada

Monday June 13, 2022 13:30 - 15:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G14.1 (Short Presentation/Présentation courte) If no one cares, why do we bother? A reflection on key informant interviews.

Considered a staple of any program evaluation methodology, interviews are also a particularly time-consuming research method. Designing instruments, conducting interviews, preparing notes, proceeding with a thematic analysis, and reporting on the resulting insights typically engage considerable resources. Yet, findings resulting from this laborious process are often given limited credit, and may even be received with suspicion (but how many people said this?). This presentation explores some of the factors that have led to this outcome, and revisit the actual value of interviews. In particular, the presentation will explore the role that interviews may play in providing a voice to those whose views may not be collected through other methods, contributing to the validity of an evaluation from a social justice perspective. As most participants are expected to have conducted some interviews, a participatory approach will be used to allow for an engaging discussion.

Speakers/Présentateur·trice·s

François Dumaine

Partner, PRA Inc.

Monday June 13, 2022 13:30 - 15:00 CDT
Sweet Grass

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G14.2 (Storytelling/Récit) Collaborative design of a conservation agriculture programme evaluation

This case study illustrates how a diversity of primary intended users co-designed and implemented the final evaluation of the Scaling-Up Conservation Agriculture in East Africa (SUCA) Program. We describe how the evaluation design accommodated the diversity of collaborators: staff of eleven NGOs in East Africa, Kenya and Winnipeg based evaluators with the Canadian Foodgrains Bank, and an external consultant. The collaborative process confirmed a sense of ownership by the primary evaluation users over the evaluation design. The evaluation design combined outcome and learning uses that took advantages of the implementing organizations' commitment to learning. The program supported local partners with a target of 50,000 male and female farmers practicing a minimum of 2 of 3 conservation agriculture principles, and to improve food security and sustainable livelihoods for 18,000 of these farmers' households across three countries. The session will summarize an article being published.

Speakers/Présentateur·trice·s

Dr. Cynthia Neudoerffer

Monitoring & Evaluation Coordinator, Canadian Foodgrains Bank

Cynthia Neudoerffer is the Monitoring and Evaluation Coordinator at Canadian Foodgrains Bank, based in Winnipeg, Canada. She is a professional engineer (registered in Ontario) and holds a PhD in Inter-discipinary Rural Studies. She provides technical guidance on Results Based Management... Read More →

Ricardo Ramirez

DECI Project, independent

Monday June 13, 2022 13:30 - 15:00 CDT
Sweet Grass

Sweet Grass, Diversity of Evaluators/Diversité d'évaluateur·trice·s

13:30 CDT

G14.3 (Short Presentation/Présentation courte) Uniting our impact in the youth sport for development sector

The youth sport-for-development (SfD) field has the potential to impact youth social, psychological & emotional well-being. In 2021, MLSE Foundation awarded grants to 24 youth sport organizations across Ontario focused on SfD. These grantees offer a diverse range of sport programs to youth facing barriers to positive development, with objectives such as preparing newcomer youth for school and creating employment opportunities for individuals with developmental disorders. In addition to financial support, grantees were provided with a variety of evaluation-focused resources to advance their capacity to sustain and improve their programs long-term, such as developing a logic model and consolidating outcomes to provide a collective impact of our work. The findings and recommendations are being utilized to develop effective sport programs. Attendees can expect to leave the presentation with practical ideas to build evaluative capacity work with the diverse organizations in the sector.

Speakers/Présentateur·trice·s

Olu Paynter

Coordinator, Research and Evaluation, MLSE Foundation

Olu Paynter is a Research Coordinator with Maple Leaf Sports & Entertainment (MLSE) Foundation and is responsible for supporting not-for-profit Sport for Development Organizations with evaluation plans, techniques, and the resources to collect relevant data. Olu is also a consultant... Read More →

Patrick Antonio

Manager, Scoreboard, MLSE Foundation

Dr. Patrick Antonio is the Manager of ScoreboardTM at Maple Leaf Sports & Entertainment (MLSE) Foundation. He is responsible for developing, implementing, and scaling the digital platform (Scoreboard) designed to increase engagement in youth sport and physical activity within the... Read More →

Monday June 13, 2022 13:30 - 15:00 CDT
Sweet Grass

Sweet Grass, Diversity of Evaluators/Diversité d'évaluateur·trice·s

13:30 CDT

G4.1 (Short Presentation/Présentation courte) What does an empowerment evaluation involving youth look like in practice?

Literature on youth involvement in program evaluation highlights the multiple benefits for involving youth in evaluation, which include benefits to the program, organization running a program, to funding agencies and to the youth program beneficiaries. This presentation focuses on what youth involvement in program evaluation can look like in practice. The presentation presents a case study of an empowerment evaluation (EE), a particular type of collaborative evaluation, to involve youth in the evaluation of their STEM program. We discuss the theory behind youth involvement in collaborative evaluation, the EE model and provide details of how EE is structured to involve youth in evaluation as co-evaluators. In particular, we describe how Fetterman's EE steps of mission development, taking stock and planning for the future can be expanded and integrated to encourage and reflect youth involvement. The presentation draws on the perspectives of evaluation scholars, evaluators, and youth.

Speakers/Présentateur·trice·s

Sarah Heath

Instructor III, University of Winnipeg

Sarah Heath holds a PhD in Education with a specialization in program evaluation from the University of Ottawa. Her dissertation focused on the involvement of youth in collaborative program evaluation. Dr. Heath is a collaborator on the SSHRC-funded Manitoba Northern and Rural Institute... Read More →

Monday June 13, 2022 13:30 - 15:00 CDT
Willow

WIllow, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G4.2 (Short Presentation/Présentation courte) Truth and Reconciliation for Evaluators - Introduction to the new CES E-Institute Course

The CES e-Institute has partnered with Hotıì ts’eeda and Reciprocal Consulting to offer an
online course to support the journey to truth and reconciliation for non-Indigenous evaluators.
This session will provide an overview of the course highlighting the deliberate design that
challenges, disrupts, and re-orients evaluators wanting to practice evaluation in a culturally
responsive way.

Speakers/Présentateur·trice·s

Marla Steinberg

Dr., Evaluation Consultant

Marla Steinberg, Ph.D., is a professional evaluator living and working in Victoria British Columbia. For over twenty-five years she has been helping funders, foundations, governments, community-based organizations and practitioners measure their impact and find ways to improve their... Read More →

Monday June 13, 2022 13:30 - 15:00 CDT
Willow

WIllow, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G4.3 (Short Presentation/Présentation courte) Evaluating Compliance with Human Rights Standards to Advance Children's Rights in Manitoba

The Manitoba Advocate for Children and Youth (MACY) is an independent office of the Manitoba Legislative Assembly that represents the rights and diversity of experiences/interests of children, youth, and young adults. Empowered by The Advocate for Children and Youth Act (ACYA), MACY provides direct advocacy, conducts research, investigates serious injuries and deaths, issues recommendations, and monitors/reports on levels of compliance with recommendations issued under the ACYA. This presentation examines how MACY employs a new and innovative technical compliance monitoring framework to evaluate progress on recommendations to advance children's rights. Using a situational approach that considers context and resource capacity in improving public services, it assesses whether government actions are compliant with human rights standards, to promote accountability and transparency in child-serving systems. A key component of this model relies on building strong interpersonal relationships.

Speakers/Présentateur·trice·s

Alison Carrey

Office of the Manitoba Advocate for Children and Youth

Alison Carrey (she/her) is a Policy Analyst at the Office of the Manitoba Advocate for Children and Youth. Alison has a BA Honours in Criminal Justice (University of Winnipeg) and an MA in Socio-Legal Studies (York University).

Monday June 13, 2022 13:30 - 15:00 CDT
Willow

WIllow, Diversity within evaluation/Diversité au sein de l'évaluation

15:00 CDT

(Food/Alimentation) Break/Pause

Monday June 13, 2022 15:00 - 15:30 CDT
Conference Foyer/Hall du congrès

Conference Foyer/Hall du congrès

15:30 CDT

Fellows Education/Événement des membres titulaires

This is an opportunity for emerging evaluators to come ask questions from the Fellows and give suggestions as to things the CES might do to better support those new to the field.

Speakers/Présentateur·trice·s

Larry Bremner

Proactive Information Services Inc.

Larry has worked in evaluation for 40+ years, across Canada & internationally. In 1984, he established Proactive to serve the non-profit & public sectors. A former CES National President, as Past President he was the driving force behind creation of the global network EvalIndigenous... Read More →

Monday June 13, 2022 15:30 - 17:00 CDT
Jack Pine

Jack Pine, Diversity of Evaluators/Diversité d'évaluateur·trice·s

15:30 CDT

Solo (Expert tutorial/Tutoriel d'expert·e) Developing Effective Community Partnerships

Conducting community-based participatory evaluations and working toward community change requires developing effective community partnerships. Using a participatory approach, two community psychologists will share their empirical and experiential knowledge based on over 60 years as professional evaluators working with communities and coalitions. This tutorial will include strategies to (really) learn about communities, how to effectively approach communities as a collaborator, how to foster true collaboration, how to ensure inclusion of community diversity, and how to negotiate a beneficial working relationship with community members. Presenters will share examples of success and not-so-successful stories from their work and invite participants to do the same. The evaluators will use exercises, small group discussion, and minimize lecture time. Participants will leave with a handout and resources.

Speakers/Présentateur·trice·s

Ann Price

President, Community Evaluation Solutions, Inc

Susan Wolfe

CEO/Community Consultant, Susan Wolfe and Associates

Monday June 13, 2022 15:30 - 17:00 CDT
Manitoba Maple

Manitoba Maple, Diversity within evaluation/Diversité au sein de l'évaluation

15:30 CDT

G20.1 (Short Presentation/Présentation courte) Evaluation and Gender-Based Analysis Plus - A True Friendship

This short presentation will examine how Gender Based Analysis Plus (GBA+) has been considered in recent Infrastructure Canada evaluations and present some examples of how GBA+ can be applied to assess how diverse groups of people are impacted by the programs that we evaluate. In particular, staff have examined how communities may be experiencing government programs differently. Examples from previous evaluations that looked at the impact of population size and remoteness, socio-economic status and proximity to public transit, Indigenous communities and drinking water advisories will be presented. Taking the approaches that INFC evaluation team did for these evaluations also provided some lessons learned, demonstrating the potential for GBA+ to inform and improve future programs. Those conclusions and considerations will be shared to assist other evaluators in doing similar work.

Speakers/Présentateur·trice·s

Judy Lifshitz

Team Lead, Infrastructure Canada

Judy Lifshitz P.M.P, C.E is a senior evaluator working in the Federal government for the past 13 years. Currently, at Infrastructure Canada she oversees a team of evaluators (internal and external) through the planning, conducting and reporting phases of evaluation projects. In her... Read More →

Monday June 13, 2022 15:30 - 17:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

15:30 CDT

G20.2 (Short Presentation/Présentation courte) Reflections on Conducting a Feminist Evaluation of the Women's Voice and Leadership Program at Global Affairs Canada

The formative evaluation of the Women's Voice and Leadership program has been an opportunity for Global Affairs Canada to adopt a feminist approach that amplifies the lived experiences of its participants. This presentation will explore how grounding an evaluation in feminist principles can support the active participation of a diversity of stakeholders in all phases of an evaluation, contributing to stronger results. The presentation will consider the challenges of operationalizing a feminist approach, including how COVID19 has constrained traditional participatory approaches while presenting new, virtual data collection and stakeholder engagement opportunities. This evaluation also included four locally-designed and implemented case studies, and the presentation will integrate reflections from the local evaluators in Sri Lanka, Ukraine, Guatemala and Nigeria. Their experiences reflect the different ways that feminist evaluation can be understood, and implemented, in diverse contexts.

Speakers/Présentateur·trice·s

Colton Brydges

Senior Evaluation Officer, Global Affairs Canada

Colton Brydges is a Senior Evaluation Officer with Global Affairs Canada's Evaluation Division. Colton's evaluation work to date has focused on international assistance programming, touching on a range of countries and thematic areas. Colton holds a Master's in International Development... Read More →

Monday June 13, 2022 15:30 - 17:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

15:30 CDT

Solo (Presentation/Présentation) Incorporating the Public Health Care Spending in Cost-benefit Analysis in the Context of Labour Market Development Agreements Evaluation

This study proposes a novel methodology to incorporate public health care spending in the cost benefit analysis of Labour Market Development Agreements Evaluation. These agreements foster labour market inclusion of various groups, including vulnerable population with relatively weaker labour market attachment in Canada. Cost-benefit analysis of previous evaluations were limited to the higher earnings profile of participants. However, literature suggest that positive outcomes associated with increased labour market attachment can positively affect health outcomes. This study uses labour market program data platform and average public health care cost data from a report of the Canadian Institute for Health Information to estimate the impact of increased labour market attachment on government's health care expenditures. The findings suggest that public health care cost decreases on average by $29 per year over a four-year post-participation period under Skills Development interventi

Speakers/Présentateur·trice·s

Leeroy Rikhi

Employment and Social Development Canada

Momath Wilane

Employment and social development Canada

Jamil Sayeed

Employment and Social Development Canada

Monday June 13, 2022 15:30 - 17:00 CDT
Jack Pine

Sage, Diversity of Evaluators/Diversité d'évaluateur·trice·s

15:30 CDT

G15.1 (Short Presentation/Présentation courte) Evaluation sensible au genre en situation de crise en Afrique

La pandémie du coronavirus (COVID-19) a eu diverses conséquences pour les hommes et les femmes dans le monde. Plusieurs études montrent que dans l'ensemble les taux de mortalité sont plus élevés pour les hommes et les conséquences socio-économiques sont très marquées pour les femmes (OECD, 2020). En Afrique sub-saharienne, les effets de la pandémie ont accentué les fragilités et les inégalités, notamment celles qui affectent les femmes. Cela se justifie par le fait que (i) les femmes vivant cette région sont au cœur de la réponse à l'urgence sanitaire dans la mesure où elles constituent l'essentiel de la main d'œuvre des secteurs de la santé et des services sociaux dans la région, ce qui les expose à un risque accru de contracter le virus. Les mesures de confinement mises en place dans la plupart des pays afin de limiter la propagation de la pandémie ont accru le travail domestique non rémunéré des femmes et leur ont fait supporter des charges supplémentaires.

Speakers/Présentateur·trice·s

Samuel Kouakou

President, Ivorian Initiative for Evaluation (2IEval)

Monday June 13, 2022 15:30 - 17:00 CDT
Sweet Grass

Sweet Grass, Diversity of Evaluators/Diversité d'évaluateur·trice·s

15:30 CDT

G15.2 (Short Presentation/Présentation courte) Identifier les conditions gagnantes pour renforcer les capacités en évaluation: une analyse qualitative comparée (QCA) auprès d'organismes communautaires

Les organismes communautaires sont maintenant nombreux à souhaiter renforcer leurs capacités en évaluation (CÉ) afin de mieux guider leurs décisions et leurs actions. Depuis 2019, le LaboÉval offre aux organismes communautaires participants de la formation et du soutien visant à renforcer leurs capacités respectives en évaluation. À partir de l’étude approfondie d’un échantillon composé de dix organismes participants, cette communication propose une analyse détaillée des conditions et des facteurs qui soutiennent le renforcement des CÉ du milieu communautaire. Plus précisément, l’analyse qualitative comparée (ou QCA), une méthode rigoureuse encore peu utilisée en évaluation, a permis de déceler des combinaisons causales expliquant le succès des efforts déployés pour augmenter les CÉ des organismes participants. Les combinaisons identifiées ainsi que leurs implications pratiques seront discutées suite à une brève présentation du LaboÉval et de certains concepts de base de la QCA.

Speakers/Présentateur·trice·s

David Buetti

University of Ottawa

David Buetti, Ph. D., est un spécialiste en évaluation et en développement communautaire. Sa thèse de doctorat a porté sur l'identification et la mesure des capacités en évaluation du milieu communautaire. Il travaille comme associé de recherche pour le LaboEval, un laboratoire... Read More →

Isabelle Bourgeois

Dr., University of Ottawa

Isabelle Bourgeois, Ph. D., est professeure agrégée à la Faculté d'éducation de l'Université d'Ottawa, et est la chercheure principale du LaboEval. Ses activités de recherche portent sur la mesure et le renforcement des capacités organisationnelles en évaluation (CE) dans... Read More →

Monday June 13, 2022 15:30 - 17:00 CDT
Sweet Grass

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

15:30 CDT

G15.3 (Storytelling/Récit) Comment former les évaluateurs émergents aux profils diversifiés pour une meilleure intégration dans les milieux professionnels ?

Nos universités accueillent des étudiants d’une riche diversité. Celle-ci répond aux besoins des milieux professionnels. Notre expérience en tant que co-responsables d’un programme visant une spécialisation en évaluation nous met face à un défi important : celui de réussir à intégrer ces étudiants dans les milieux professionnels en évaluation, en les accompagnant dans leurs apprentissages. L’objectif de cet exposé est d’échanger avec les milieux académiques en français et en anglais avec des expériences similaires (étudiants autochtones, étrangers etc). La diversité de profils d’apprenant de nos étudiants reflète les savoirs issus de leurs institutions de formation. Il s’agit alors de réussir à identifier les approches pédagogiques permettant d’atteindre les compétences pour devenir évaluateurs qualifiés selon les standards canadiens. Nous réfléchissons à intégrer des éléments sur l’interculturalité et sur les relations interpersonnelles au cœur de nos cours fondamentaux en évaluation.

Speakers/Présentateur·trice·s

Sherri Bisset

Dr., Innovation Discipline

Dre Sherri Bisset cumule 15 ans d'expérience en évaluation de programmes et en recherche interventionnelle en promotion de la santé. Elle est également évaluatrice qualifiée et a été présidente du chapitre québécois. Professeure adjointe de clinique à l'École de sant... Read More →

Lara Gautier

Université de Montréal

Lara Gautier est professeure adjointe au Département de Gestion, Évaluation et Politique de Santé de l'École de Santé Publique de l'Université de Montréal. Formée en santé publique, sciences politiques et en socioéconomie, ses intérêts de recherche se situent au croisement... Read More →

Monday June 13, 2022 15:30 - 17:00 CDT
Sweet Grass

Sweet Grass, Diversity of Evaluators/Diversité d'évaluateur·trice·s

15:30 CDT

G20.3 (Storytelling/Récit) Modifying Storytelling Methodology for the Summative Evaluation in Inukjuak, Nunavik - UESCO Futures Literacy Modified

Conference Theme 1 - Exploring the diversity of evaluator roles (e.g., advocate, impartial third party, educator), and evaluation approaches In addition to panel members listed, we will invite Inuit Project Manager for the Projet and a community partner. FDR's approach to storytelling for the Summative Evaluation of Makivik Corporation's Pirursiivik Project in Inukjuak, Nunavik, drew on the concept of Futures Literacy (FL) as conceptualized by UNESCO (UNESCO 2021). FDR in consultation with field partners and project participants was able to modify the storytelling methodology to better match the needs of the community in Inukjuak and to tell the story from the lens of Indigenous people there. Individual and institutional biases and various evaluator roles were explored.

Speakers/Présentateur·trice·s

Harry Cummings

Dr., Harry Cummings and Associates

Harry Cummings is a fomer President of Canadian Evaluation Society and a Credentialed Evaluator. For this submission Harry had worked as an affiliated Feminist Data and Research Advisor.

Monday June 13, 2022 15:30 - 17:00 CDT
Sage

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

15:30 CDT

G12.1 (Short Presentation/Présentation courte) Into the Great Unknown: How to Navigate a Program Evaluation on an Unfamiliar Topic

As evaluators, it is rare that our education and work experience make us highly knowledgeable about the fields that we are called upon to evaluate (e.g., space science, youth justice, public health, etc.), or the context in which we are called to work (like COVID-19). While this can pose a number of challenges for the evaluation, it can also present opportunities. Drawing on their past evaluation experiences, this session will introduce some of the approaches that Emily and Susanna have used to help navigate the challenges of unfamiliar topics, while still collecting the information necessary to produce a robust and useful evaluation report. The session will highlight how evaluators can draw on their unique strengths and experiences to lead successful evaluations. The discussion portion of the session will then provide opportunities for diverse participants to share their own experiences and insights, including best practices they have found, and challenges that they have encountered.

Speakers/Présentateur·trice·s

Susanna Beaudin

Senior Research Associate, PRA Inc.

Susanna Beaudin is a senior research associate at PRA. Over the past five years, she has worked on numerous evaluations in a wide variety of sectors including public safety, GBA Plus, justice, the environment and official languages. She holds a master's degree in International Affairs... Read More →

Emily Brennan

Senior Evaluation Manager, Agriculture and Agri-Food Canada

Emily is a new Senior Evaluation Manager at Agriculture and Agri-Food Canada, and previously worked as a Senior Research Associate at PRA Inc. She has been working as an evaluator for the last six years. She strives to implement a collaborative approach to all of her work. Her evaluation... Read More →

Monday June 13, 2022 15:30 - 17:00 CDT
Willow

White Spruce, Diversity of Evaluators/Diversité d'évaluateur·trice·s

15:30 CDT

G12.2 (Storytelling/Récit) Finding Our Way: Reflections on Diversity

In the final year of the Bachelor of Child Studies at Mount Royal University in Calgary, students take a Capstone course in which they work together with community partners on innovative and impactful projects. Our Capstone of 11 students partnered with the Evaluation Capacity Network to help build evaluation capacity in the early childhood field across Alberta. We have navigated the diversity of two distinct disciplines, a range of backgrounds and professional experiences, a multitude of supervisors and diverse community partners and a variety of evaluation approaches and frameworks. This presentation will focus on the challenges, successes, learnings and growth that we experienced as we worked through four evaluations and made connections back to the overall network. Despite the diversity and complexity of initiative, we learned that having a strong core team is what made the Capstone successful. Student, faculty and community partner experiences will be part of the presentation.

Speakers/Présentateur·trice·s

Leah Gavel

Mount Royal University

Leah Gavel is a 4th-year student in Mount Royal University's BCST program and has over 10 years of experience working with children and youth. Leah is drawn to research and evaluation by its ability to collaborate with stakeholders. Leah is planning to get an MD in Psychology after... Read More →

Linh Le

Mount Royal University

Linh Le is a fourth-year student in the Bachelor of Child Studies, majoring in Early Learning and Child Care at Mount Royal University in Calgary, Alberta. She is a foreign student in Canada from Vietnam and this is currently her sixth year in Canada. She is interested in creating... Read More →

Monica Pauls

Associate Professor, Mount Royal University

Monica Pauls is an Associate Professor in the Bachelor of Child Studies at Mount Royal University in Calgary. She is currently a doctoral candidate in Sociology at the University of Calgary. Monica's research interests include youth activism, social media and social change. Monica... Read More →

Nicole Homick

Mount Royal University Student

Nicole Homick is a fourth-year student in the Bachelor of Child Studies at Mount Royal University in Calgary. Nicole has 15 years of experience working with children and families in various capacities. Her research topics of interest are inclusion implementation, coaching and mentoring... Read More →

Monday June 13, 2022 15:30 - 17:00 CDT
Willow

White Spruce, Diversity of Evaluators/Diversité d'évaluateur·trice·s

15:30 CDT

G12.4 (Short Presentation/Présentation courte) Evaluators as educators and learners: Supporting evaluation capacity-building in diverse contexts

Being of practical service to clients is an essential part of the evaluator role. Rather than being commissioned solely to conduct third-party evaluations "for" or "on behalf of" clients, we are increasingly called upon to help bolster organizations' capacity to conduct their own internal evaluation work and/or educate them to work in an informed way with external evaluators. The presenters will share their experiences helping to build evaluation capacity across a diverse range of organizations with varying levels of evaluation familiarity, experience, and comfort. As practical examples, they will highlight their strengths-based approach to co-developing a toolkit focused on the basics of evaluation and their experience designing and delivering bespoke workshops to address organization-specific evaluation challenges. In addition, they will reflect on their own learning from the capacity-building process, plus challenges as they stepped outside their comfort zones for this work.

Speakers/Présentateur·trice·s

Jacey Payne

Senior Research Associate, Social Research and Demonstration Corporation (SRDC)

Jacey Payne is a Calgary-based Senior Research Associate with the Social Research and Demonstration Corporation (SRDC). Working in the research and evaluation sector for over 15 years, she has provided consulting services in areas including health, human services, education, and employment... Read More →

Andrea Bobadilla

Social Research and Demonstration Corporation

Monday June 13, 2022 15:30 - 17:00 CDT
Willow

White Spruce, Diversity within evaluation/Diversité au sein de l'évaluation

17:00 CDT

(Social) Case Competition Awards Reception/Réception du concours d'étude de cas

Monday June 13, 2022 17:00 - 18:00 CDT
Hospitality Suite/Suite de réception

Hospitality Suite/Suite de réception

18:15 CDT

(Preregistered social event/Événement social avec préinscription) Walking Tour and Spaghetti Factory Dinner/Tour guidé et souper à la Spaghetti Factory

Join local hosts on an informative walk from The Fairmont Hotel Winnipeg to The Forks National Historic Site, highlighting historical, cultural and architectural significance of the area adjacent to the confluence of the Red and Assiniboine Rivers.

Des hôtes locaux nous rejoindrons pour une promenade informative de l’hôtel Fairmont au lieu historique national de La Fourche, soulignant l’importance historique, culturelle et architecturale de la zone adjacente au confluent des rivières Rouge et Assiniboine.

Monday June 13, 2022 18:15 - 21:30 CDT
Hospitality Desk/Bureau

Hospitality Desk/Bureau

18:30 CDT

(Social) Private Shopping/Magasinage privé

At the Forks... On June 13, embark on a private shopping experience for all CES attendees at The Forks Trading Company from 6:30 p.m. to 8:30 p.m. where you can buy a little piece of Winnipeg and sample food and other products by local Winnipeg Makers.

À La Fourche... Le 13 juin, participez à une expérience de magasinage privée pour tou·te·s les participant·e·s à C2022 avec The Forks Trading Company, de 18 h 30 à 20 h 30; vous pourrez acheter un petit souvenir de Winnipeg et déguster des aliments et d’autres produits des artisans locaux de Winnipeg.

Monday June 13, 2022 18:30 - 20:30 CDT
Offsite/En ville

Offsite/En ville

07:00 CDT

(Social) Morning Movement /Exercices matinaux

Tuesday June 14, 2022 07:00 - 08:00 CDT
Hospitality Desk/Bureau

Hospitality Desk/Bureau

08:00 CDT

(Food/Alimentation) Breakfast/Déjeuner

Tuesday June 14, 2022 08:00 - 09:00 CDT
Tamarak

Tamarak

09:00 CDT

(Keynote/Conférence principale) Working Nation to Nation: Using Evaluation as a Path to Truth and Reconciliation / Relations de nation à nation : utiliser l'évaluation comme voie vers la vérité et la réconciliation

Close captioning and translation at https://live.syncwords.com/w-L2cdGX/
Captation et traduction à https://live.syncwords.com/w-L2cdGX/

Speakers/Présentateur·trice·s

Andrealisa Belzer, CE

Sr Evaluation Advisor, Indigenous Services Canada

Larry Bremner

Proactive Information Services Inc.

Nicole Bowman

President of BPC, Bowman Performance Consulting

Dr. Nicole Bowman (Lunaape/Mohican) is a traditional Ndulunaapeewi Kwe (Lunaape woman), authentically engaged community member and a scientific and evaluation innovator and advocate. Her academic lodge sits at the intersection of truth, spirituality, traditional knowledge, sovereignty... Read More →

Vanessa Nevin

Director of Health, Atlantic Policy Congress of First Nations Chiefs Secretariat

Vanessa Nevin is a member of Sipekne’katik First Nation within Mi’kmaki. She is working currently as the Director of Health at the Atlantic Policy Congress of First Nations Secretariat (APC). Vanessa leads a dynamic health department that works to improve the health and wellbeing... Read More →

Mindy Denny

Director of Information Governance & Data Projects, Union of Nova Scotia Mi'kmaq

Mindy Denny is Mi’kmaw and a member of the Eskasoni First Nation in Unama’ki (Cape Breton, Nova Scotia). Mindy works with the Union of Nova Scotia Mi’kmaq, where she is the Director of the Information Governance & Data Projects department. Her key work involves engaging with... Read More →

Tuesday June 14, 2022 09:00 - 10:00 CDT
Tamarak

Tamarak

10:00 CDT

(Food/Alimentation) Break/Pause

Tuesday June 14, 2022 10:00 - 10:30 CDT
Conference Foyer/Hall du congrès

Conference Foyer/Hall du congrès

10:30 CDT

Solo (Debate/Débat) The Fellows Debate: Be it resolved evaluation is more art than science

The CES Fellowship is the highest honour bestowed on a CES member. The Fellowship recognizes leadership and prominence in the field of evaluation, significant lifetime achievements, and exceptional service to the profession. The CES Fellowship includes 8 former CES National Presidents, 4 former National Vice-Presidents, and many Fellows who have been keynote speakers at CES and other international evaluation conferences. As part of the Fellows’ Conference Strand, in this inaugural Fellows’ debate two prominent Fellows; Gail Barrington (Former CES National Vice-President) and Francois Dumaine (Former CES National President) will debate the resolution: Be it resolved evaluation is more art than science. This resolution will elicit examples spanning the diversity of evaluation approaches and practices. The debate format allows for a 30-minute Q&A period. Fellows’ panels at previous CES conferences have been extremely well attended and many questions were directed at the CES Fellows, suggesting this format will be well-received by attendees.

Speakers/Présentateur·trice·s

Gail Vallance Barrington

I am Gail Vallance Barrington, a CE and certified teacher. As an independent consultant, I have completed over 130 evaluation studies. I have written articles, book chapters, and two books, Consulting Start-up and Management: A Guide for Evaluators and Applied Researchers (SAGE, 2012... Read More →

François Dumaine

Partner, PRA Inc.

Linda Lee

Proactive Information Services Inc.

Linda Lee CE FCES has worked in evaluation and research for 40+ years. She has been a keynote speaker, presented papers and facilitated workshops at many national and international conferences. She has worked on evaluations across Canada and internationally. Linda, a former CES National... Read More →

Tuesday June 14, 2022 10:30 - 12:00 CDT
Cottonwood

Cottonwood, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

Solo (Expert tutorial/Tutoriel d'expert·e) From Knowledge to Action: Building knowledge and learning strategies for greater influence

The knowledge and learning drawn from evaluations has great power to influence change, if wielded with intention. This expert tutorial will focus on evaluators' instrumental role in 'closing the loop' by supporting knowledge translation, dissemination and overall learning processes at the community and institutional levels. Oxfam Canada will present its learning-centered Feminist Knowledge Strategy as an example of how to foster community empowerment and benefits throughout all stages of research, monitoring, and evaluations, while challenging power structures and gendered inequalities. The discussion will highlight case studies from the Creating Spaces To End Violence Against Women and Girls project in South and South East Asia on how to develop and implement learning strategies, and apply knowledge to ignite programming and advocacy efforts.

Speakers/Présentateur·trice·s

Anna Du Vent

Oxfam Canada

Anna is a feminist data enthusiast seeking to challenge organizations to think about knowledge and power differently. She is current a Monitoring, Evaluation, and Learning Specialist at Oxfam Canada with experience leading program, research and evaluation teams focused on gender equality... Read More →

Tuesday June 14, 2022 10:30 - 12:00 CDT
Manitoba Maple

Manitoba Maple, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

G7.1 (Short Presentation/Présentation courte) "Knowledge speaks, wisdom listens": Evaluation and learning from the wisdom of others

Staff, in a supported housing program at a Manitoban medium-sized community organization, utilize lived experience from personal and professional perspectives, in their fast and slow decision-making. For this organization, useful and inclusive evaluation involves understanding lived experience, which requires qualitative methods, such as talking and reflection. The evaluation process brings together a diversity of experiences, which enhances organizational (e.g., training and educational opportunities) and individual staff members' resources and provides a unique opportunity for staff to feel heard and supported. To understand what it entails for evaluation to be utilized to learn from the wisdom of others, we conducted interviews with 19+ staff in a variety of front-line, administrative, and managerial roles. In this presentation, we will share our research on evaluation, focusing on the value of a listening-oriented evaluation process for dynamic community organizations.

Speakers/Présentateur·trice·s

Javier Mignone

University of Manitoba

Javier Mignone is professor in the Department of Community Health Sciences, University of Manitoba. Javier teaches undergraduate and graduate courses, including program evaluation and social development. He conducts research and development projects on intercultural health and health... Read More →

Andrea Ayo

Institute of Urban Studies

Andrea Ayo is a Research Assistant at the Institute of Urban Studies, a research arm of the University of Winnipeg. She is passionate about working in and with community-centred organizations. She recently completed her undergraduate degree in International Development Studies, with... Read More →

Aynslie Hinds

University of Winnipeg

Dr. Aynslie Hinds is a Senior Analyst in Institutional Analysis at the University of Winnipeg. She also coordinates the Certificate in Applied Psychology (CiAP) program and co-chairs Community Hub-Information and Research Partnerships (CHIRP). In these roles, she supervises students... Read More →

Carolyn Peters

New Directions

Carolyn Peters is a Director at New Directions in Manitoba overseeing Program Evaluation, Organizational Development and Training. She has a background in Social Work specializing in direct service with oppressed populations and research in a practice setting most of her career. She... Read More →

Tuesday June 14, 2022 10:30 - 12:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

G7.2 (Short Presentation/Présentation courte) Not just a report: The unintended and unexpected outcomes of the evaluation process

Lived experience involves knowledge and understanding gained about the world from first-hand experience and includes life circumstances, life events, choices, beliefs, and instincts. When evaluation involves, considers, and interprets lived experience, the evaluation process may feel therapeutic for staff of the organization undergoing evaluation. Nineteen staff in a variety of roles at a Manitoban medium-sized community organization in a supported housing program were interviewed about how they collect and use information, as well as the decisions they make, and how they make them. This presentation will focus on the emotional aspects of using lived experience as an organizational and personal resource, and the revelatory and cathartic effect of evaluation. Our research on evaluation offers a unique perspective as we will reflect on the unexpected and unintended impact of evaluation on those contributing to its process.

Speakers/Présentateur·trice·s

Andrea Ayo

Institute of Urban Studies

Aynslie Hinds

University of Winnipeg

Carolyn Peters

New Directions

Tuesday June 14, 2022 10:30 - 12:00 CDT
Sage

Sage, Diversity of Evaluators/Diversité d'évaluateur·trice·s

10:30 CDT

G7.3 (Short Presentation/Présentation courte) Using Expert Review Committees Increase Diversity and Utilization! The Experience of the National Research Council Canada

"Evaluators don't need to be subject matter experts." How many times have we heard or said this assertion when conducting evaluations? But, what if the program is scientifically, technically, sociologically or contextually complex and evaluation questions are about excellence? We found that our tailored consultation process, composed of independent scientific and industry experts, fills important knowledge, context and diversity gaps especially in assessing areas of research excellence and positioning for future success. This approach, considered as one line of evidence, increases attention, involvement and buy-in and leads to evaluations that are likely to be more respected and used. In this presentation, we will showcase our experience with an expert consultation method using two formats: a standard in-person approach and a newly adapted virtual version. Join us to hear and share about the benefits of expert input to your evaluation, as well as pros and cons of both.

Speakers/Présentateur·trice·s

Frédéric Bertrand

Evaluation Manager, National Research Council Canada (NRC)

Evaluation Manager at the National Research Council since March 2021, I am working remotely from the Greater Montreal area. Previously, I acted as a consultant contributing to evaluations of scientific research programs and organizations. In addition, I have been involved with the... Read More →

Kathryn Radford

Evaluation Manager, National Research Council

I joined the National Research Council in Ottawa as Evaluation Manager in April 2020, just at the start of the pandemic. Like many people, I moved from working in an office to working from home. Allow me to share my experiences in moving to online scientific peer review committees... Read More →

Tuesday June 14, 2022 10:30 - 12:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

G18.1 (Short Presentation/Présentation courte) ReConnect Grant Program: Using diverse approaches for participant inclusion

Program evaluation findings can highly reflect on the voices and experiences of those who are directly impacted by the service or program. As part of our inclusive and equitable evaluation practice, City of Red Deer staff used a mixed methods approach with a blend of traditional and arts-based methods in our ReConnect grant program. The use of mixed methods and arts for evaluation has been ideal for an equitable evaluation as it couples the strengths of participant expression and lived experiences with quantitative measurement and analysis. The use of photographs and videos as one of methods of evaluation has helped minimize the barriers that most of the marginalized and excluded community usually face in evaluation projects. In the presentation I will describe this unique grant program and its diverse participants and the diverse approaches we have used to ensure our evaluation methods address diversity and inclusion.

Speakers/Présentateur·trice·s

Deena Giri

City of Red Deer

Deena is a current Research and Evaluation Specialist at the City of Red Deer, Alberta. Immigrated as an International student, she graduated from the University of Alberta with a master's in sciences degree in health promotion and socio-behavioural sciences. Deena started her career... Read More →

Tuesday June 14, 2022 10:30 - 12:00 CDT
Sweet Grass

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

G18.2 (Short Presentation/Présentation courte) Evaluation Methodology of the Youth Employment Strategy Program

This presentation discusses the method used for, and the results of the impact of the Youth Employment Strategy by five different intervention types. This is a Government of Canada program that targeting vulnerable youth with a focus on broad diversity inclusion objectives. The targeted participants are from 2010 to 2011 cohort, followed up to 2017. We applied propensity score matching combined with difference-in-difference to identify the impact of each intervention. We also applied machine learning methods to select an appropriate comparison group and to detect potential heterogeneity among program participants. The results from the evaluation indicate the impact of Skills Link interventions on participants' labour market outcomes was limited overall. In line with findings from the international literature on youth programs, interventions that took the form of work experiences were associated with better gains in labour market attachment relative to other intervention types.

Speakers/Présentateur·trice·s

Essolaba Aouli

Employment and Social Development Canada

Leeroy Rikhi

Employment and Social Development Canada

Tuesday June 14, 2022 10:30 - 12:00 CDT
Sweet Grass

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

G18.3 (Short Presentation/Présentation courte) Machine Learning for estimating heterogeneous treatment effects in program evaluations

Our study will show how the Evaluation directorate at Employment and Social Development Canada uses rich administrative data and Modified Causal Forests, a causal machine learning estimator, to inform policy development through impact evaluations. We will illustrate our implementation of the innovative Modified Causal Forests algorithm to estimate individualized treatment effects, thereby learning what works for whom. This endeavour is fully aligned with the Government of Canada's commitment to implement a Gender-Based Analysis+ lens in evaluation work, ensuring that differential impacts on people of various sociodemographic backgrounds are considered in policy and program development.

Speakers/Présentateur·trice·s

Leeroy Rikhi

Employment and Social Development Canada

Tuesday June 14, 2022 10:30 - 12:00 CDT
Sweet Grass

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

G1.1 (Short Presentation/Présentation courte) An Empirical Exploration of Evaluators' Values and Practice

Psychological theory suggest that evaluators' individual values and traits play a fundamental role in evaluation practice, though few empirical studies have explored those constructs in evaluators. This paper describes an empirical study on evaluators' individual, work, and political values, as well as their personality traits to predict evaluation practice and methodological orientation. The results suggest evaluators value benevolence, achievement, and universalism; they lean socially liberal but are slightly more conservative on fiscal issues; and they tend to be conscientious, agreeable, and open to new experiences. In the workplace, evaluators value competence and opportunities for growth, as well as status and independence. These constructs did not statistically predict evaluation practice, though some workplace values and individual values predicted quantitative methodological orientation.

Speakers/Présentateur·trice·s

John LaVelle

University of Minnesota

John LaVelle is Assistant Professor of Evaluation Studies at the University of Minnesota. His research is on evaluator values and ethics, the impostor phenomenon, evaluation theory, and evaluator education systems.

Tuesday June 14, 2022 10:30 - 12:00 CDT
Willow

White Spruce, Diversity of Evaluators/Diversité d'évaluateur·trice·s

10:30 CDT

G1.2 (Short Presentation/Présentation courte) Comparing and Contrasting the EDI Language Embedded in the Competencies of the Canadian Evaluation Society and the American Evaluation Association

Professional organizations for evaluators have developed sets of competencies to improve evaluation practice. Competencies are dynamic, contextual, and expected to evolve. The recent shift towards enacting Equity, Diversity, and Inclusion (EDI) in evaluation practice provides us with an opportunity to consider how we have attended to EDI so far, and how we might enhance our efforts in the future. Examining the forms of EDI language embedded in the competencies of CES and AEA will prompt discussion of questions for the evaluation field, such as: What words, phrases, or themes might we recognize as EDI language? To what extent is EDI language explicitly, or implicitly, present in the competencies promoted by our professional organizations? What changes should be considered for future iterations to better emphasize EDI? The goal of this short presentation is to provoke conversation about how we ensure EDI is an explicit element in foundational documents that define our work as evaluators.

Speakers/Présentateur·trice·s

Tyler Clark

University of North Carolina at Greensboro

Ms. Tyler Clark is a native of Charlotte, NC. and is a two-time alumnus of North Carolina Central University (NCCU), where she was awarded a B.A. in Psychology, B.S. in Criminal Justice (May 2018), and an M.S. in Criminal Justice (May 2020). While studying for her Master’s, she... Read More →

Samantha Bradley

University of North Carolina at Greensboro

Samantha Bradley is an institutional researcher at Winston-Salem State University. Her work focuses on improving higher education outcomes for underrepresented student populations. Samantha has a background in applied economics and is currently pursuing her Ph.D. in Educational Research... Read More →

Tuesday June 14, 2022 10:30 - 12:00 CDT
Willow

White Spruce, Diversity within evaluation/Diversité au sein de l'évaluation

10:30 CDT

G1.3 (Short Presentation/Présentation courte) Diversity in Designations: What Different Professional Designations Bring to the Practice of Evaluation

So you are an evaluator. What perspectives do you bring to the table based on your professional background? Are you a Credentialed Evaluator? Does it matter? What if you are a Project Management Professional (PMP)? A Certified Management Consultant (CMC)? A Chartered Professional Accountant (CPA)? Does it matter? If you possess one of these other designations, do you practice evaluation differently? This short presentation will explore the value of different professional viewpoints and training on the practice of evaluation.

Speakers/Présentateur·trice·s

Sandy Moir

Goss Gilroy Inc.

Sandy has been practicing evaluation in Canada for over 25 years. Her evaluation practice focused on public sector clients in a range of sectors. She is the managing partner of the Ottawa office of Goss Gilroy Inc. In addition to being a Credentialed Evaluator, Sandy is a Fellow Certified... Read More →

Tuesday June 14, 2022 10:30 - 12:00 CDT
Willow

White Spruce, Diversity of Evaluators/Diversité d'évaluateur·trice·s

12:00 CDT

(Food/Alimentation) CES Awards Luncheon/Dîner de remise de prix SCÉ

Close captioning and translation at https://live.syncwords.com/w-oZuiA6/
Captation et traduction à https://live.syncwords.com/w-oZuiA6/

Tuesday June 14, 2022 12:00 - 13:30 CDT
Tamarak

Tamarak

13:30 CDT

Solo (Expert tutorial/Tutoriel d'expert·e) Weathering: A Missing Tool in Our Understanding of Population Health Inequities

Canadian evaluators who research population health inequities are missing out on a powerful tool developed by American researchers: the Weathering Hypothesis, a multi-disciplinary analytic framework created in the 1990s at the Institute for Social Research, the world's largest social science survey and research organization. Initially controversial, it is now a pillar of American population health research. It was used to validate the policies behind Obamacare, for example, by identifying and eliminating discriminatory practices entrenched in policy. Canadian evaluators need this tool. This presentation will show how it could be used to better serve the program recipients we evaluate, through greater understanding of strategies communities use to mitigate racialized stressors, and how public policies sometimes disturb the protections these strategies built. The audience will be invited to contribute their views to a case scenario that showcases the power of the Weathering tool.

Speakers/Présentateur·trice·s

Christine Schippers

Research Associate, PRA Inc.

Christine Schippers is a research associate at PRA Inc. Christine has an extensive research background in economics, statistics, and population health, with a focus on poverty of vulnerable groups. She has previously been an instructor at the University of Manitoba, a Research Fellow... Read More →

Tuesday June 14, 2022 13:30 - 15:00 CDT
Cottonwood

Cottonwood, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

Solo (Leading Edge Panel/Panel d'avant-garde) Renewing Relationships: Indigenous and Federal Perspectives on Co-Development

This panel focuses on lessons learned over five years of experience in the federal government, working on performance measurement and evaluation for Indigenous programming. In this presentation, the panelists will outline common critical mistakes that government evaluations and evaluators tend to make when working with Indigenous and community partners, relaying information through pictures, story-telling and real life examples. The presentation provides space for discussion and reflection on how cultural and relational practices and assumptions can impact the design, delivery and outcomes of federal evaluations and performance measurement frameworks, and ultimately how evaluation and performance measurement is a proxy for broader treaty relationships between Indigenous Peoples and the Crown.

Speakers/Présentateur·trice·s

Marina Santilli

Claudia Gbetholancy

Results Analyst, Indigenous Services Canada

Experienced in performance measurement and climate change adaptation planning. Passionate about environmental justice and using participatory methods in decision-making processes. Co-led various research projects for both municipal and federal governments. Currently works for Indigenous... Read More →

Julie Eldridge

Evaluator, Government of Canada

Learned the hard way how not to co-develop. Academic background in sociology and philosophy sparked interest in cultural understandings of relationship and how these impact policy and evaluation. Working in the federal government on performance measurement and evaluation since 2017... Read More →

Tuesday June 14, 2022 13:30 - 15:00 CDT
Manitoba Maple

Manitoba Maple, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G19.1 (Short Presentation/Présentation courte) Accessibility--Using a Mixed-Methods Approach to Understand the Interwoven Experiences of Persons with Disabilities.

This presentation will demonstrate how using a mixed-methods approach could be used to ensure that diverse and interwoven experiences (e.g. by age group, region, type of disability, organization and community) of Canadians are considered in the context of the Enabling Accessibility Fund (EAF) evaluation. The EAF is a federal program that provides funding for construction and communication technology projects to make Canadian communities and workplaces more accessible for Persons with Disabilities (PwD). The lines of evidence include a geo-spatial funding analysis, internal and external key informant interviews, community case studies, expert panels and survey of all EAF organization applicants. For example, to understand intersectional experiences, the community case studies will reach the end-users themselves and the expert panels will help us understand accessibility needs facing PwD across the lifespan. The lines of evidence have been adapted to ensure inclusiveness.

Speakers/Présentateur·trice·s

Abhi Bhandari

Evaluation Officer, Employment and Social Development Canada

Abhi Bhandari is an Evaluation Officer within the Evaluation Directorate of Employment and Social Development Canada (ESDC). She is currently leading the evaluations of the Enabling Accessibility Fund and the Canada Service Corps. Her educational background is in Health Sciences and... Read More →

François Laferrière

Employment and Social Development Canada

François Laferrière is a Program Evaluator at Employment and Social Development Canada (ESDC). He is evaluating partnership programs such as the Enabling Accessibility Fund and the Supports for Student Learning Program. He obtained a Master's degree in Public Administration with... Read More →

Tuesday June 14, 2022 13:30 - 15:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G19.2 (Short Presentation/Présentation courte) Understanding youth program quality through a novel observation-based methodological approach

This presentation will discuss key insights and implications from the MLSE LaunchPad (MLSELP) experience with piloting a novel observation tool as part of a diverse, mixed-methods approach to evaluating program quality. The Program Quality Assessment in Youth Sport (PQAYS) instrument is a novel observational measure intended to assess program quality related to safe spaces, psychosocial and life skills development within youth sport. From 2019-2021, MLSELP piloted the implementation of PQAYS for utility as a program quality evaluation measure in a diverse, multi-cultural and community-based youth environment. Following its implementation in 34 individual programs over 3 years at its Sport For Development facility, MLSELP has generated key insights and lessons for integrating an observation-based methodology as part of a data-driven quality-improvement culture; and critical feedback for adapting the PQAYS measure for practical use in the evaluation of community-based youth programs.

Speakers/Présentateur·trice·s

Bryan Heal

Manager, Research & Evaluation, MLSE Foundation / MLSE LaunchPad

Bryan (he/him) is the Change the Game Research Lead for MLSE Foundation, and manages the Evaluation portfolio at MLSE LaunchPad, a Sport For Development facility in downtown Toronto built to advance positive development outcomes for youth facing barriers. Bryan manages the evaluation... Read More →

Jackie Robinson

Coordinator, Research & Evaluation, MLSE LaunchPad

Jackie (she/her) coordinates the Research & Evaluation portfolio at MLSE LaunchPad, a Sport For Development facility in downtown Toronto built to advance positive development outcomes for youth facing barriers. Jackie coordinates and leads the analysis of over 200 youth program... Read More →

Tuesday June 14, 2022 13:30 - 15:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G19.3 (Short Presentation/Présentation courte) Do drug treatment court ratings differ by participant, graduate, drop out or discharge status? A mixed methods analysis

Past drug treatment court evaluation research has focused on graduates, who generally show low reoffence rates and give high ratings to the drug treatment court experience. Attrition, however, has been a problem for Canadian drug courts and less is known about the attitudes and feelings from those who leave or are discharged from programs. As part of a large scale evaluation, Likert survey and open-ended depth interview questions were used to assess and contrast participant perceptions of a Canadian DTC based on program status. Both quantitative and qualitative results did show variation, and had clear policy implications. The strengths in the diversity of methodological approaches are also discussed.

Speakers/Présentateur·trice·s

Michael Weinrath

Dr., University of Winninpeg

Michael Weinrath is Professor and founding Chair of Criminal Justice, and Director of the Justice Research Institute at the University of Winnipeg. He teaches program evaluation and criminal justice policy courses for honours and graduate students. Over the past 10 years his major... Read More →

Sarah Heath

Instructor III, University of Winnipeg

Tuesday June 14, 2022 13:30 - 15:00 CDT
Sage

Sage, Diversity of Evaluators/Diversité d'évaluateur·trice·s

13:30 CDT

G8.1 (Storytelling/Récit) The Second-Best Way to Do Things': Stories of a multidisciplinary, participatory, online evaluation of wellness programming for people living with dementia

A master of social work student, a psychology professor, a leader in community-based research, and an emerging public health evaluator make up the diverse team that collaborated with a steering committee of staff and Executive Directors from three local Alzheimer Societies to assess the implementation and outcomes of virtual wellness programing for people living with dementia and their care partners compared to in-person programming. This presentation will tell the story of how a diverse group of evaluators and stakeholders wove their perspectives and respective agendas together through a community-based, participatory evaluation to formulate fresh insights into the benefits of various modalities of programming for people living with dementia. In this story telling session, we will share the successes and challenges the evaluation team faced negotiating the evaluation scope and logistics of planning and doing a participatory evaluation with people living with dementia -- all online.

Speakers/Présentateur·trice·s

Hsiao d’Ailly

Associate Professor, Renison at University of Waterloo

Amanda Demmer

University of Waterloo; Centre for Community-Based Research

Amanda Demmer is a PhD student in Public Health at the University of Waterloo. She holds an M.A in Community Psychology supplemented by years of community-based research and evaluation experience and applied experience coordinating and evaluating suicide prevention initiatives within... Read More →

Tuesday June 14, 2022 13:30 - 15:00 CDT
Sweet Grass

Sweet Grass, Diversity of Evaluators/Diversité d'évaluateur·trice·s

13:30 CDT

G8.2 (Short Presentation/Présentation courte) Serving People Living with Dementia under Covid-19: From In-Person to Virtual

Is it possible to continue to serve and to reduce the social isolation of those living with dementia and their care partners during the COVID-19 pandemic where mandated social isolation is implemented as a strategy to mitigate the virus' spread? This presentation will highlight the findings of an evaluation of social recreational programs designed to enhance physical and mental stimulation and to prevent the social isolation of people living with dementia and their caregivers. These programs offered by three Alzheimer's societies in Ontario, funded by The Trillium Foundation, and the community-based evaluation itself, were forced to quickly adapt to challenges imposed on them by COVID-19. We will present both formative and summative findings on program implementation and outcomes before and after COVID-19 and will explore the strengths, limitations, and new opportunities of remote programming for people living with dementia and their care partners.

Speakers/Présentateur·trice·s

Hsiao d’Ailly

Associate Professor, Renison at University of Waterloo

Amanda Demmer

University of Waterloo; Centre for Community-Based Research

Tuesday June 14, 2022 13:30 - 15:00 CDT
Sweet Grass

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G8.3 (Short Presentation/Présentation courte) Evaluation as a cloaking device: how common evaluation methods help keep diverse populations invisible

Full and respectful consideration of the perspectives of diverse populations in program evaluations requires data collection and analysis at a level of granularity precluded by the academic and positivist underpinnings of many evaluation methods. For example, the continued heraldry of quantitative methods is serving to help evaluators keep diverse populations invisible by aggregating sub-populations to achieve the holy grail of 'N > n'. In reality, total populations (N) which feature diversity can often be smaller than the statistically required sample size (n). In this presentation, Paul will illustrate, with a logic model metaphor, how the reach of summative evaluations can be stunted by an institutional insistence on statistically-inferred indicators. The presentation will also consider how evaluation data that respects the perspectives of diverse populations, and helps to make diverse populations visible, can be generated from already-existing data sources with the right tools.

Speakers/Présentateur·trice·s

Paul Kishchuk

Vector Research

Paul Kishchuk, MA CE is a public finance economist and founded Vector Research, an independent economic research consultancy in Whitehorse, Yukon in 1999, through which he has completed more than 285 research and evaluation assignments. Paul's lived experience in a remote and data-sparse... Read More →

Tuesday June 14, 2022 13:30 - 15:00 CDT
Sweet Grass

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G17.1 (Short Presentation/Présentation courte) Rapid Evaluations in the Context of COVID-19 Global Pandemic and Social Justice Concerns

Program evaluation has transformed from a sideline activity conducted by part-time academics in the '60s into a field of professional practice replete with theories, competencies, ethical guidelines, standards, and robust approaches. With the global spread of COVID-19 and protests against social injustices resulting from the murder of George Floyd, many organizations have turned to rapid evaluation(RE) approaches to provide timely feedback on response and recovery, pushing the boundaries of evaluation practice beyond the usual formative, summative, and impact evaluations. Participants will discuss and reflect on: a)how can evaluators use AEA competencies and Culturally Responsive Evaluation lens in RE to serve vulnerable populations, b)how can evaluators use the lessons learned from RE to address the challenges of globalization, social change, social justice, equity, and sustainability, c) how might educators prepare emerging evaluators in the use of RE focusing on social justice.

Speakers/Présentateur·trice·s

Stacy Huff

Graduate Student, University of North Carolina at Greensboro

I am a third-year M.S./Ph.D. graduate student in Educational Research, Assessment, and Measurement with a concentration in Program Evaluation. Prior to coming to grad school, I spent 25 years in the veterinary field in Emergency and Critical Care. I am interested in the evaluation... Read More →

Tuesday June 14, 2022 13:30 - 15:00 CDT
Willow

WIllow, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G17.2 (Storytelling/Récit) Adventures in survey co-design

Join us for the story of our recent entry into the world of collaborative survey design. As part of ongoing work with the Canadian Women's Foundation, Emily and Zia brought their knowledge of survey design to the development of a survey on the effects of COVID-19 on Indigenous youth across Canada. To design this survey, they worked with personnel from the Foundation and with Indigenous youth from across Canada. Each individual in this group brought their unique knowledge and experience to the table, with the youth members of the group leading the way, supported by the other group members. This was our first foray in to collaborative survey design. While there were challenges along the way, this experience was also uniquely rewarding. We would like to share our approach, and the lessons that we learned along the way.

Speakers/Présentateur·trice·s

Emily Brennan

Senior Evaluation Manager, Agriculture and Agri-Food Canada

Tuesday June 14, 2022 13:30 - 15:00 CDT
Willow

WIllow, Diversity within evaluation/Diversité au sein de l'évaluation

13:30 CDT

G17.3 (Short Presentation/Présentation courte) Integrating social justice and culturally responsive and equitable evaluation practices in meta-evaluation: Learning from the UN evaluation quality assessments

Meta-evaluation refers to “evaluating evaluations” or assessing the quality of evaluations for its adherence to established evaluation standards to increase the trust of the intended users of the evaluations. While meta-evaluation is widely regarded as an essential tool for the evaluation practice, our current study shows much diversity in the practice and tools for meta-evaluation, such as Stufflebeam’s Checklist (Stufflebeam, 2001) and the United Nations Evaluation Groups’ Evaluation Quality Assessments (UNEG/EQAs) to name a few. Despite the robustness of the EQAs and their human rights focus, there is a need to integrate social justice and culturally responsive and equitable evaluation (CREE) practices to the UNEG EQAs. Participants in this session will discuss and reflect on (a) the main components of UNEG EQAs, (b) the key lessons from the UNEG EQAs, and (c) brainstorm ways to integrate social justice and CREE in the EQAs in line with the current focus on addressing systemic racism and diversity in evaluations.

Speakers/Présentateur·trice·s

Sandra Ayoo

Assistant Professor, University of North Carolina Greensboro

Sandra Ayoo is an Assistant Professor of Educational Research Methodology at the University of North Carolina Greensboro where she teaches program evaluation and research methodology. She is a researcher, teacher, and practitioner of program evaluation and her scholarship focuses... Read More →

Meghan Leeming

University of North Carolina at Greensboro

Tuesday June 14, 2022 13:30 - 15:00 CDT
Willow

WIllow, Diversity within evaluation/Diversité au sein de l'évaluation

15:00 CDT

(Food/Alimentation) Break/Pause

Tuesday June 14, 2022 15:00 - 15:30 CDT
Conference Foyer/Hall du congrès

Conference Foyer/Hall du congrès

15:30 CDT

Solo (Leading Edge Panel/Panel d'avant-garde) Diverse Experiences with Participatory Evaluation

We propose a panel of diverse individuals (i.e., evaluators, community members and program staff) sharing their experiences regarding employing a participatory approach to federal evaluations of Indigenous programs that recognize the relevant TRC Calls to Action, the UN Declaration on the Rights of Indigenous Peoples and Evaluation guiding principles for Indigenous context (Maddox et al., 2021). This approach is uncommon in the Canadian Federal Government and supports the co-development and implementation of evaluations. The focus is on reconciliation and building stronger relationships among Indigenous and non-Indigenous people involved in evaluations. There is a development of common ground, as well as shared goals and approaches that balance tensions and power dynamics during all phases of an evaluation. The result are culturally safe evaluations as identified by Indigenous communities that consider the community's knowledge to provide relevant findings and recommendations.

Speakers/Présentateur·trice·s

Andrea Johnston

CEO, Johnston Research Inc.

Johnston has been employed full-time as an evaluation manager working towards a future that changes the relationships between Indigenous Peoples and funding agents to operationalize an environment in which Indigenous Peoples can set their own parameters. In 2016, Johnston launched... Read More →

Elyse McCall-Thomas

Evaluation Manager / PhD Student, NSERC / University of Ottawa

I have worked in the field of program evaluation for over fourteen years, starting as a consultant before joining the federal government as an internal evaluator. Currently, I am an Evaluation Manager with the Natural Sciences and Engineering Research Council. I am also a second-year... Read More →

Litsa Papathanasopoulou

Evaluation Manager, DFO

I am an Evaluation Manager with the Public Health Agency of Canada. I have been in the evaluation field in the federal government milieu for twenty years. During this time, I acquired an extensive knowledge and experience working collaboratively with Indigenous people to improve evaluation... Read More →

Tuesday June 14, 2022 15:30 - 17:00 CDT
Cottonwood

Cottonwood, Diversity within evaluation/Diversité au sein de l'évaluation

15:30 CDT

Solo (Debate/Débat) No sustainability without equity. No equity without sustainability

Sustainability and equity are the major global, national and local issues of the day. The panelists regard sustainability and equity as coupled matters with similar origins and with some similarities for evaluation. We also regard the two matters as non-negotiable both needing to be addressed in every evaluation. Panelists will first discuss and invite the audience to consider the concept that sustainability and equity matters are coupled matters with similar origins. Then the panel will provide their core elements for a checklist of what elements need to be addressed to incorporate sustainability and equity in evaluation. The audience will be asked to comment on these and provide additional core priorities. The results will be compiled and shared back with attendees following the conference. The panelists are recognised global leaders in sustainability and equity and look forward to sharing and learning with participants.

Speakers/Présentateur·trice·s

Larry Bremner

Proactive Information Services Inc.

François Dumaine

Partner, PRA Inc.

Debbie DeLancey

D.J. DeLancey Consulting

Andy Rowe

Dr, Footprints Evaluation

Nicole Bowman

President of BPC, Bowman Performance Consulting

Tuesday June 14, 2022 15:30 - 17:00 CDT
Manitoba Maple

Manitoba Maple, Diversity within evaluation/Diversité au sein de l'évaluation

15:30 CDT

G6.1 (Short Presentation/Présentation courte) A Pan-Canadian, Multi-Agency Approach to Federal Impact Evaluation in the Mid-term Assessment of the Regional Economic Growth through Innovation (REGI) Program

This presentation will offer insights into an innovative approach to federal impact evaluation which brought together internal evaluators from six regional development agencies (RDAs) across Canada guided by an external evaluator consultant. The goal was to conduct a Mid-term Assessment of the Regional Economic Growth through Innovation (REGI) Program, a nationally coordinated, regionally delivered initiative. The presentation will cover the approach to ensure meaningful collaboration and co-creation of the assessment and how data was collected and aggregated across the diverse agencies. The presentation will also identify lessons learned and areas for improvement in the approach. The presentation aligns with the conference sub-theme of "Diversity of Evaluators" due to the diverse geographic representation of evaluators (coast-to-coast-to-coast) and organizational contexts in which they operate.

Speakers/Présentateur·trice·s

Beth Garner

Principal, Ference & Company Consulting Ltd.

Beth Garner is a Principal with Ference & Company Consulting Ltd. For over ten years, Beth has undertaken evaluations, performance measurement, and stakeholder engagement for various government and non-profit organizations in the areas of innovation, economic and social development... Read More →

Tuesday June 14, 2022 15:30 - 17:00 CDT
Sage

Sage, Diversity of Evaluators/Diversité d'évaluateur·trice·s

15:30 CDT

G6.2 (Short Presentation/Présentation courte) Program vs. Thematic Evaluations: Insights from CMHC on Evaluating Canada's National Housing Strategy

Imagine this: Your organization has been given $70+ billion across 20+ new and diverse programs under a new federal strategy with interdependent objectives, including targets for different population groups. Programs have ten years of funding and you have to return to Cabinet triennially to report on progress with the opportunity to course correct. How would you evaluate this undertaking? This session will illustrate how Canada Mortgage and Housing Corporation's (CMHC) evaluation team tackled this challenge via the development of an evaluation roadmap which incorporated both program-specific evaluations and thematic evaluations. This allowed CMHC to put in place a solid plan to assess the diversity of programs, collective impacts and efficiency. Join us to learn more about our journey of going beyond the typical departmental evaluation plan to ensure evaluation coverage of complex programs, initiatives and thematic outcomes.

Speakers/Présentateur·trice·s

Brian Diener

CMHC

Brian (he/him) heads the evaluation function at Canada Mortgage and Housing Corporation (CMHC). Prior to this, he led evaluations for Immigration, Refugees and Citizenship Canada and other organizations. He holds an MBA, a MSc in International Public Policy and a Bachelor of Comm... Read More →

Katie Gilhooly

Canada Mortgage and Housing Corporation

Katie Gilhooly (she/her) is a Senior Evaluator at Canada Mortgage and Housing Corporation with a decade of professional experience across performance measurement, evaluation and policy. She completed the graduate diploma in Policy and Program Evaluation (DPE) and holds a Canadian... Read More →

Sharon Lécuyer

Canada Mortgage and Housing Corporation

Sharon Lécuyer (she/her) is a Senior Evaluator at Canada Mortgage and Housing Corporation. She is an emerging evaluator bringing more than a decade of professional experience in program delivery and policy development. She holds a Master of Arts in Public Administration and a Bachelor... Read More →

Tuesday June 14, 2022 15:30 - 17:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

15:30 CDT

G6.3 (Short Presentation/Présentation courte) Impact of the 2016 Policy on Results on the federal evaluation practices

The objective of the research is to describe the impact of the 2016 Policy on Results on federal evaluation practices, their diversity as well as the usefulness and the utilization of evaluation findings. In 2019, we reported on a comparison of 60 characteristics of 64 evaluation reports produced in 2014-2015 (before the 2009 policy was morphed into the 2016 policy) and in 2017-2018 (~after). We have replicated the core design with 32 2021 reports, adding another time point to the analysis. We also documented evidence of consideration given by the evaluation to four government-wide policies/priorities: SGBA+ (including at-risk, vulnerable, and target populations), official languages, sustainability, and reconciliation. This three time point comparison over six years is an innovative approach to documenting how institutional mechanisms can modify the practice of evaluation.

Cette recherche veut décrire l'impact de la Politique des résultats de 2016 sur les pratiques d'évaluation fédérales, leur diversité ainsi que l'utilité et l'utilisation des résultats d'évaluation. En 2019, nous avons fait état d'une comparaison de 60 caractéristiques dans 64 rapports d'évaluation produits en 2014-2015 (avant que la politique de 2009 ne se transforme en politique de 2016) et en 2017-2018 (~après). Nous avons reproduit ce modèle avec 32 rapports additionnels de 2021, ajoutant un autre point dans le temps à l'analyse. Nous avons également documenté la considération accordée par ces évaluations à quatre politiques ou priorités gouvernementales : ACSG+ (y compris les populations à risque, vulnérables et ciblées), les langues officielles, la durabilité environnementale et la réconciliation. Cette comparaison de trois points dans le temps sur une période de six ans est une approche innovante pour documenter comment les mécanismes institutionnels peuvent modifier la pratique de l'évaluation.

Speakers/Présentateur·trice·s

Simon Roy

Dr, Goss Gilroy Associate

Shelley Borys

Director General, Evaluation and Chief Audit Executive, Public Health Agency of Canada and Health Canada

Shelley Borys is the Director General, Evaluation, and Chief Audit Executive in the Office of Audit and Evaluation, a shared service for the Public Health Agency of Canada and Health Canada. She holds a PhD in Social Psychology from the University of Waterloo. Shelley is a Credentialed... Read More →

Benoît Gauthier

Circum inc.

Tuesday June 14, 2022 15:30 - 17:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

15:30 CDT

G5.1 (Short Presentation/Présentation courte) Using oral histories within a "counter-research" undergraduate program: A new method for gaining deeper insight into and through interwoven experiences

When studying programs inherently related to equity and challenging the dominant research perspective, it is antithetical to use traditional methodologies. Evaluation methods should be as culturally responsive as the programs themselves. To re-conceptualize evaluation to be more inclusive, it is critical to give voice to historically underrepresented and marginalized individuals to honor their experiences, cultures and identities. One way is through oral histories--interviews to elicit memories and personal stories about significant events and experiences. This presentation will describe their use to evaluate a "counter-research" program seeking parity of minority representation in culturally-responsive research by training undergraduates to conduct research challenging dominant paradigms. We will introduce this innovative method, our findings and share our team's successes, failures and recommendations so that participants' and evaluators' interwoven experiences can guide future work.

Speakers/Présentateur·trice·s

Shannon Sharp

Evaluation Associate, University of Mississippi

Shannon Sharp is an Evaluation Associate at the Center for Research Evaluation. After receiving her doctorate in School Psychology, Dr. Sharp practiced as a school psychologist for eight years before joining the Center for Research Evaluation as an evaluator in 2017. Her project work... Read More →

Tuesday June 14, 2022 15:30 - 17:00 CDT
Sweet Grass

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

15:30 CDT

G5.3 (Short Presentation/Présentation courte) Understanding Diverse Experiences Through Surveying- Limitations and Opportunities

The City of Red Deer completes a bi-annual community wide survey on topics of safety and policing to assist in setting RCMP policing priorities. Historically this survey did not include any questions on identity. Consequently, there was no ability to conduct analysis on differences in sense of safety between population groups who historically have experience safety differently: people of colour, Indigenous people, people with a disability, etc. As of 2022, the City is now moving forward with identifying questions. This presentation will review how the survey tool was created, and how these additional questions allow for a more nuanced assessment of safety that captures historically marginalized voices and experiences. The presentation will provide an overview of approaches for collecting and analyzing survey data in a way that considers diverse populations and seeks to be inclusive of marginalized experiences.

Speakers/Présentateur·trice·s

Kate McBride-Staples

Research and Evaluation Coordinator, City of Red Deer

Kate McBride-Staples (she/her/elle) lives in rural Alberta and works as the Research and Evaluation Coordinator for the City of Red Deer. Her focus areas include community safety, community data collection, and program impact assessment. Her professional training is as an epidemiologist... Read More →

Tuesday June 14, 2022 15:30 - 17:00 CDT
Sweet Grass

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

17:00 CDT

(Social) CES National Mix and Mingle/SCÉ Nationale, rencontres et échanges

Tuesday June 14, 2022 17:00 - 18:00 CDT
Tamarak

Tamarak

18:45 CDT

(Social) Dine Around Town/Dîner en ville

Tuesday June 14, 2022 18:45 - 21:00 CDT
Offsite/En ville

Offsite/En ville

21:00 CDT

(Social) Party in the 'Peg/Ébrouement à Winnipeg

Tuesday June 14, 2022 21:00 - Wednesday June 15, 2022 00:00 CDT
Hospitality Suite/Suite de réception

Hospitality Suite/Suite de réception

06:00 CDT

(Social) Store your luggage / Rangez votre valise

Wednesday June 15, 2022 06:00 - 12:00 CDT
Prairie Grass

Prairie Grass

07:00 CDT

(Thematic breakfast/Déjeuner thématique) Approaches to engaging and co-developing Indigenous Program Evaluations

This presentation will facilitate an interactive discussion while incorporating a variety of perspectives and interwoven experiences from evaluators, analysts and Indigenous peoples to share and further build knowledge that will help inform collaborative approaches to Indigenous program evaluations. Through this discussion, there will be an opportunity to share and gather insights and diverse perspectives into best practices to more effectively understand the unique experiences of Indigenous Peoples in the context of Indigenous program evaluations. The discussion will also draw on research, approaches and lessons learned from the evaluations of the Skills and Partnership Fund and the Indigenous Early Learning and Child Care Transformation Initiative. Based on experiences and perspectives shared, this discussion will seek to gather new knowledge and guiding principles to evaluation approaches that are meaningful to Indigenous partners and the evaluation community.

Speakers/Présentateur·trice·s

Thulasi Venkateswaran

Evaluation Officer, Employment and Social Development Canada

Thulasi has been working with Employment and Social Development Canada since 2016. She has led the research of the Nunavut Inuit labour force to help fulfil Nunavut Agreement obligations, & has also worked on several federal government program evaluations such as the Foreign Credential... Read More →

Wednesday June 15, 2022 07:00 - 08:15 CDT
Tamarak

Tamarak, Diversity within evaluation/Diversité au sein de l'évaluation

07:00 CDT

(Thematic breakfast/Déjeuner thématique) Get your cross trainers on! Rapid evaluation with diverse groups with complex challenges

It seems that, these days, there are many calls for evaluations to be completed in 3-4 months, so we think there is a lot we can learn from one another to assist clients and funders. We will start discussion with a case -evaluating outcomes with newcomer (immigrant and refugee) families learning online during covid-19. The project involved 5 diverse NGOs, in 5 different cities, under 3 provincial and 2 national government agencies. The newcomers (almost 1/2 refugees) came from 8 countries, spoke 12 languages and had many challenges associated with resettlement. We had less than 3 months to complete the evaluation, but successful in documenting outcomes and generating implications and recommendations. In addition to the final report circulation, we've used the knowledge in three presentations with clients to policy/funding and practice groups. We'll share what we've learned and look forward to hearing the wisdom and questions of others.

Speakers/Présentateur·trice·s

Eugene Krupa

Co-Principal, senior Evaluator, Catalyst R&D

Krupa is Co-Principal of Catalyst Research and Dev’t (with Kate Woodman) and held academic positions in Alberta, BC and Zambia. Catalyst specializes in evaluating innovations in health & human services, and building capacity of communities and organizations. Gene has a background... Read More →

Wednesday June 15, 2022 07:00 - 08:15 CDT
Tamarak

Tamarak, Diversity within evaluation/Diversité au sein de l'évaluation

07:00 CDT

(Thematic breakfast/Déjeuner thématique) Global, National, and Chapter collaboration toward synergy and diversity in Indigenous evaluation professional development

The presenters will draw on their longstanding collaboration to outline successes and challenges they have experienced in supporting global, national, and regional Indigenous-specific evaluation capacity infrastructure to promote professional evaluation that is by, for, and as Indigenous evaluators. Participants will be invited to reflect on ways that CES members of diverse backgrounds can find their "seat in the bus" to support distinct, sovereign, decolonized, culturally regenerative, practice by First Voice evaluators to accelerate progress on system transformation necessary to achieve interrelated priorities for equity and sustainability outlined in the UN Declaration of the Rights of Indigenous Peoples, Canada's Truth and Reconciliation Calls to Action, and Recommendations to Canada for the UN Decade for People of African Descent. The session will increase shared awareness of existing networks and supports, and solicit input on a list of priority challenges and gaps to address.

Speakers/Présentateur·trice·s

Andrealisa Belzer, CE

Sr Evaluation Advisor, Indigenous Services Canada

Wednesday June 15, 2022 07:00 - 08:15 CDT
Tamarak

Tamarak, Diversity of Evaluators/Diversité d'évaluateur·trice·s

07:00 CDT

(Thematic breakfast/Déjeuner thématique) How to develop and maintain an inclusive evaluation function, at all levels

A diverse and inclusive workplace is a key component for constructive organisational and team culture. Greater diversity and inclusion enables organizations to leverage the range of perspectives needed to address today's complex challenges. A successful, diverse and inclusive evaluation function drives innovation, increases the intersecting identity factors analysis and creates a healthy and respectful workplace. It also helps evaluators at all levels to recognize their own beliefs and bias which impact their relationships, decisions, and analysis. This thematic breakfast session will be an opportunity to share strategies and recruitment practices for diversifying the evaluation workforce and the key challenges to develop and maintain a professional inclusive evaluation function. During the discussion, we will explore how the evaluation team can reflect on their own beliefs and bias in using intersecting identity factors which could impact the evaluation projects.

Speakers/Présentateur·trice·s

Judy Lifshitz

Team Lead, Infrastructure Canada

Rakhi McAuliffe

Evaluation Officer, Infrastructure Canada

Wednesday June 15, 2022 07:00 - 08:15 CDT
Tamarak

Tamarak, Diversity of Evaluators/Diversité d'évaluateur·trice·s

07:00 CDT

(Thematic breakfast/Déjeuner thématique) Institutionalization process of the evaluation of public policies in Africa: Experience from Côte d'Ivoire

Institutionalization process of the evaluation of public policies in Africa: Experience of the Ivory Coast The process of institutionalizing the evaluation of public policies is led by the government of Côte d'Ivoire. The government expressed a firm political will to carry out a number of reforms, including the drafting of a law on the evaluation of public policies by the parliament. After the integration of evaluation into the new constitution of November 2016, the parliament just set up its commission for the evaluation of public policies for the very first time. This process poses several challenges compared to the experiences of several other countries.

Speakers/Présentateur·trice·s

Samuel Kouakou

President, Ivorian Initiative for Evaluation (2IEval)

Wednesday June 15, 2022 07:00 - 08:15 CDT
Tamarak

Tamarak, Diversity within evaluation/Diversité au sein de l'évaluation

07:00 CDT

(Thematic breakfast/Déjeuner thématique) L'internationnalisation de l'évaluation: regards croisés

L'internationalisation est omniprésente dans la quasi-totalité des domaines d'intervention, y compris dans le domaine de l'évaluation. L'institutionnalisation de la gestion axée sur les résultats (GAR) à l'échelle mondiale a placé la fonction d'évaluation comme un enjeu de de la performance de de l'action publique. Dans la perspective d'une internationalisation de l'évaluation quels sont les divers enjeux? Quels défis pour la collaboration internationale? Comment tirer parti du facteur culturel?

Speakers/Présentateur·trice·s

Anna Gueye

Gesteval Inc.

Anna Gueye est une évaluatrice accréditée par la Société canadienne de l’évaluation. Elle est formatrice en évaluation de politiques et chargée d’enseignement à l’École nationale d’Administration publique. Présidente de Services Conseils Gesteval Inc., un consortium... Read More →

Wednesday June 15, 2022 07:00 - 08:15 CDT
Tamarak

Tamarak, Diversity within evaluation/Diversité au sein de l'évaluation

07:00 CDT

(Thematic breakfast/Déjeuner thématique) Reflect on the Past to Inform the Future: a Mixed Dialogue Between SEEs and Experienced Evaluators

As the field of evaluation changes and grows, there is value in reflecting on learned experiences from seasoned evaluators. Yet, utilizing new insights and fresh perspectives from student and emerging evaluators (SEEs) is equally important to guide the future of the field. This interactive thematic breakfast aims to combine learnings from seasoned evaluators with innovative perspectives of emerging evaluators to visualize what the future of evaluation could look like and how this may be actualized. The aim is to move the dialogue away from SEEs only needing support and instead ask the question "what can SEEs bring to the future of evaluation?". Opportunity will be given for these combined perspectives to co-develop ideas for realizing the full potential of SEEs within the ever-growing field of evaluation as well as to reflect on the trajectory of evaluation over the last 5-20 years and identify ways to move evaluation into the future.

Speakers/Présentateur·trice·s

Cassandra Parsons

Evaluation Consultant, Ference & Company

Cassandra Parsons is an emerging evaluator working as an Evaluation Consultant in Vancouver, BC. She is the student and emerging evaluator representative for CES-BC and CES National. Cassandra has a diversity of experience in areas such as global health, health equity, HIV/AIDS prevention... Read More →

Maria Montenegro

Virtual Health Evaluation Lead, Fraser Health Authority and Chair, EvalYouth North America

Maria is a Credentialed Evaluator with an MSc from the University of Alberta. Maria has experience in various areas including health, immigration and settlement, education, and gender. She is an Evaluation Lead, Virtual Health at Fraser Health Authority. Maria is the Chair and Founder... Read More →

Wednesday June 15, 2022 07:00 - 08:15 CDT
Tamarak

Tamarak, Diversity of Evaluators/Diversité d'évaluateur·trice·s

07:00 CDT

(Thematic breakfast/Déjeuner thématique) We Can Do It: Mainstreaming Sustainability in all Evaluation

While there is now greater awareness and willingness to address environmental sustainability in evaluations, there is considerable uncertainty and concern about how to do this. The hosts of this thematic breakfast are globally recognised as two of the most experienced and recognised evaluators addressing sustainability and climate. We are convinced that sustainability has an important place in all evaluations and that this is entirely within the reach of evaluation. After all, since we depend on natural systems for our existence, and all that we do, it seems reasonable to say that everything we do affects natural systems. That includes the interventions we evaluate. Evaluation has not been very interested in natural systems but global awareness of the urgent need for sustainability is changing this. Andy and Juha look forward to discussing sustainability and evaluation with participants and sharing experiences and suggestions for incorporating sustainability into your evaluations.

Speakers/Présentateur·trice·s

Andy Rowe

Dr, Footprints Evaluation

Wednesday June 15, 2022 07:00 - 08:15 CDT
Tamarak

Tamarak, Diversity within evaluation/Diversité au sein de l'évaluation

08:30 CDT

Solo (Expert tutorial/Tutoriel d'expert·e) Learning from Implementing Feminist Approaches in Formative Evaluation

Oxfam applies a feminist lens in monitoring, evaluation and learning (MEL) to think differently about knowledge and power. Using this lens, the Capacity Assessment Tool for Gender-Just Organizational Strengthening (CAT4GJO) is a set of tools to support capacity strengthening for women's rights with civil society partners. The CAT4GJO informs MEL processes in Oxfam's Women's Voice and Leadership projects in Asia and Latin America. This presentation will probe the experiences of staff and stakeholders, involved at various levels in these projects, to gather lessons from seeking to implement feminist MEAL through the CAT4GJO. This will include focusing on the experiences of Indigenous and women-led organizations, with varying levels of skills and resources undertaking feminist evaluations. Speakers will unpack why using a broad definition of evaluator and contextualizing tools is integral to understanding how feminist approaches can contribute to organizational growth.

Speakers/Présentateur·trice·s

Anna Du Vent

Oxfam Canada

Carla Caxaj

Oxfam Canada

Carla is a Monitoring, Evaluation, & Learning (MEL) Program Officer in Oxfam Canada's Women's Economic Empowerment & Transformative Leadership Unit. Currently, she is the MEL officer for the Women's Voice and Leadership, and Camino Verde projects in Guatemala. She has a specialisation... Read More →

Richa Sharma

Oxfam Canada

Richa is a Monitoring, Evaluation, & Learning Program Officer in Oxfam Canada's Women's Economic Empowerment & Transformative Leadership Unit. Currently, she is the MEL officer for the Women's Voice and Leadership (Pakistan), and Securing Rights project in Bangladesh. She has research... Read More →

Wednesday June 15, 2022 08:30 - 10:00 CDT
Cottonwood

Cottonwood, Diversity of Evaluators/Diversité d'évaluateur·trice·s

08:30 CDT

G99.1 (Ignite Presentation/Présentation éclair) Evaluation Capacity Building as a Complex System

Barbrook-Johnson et al. (2021) issued a "call for more radical and ambitious interpretations of what complexity means for evaluation scholarship and practitioners" (p. 14). I respond to this call by developing the argument that evaluation capacity building (ECB) is a complex system focused on building capacity to support learning and evaluation use. The application of complexity perspectives to ECB may provide a greater understanding of educational change (Jacobson et al., 2019) and thus the most beneficial ECB strategies to employ at any given time. Recognizing and treating ECB as a complex system encourages an adaptive and dynamic approach to building evaluation capacity that is responsive to contextual characteristics. This in turn allows the evaluator to assume a variety of roles and to tailor their ECB strategies to the needs and context of individual evaluations including the use of appropriate evaluation approaches.

Speakers/Présentateur·trice·s

Elyse McCall-Thomas

Evaluation Manager / PhD Student, NSERC / University of Ottawa

Wednesday June 15, 2022 08:30 - 10:00 CDT
Manitoba Maple

Manitoba Maple, Diversity within evaluation/Diversité au sein de l'évaluation

08:30 CDT

G99.2 (Ignite Presentation/Présentation éclair) Incorporating Diverse Stories into Health Evaluation

Reporting quantitative outcomes for patients can provide interesting information about the clinical successes of a health program, but how do we capture the diverse experiences of patients? Reporting qualitative outcomes for patients through semi-structured interviews or focus groups can be valuable to outlining shared experiences with a program, but is that enough to capture the nuances of patient care? Our firm recently conducted several in-depth patient narrative interviews to capture the experience of patients in a new way. We explored their cancer journey using an open-ended guide and utilized their experience with the program to create short stories to include in the evaluation, alongside more traditional methods. The stories provided rich information to the reader and contextualized the successes in a new way. During this presentation, we will explore what worked well, what didn't, and how we might incorporate the diverse experiences of patients into future work using stories.

Speakers/Présentateur·trice·s

Sarah Boorman

Principal, Ference & Company Consulting

Sarah Boorman is a Principal with an evaluation consulting firm, Ference & Company Consulting Ltd. As a Credentialed Evaluator (CE) and Project Management Professional (PMP), she specializes in leading provincial and national level evaluations of health and social policies, initiatives... Read More →

Wednesday June 15, 2022 08:30 - 10:00 CDT
Manitoba Maple

Manitoba Maple, Diversity within evaluation/Diversité au sein de l'évaluation

08:30 CDT

G99.3 (Ignite Presentation/Présentation éclair) Systems Approaches to Homelessness: Lesson Learned From Calgary's Homeless Serving System

This presentation summarizes lessons learned from a research study on the evolution of the homeless serving system of care (HSSC) in Calgary, Alberta. The case study uses qualitative research methods: a literature review, document reviews, and key informant interviews to explore various aspects of Calgary's HSSC from a collective governance lens. Eco-systems integration i.e. alignments in services and practices between the different sectors involved in homelessness (e.g., housing, health, and justice systems) relies on various aspects of collaboration, diversity and inclusion in implementing more coordinated solutions to homelessness. The research is more closely aligned with the first stream "Diversity Within Evaluation". The presentation will summarize key lessons learned and introduce a preliminary logic model for situating system-wide changes in the broader multi-level governance context.

Speakers/Présentateur·trice·s

Rea Mishaxhiu

Crown-Indigenous Relations and Northern Affairs Canada

Rea is an Ottawa-based credentialed evaluator (CE) and an Associate Evaluation Officer with Crown-Indigenous Relations and Northern Affairs Canada. She recently completed a Master in Public Policy and Administration at Carleton University, where she conducted a case study on the homeless... Read More →

Wednesday June 15, 2022 08:30 - 10:00 CDT
Manitoba Maple

Manitoba Maple, Diversity within evaluation/Diversité au sein de l'évaluation

08:30 CDT

G99.4 (Ignite Presentation/Présentation éclair) Canine Spin on Qualitative vs Quantitative: Evaluation from a Different Perspective

This Ignite presentation will use two canine competitions to explore the intersection of qualitative and quantitative methods. The dog show conformation ring appears to be an example of an event which is judged through a qualitative lens, while agility (a timed course with obstacles, such as jumps and tunnels) is judged quantitatively. However, all is not as it seems! While not immediately obvious, both events have both quantitative and qualitative aspects. This presentation will use conformation and agility as an analogy for the overlapping of qualitative and quantitative approaches which appear in our diverse evaluation endeavours.

Speakers/Présentateur·trice·s

Linda Lee

Proactive Information Services Inc.

Wednesday June 15, 2022 08:30 - 10:00 CDT
Manitoba Maple

Manitoba Maple, Diversity within evaluation/Diversité au sein de l'évaluation

08:30 CDT

G99.5 (Ignite Presentation/Présentation éclair) Findings from a National Students and Emerging Evaluators Needs Assessment Survey

Between September and December 2021, the Canadian Evaluation Society distributed a needs assessment survey for students and emerging evaluators across Canada. Over 100 students and emerging evaluators across 10 provinces/territories shared feedback about their experience entering the Canadian evaluation field. This presentation shares findings from the needs assessment to highlight barriers and facilitators to entering the field of evaluation, as well as describes key challenges this population face in Canada today. In-line with the conference themes, this presentation knits together insight from individuals entering the field of evaluation to guide future support for emerging evaluators in Canada.

Speakers/Présentateur·trice·s

Cassandra Parsons

Evaluation Consultant, Ference & Company

Wednesday June 15, 2022 08:30 - 10:00 CDT
Manitoba Maple

Manitoba Maple, Diversity of Evaluators/Diversité d'évaluateur·trice·s

08:30 CDT

G99.6 (Ignite Presentation/Présentation éclair) Shooting Ourselves in the Foot

This year's conference theme is diversity. This is not surprising since the recognition of the value of diversity has finally come into the mainstream. But have you noticed that government RFPs in evaluation end up churning out the same old evaluators and same old teams? How can suppliers offer diversity in evaluation teams when we have to meet criteria that value years of experience (=older evaluators with pages of project experience), education (=evaluators with recognized Canadian degrees) and security clearance (=evaluators who have not recently lived outside of Canada)? This ignite presentation will explore these barriers and make a plea for diversity in contracting.

Speakers/Présentateur·trice·s

Sandy Moir

Goss Gilroy Inc.

Wednesday June 15, 2022 08:30 - 10:00 CDT
Manitoba Maple

Manitoba Maple, Diversity of Evaluators/Diversité d'évaluateur·trice·s

08:30 CDT

G21.1 (Short Presentation/Présentation courte) CJPE New Section: Roots and Relations -- A Gathering of Relatives

Come learn about CJPE's new permanent section in the journal called "Roots and Relations" (R&R). Co-founders and Associate Editors of R&R, Larry Bremner and Nicole Bowman will be there to share the purposes, processes, and hopes for this promising and innovative contribution to CJPE and the field of evaluation. CJPE Editors and R&R reviewers and advisors will also be there to visit, listen to ideas or areas of concern, and will be open to suggestions, ideas for development, and other promising strategies that the Indigenous communities would like to have considered for R&R. We also very much welcome and hold space for intersectional Indigenous contributors (e.g., LGBTQ2S, bi or multi-cultural) and participants to this session. We welcome community members, practitioners, and multi-disciplinary relatives from Indigenous communities. If you are interested in being a mentor, advisor, a reviewer, or being involved in another way for R&R, this is the session to attend.

Speakers/Présentateur·trice·s

Larry Bremner

Proactive Information Services Inc.

Nicole Bowman

President of BPC, Bowman Performance Consulting

Wednesday June 15, 2022 08:30 - 10:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

08:30 CDT

G21.2 (Short Presentation/Présentation courte) Virtual Focus Groups: Hasta La Vista Baby? ...I'll Be Back!

While the world has not ended as prophesized in the Terminator movies, the pandemic has resulted in the need for us humans to increase our technological capacity. One area of evident improvement is video conferencing, which has allowed evaluations to undertake focus groups virtually as needed. While it is expected that we will be able to gather safely again one day following the pandemic, but before Skynet goes online, there are still advantages to holding focus groups virtually. This presentation will highlight the pros and cons of in-person and virtual focus groups, as well as, discuss the differences in process and moderation between the two methods. "Come with me if you want to live," I mean, come with me on this journey to experience all there is to know about this diverse methodological option for evaluation now and going forward.

Speakers/Présentateur·trice·s

Paul Simpson

PRA Inc.

Wednesday June 15, 2022 08:30 - 10:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

08:30 CDT

G21.3 (Short Presentation/Présentation courte) Reconcili-ACTION at Ottawa Public Health: Amplifying the call to action through evaluation

As part of its commitment to reconciliation, Ottawa Public Health (OPH) pledged to design a culturally-responsive evaluation framework to monitor organizational progress and impacts of reconciliation on First Nations, Inuit and Metis Peoples in Ottawa. Informed by the collective feedback from First Nations consultants, Indigenous partners and public health employees, the framework was designed to respect and reflect Indigenous values, principles, priorities and practices. Elder Albert Marshall's (2004) Two-Eyed Seeing concept was employed in designing the framework, which includes mixed methods data collection connecting Indigenous and Western practices. This poster presentation showcases OPH's reconciliation journey, including the process and lessons learned as part of the development of the evaluation framework and how evaluation methods are enabling OPH to call others to the circle to advance reconciliation.

Speakers/Présentateur·trice·s

Kimberley Trotter

Knowledge Exchange Specialist, Ottawa Public Health

Kim Trotter is a settler Canadian residing on the traditional territory of the Algonquin Anishinaabeg People. As a seasoned nurse, Kim has had the privilege to work with, and learn from, First Nations, Inuit and Métis peoples in Quebec, Ontario and Manitoba. Kim is currently a Knowledge... Read More →

Katrina Dumont

Program Planning and Evaluation Officer, Ottawa Public Health

Katrina Dumont is a Program Planning and Evaluation Officer at Ottawa Public Health. Her current work focuses on supporting strategic planning efforts. Previously, Katrina worked at the Canadian Foundation for Healthcare Improvement where she was the Evaluation Lead for four programs... Read More →

Wednesday June 15, 2022 08:30 - 10:00 CDT
Sage

Sage, Diversity within evaluation/Diversité au sein de l'évaluation

08:30 CDT

G9.2 (Short Presentation/Présentation courte) Learning Through Doing: Evolution of an Evaluation Practicum Mid-Pandemic

The pandemic has presented unique challenges for new and emerging evaluators who wish to gain evaluation experience. In this presentation we reflect on a unique evaluation practicum case study through the perspective of an emerging evaluator and an evaluation practicum supervisor. This presentation describes the practicum, which included a combination of participation in online evaluation training, casework in teams, small consultations with organizations seeking evaluation assistance, and in person collaborative evaluation exercises. We offer the strengths and limitations of this innovative approach to evaluation training, identify the ways that this approach removed pandemic barriers to experiential learning and suggest lessons learned to those studying and teaching evaluation in pandemic times.

Speakers/Présentateur·trice·s

Breanna Baillie

Sarah Heath

Instructor III, University of Winnipeg

Wednesday June 15, 2022 08:30 - 10:00 CDT
Sweet Grass

Sweet Grass, Diversity within evaluation/Diversité au sein de l'évaluation

08:30 CDT

G9.3 (Storytelling/Récit) Mismatched Evaluator-Funder Values: Know When to Fold

The presenters were contracted to evaluate a foundation initiative to help rural communities develop inclusive community collaboratives focused on improving community well-being. The kick-off meeting for this project included a discussion and clarification of values and expectations. The evaluation began with the evaluation team conducting a baseline assessment of the current state of each community and its collaboration capacity against which to measure progress regarding inclusiveness and collaboration. The baseline also included gathering information to better understand the contextual factors that may impact collaborative development or functioning. The baseline assessment uncovered problematic practices and reported them to the funder. This presentation will tell the story of the assessment and some key findings, and the reaction to the findings that ultimately led the team to resign based on the mismatch in values and expectations in contrast to the initial conversations.

Speakers/Présentateur·trice·s

Ann Price

President, Community Evaluation Solutions, Inc

Susan Wolfe

CEO/Community Consultant, Susan Wolfe and Associates

Wednesday June 15, 2022 08:30 - 10:00 CDT
Sweet Grass

Sweet Grass, Diversity of Evaluators/Diversité d'évaluateur·trice·s

10:00 CDT

(Food/Alimentation) Break/Pause

Wednesday June 15, 2022 10:00 - 10:30 CDT
Conference Foyer/Hall du congrès

Conference Foyer/Hall du congrès

10:30 CDT

(Keynote/Conférence principale) CES Fellow Reflections / Réflexions de la part de membres titulaires de la SCÉ

Speakers/Présentateur·trice·s

Larry Bremner

Proactive Information Services Inc.

François Dumaine

Partner, PRA Inc.

Debbie DeLancey

D.J. DeLancey Consulting

Wednesday June 15, 2022 10:30 - 11:00 CDT
Tamarak

Tamarak

11:00 CDT

(Keynote/Conférence principale) Closing Keynote/Conférence de clôture

Close captioning and translation at https://live.syncwords.com/w-zGYdAw/
Captation et traduction à https://live.syncwords.com/w-zGYdAw/

Speakers/Présentateur·trice·s

Veronica Olazabal

Chief Impact & Evaluation Office; President, American Evaluation Association, The BHP Foundation

Veronica Olazabal is Chief Impact and Evaluation Officer at The BHP Foundation, President of the American Evaluation Association (AEA) and Adjunct Associate Professor at Columbia University’s School of International and Public Affairs. Her professional background range ~20 years... Read More →

Wednesday June 15, 2022 11:00 - 11:45 CDT
Tamarak

Tamarak

11:45 CDT

(Keynote/Conférence principale) Closing remarks and prayer / Remarques et prière de clôture

Speakers/Présentateur·trice·s

Tamarak