All posts by Chi Yan Lam

About Chi Yan Lam

Chi Yan Lam, PhD(c), CE, is a program evaluator, researcher, and educator. He works closely with social innovators and public sector leaders across Canada to bring analysis and strategy to bear on program development, evaluation, and decision-making.

Key Journals on Evaluation

The following is a list of journals that focus on evaluation theory and its application in various contexts and settings. These titles are recommended for readers looking to develop a grounding in evaluation theory and methodology.

American Journal of Evaluation
Each issue of the American Journal of Evaluation (AJE) explores decisions and challenges related to conceptualizing, designing and conducting evaluations. Four times/year it offers original, peer-reviewed, articles about the methods, theory, ethics, politics, and practice of evaluation.

Canadian Journal of Program Evaluation
The Canadian Journal of Program Evaluation seeks to promote the theory and practice of program evaluation. To this end CJPE publishes full-length articles on all aspects of the theory and practice of evaluation and shorter Evaluation Practice Notes which share practical knowledge, experiences and lessons learned. A third section of the journal, Addressing Challenges in Evaluation Practice, presents real-life cases written by evaluation practitioners. The journal has a particular interest in articles reporting original empirical research on evaluation. CJPE is a completely bilingual journal, publishing in both English and French. Readership includes academics, practitioners and policymakers. CJPE attracts authors and readers internationally.

New Directions for Evaluation
New Directions for Evaluation, a quarterly sourcebook, is one of two official journals of the American Evaluation Association. The journal publishes works on all aspects of evaluation, with an emphasis on presenting timely and thoughtful reflections on leading-edge issues of evaluation and the organizational, cultural, and societal context within which evaluations occur. 

Evaluation
The journal Evaluation launched in 1995, publishes fully refereed papers and aims to advance the theory, methodology and practice of evaluation. We favour articles that bridge theory and practice whether through generalizable and exemplary cases or theoretical and methodological innovation

Journal of MultiDisciplinary Evaluation 
The Journal of MultiDisciplinary Evaluation is a free, online journal published by the Evaluation Center at Western Michigan University. The journal focuses on news and thinking of the profession and discipline of evaluation.

Evaluation and Program Planning
Evaluation and Program Planning is based on the principle that the techniques and methods of evaluation and planning transcend the boundaries of specific fields and that relevant contributions to these areas come from people representing many different positions, intellectual traditions, and interests. The primary goals of the journal are to assist evaluators and planners to improve the practice of their professions, to develop their skills, and to improve their knowledge base.

Practical Assessment, Research & Evaluation
Practical Assessment, Research & Evaluation is a peer-reviewed online journal whose purpose is to provide free access to articles that can have a positive impact on assessment, research, evaluation, and teaching practice.

PME 828 – Conducting Quality Program Evaluations

Program personnel plan and implement programs as a way to address the needs of intended users and their communities. These needs may be rooted in learning, recreation, behaviour, culture and health and well-being to name a few. Programs are often spearheaded by individual champions or working groups. Their focus is typically on program activities and the challenge of implementation. Rarely do they plan to learn, in any systematic way, about program design, about how the program is working or about how it could be improved following implementation.  This is an issue especially when data-informed decisions have to be made about whether programs should be supported, expanded, downsized or allowed to end.

The overall learning objectives for PME 828 are to:

  • Understand the history and purposes of educational evaluation 
  • Understand the factors and practices that contribute to a quality evaluation
  • Understand current models of evaluation and the needs they serve 
  • Make decisions that lead to a defensible design for systematic evaluative inquiry
  • Collect data about program processes, stakeholder  learning or performance and accurately interpret their meaning.Make appropriate judgments about the meaning of evaluation findings
  • Critique (using current literature and principles of evaluative inquiry examined in GDPI 802) the purposes, processes and methods proposed for evaluating a program
  • Take leadership in promoting effective evidence-informed program decision-making 
  • Report effectively to program or  organizational stakeholders what could be learned through evaluation activities 
  • Take leadership in promoting evidence-informed program and policy decision-making 
  • Mentor colleagues interested in learning more about their programs
  • Be able to argue the strengths and limitations of evidence-informed decision making 

Course is offered in:
Professional Master of Education, Faculty of Education, Queen’s University

Course authors: Drs. Lyn Shulha, Michelle Searle, Chi Yan Lam

Sessions Taught: Winter 2020, Winter 2019, Winter 2018, Fall 2017, Summer 2017

PME 827 – Planning and Implementing Effective Classroom Assessment

The assessment of learning and achievement is a global phenomenon. Over the last 2 decades there has been considerable research demonstrating how well conceived assessment can be a powerful force both in supporting learning, and as a mechanism for individual empowerment. Yet, no matter how clearly assessment policies are defined, how concisely strategies for implementing assessment are described, and how many sample instruments are available for adoption or adaptation, the planning and implementing of assessment in classrooms continues to be a complex “wicked problem”.

Students who complete this course will develop a clear sense of the multiple and overlapping purposes for assessment and be better positioned to reason, justify, and communicate their decision-making in assessment to students, their parents and administrators. In addition, they will become more confident in using assessment information to create summaries of student achievement.

The overall learning objectives for PME 827 are to:

  • To examine theories that underpin contemporary assessment practices.
  • To explore the complexity of planning and implementing classroom assessment.
  • To develop the competencies necessary to become a school leader in classroom assessment.
  • To examine issue of quality–validity, reliability, and fairness–in classroom assessment.

This course is offered in:
Professional Master of Education, Faculty of Education, Queen’s University

Course authors: Drs. Chi Yan Lam, and Stefan Merchant

Sessions Taught: Fall 2016, Summer 2017, Winter 2016

GDPI/PME 802 – Program Inquiry and Evaluation

This course will guide students in conducting systematic evaluative inquiry in support of data-informed program decision-making. Students will examine the multiple purposes of program evaluation, applying the principles, methods and logic inherent in the needs of targeted program personnel and decision makers.

The overall learning objectives for GDPI 802 are:

  • To develop an understanding of theoretical frameworks for inquiring into social programs, their organizational structures, learning activities, and outputs
  • To develop an understanding of how to inquire into social programs and develop program evaluation plans
  • To develop engage in professional dialogues about program inquiry and evaluation within a community of evaluation practice
  • To articulate complex understandings and response to dilemmas in program evaluation theory and practice
  • To apply understandings about program evaluation theory to a specific context of evaluation practice

Course Offered in:

Sessions Taught: Summer 2019, Winter 2019, Fall 2019, Summer 2018

Policy Evaluation: The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation

Suggested Citation: Lam, C. Y. (2018). Policy evaluation. In The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation (pp. 1258-1261)Thousand Oaks, SAGE.

“Policy evaluation refers to the systematic investigation and determination of value of a policy— and can take place in various sectors, including education. Policy evaluators apply evaluation methodologies and employ social scientific research methodologies to answer evaluation questions in support of policy making, policy development, and policy decision making. Policy evaluation is rarely undertaken for its own sake but mostly conducted in connection to the policy cycle. Because of the inherent political nature of policies, policy evaluation is increasingly undertaken by policy actors—both governmental and nongovernmental parties— with interests in shaping the political agenda. Hence, awareness into both the technical and political dimensions of policy evaluation is important to its understanding and execution.”

Consumer-Oriented Evaluation Approach: The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation

Suggested Citation: Lam, C. Y. (2018). Consumer-Oriented Evaluation Approach. In The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation (pp. 390-392). Thousand Oaks, SAGE. 

“The consumer-oriented approach to evaluation is the evaluation orientation advocated by evaluation expert and philosopher Michael Scriven. The approach stems from the belief that evaluation ought to serve the consumer, that is, the ultimate end user of the particular object under evaluation, the evaluand—be it a program, a curriculum, a policy, a product, or a service. This entry first discusses the history and the key aspects of the consumer-oriented evaluation approach, including the centrality of the consumer, the goal of the evaluation, and the role of the evaluation and the evaluator. It then looks at the techniques used in consumer- oriented evaluation, the checklist developed by Scriven for this evaluation approach, and the advantages and challenges of the approach.

The consumer-oriented evaluation approach arose in the 1960s in reaction to the then- prevailing stances that saw evaluation as an exercise in value-free measurement of whether program goals were achieved. The consumer-oriented evaluation approach reminds evaluators, and those who commission and use evaluation, that an evaluation ought to produce a determination about the merit, worth, and/or significance of the evaluand and that the basis of evaluation ought to be referenced to the needs of consumers.”