Categories
Classroom Assessment

Three key ideas on assessing student learning from week one of #PME827

close up of girl writingPhoto by Pixabay on Pexels.com

Our learning in classroom assessment is off to a great start. You, too, can follow along here. For the first week, we examined the fundamental role of assessment in instruction. In doing so, I was inspired to create the following graphics to capture three ideas arising from that study.

  1. The role of assessment in instruction.

In teaching as in surgery, the trained professional engages in purposeful acts. A teacher teaches and a surgeon, well, performs surgery. Performance of those actions are rarely an end, but a deliberate means to achieving some purposeful ends. Accordingly, the professional is intrinsically motivated to find out if their efforts have amounted to some intended effects—i.e., the teacher wants to know if their student has understood what was taught to them, and the surgeon wants to know if their repair has improved function and wellbeing. So, in contexts of teaching and learning, assessment is the means by which we arrive at that understanding. So, put differently, assessment is the systematic collection of information about student learning so we may answer questions of educational importance.

2. Assessment as Inquiry

When we assess, we are conducting an inquiry. We try to make visible what students have learned via the use of some instruments (e.g., a test, a portfolio). This results in information about what students have learned. We can then compare that information against what we have predetermined to be meritorious demonstration of the assessed performance (referencing to curriculum standards, other test-takers, or to prior performance). In doing so, we arrive at some claim about the learner’s performance (e.g., pass/fail; admission/not; promotion to the next grade/retention).

3. Assessment is purpose-driven.

The purpose for assessing learning determines other assessment decisions. These decisions all ‘hang together’.

Categories
Classroom Assessment

Come learn about Classroom Assessment with my #PME827 students!

I’ve often said that the ability to implement effective classroom assessment is a secret superpower that all effective teachers share.

Starting on Monday, July 6, 2020, my students and I will begin a seven-week course of study on effective classroom assessment: PME 827–Planning and Implementing Effective Classroom Assessment. It’s an introductory, graduate-level course which surveys the literature on classroom assessment.

Join us as we take a deep dive into the literature and explore key concepts and practices associated with effective classroom assessment. I’ll be taking our exploration to Twitter, so follow along by using the hashtag #PME827. Each week, I’ll share with you one key idea that we’d be studying. I’ll also be tweeting out links to key articles you should know about!

If you have questions, don’t be afraid to tweet me @chiyanlam!

Categories
Aging and Health Teaching

AGHE 800 – Evaluating Aging-Related Programs and Services

To enrol in this course, please visit: https://www.rehab.queensu.ca/academic-programs/aghe

This course introduces learners to evaluation principles and practice as applied to programs that address social, physical and economic determinants of wellness and participation for older adults. Learners will acquire skills necessary to identify and apply program evaluation methods to inform ongoing program development. Topics will include theoretical aspects of program evaluation, as well as strategies for program development, monitoring and change with a focus on participatory approaches. Current debates in the field will be discussed, with particular attention to issues underlying research and evaluation with older adult populations in community and institutional contexts. Attention will be given to knowledge mobilization strategies that foster inclusion, empowerment and innovation. 

After completing this course, students are expected to be able to demonstrate: 

  1. Skills in articulating and critiquing various approaches to program evaluation and development. 
  2. The ability to describe programs through depiction of the underlying program logic. 
  3. The ability to critique existing evaluation plans and reports in the context of aging-related programs and services. 
  4. The ability to prepare a plan for the evaluation of age-related programs and services. 
  5. Skills in applying evaluation knowledge to decision making. 
  6. Knowledge and understanding of strategies for reporting evaluation results to program administrators and other stakeholders for maximum utilization. 

Course is offered in:
Master of Science in Aging and Health, Faculty of Health Sciences, Queen’s University

Sessions taught: Winter 2020, Winter 2019

Categories
Classroom Assessment Teaching

Key Journals on Assessment

The following is a list of journals which focuses on classroom assessment and assessment in general. Descriptions presented describe the aim and scope of the respective journals. These titles are recommended for readers looking to develop a grounding in assessment theory and practice.

Practical Assessment, Research & Evaluation
Practical Assessment, Research & Evaluation (PARE) is an on-line journal providing access to refereed articles that can have a positive impact on assessment, research, and evaluation. Manuscripts published in PARE are scholarly syntheses of research and ideas about methodological issues and practices. PARE is intended to help members of the community keep up-to-date with effective methods, trends, and research developments from a variety of settings. PARE is committed to highlighting research that focuses on equity in assessment and measurement. To that end, we encourage the submission of manuscripts that address issues of equity and fairness – particularly articles that focuses on the assessment experiences and outcomes for historically marginalized populations.

Assessment Matters
Assessment Matters is a refereed, international journal that pushes the thinking in assessment in education research, policy and practice.

Assessment in Education: Principles, Policy & Practice
Recent decades have witnessed significant developments in the field of educational assessment. New approaches to the assessment of student achievement have been complemented by the increasing prominence of educational assessment as a policy issue. In particular, there has been a growth of interest in modes of assessment that promote, as well as measure, standards and quality. These have profound implications for individual learners, institutions and the educational system itself.
Assessment in Education provides a focus for scholarly output in the field of assessment. The journal is explicitly international in focus and encourages contributions from a wide range of assessment systems and cultures. The journal’s intention is to explore both commonalities and differences in policy and practice. 

Educational Assessment, Evaluation and Accountability
The main objective of this international journal is to advance knowledge and dissemination of research on and about assessment, evaluation and accountability of all kinds and on various levels as well as in all fields of education. The journal provides readers with an understanding of the rich contextual nature of evaluation, assessment and accountability in education. The journal is theory-oriented and methodology-based and seeks to connect research, policy making and practice. 

Studies in Educational Evaluation
Studies in Educational Evaluation publishes original reports of evaluation studies. 
Four types of articles are published by the journal: 
(a) Empirical evaluation studies representing evaluation practice in educational systems around the world; 
(b) Theoretical reflections and empirical studies related to issues involved in the evaluation of educational programs, educational institutions, educational personnel and student assessment (emphasis added); 
(c) Articles summarizing the state-of-the-art concerning specific topics in evaluation in general or in a particular country or group of countries; 
(d) Book reviews and brief abstracts of evaluation studies.

Canadian Journal of Educational Administration and Policy
The Canadian Journal for Educational Administration and Policy (CJEAP) is devoted to scholarly and critical works in the field of educational administration. Thearticles that it accepts for review must address administrative or policy issues relevant to education and educational systems in Canada. Comparative articles as well as expositions on educational administration in relation to sociology, philosophy, history, and theory are welcomed. It is a forum for articles addressing the administration of schools and higher education institutions, Indigenous educational contexts, and the education of adult learners. CJEAP accepts contributions from scholars, policy makers, graduate students, practitioners, and researchers in education and related fields.

Educational Assessment

Educational Assessment publishes original research on the design, analysis and use of assessment to deepen understanding of the performance and quality of individuals, groups, and programs in educational settings. It welcomes rigorous theoretical pieces and empirical studies (quantitative or qualitative) that can inform important national and international discussions in educational policy and practice.

Acknowledgement: Thank you to Dr. Man-Wai Chu for her additions to this list.

Categories
Social Innovation

Key Journals on Evaluation

The following is a list of journals that focus on evaluation theory and its application in various contexts and settings. These titles are recommended for readers looking to develop a grounding in evaluation theory and methodology.

American Journal of Evaluation
Each issue of the American Journal of Evaluation (AJE) explores decisions and challenges related to conceptualizing, designing and conducting evaluations. Four times/year it offers original, peer-reviewed, articles about the methods, theory, ethics, politics, and practice of evaluation.

Canadian Journal of Program Evaluation
The Canadian Journal of Program Evaluation seeks to promote the theory and practice of program evaluation. To this end CJPE publishes full-length articles on all aspects of the theory and practice of evaluation and shorter Evaluation Practice Notes which share practical knowledge, experiences and lessons learned. A third section of the journal, Addressing Challenges in Evaluation Practice, presents real-life cases written by evaluation practitioners. The journal has a particular interest in articles reporting original empirical research on evaluation. CJPE is a completely bilingual journal, publishing in both English and French. Readership includes academics, practitioners and policymakers. CJPE attracts authors and readers internationally.

New Directions for Evaluation
New Directions for Evaluation, a quarterly sourcebook, is one of two official journals of the American Evaluation Association. The journal publishes works on all aspects of evaluation, with an emphasis on presenting timely and thoughtful reflections on leading-edge issues of evaluation and the organizational, cultural, and societal context within which evaluations occur. 

Evaluation
The journal Evaluation launched in 1995, publishes fully refereed papers and aims to advance the theory, methodology and practice of evaluation. We favour articles that bridge theory and practice whether through generalizable and exemplary cases or theoretical and methodological innovation

Journal of MultiDisciplinary Evaluation 
The Journal of MultiDisciplinary Evaluation is a free, online journal published by the Evaluation Center at Western Michigan University. The journal focuses on news and thinking of the profession and discipline of evaluation.

Evaluation and Program Planning
Evaluation and Program Planning is based on the principle that the techniques and methods of evaluation and planning transcend the boundaries of specific fields and that relevant contributions to these areas come from people representing many different positions, intellectual traditions, and interests. The primary goals of the journal are to assist evaluators and planners to improve the practice of their professions, to develop their skills, and to improve their knowledge base.

Practical Assessment, Research & Evaluation
Practical Assessment, Research & Evaluation is a peer-reviewed online journal whose purpose is to provide free access to articles that can have a positive impact on assessment, research, evaluation, and teaching practice.

Categories
Teaching

PME 828 – Conducting Quality Program Evaluations

Program personnel plan and implement programs as a way to address the needs of intended users and their communities. These needs may be rooted in learning, recreation, behaviour, culture and health and well-being to name a few. Programs are often spearheaded by individual champions or working groups. Their focus is typically on program activities and the challenge of implementation. Rarely do they plan to learn, in any systematic way, about program design, about how the program is working or about how it could be improved following implementation.  This is an issue especially when data-informed decisions have to be made about whether programs should be supported, expanded, downsized or allowed to end.

The overall learning objectives for PME 828 are to:

  • Understand the history and purposes of educational evaluation 
  • Understand the factors and practices that contribute to a quality evaluation
  • Understand current models of evaluation and the needs they serve 
  • Make decisions that lead to a defensible design for systematic evaluative inquiry
  • Collect data about program processes, stakeholder  learning or performance and accurately interpret their meaning.Make appropriate judgments about the meaning of evaluation findings
  • Critique (using current literature and principles of evaluative inquiry examined in GDPI 802) the purposes, processes and methods proposed for evaluating a program
  • Take leadership in promoting effective evidence-informed program decision-making 
  • Report effectively to program or  organizational stakeholders what could be learned through evaluation activities 
  • Take leadership in promoting evidence-informed program and policy decision-making 
  • Mentor colleagues interested in learning more about their programs
  • Be able to argue the strengths and limitations of evidence-informed decision making 

Course is offered in:
Professional Master of Education, Faculty of Education, Queen’s University

Course authors: Drs. Lyn Shulha, Michelle Searle, Chi Yan Lam

Sessions Taught: Winter 2020, Winter 2019, Winter 2018, Fall 2017, Summer 2017