A young program evaluator was casted into the deep waters of innovation and discovered the potentials of developmental evaluation and design.
One afternoon during the winter term of my first year in graduate school, my advisor asked if I had a second to spare. Standing in a darkened, narrow corridor, in one of those serendipitous moments, she asked if I could help her re-think the undergraduate training of preservice teachers. For five years, she and a colleague had been working on developing a course in classroom assessment that was mandatory for all 700 students enrolled each year. Despite their efforts, the a different quality of learning was desired. Their instruction was constrained by limited financial resources, restrictive lecture-style instruction, and a meagre allotment of 7 hours for instruction. They had wanted to experiment with alternative ways of structuring learning in classroom assessment. I was intrigued by the problem and the prospects of making a small contribution to a practical problem. I agreed. As was typical of conversations with my advisor, I left with a sense of intellectual bewilderment and stimulation.
My curiosity in using evaluation as a vehicle for problem-solving and social change thus began. What I had originally thought to be a simple project involving some literature review, analysis into our particular program context, and maybe some coaching sessions, blossomed into a full-fledged developmental evaluation project. A few months later from that first interaction I found myself helping to pilot a blended learning mini-course that saw the integration of microblogging to connect teacher candidates, who were by then on field placement, with their peers and with faculty mentors. Just when they were grappling with doing assessment in their respective classroom, the instructional team had the opportunity to guide and inject new thinking about assessment. Assessment suddenly sprung to life became a practical and situated professional practice.
What was remarkable about that project was the role that evaluation–specifically, developmental evaluation–played. Developmental evaluation not only provided a means towards anchoring our decision-making through ‘best-available data’, it also helped us to continually develop the program through incremental learning, itself a developmental process. It also occurred to us that incorporating some sort of evaluation exercise as a way of reality-testing was the prudent and responsible way to go about piloting an unknown, untested way of teaching; DE offered that.
Emerging from this substantial experience myself, I subsequently went back, analyzed the project through a researcher lens, and wrote it up in the form of a case study. This project proved not only intellectually gratifying but also viscerally draining. During this project, I experienced anxiety. I experienced emotions like feeling lost and feeling stuck. In the write-up, I explained that these emotions resulted from the uncertainty associated with innovating. Often we see and talk about evaluation as if it’s a systematic, clinical procedure, and far too often we neglect the emotional component of leading and participating in evaluations.
In trying to unpack the evaluation for the case study, I contended with the notion that what I had participated in was indeed not an evaluation. Indeed, the project (the evaluation) and the evaluand (the program) lacked many of the hallmarks of a program evaluation: clear, specific, and measurable goals; an operational program; or program participants. But a closer look at some of the processes and activities told a different story. What had transpired wasindeed an evaluative exercise and evaluative thinking played a prominent part in moving the project forward.
But there was something more…
the way we systematically explored options and made decisions about what might had been appropriate for use in our particular context… the aim to ‘do different’… and the permission to be vision-driven and participant-driven…oh, and the way in which we went from ‘nothing to something’…. how we embraced uncertainty, how I led and coached in service to my clients and participants… and creating the space to think creatively and innovatively…
… and that something was design.
Dr. Chi Yan Lam 