I recently received an e-mail from a fellow doctoral student asking me to explain Scriven’s notion of merit/worth/significance. One part of her dissertation is around determining the value of test preparation training (e.g. MCAT/GMAT/LSAT prep courses) among language learners. One of her committee members suggested that she use M/W/S as a framework for tackling this aspect of her work. So, I wrote back to her, saying, why don’t we Skype and talk about this.
I’ve been thinking about this problem since. As an evaluator, I am reminded that one of the basic purposes in evaluation is the determination of merit/worth/significance of something. And, we typically refer to whatever we are evaluating (the ‘something’) as the evaluand. This classical definition of evaluation constitutes a part of what Scriven (1991) refers to as the logic of evaluation in a paper by the same name in the Evaluation Thesaurus. The logic of evaluation is a seminal contribution to the field as it gets at the core of what makes evaluation unique as compared to, say, research–evaluation allows us to make evaluative claims. The distinction between M/W/S and its application in evaluation is an important one, but finding accessible writing on this topic is difficult. Perhaps, m/w/s is so obvious to everyone else but me :). Hopefully not.
So… what’s merit, worth, and significance?
Merit, worth, and significance can be easily explained by reference to evaluating anapple. Say you’re at a grocery store. The decision you’ll have to make is to buy an apple.
Merit
Merit has to do with the intrinsic properties, characteristics, or attributes of an evaluand. When buying an apple, most people would prefer an apple that is not rotten, is sweet to taste, and is not otherwise damaged or deformed. That’s typically what people would look for if the apple were to be eaten on its own. But, what if you were buying the apple to make an apple pie? Then, you may wish to buy an apple that is not sweet but tart. So, as we can see, what we value to be desirable attributes of an object depends on other contextual factors.
Here is another example. A car has merit if it is reliable (i.e. does not break down while you’re driving down the highway; predictable), is safe (i.e. has adequate safety features and operates as intended), and is powerful relative to its intended application (i.e. say, a commuter car vs a pick-up truck to haul construction material). Now, you may say, a car has merit only if it has an integrated air conditioning unit or a stereo system. A design-conscious person may insist that a car be visually appealing. Increasingly, drivers want good fuel consumption. Different people may hold different views of what constitutes merit. In other words, an evaluand may be evaluated against different dimensions of quality, i.e. criteria. Part of what makes evaluation fun is surfacing the criteria that one might use to evaluate an evaluand. What’s ‘good’ to you is not necessarily ‘good’ to me. That’s why there are so many kinds of cars out there.
In a program evaluation, we typically think of a program as having merit if: 1) it does what it sets out to do, i.e. achieves its intended outcomes, and that 2) it makes a meaningful difference as a consequence to its operation.
Worth
Now, worth is a trickier concept. In everyday parlance, we might say that an apple (assuming that is ‘good’) is worth something; that ‘something’ is typically expressed in some monetary value (e.g. this apple is worth $2.00; that car is worth $24,999.) So, worth is the value of an evaluand that is expressed as an equivalence to something else. We may say… that this activity is worth ‘my time’. Whereas merit can be difficult to measure, worth is usually expressed in some more easily measurable unit.
Another way to think about worth is in a comparative situation. Let say you’re evaluating two instances of the same program: Program Breakfast-for-all at Site A and Site B. While they may both have merits, the worth of the program at Site A may be different from Site B depending on its impact on the constituents. Worth between two comparable, but different programs may also differ if one is cheaper to run (so one is worth more than the other).
Finally, significance.
Significance is the fuzziest of the three. Significance refers to the values and meanings that one ascribe to an evaluand. Typically, one can learn about the significance of something by asking questions about: What makes this evaluand special? What meaning does it hold for particular individuals?
Ask any young bride about her diamond ring. While it may not feature a big diamond (so, the ring is of limited worth), it probably holds great significance. A young college graduate may be driving a high-mileage car that is nearing the end of its service life. We might speculate that the car has limited merit (i.e. the transmission is wonky, the body is rusting, but the car is still roadworthy), and as a result is of limited worth to any body, but to the college graduate it may hold significance for his/her livelihood depends on it to get him to work everyday.
Notice that significance often have little to do with merit. Indeed, a program may be shown to have limited impact on a community, but it may hold great significance for its symbolic value. We may say that “it matters! Even if it is to a few.” As another example, a program may be shown to be inefficacious, but if it is the only program of its kind that serves a specific need for a vulnerable population, that’s significance to know, isn’t it?
So what?
Knowing m/w/s well enables us not only to unpack what others mean by ‘good’, but it also helps in raising questions around understanding quality, say, when designing an interview guide or constructing survey questions.
Question for you: Is this how you understand merit/worth/significance? Might you have other powerful ways of explaining m/w/s to others? Comment below. Thanks for reading!
PS: For all you educators out there, is a grade an indication of merit, worth, or significance, or any/all of three?
Dr. Chi Yan Lam 