(still penning this..)
When we determined what we thought should be the
standard for public school (vs the state standards etc) - it took us a while to come up with a good word for it.
Assessment seemed too fixed.
Merit was a possibility.
(can't remember) seemed too negative.
For some reason.. validity resonated well.
Students wanted to be validated for what they were doing..
Teachers seek to be validated for their efforts and talents.
Little did we know... how well the term would continue to resonate.
Today - I've been reading Roger Martin's
graciously given to me by @polarunlimited
Martin distinguishes between reliability and validity - even saying the distinction is at the heart of the innovation dilemma. (p. 37)
Martin says (p. 37) that:
the goal of reliability is to produce consistent, predictable outcomes... and that it is achieved by narrowing the scope of the test to what can be measured in a replicable, quantitative way and by eliminating as much subjectivity, judgment, and bias as possible.
the goal of validity is to produce outcomes that meet a desired objective. it produces a result that is shown, through the passage of time, to be correct.... and that it is difficult to achieve with only quantitative measures, because those measures strip away nuance and context.
He goes on to say how reliability rules... in business... but he adds ...
p. 42:
Companies that devote all their resources to reliability lack the tools to pursue outcomes that are valid, that is , that produce a desired result. ....little wonder that those same organizations don't know how to manage validity-seeking activities to generate lasting business value.
whoa.. we called our beta tool - seeking validation.
p. 43:
What organizations dedicated to running reliable algorithms often fail to realize is that while they reduce the risk of small variations in their businesses, they increase the risk of cataclysmic events that occur when the future no longer resembles the past and the algorithm is no longer relevant or useful.
p. 45:
Because it is so well suited to satisfying the organization demand for proof, reliability almost always trumps validity. But it is all too often a hollow victory. When the future takes a different course than the path the data predicted for it, all the proofs in the world are unavailing.
he marks two groups
| analytical thinking (proof) | intuitive thinking (gut) |
| exploitation (residing in, honing existing knowledge) | exploration (moving to different level of knowledge) |
| reliability | validity |
| administration | invention |
| management | innovation |
| mastery | originality |
| art of knowing w/o reasoning |
Martin says that what is needed is a balance of the two, he terms this design thinking. He says at the heart of design thinking is abductive logic (p. 25) where steps can't be proved only validated with time. He says that it is imperative that people know it is safe and rewarding to bring forward an abductive argument. (p.28)
ie: Steve Jobs role wasn't the one to realize the innovative products, his role: he created an organization that placed "insanely great" design at the top of its hierarchy of values, and he gave the green light to spend the resoures necessary to make lasting successes of his designers' innovations.
Martin calls a design thinker - a first class noticer.
He says that reliability alone is the chief limiter of success
but that validity - which at first seems to be the enemy of reliability, is the force that, when paired with reliability, creates a winning advantage. (p. 31)
p. 56 via Mihnea Moldoveanu:
The validity seeker, unlike the reliability seeker, treats past predictive successes as hypotheses to be carefully tested before using them to getnerate predictions that are expted to be valid. Hence, the real empiricist is "a first-rate noticer" of precisely the anomalies that would cause him or her to throw out the "all things are equal" assumption. ________________________________________________