Conative Bias in the SAT?
Education Testing Service, the company that publishes the SAT exam had asked me to do a presentation for its test development team. They said they’d been trying to develop a test based on conation for many years and had not figured out how to do it. The guy who invited me explained that they hoped I would share with their professional team how I had created the Kolbe Index.
My favorite way of demonstrating the predictive validity of the Kolbe Index is to do Glop Shop, a three-minute activity during which three people I’ve never met create a hands-on solution for a problem with no right answer. I provide a challenge, a bag of assorted junk they must use to achieve the goal, and when they are out of the room, give predictions for their actions, reactions and interactions during the exercise.
I selected the top SAT developer to participate because of his surprising MO: Initiating Follow Thru and Implementor, CounterAct Fact Finder and Quick Start. Less than a minute into the activity he showed a strong bent toward the opposite MO.
Totally trusting my instincts, I stopped the activity (only time I have ever done that), and asked him:
“Why did you cheat when you took the Kolbe Index?”
Caught by surprise he blurted out:
“Because I wanted to see what would happen if I answered in the most socially undesirable way possible.”
I asked why he had such a strong bias against a perfectly wonderful conative strength – and wondered aloud what impact that might have on his work on the SAT.
Only a few people in the room seemed incensed by his lack of truthfulness in filling out the Kolbe Index questions and demographic information (he said he was a female, for instance). Aghast that one test developer would be so disrespectful of another’s data, I no longer felt comfortable sharing my work with those in the room. I ended my presentation.
Studies we then did, using data provided by universities, showed there is a difference in SAT scores by modes of conative initiation on the Kolbe Index. People who initiate in Follow Thru and Implementor did not receive as high scores on the SAT as people who initiate in Fact Finder and Quick Start. Since there is no correlation between IQ and Kolbe Index results, my instincts tell me that the SAT has a conative bias. The very one expressed by its then lead developer.