I’ve written a good number of tests in my years. In my years of management and training development, I have also had the opportunity to see how others approach assessment. While I would not say I write the best tests, I always thought I did a pretty good job producing balanced and representative assessment tools. I even took a research methodology course in University that taught the art of writing unbiased questions–skills I’ve held close throughout the years.
And then I discovered this thing called “Table of Specifications” (http://www.slideshare.net/davejaymanriquez/table-of-specifications-assesment-of-learning).
This changed my way of writing testing instruments.
By using a table of specification you can ensure you are making a test that is representative of the instructional time spent on outcomes.
I used this to create a test on a topic for which I have previously created tests. Immediately I could see that the number and variety of questions I was choosing did not align with the instructional time spent on some outcomes. This was particularly true for easier learning items, such as terminology. Often times, I would skip over testing these items as I considered them necessary for more advanced items, but too easy to be tested directly. To accomplish a similar end, I instead altered the value of question types. This meant I could test for easier items, like terminology, but these items would be worth only 1 mark each. A more complicated problem solving question would then be worth 5 or more marks.
While I am happy to have discovered this method, I can see some potential limitations in our technology driven world. I try to do almost everything paperless, so I generally leverage our Learning Management Systems testing features. While there are features for building questions pools (which is way easier than if you were doing this by paper), other items, such as question types or changing question value may be more difficult. For an example, my LMS does not support short answer or essay type questions.
In the over all scheme of curriculum development, I do think the table of specification has a special place. While creating course outcomes, lesson plans, and assessment it is important to keep in mind that the instructional time devoted to outcomes aligns with the value–both in assessment and goal setting. This is definitely a tool that is staying in my repertoire.