When editing a test, the “Set scoring info...” menu item on the Test menu lets you choose whether or not to save the results of each practice session in a log file. The results are saved the first time the person selects “Check answers...” from the Practice menu. This log file can be analyzed using Tester.
Saving the results of each practice session is very useful for teachers who use Tester in the classroom because they can then monitor which questions are especially easy or difficult. This in turn helps them hone the test questions and figure out what the class is having the most difficulty with.
When the option is on, a log file will be created the first time a person practices the test. The file’s name will be the test name with “.log” appended. After each subsequent practice session, the latest results will be appended to the file.
When you open a log file, the resulting window displays the number of practice sessions stored in the log file, the average number of correct answers per session, and, for reference, the minimum score required to pass.
The Analysis menu lets you analyze the log file and generate reports.
Student Score Histogram
The “Student score histogram...” item displays a histogram of the scores from all the practice sessions. The vertical gray line marks the minimum passing score.
To change the scales on an axis, double click on the scale. The resulting dialog window lets you adjust the appropriate values. For the horizontal scale, you can change the minimum, the maximum, and the step size. For the vertical scale, you can only change the maximum and the step size because the minimum is always zero. In both dialog windows, clicking on the “Defaults” button reverts both scales to their initial ranges.
You can print the histogram with the “Print...” item on the File menu. Regardless of the size of the window, the histogram will be printed to fill the page.
Question Response Histograms
The “Question response histograms...” item creates a text file displaying a histogram of the responses to each question. Here is an example of what it looks like:
Question #1
0: 2
1: 0
2: 8 *
3: 1
4: 2
Question #2
0: 0
1: 1
2: 4
3: 3
4: 7 *
The question number is the index number of the question in the test. This corresponds to the setting of the scroll bar when editing or browsing the question. The answer index is the index number of the answer when editing or browsing the question. The asterisk marks the correct answer. Answer index 0 counts the number of people who didn’t answer the question.
If there are actually more answers than are displayed in the file, it means that those extra answers were never selected.
Question Response Histograms can be used to find questions that are either too easy or too hard. This helps the teacher hone the question pool. There is, however, an easier way to search for questions that are too easy or too hard: let Tester do it with the next menu item.
Search for Poor Questions
The “Search for poor questions...” item creates a text file displaying the indices of the questions that match your definition of “too easy” or “too hard.” You specify the minimum percentage responses correct that constitutes “too easy” and the maximum percentage responses correct that constitutes “too hard.” The defaults are 90% and 10%, which means that a question will be deemed “too easy” if more than 90% of the responses were correct and will be deemed “too hard” if less than 10% of the responses were correct. Here is an example of what the report looks like:
Too easy:
3: 92%
Too hard (perhaps they should be discussed more carefully):
1: 8%
11: 0%
14: 6%
No reliable information:
12
For the questions that are deemed to easy or too hard, the actual percentage responses correct is displayed after the question index. The question index is the index number of the question in the test. This corresponds to the setting of the scroll bar when editing or browsing the question.
Some questions may never have been encountered during practice sessions. (This can occur if the questions are selected from a very large pool.) In this case, Tester has no information about how people answer these questions, so it notifies you by displaying their indices in the last section. This section will only be included if there are any such questions.
Detailed Responses
The “Detailed responses...” item creates a text file displaying each person’s response to each question. Here is an example of what it looks like:
Mon, May 30, 1994 11:51 AM
6 questions
2 correct
2 incorrect
1 unanswered
1 unanswerable
3: 3 / 3
4: 0 / 2
8: 0 / 0
15: 2 / 1
17: 1 / 1
22: 2 / 3
The date and time are included so that you can keep track of when people are using Tester. This can be especially useful if Tester is set up in an open-access computer lab.
The first few lines give you an overview of the session. These are the results that are displayed in the dialog window after checking the person’s answers. You can use these to decide whether the details of the session are interesting or not.
The question number is the index number of the question in the test. This corresponds to the setting of the scroll bar when editing or browsing the question. The questions are always listed in increasing index number order, and only the questions that were actually displayed during the practice session will be included.
The person’s answer is the index number of the answer that the person selected. The first answer in the answer list (during editing or browsing) has index number 1.
The correct answer is the index number of the correct answer. This is provided so that you can quickly scan through the list (which can get quite long) for questions that were answered incorrectly.
If the person did not answer the question or the question was not multiple-choice, the person’s answer will be zero. If the question was not multiple-choice, the correct answer will also be zero.
