The failure of a decision support system: Inconsistency in test grading by teachers

This paper examines the interaction between the contribution of established criteria to objective assessment of examinations, and the effect of mental fatigue induced during the process. Copies of 31 compositions, previously awarded a grade of 80 percent by specialists, were stacked in three randomly arranged sequences. Sixty teachers graded the compositions in sequential order. Thereafter, they re-evaluated every essay according to each of six criteria. These scores were used to generate a series of accumulative grades. In both scoring procedures, the grades rose in a time dependent fashion, indicating impairment of judgment, apparently as a result of mental fatigue. Differences between the two assessments suggest that the criteria were not used efficiently in the original evaluation of the teachers. This may be
explained by the phenomenon of bounded rationality.