Quantitative and qualitative initial validation of the accountable talk questionnaire

Introduction: Accountable Talk (AT) has been extensively studied as a tool for
improving argumentation and respectful discourse. While several research tools
exist for evaluating AT, as of yet there no self-report assessment tool to measure
AT directly, on a large-scale basis, and without significant costs. The aim of the
current study was to develop and provide initial validation for a self-report AT
questionnaire (ATQ).
Methods: One hundred students aged 11–12 years participated in the study.
Exploratory Factor Analysis, content validity, and test- retest reliability were
assessed. 50 students were also randomly assigned to the qualitative part of the
study. These students were recorded while discussing in small groups a topic
provided by the experimenter (animal research). The qualitative data was coded
and then correlated with the quantitative data obtained from the self-report
questionnaire.
Results: The results indicated that a 12-items questionnaire can reliably assess
three separate and independent qualities of AT: accountability to a learning
community, accountability to rigorous thinking, and accountability to accurate
knowledge. The reliability of the ATQ was high with a = 0.80. The Test-retest
reliability was assessed at two time points separated by a 3-week interval with
Pearson correlation. Excellent correlations (r >0.98) between the ATQscores were
found. The correlation coecients between the three components of the self-
reported ATQ and the observed data obtained from the group discussions were
significant positive medium-high.
Discussion: We have shown that a concise 12-item questionnaire can assess the
three main components of Accountable Talk within the framework of respectful
discourse. The questionnaire showed good reliability and structural validity,
with weak correlations between sub-topics supporting the distinction between
di􀀀erent aspects of Accountable Talk. We suggest that the ATQ can be used to
evaluate the e􀀀ectiveness of interventio