Incorporating Inquiry-Based Class Sessions with Computer Assisted Instruction
4. Results
The following preliminary results are available. Analysis of data gathered continues. The one-year delayed Post-Test data has yet to be gathered.
Result 1. As hypothesized (H1), there were no significant differences between treatments in terms of Final Scores (determining grades). See chart below. In particular, there were no significant differences between treatments in terms of the Test Sum (sum of the four test scores), the most rigorously determined part of the Final Score. Analysis of between-subjects effects for Group, Instructor, and Group*Instructor were not significant at p≤0.05 (Scheffe). (N=245: Group=80; Lecture=77; Quiz\Lecture=88. An additional 39 students did not take all tests, either dropping after the first few days, officially withdrawing, or quit attending.)
Result 2. In support of hypotheses H2 and H4, there were significant differences between the Group treatment and each of the other two treatments in terms of Pre-Test and Post-Test Scores. The Pre- and Post-Tests consisted of three problems, each scored on a rubric which awarded on each problem 0-1 point for Problem Identification, 0-1-2 points for Evidence of Problem-Solving, and 0-1-2 points for Explanation, for a total of 15 points possible on the test. Raters participated in three practice sessions of rating the same responses to problems, and subsequent discussions resolving differences, prior to grading the Pre- and Post-Tests (note one exception). Repeated Measures ANOVA (Wilks' Lambda), and univariate analysis of difference scores, each indicated significant differences at p<0.05. In particular, the Time main effect (Pre- to Post-) was significant (λ=0.50), as was the Time*Group interaction effect (λ=0.86). Moreover, the between-subjects effects for Group versus each of the other treatments was significant, while the between-subjects effect for Lecture versus Quiz\Lecture was not significant, at p<0.05 (Bonferroni). (N=214: Group=71; Lecture=68; Quiz\Lecture=75.) See chart below for comparison.
Result 3. Hypothesis H3 was not supported. The responses to the Survey of Mathematical Efficacy, given Pre- and Post- were subjected to factor analysis. Five factors were evident; 7 (out of 34) questions were excluded from the analysis as not meeting the threshold of significant contribution. All treatments showed significant improvement in mathematics self-efficacy, with no significant between-treatments effects overall, nor in any single factor. Apparently, succeeding at a mathematics course in and of itself improves self-efficacy and washes out any other effects. Further study is required.
Further Analysis. More detailed analysis is forthcoming on other interaction effects including Instructor and Section (time of day). We will also do further analysis on the Pre- and Post-Test subscores in order to tease apart Hypotheses H2 and H4. Inter-rater reliability on the Pre- and Post-Test scoring has yet to be verified. Preliminary analysis of comments from focus groups is expected to suggest a closer look at other statistics.
