The Brigham Young University (BYU) statistics department is performing experiments to compare teaching methods. Response variables include students’ final-exam scores and a measure of their attitude toward statistics. One study compares two levels of technology for large lectures: standard (overhead projectors and chalk) and multimedia. There are 8 lecture sections of a basic statistics course at BYU, each with about 200 students. There are four instructors, each of whom teaches two sections.
Suppose the sections and lecturers are as follows: (a) Suppose we randomly assign two lecturers to use standard technology in their sections and the other two lecturers to use multimedia technology. Explain how this could lead to confounding. (b) Describe a better design for this experiment.
The Correct Answer and Explanation is:
(a) Confounding in the Given Design
If we randomly assign two lecturers to use standard technology in their sections and the other two lecturers to use multimedia technology, this creates a confounding between teaching method and instructor. In this setup, it’s impossible to distinguish whether differences in student outcomes are due to the teaching method or the instructor’s effectiveness. For example, if students taught by the instructors using multimedia perform better, we cannot tell whether this is due to the multimedia tools or simply because those instructors are more engaging, experienced, or effective regardless of technology. Since each instructor only uses one method, the effects of instructor and technology are mixed, making causal conclusions invalid.
(b) Better Experimental Design
A better design is to use a randomized block design where each of the four instructors teaches one section with standard technology and one with multimedia technology. This way, each instructor serves as their own control, and differences in teaching style or experience are accounted for. The comparison of teaching methods then occurs within each instructor’s sections, reducing variability due to instructor differences and eliminating confounding.
In this improved design, the eight sections (two per instructor) are randomly assigned technology methods, ensuring balanced treatment assignment across instructors. This structure isolates the effect of technology on student outcomes because each instructor’s performance with both methods can be directly compared. Additionally, if student characteristics are similar across sections (since each has about 200 students and belongs to the same course), random assignment ensures comparability across groups.
Summary
Confounding occurs when the effects of two variables—here, teaching method and instructor—cannot be separated. In the original design, assigning each instructor to only one teaching method leads to such confounding, as it becomes impossible to tell whether differences in student performance are due to the method or the instructor. A more effective design uses a randomized block approach, where each instructor teaches one section with each method. This balances instructional influence across both treatment types, allowing clearer assessment of the technology’s impact.
