Surveys of Learning Reveal Strengths, Weaknesses
May 10, 2010
Over the past two decades, large-scale surveys have generated a wealth of generalizable evidence on instruction. But until recently, this vast body of research has not been summarized systematically.
UW–Madison education professor Eric Camburn and Sociology graduate student Seong Won Han set out to remedy that. They reviewed and analyzed evidence on instruction from all surveys conducted between 1987 and 2005 that used nationally representative samples to measure classroom instruction.
Camburn and Han reviewed 145 studies that used data from 19 national surveys to investigate instruction. They found that
More than half the studies used data at least a decade old;
Few studies examined instruction during important transition years such as 6th and 9th grade; and
Subject-area emphasis was lopsided (mathematics and science instruction received much greater attention than English/language arts and social studies).
The review also confirmed that students of low socioeconomic status (SES) received diminished learning opportunities compared to their more affluent peers.
Camburn identified positive relationships between six dimensions of instruction and student achievement (see sidebar: “Dimensions of Instruction”). The dimension of instruction with the greatest number of positive results was interactions for learning. That refers to the ways students tap informational, human, or material resources to support their learning.
The review also showed several positive relationships between focus of learning (content coverage) and student achievement. Among the nine studies reporting a significant relationship, five examined the amount of content covered, and four examined the coverage of specific topics or the integration of topics.
Type of cognitive activity was shown to be positively associated with achievement in seven studies.
Five studies documented a positive relationship betweentime on learning and achievement, while one study found a negative relationship.
Finally, five studies found a positive correlation between achievement and control of learning (i.e., whether the pace of instruction is controlled by students or the teacher).
Level of rigor varies
Camburn also evaluated the studies’ rigor, using two indicators: analytic approach and publication venue. Regarding the former, 49 studies used inferential statistics to investigate some aspect of instruction. (Inferential statistics are preferable to descriptive analytic techniques because they take sampling error into account and are better suited to understanding complex relationships among multiple variables.)
With regard to publication venue, only 26% of the studies were published in refereed journals. A majority (59%) of the studies were reports that tend to not be subjected to the peer-review process.
Unequal opportunity to learn
The review showed that learning opportunities were unevenly distributed among low- and high-income students. Lower SES students received less time on instruction overall and were exposed to a smaller proportion of mathematics textbooks. These students also were less likely (a) to be exposed to authentic instruction, (b) to engage in meta-cognition and problem solving, and (c) to read trade books.
Content coverage lopsided
Mathematics instruction received the greatest attention (37% of the studies reviewed). Science instruction was the focus of 17% of the studies, and science and math together accounted for 66% of all studies. English/language arts accounted for 19% of the studies, and social studies instruction, only 5%.
Implications for research on instruction
Based on this analysis, Camburn calls for more research on instruction at key transition points, particularly Grades 5, 6, and 9. More research is also needed on high school instruction, particularly Grades 9–11. And more research is needed on instruction in English/language arts and social studies.
Dimensions of Instruction
- Interactions for learning (86% of all studies examined): The ways students interact with each other and with teachers, or with instructional tools and artifacts (e.g., computers, manipulatives, textbooks, written assignments).
- Time on learning (61%): The amount of time students and teachers spend on learning.
- Focus of learning (55%): The academic content of instruction (e.g., subject areas and topics).
- Grouping for learning (41%): Whole class instruction, ability grouping, and student collaboration.
- Cognitive activity (41%): The kinds of cognitive activity demanded of students during instruction (e.g., memorization, application of specific skills, understanding of relationships).
- Control of learning (33%): The degree to which instruction is controlled by students or teachers.
- Support for learning (6%): Cognitive and emotional support.
Camburn also encourages fellow education researchers to engage in more scientifically rigorous inquiries when they use large-scale survey data to study instruction.
See Camburn, E.M., and Han, S.W. (in press). Two decades of generalizable evidence on U.S. instruction from National surveys. Teachers College Record.
