The analysis of teaching from video taped lessons is difficult, and compounded if cross-cultural comparisons are conducted. A reliable coding system must be deployed so both common and unique cultural features of teaching are adequately captured.

In the first TIMSS 1995 Video Study the coding strategy involved a long and difficult building of a "bottom-up" coding system that could be applied to lesson tapes from the three countries in the study: Germany, Japan, and the United States. A team of code developers, with representatives from each country, wrote definitions for codes that communicated what "counts as," for example, an open-ended question, seatwork activity, use of instructional aids (e.g. audio-visual equipment), and many other individual features that described what they saw on the tapes. They refined and produced a reliable coding system in which two independent coders could make the same decision at least 80% of the time. The codes provided the team with quantitative indicators of how often specific teaching features appeared on each tape.

Although the coding system captured many of the individual features of interest, individual features do not tell the whole story. Stigler and Hiebert note, in their retrospective account of the video study, that "What is important is how the features fit together to form a whole. How does one feature connect with the next one; how does an activity near the end of the lesson link back with one at the beginning? This is a very different way to think about teaching. It means that individual features make sense only in terms of how they relate with others that surround them" (The teaching gap, Free Press, Summer 1999).

As an example, they cited the coded category of teaching aids used in Japan and the U.S. Japanese teachers use only chalkboards whereas U. S. teachers frequently use overhead projectors. To simply count this as a technology difference misses a major distinction in teaching systems. Japanese teachers could not use overheads because they record on their chalkboards a running account of the lesson which they and the students use throughout the lesson. The Japanese teachers emphasize relating ideas within a lesson. A different system of teaching was observed in the U.S., a system that places greater emphasis on collecting and holding students’ attention.

Stigler and Hiebert came to the conclusion that a "bottom-up" analysis strategy, by itself, is insufficient because "Teaching is a [cultural] system . . . not a loose mixture of individual features thrown together by the teacher" (The teaching gap). They liken the individual features captured in their codes to a mountain range poking above the surface of the water: "The videotapes provide views of these mountaintop islands. But still hidden, underneath the surface, are the mountain ranges."

Based on the experiences and findings of the first TIMSS 1995 Video Study, we elected to begin code development by generating tentative models of mathematics lessons akin to those which Stigler and Hiebert concluded were essential for interpreting the results of their more detailed coding procedure.

Rather than begin with development of specific codes, in TIMSS 1999 we would begin with development of a more holistic model of teaching in each of the participating countries. In this way we hoped to achieve two goals:

1. Identify the specific variables to be included in the quantitative code that are needed to faithfully represent key elements of teaching in every country.
2. Develop a holistic interpretive framework for each country to which specific teaching codes can be linked. We refer to this as "conserving" for each country the context or meaning of a given analytic code, for example the meaning of the use of chalkboards and overhead projectors.

The figure below provides a metaphor for this dual coding strategy. Each country has its own unique shape (or teaching system). By using a common framework incorporating six dimensions of classroom lessons, we can build descriptions of the system along each dimension that conserve some of the unique features (the same rows have differently-shaped boundaries in each country) while sharing some common elements (codes can be developed within each dimension that look the same across countries). The important point is that codes or indicators get their meaning from the role they play in the system, so it is critical to keep the pictures of both the individual codes and the whole systems in view simultaneously.

Visual Metaphor for Analytic Framework

(click image for larger version)

To accomplish these goals, we took advantage of the 50+ field test tapes from the participating countries. Using these video tapes, we began with an inductive, top-down construction of tentative models of teaching in each country. A Country Associate led the development of an inductively derived model of a typical mathematics lesson in each country

There is an anthropological injunction to seek "insider" perspective when investigating cultural matters. Although truly cultural matters may be transparent to insiders and taken for granted, a standard protocol in the comparative culture literature is to ask insiders to respond to constructions of cultural belief and practice. After developing a tentative model of teaching in each nation, we solicited feedback from experts in each country. These materials were used to revise the models, as well as to identify elements codes that will faithfully represent teaching in each country.

The top-down phase of our code development is nearly complete. The next step we characterize as bottom-up. It is the development of specific codes that can be applied to all countries to achieve the goal of a comparative analysis (with reliable coding), and yet retain the meaningful context of each element as it relates to the system of teaching in each country.

(1) Some aspects of this approach have been described by Glaser and Strauss (1969) as the "discovery of grounded theory."

TIMSS 1999 VIDEO CODING MANUAL

The Mathematics Code Development Team wrote a manual that became an integral part of training coders as well as an important resource throughout the coding process. The manual is organized by "passes," which were meant to correspond with separate viewings of the lessons. Each pass focused on a manageable set of related codes (e.g., classroom interaction, problem-level codes, purpose). The manual contains code definitions, examples, tips for coding, and instructions for marking transcripts.

Download the TIMSS 1999 Video Coding Manual