Completed Projects

On Evaluating Curricular Effectiveness: Judging the Quality of K-12 Mathematics Evaluations

The Mathematical Sciences Education Board (MSEB) conducted a review of evaluations of 19 mathematics curriculum materials. The report breaks new ground in reviewing the literature that has accumulated on these K-12 mathematics curricula and in framing an ambitious and rigorous approach to curriculum evaluation that has relevance beyond mathematics.

The committee that produced this report consisted of mathematicians, mathematics educators and methodologists and they began with the following charge:

The committee collected, reviewed and classified almost 700 studies, solicited expert testimony during two workshops, developed an evaluation framework, established dimensions/criteria for judging different kinds of studies (e.g., content analyses, comparative studies and case studies), drew conclusions on the corpus of studies, and made recommendations for future research.

The pool of 700 studies was winnowed down to 147 that met the criteria the committee established for minimally methodologically adequate and therefore worthy of close inspection. Some of these studies were well conceived and well executed and others were not. But taken together they provided a sound basis for the committee to make the findings it did.

While reviewing these studies several common weaknesses were found such as incorrect unit of analysis, inadequate attention to implementation fidelity, lack of independent evaluators, and unspecified comparative groups. As a result, the committee determined that the corpus of evaluation studies would not allow a determination of the effectiveness of any of the 19 individual programs with a high degree of certainty. This inconclusive finding should not be interpreted to mean that these curricula are not effective, but rather that problems with the data and/or study designs prevented confident judgments about their effectiveness.

The committee also concluded that a determination of curricular effectiveness can not rest on a single scientifically valid study. Rather, if trustworthy conclusions about curricular effectiveness are to be made, a body of scientifically valid studies is needed that includes comparative studies joined by content analyses and case studies.

The report identifies standards of evidence that individual studies should meet if they are to contribute to reliable, fair and valid judgments, outlines a framework for evaluating curricular effectiveness, and recommends actions at the district, state and federal levels and for publishers. To read the entire consensus report online visit http://www.nap.edu/catalog/11025.html.

Next Steps in Mathematics Teacher Development, Grades 9-12

The MSEB convened a steering committee that planned and executed a workshop that explored the mathematical knowledge needed by high school teachers of mathematics, how teachers come to acquire that knowledge, and the roles that both pre- and in-service teacher development play in building a cadre of exemplary high school teachers of mathematics. This project followed on similar successful workshops on mathematical knowledge required to teach at the elementary school (MSEB, 2001) and middle school (MSEB, 2000) levels.

The workshop focused on what mathematics teachers actually do, how they come to learn the mathematics they need to know, their beliefs about how students should learn mathematics, and their expectations about which students should learn what kind of mathematics. The workshop explored this focus from the practicing teachers’ point of view.

The workshop product, a CD-ROM is a new format for the National Academies and was developed to try and reach our audiences in a user-friendly format. The CD includes workshop materials including the “What is Pi, Anyway?” webcast, transcripts of the talks, PowerPoint presentations, session handouts, workshop readings and Internet links. As a bonus we have included on the CD PDF versions of the two previous workshops reports in this series, Knowing and Learning Mathematics for Teaching (2001), and Mathematics Education in the Middle Grades (2000). Therefore, this CD provides one convenient location for information on elementary, middle school, and high school teacher professional development.

Talking It Through: Cross-National Conversation About Secondary Mathematics Curriculum

The MSEB and the Board on International Comparative Studies in Education (BICSE) brought together a steering committee that designed and oversaw a two-day symposium that examined
how various industrialized countries use mathematics curricula both to prepare all secondary students to be mathematically proficient citizens and to prepare students for mathematically intensive college programs and career fields. The speakers presented historical and contemporary experiences in their countries and discussed evidence for the effectiveness of these approaches. This two-day session was videotaped for the purpose of disseminating the symposium to a larger audience and was made available to the public for a one-year period. The webcast is no longer available. Click here to access the background reading materials.

Studying Classroom Teaching as a Medium for Professional Development: Proceedings of a U.S.-Japan Workshop

MSEB is pleased to announce the release of Studying Classroom Teaching as a Medium for Professional Development, which was supported by the National Science Foundation. This report includes a video of the sessions and will serve as a resource for those from both countries that are responsible for the preparation and development of mathematics teachers.

In August 2000, following the Ninth International Congress on Mathematics Education (ICME 9), the MSEB joined with the U.S. National Commission on Mathematics Instruction (UNSCMI) and Japanese mathematics educators to organize a workshop in Japan. The workshop considered the knowledge needed by mathematics teachers about content and instruction and the relation of these needs to the teacher development programs in the two countries. The experiences, research, and practices from both countries was used to develop a workshop agenda that featured country-specific practices that formed a platform to contribute to the broader thinking of mathematics educators about the professional development of mathematics teachers.

The professional development process used in this project is referred to as “lesson study,” a process that Japanese teachers engage in to systematically examine their practice, with the goal of becoming more effective. The link below takes you to more information about lesson studies from the Lesson Study Research Group at Columbia University.

http://www.tc.columbia.edu/lessonstudy/index.html

Related Projects

Mathematics Learning Study (MLS)

At the request of the U.S. Department of Education and the NSF, the Mathematics Learning Study Committee was created and charged with synthesizing the rich, diverse research on school mathematics in grades pre-K through 8 and providing research-based recommendations for teaching, teacher education, and curriculum for improving student learning. The report, entitled Adding It Up: Helping Children Learn Mathematics, which presented the findings of this committee, was published in November.

With the help from the ExxonMobil Foundation, a targeted short report Helping Children Learn Mathematics was prepared for policy makers. It incorporated the main observations and recommendations of the larger report while touching more briefly upon some of the larger report's more detailed recommendations. The targeted report was published August 2002.