The National Research Council of the National Academy of Sciences has created a committee of experts in mathematics assessment, curriculum development, curricular implementation and teaching to assess the quality of the studies about the effectiveness of the thirteen sets of mathematics curriculum materials developed with National Science Foundation support. A selection of evaluations of non-NSF supported materials will be used for comparison purposes.
The current committee, chaired by Dr. Jere Confrey of the University of Texas at Austin, is charged with the first phase of a potentially multi-phased review process. "We have been asked to prepare a short consensus report summarizing the results of our work, which includes creating an extensive bibliography of studies, mapping those studies according to their characteristics and then advising if the quality of the evidence merits a full review."
The committee is seeking a broad set of studies that meet the following critieria: 1) they study one or more of the thirteen NSF curricula as a central variable, 2) they meet the methodological canons for acceptable inquiry associated with that methodology, and 3) they have identified authorship and affiliation so as to give credibility to the work.
The Committee is soliciting evaluation studies that include: studies with specific student outcomes, content analysis studies, students of classroom implementation and school environment and studies of teacher knowledge, teacher characteristics and professional development and will add to these classifications as committee members identify additional relevant categories. Dr. Carole Lacampagne, director of the Mathematical Science Education Board stated, "We have not limited ourselves to published studies, because summative studies of these curricula are often recently completed, but as with all NRC work, the reports must meet scholarly expectations."
The Committee is also hosting a two day workshop in Washington D.C. on September 17 and 18th. This meeting permits us to hear from various curricular designers, researchers, evaluators, mathematicians and practitioners on their points of view concerning the evaluation of effectiveness. Panel members are asked to respond to the question "How would you define and/or evaluate effectiveness of a K-5, 6-8 or 9-12 NSF-supported curricula and what evidence would be needed." They are asked to identify primary and secondary variables, methods of examining and measuring those variables, research designs and other relationships under investigations.
"These are complex questions," said Dr. Confrey, "as curricular design and implementation involve many people's participation, are measured by a myriad of local and national forms of assessment, and are used across highly variable settings, differing in values, resources, cultural contexts and forms of organization. It is imperative for us as a nation to get smarter and more sophisticated in how to conduct and evaluate such studies and to learn from our current work" Confrey added that she thinks, "I believe that this NRC work can lead towards resolution of some of the debates by bringing together people and studies from a variety of perspectives and working for a common framework to establish a solid research-based foundation to improve curriculum development and evaluation and to aid schools and districts making decisions."
Dr. Michael Feuer, director of the Center for Education, added, "As long as there is a commitment to increasing the scientific evidence on questions of education, work such as this will be needed and is directly in line with the responsibilities of the NRC to provide advice to the Nation." Suggestions of studies for review should be sent to CLacampagne@nas.edu.