 |
Additional Online Case Studies & Appendices | |
|
Additional Online Case Studies & Appendices | |
|
Assessing the General Education Mathematics AbdelNaser Al-Hasan and Patricia
Jaberg At a private liberal arts college in the Midwest, an assessment plan was developed to assess the effectiveness of the introductory courses. The assessment plan concentrated on mathematical disposition, as well as skills. An attitude survey was developed to give us some information regarding the impact of mathematics anxiety and other affective factors on students’ mathematical skills and a subset of the basic skills examination was developed to test their improvement in their remedial mathematics courses that fulfill the college mathematics competency requirement. Background and Purpose Mount Mary College, located in metropolitan Milwaukee, is Wisconsin’s oldest Catholic college for women with approximately 600 full-time undergraduate, 600 part-time undergraduate, and 175 graduate students. The Department of Mathematics and Computer Science consists of four full-time and nine part-time faculty members. The department offers BS degrees in both Mathematics and Computer Science. The overarching goal for the current assessment program at Mount Mary College is to assess the effectiveness of our remedial mathematics courses and advise the department and the larger college community on how to better serve students in fulfilling the mathematics competency requirement. The current competency requirement consists of either achieving a score of 70% on the competency test upon entrance or taking a remedial mathematics course. The competency test consists of 32 multiple choice items related to basic operations with whole numbers, integers and rational numbers, ratio and proportion, number sense, and basic probability and statistics. A student scoring below 47% on the competency test is placed in a developmental mathematics course. A student scoring between 48% and 69% is placed in a liberal arts mathematics course. Initial examinations of these courses included discussions with departments that require the algebra sequence, enrollment data in the developmental and the liberal art courses since 1998, materials used for these courses, and student course evaluations. The review of these courses revealed the following:
The initial examination of course goals and enrollment data led to the conclusion that in order to develop an effective assessment program, we needed to reexamine the learning goals for each of these two courses, with a possible revision and realignment of the curriculum to ensure proper articulation. Developing the assessment program Discussions during the initial phase focused on the following: · Original course goals in relation to the overall departmental goals · The existing curriculum in relation to what it is we want students to learn · The currently existing competency and placement tests · An assessment plan for the presently-existing courses This initial phase required a great deal of discussion concerning the purpose of each course and how each course fits into a cohesive sequence. Examination of the enrollment records from the last five years revealed that only about 25% of students enrolled in these remedial courses continued into the algebra sequence. This led us to concentrate on those students who did not pass the competency test and placed into either Developmental Mathematics or Mathematics for the Liberal Arts. During this phase, we developed new goals and objectives for these two courses (see Appendix A), and began examining materials that would reflect these new goals and objectives. These curricular changes were scheduled to be in place by the fall 2003 semester. However, the assessment phase was begun immediately. A pilot assessment plan was used during fall 2002 and spring 2003 to help inform our decision-making for the future. The pilot assessment plan focused primarily on skills and attitudes. We are currently working on modifying the assessment plan to reflect the developing learning goals and curriculum changes. Since all students entering Mount Mary College take the competency test, we selected ten items from this test to create a post-course assessment test (see Appendices B and C). These items were used for the Developmental Mathematics and the Liberal Arts courses. Some items were identical, but others reflected the difference in course emphasis. Selection of the items was based on the learning goals for each course. To gauge the affective domain, students in these two courses were given the opportunity to self-assess their mathematical disposition both at the beginning and the end of the course, through an attitude survey. Details of the assessment program The attitude survey was administered in both courses. In spite of the level of the material in these courses, some students struggle in these courses and are generally intimidated by mathematics. Through discussions with departments whose students take these courses, faculty who teach these courses, and student evaluations, we felt that an attitude survey might reveal students’ beliefs about mathematics and how they approach a mathematics course. An attitude survey was constructed and focused on four areas: confidence, anxiety, persistence, and usefulness. With each of the four areas, a 5-point Likert scale allowed students to choose a descriptor that best described their self-perceived ranking in relation to these four areas. The four constructs along with the extreme descriptors are provided below: Confidence 1= Distinct lack of confidence in learning mathematics 5 = Confident in one’s ability to learn math and perform well on mathematical tasks Anxiety 1 = Dread, anxiety and nervousness related to doing mathematics 5 = Feels at ease in a situation requiring mathematical thinking Persistence 1 = Lack of involvement in problem solving; easily gives up 5 = Keeps persisting in order to complete a mathematical task or problem Usefulness 1 = Believes mathematics is not useful now or in the future 5 = Believes mathematics is currently useful and will be important in future activities and career The descriptors for “Usefulness” was changed during the spring semester to reflect the use of mathematics in one’s personal life as follow 1 = Believes mathematics is not useful to me now or in the future 5 = Believes mathematics is currently useful and will be important in future activities, both personally and professionally This attitude survey has since been modified several times so that the wording is in the active rather than passive voice (see Appendix D). Additionally, students were also asked to write a brief statement to explain their choice on each scale. Research has shown that most people suffer some mathematics anxiety, but “it disables the less powerful – that is, women and minorities – more” (Tobias, 1993, p. 9). The attitude survey, then, was designed to roughly measure students’ perceptions about themselves in relation to mathematics. The anticipation is that instructors can assist those with high anxiety and low confidence, once they are identified. Through minimal intervention and encouragement, these perceptions, hopefully, can be altered to generate more success in taking mathematics. Revisions based on initial discussions A second round of discussions resulted in a decision to identify and separate the population that did not pass the competency test into three groups. Group one consists of those students who will ultimately take an algebra course. The Developmental Mathematics course will be renamed as Prealgebra, and its curriculum will provide a better bridge to the algebra sequence. Group two are those students who will only need a mathematics course to fulfill the math competency and they will be placed in a course that will be called Quantitative Reasoning. This course will replace the Mathematics for the Liberal Arts course and its curriculum will reflect this emphasis. The third group consists of students who are not ready to take either course. This last group of students needs to obtain basic arithmetic skills before continuing with other mathematics courses. Therefore, a one-credit preparatory workshop was developed. This workshop will serve as a prerequisite for Prealgebra or Quantitative Reasoning. The realignment of courses and curriculum choices were based on our student population and learning goals addressed in the initial phase. Findings Students were given the attitude survey at the beginning and end of the semester-long courses. Responses from students who dropped the course during the semester were eliminated from the initial analysis. We also administered the post-course assessment as described earlier in both lower-level courses. During the spring semester, the assessment was administered with and without a calculator in the developmental course. Students often reported that they failed to do well on the competency test due to the lack of the ability to use a calculator. We wished to see if the availability of a calculator made a difference in the results. Results of the attitude survey are shown below:
Based on data collected, we are pleased that students are reporting improved results in all categories. However, these results are only self-reported. It was important to also read the rationales provided to gain further insight in relation to these results. In other words, did something in the course impact these changes? Here is a sample of written responses from the end of the semester: · I think my confidence has improved slightly over the semester; I feel better about coming to class. · Has improved because I feel I understand things better now. Feel comfortable asking when I don’t understand things. I understand it and try to jump right in. Can ask questions without feeling dumb. I am trying more and paying more attention so I can understand and not dread. · Sometimes I doubt myself on certain problems with homework or tests. Otherwise I feel much better about my ability to do well on my work. · I know that the things that we are learning in here will be useful one day, so I work hard to understand them and use them. · After this course I don’t even use my calculator as much, so I feel confident. · If I don’t get a problem correct I used to just leave it alone but now I try it again and if I don’t understand I come back to it later or I ask for help. · I still dread math tests/quizzes, but my nervousness has decreased I think. These comments support the conclusion that the courses positively changed student perceptions about their ability to do and use mathematics. Other comments indicate that these courses do not influence all students in a positive way. One comment, “Sometimes I feel confused or frustrated with my math problems but its math; math isn’t supposed to be easy or fun,” indicates an unchanged perception of mathematics. Other comments such as “I’m going to be a history teacher so I might have to add or subtract years!” and “I’m not going to go into a mathematics career,” clearly indicate the need to relate the mathematics that students are learning to real-life situations so that students see mathematics as an important part of their liberal arts education. We also then examined the attitudes of students who dropped the course. The results of the attitude survey of these students are shown below:
The data was examined to see if students who dropped the course began their mathematics course with less confidence, higher anxiety, and so forth. From the limited number of students who dropped, there was not a clear difference, although we did observe a slightly higher level of anxiety. We will continue to collect and examine such data. The results of the placement testing and post-course assessment test are reported for each course (see appendix E). Overall, when test items were compared on a one-to-one-basis, we were not extremely encouraged by the results. Although the percentage of students selecting the correct response increased, this increase was minimal on some items. Items that were more computational in nature showed greater gains, but questions that required the application of more critical thinking or problem solving did not show that same level of improvement. Note that in the spring semester, the items for the developmental mathematics course were administered first without a calculator and then with a calculator. The items that require a determination of what to do with numbers (such as using a ratio, or taking two numbers to determine a percentage) did not fare well on the assessment. Overall, there was not a great increase in success with the items when a calculator was permitted. The exceptions were items related to computation and comparison of numbers and calculating unit prices. To our surprise, the question that asked students to determine how many eggs are left over when 115 eggs are put in cartons of 12 eggs, produced poorer results when students actually had assess to the calculator. It appears they were unable to interpret the decimal results when dividing 115 by 12. The multiple choice format allows respondents to choose the correct answer often based on number sense, rather than actual computation, but it appears that students often did not use this approach. This finding is disturbing and will be addressed in the future. Students enrolled in Mathematics for the Liberal Arts performed better on the post-course assessment than those in the developmental course, but these students also scored higher on the initial competency examination so it is not proper to conclude that the course created these more acceptable results. The item-by-item results can also be found in Appendix E. Note that this assessment was done without calculator. Use of findings Initial examination of courses led to a decision to revamp the learning goals and curriculum of all remedial mathematics courses and to add a preparatory remedial workshop for students who score low on the competency examination. Additionally, students will be placed in the Quantitative Reasoning course if they do not need the algebra sequence for their major program. These changes were deemed necessary in relation to the needs of our population. The pilot assessment program has given us valuable information and experience in designing a feasible assessment plan. The attitude survey revealed that we need to continue to make efforts to address the affective domain, and to continue to provide a supportive learning environment for these students. We also need to help students develop an appreciation for mathematics in their daily life by emphasizing real-life applications and the connections to their major. The post-course assessment of skills certainly points to the need for students to continue to develop number sense. This needs to be a pervasive theme in their coursework in the mathematics department. Next Steps and Recommendations The new structuring of courses will need ongoing assessment to monitor the success of our students and the impact of these changes. We need to assess the effectiveness of the workshop in preparing students for these remedial courses, and the effectiveness of the new Prealgebra in preparing students for the algebra sequence. We will compare those taking the workshop relative to those students who did not take the workshop. We will continue to develop both the attitude survey and skills assessment and expand the assessment program to include a portfolio of open-ended tasks or projects to assess the student’s ability to solve problems and use real-world data. The portfolio might also include a self-assessment component. We are currently discussing ways to improve or possibly create a new competency exam that will be aligned with the new goals and objectives set forth for these courses. Since these remedial courses are primarily taught by part time faculty, we have determined it is important to provide these instructors with a rationale and information in relation to the assessment program, the changes in the course descriptions and goals, and research information on helping students deal with mathematics anxiety. We were awarded a grant from the college to hold a day-long professional development opportunity for the part-time faculty before the beginning of the fall 2003 semester, and many of the findings and concerns from our assessment plan were shared at that time. References Tobias, S. (1993). Overcoming Math Anxiety. New York: W. W. Norton & Company. APPENDIX Appendix A: Our new goals for the course Appendix B: Post-Course Assessment for Developmental Mathematics Appendix C: Post-Course Assessment for Mathematics for the Liberal Arts Appendix D: Attitudes Survey Appendix E: Placement and Post-Course Assessment Test Results |