Queena N. Lee-Chua, Ph.D.
Ateneo de Manila University, Philippines
Note: Although this article discusses problem solving in the context of an introductory probability course, the discussion of the kinds of errors students make, and the methods the author uses to improve students' problem solving skills could be applied, with small changes, to a variety of introductory courses which have a substantial problem-solving component. B. Gold, editor, Innovative Teaching Exchange
The study of probability is often deceptive: on the surface, it seems close to everyday experience, where terms such as "randomness," "chance," and so forth are used by laypeople -- the media, ratings boards, athletes -- to justify one action over another. However, they tend to use these words loosely, and in the process, fall prey to fallacies and errors.
The mathematical notion of probability is a different case. Terms are well-defined, rules formulated and proven, reason preferred over intuition. Yet because of this dual nature of probability, students often find the topic difficult, especially when concepts such as conditional probability, Bayes' rule and their ilk are thrown in.
I have been handling an introductory probability course for engineering and mathematics juniors for five years. The majority are quite diligent, and would listen attentively, take notes, read the text  -- or so they claim. For the most part, they manage to solve "simple" problems (with one to two pieces of information, which can be directly substituted into the formula or equation). For instance, after some thought, they can easily solve a problem such as: "A truth serum given to a suspect is known to be 90% reliable when the person is guilty. If you are guilty and you are given the serum, what is the probability that you will go free?" [From knowledge of the formula, or "common sense", as they claim, they will usually answer: 1 - 0.9 = 0.1, or 10%]
However, when additional pieces of information are given, and the problem gets more complicated, students tend to make several errors:
they look for key words and apply what they believe to be the corresponding operation (a favorite is equating the word "and" with the operation of multiplication, even when the events are not independent)
they strive to use all numerical pieces of information given in the problem, and go through their list of formulas to see which can accommodate all of them (without distinguishing between relevant or irrelevant information)
they imitate the problem-solving procedure of a superficially similar problem (treating the act of throwing different-colored dice the same as that of throwing dice of the same color -- only because "the problem involves dice anyway, and didn't we already tackle dice in class?")
they regress to the "brute-force approach" -- enumerating all possible cases and manually counting each one, especially when the question involves combinatorics (this of course defeats the purpose of learning probability rules).
For instance, students make several mistakes when faced with the following Bayes' theorem problem: "A truth serum given to a suspect is known to be 90% reliable when the person is guilty and 99% reliable when the person is innocent. If the suspect was selected from a group of suspects of which only 5% have ever committed a crime, and the serum indicates that he is guilty, what is the probability that he is innocent?" Mistakes range from multiplying 90% and 99% or subtracting 5% from 90%, to adding the three numbers or trying to list all possible formulas for solution.
Apparently, many of them do not have a solid conceptual understanding of basic probability concepts, and resort to mindless substitution in formulas and frantic number crunching instead. So I was faced with this fundamental question: how do I encourage my students to read the problem carefully, and analyze the various pieces of information given, so that they will come to a clear understanding of the problem? Once they understand the problem, they can then use the rules and theorems they have learned to tackle it well.
From cognitive psychology, I know that asking good questions  and rewording  encourage students to concentrate on the problem and to analyze the various concepts given. I decided to train my class in those two strategies.
To train them in the former, I adopted parts of Graesser and McMahen's  method of using anomalous information to trigger student questions and modified their experiment to suit the classroom setting. They found (p. 137) that questions would be triggered when the problem contains anomalous information in which information was either deleted or added, as defined below:
Deletion: "A critical piece of information was removed so that it would be very difficult, if not impossible, to solve the problem." For instance, when the information "only 5% of the subjects ever committed a crime" was replaced by " only a few of which have ever committed a crime", then it cannot be solved.
Contradiction: "A sentence that directly contradicted one or more of the earlier statements was added to the original version." Again, the problem cannot be solved unless the contradiction is resolved. For instance, when this information was added ("furthermore, 1% of the guilty are judged innocent by the serum"), the problem now contains two contradictory pieces of information.
Salient irrelevancy: "A sentence that was obviously irrelevant to the quantitative solution of the problem was added to the original version...[and often] was not thematically relevant to the semantic context of the problem." For instance, a salient irrelevant piece of information would be "25% of the suspects are scared to die."
Subtle irrelevancy: "A sentence that was irrelevant to the quantitative solution to the problem but was thematically relevant to the semantic context was added to the original version." This is often harder to detect than the salient irrelevancy version. For instance, a subtly irrelevant piece of information would be "and only 1% have ever been found guilty" added to the conditional clause of the last statement.
How do I proceed? First, I tell the class that they should "concentrate on and pay careful attention to the problem, not just substitute numbers into formulas." Then I randomly distribute the four different versions of the problem -- one version per student (in an average class of 40, this works nicely: 10 students work on each version of the problem). I then give instructions: "Solve the problem, but if you run into some difficulties, jot down on any questions and observations you may have -- which you think may help you solve the problem better. By the way, I will give points for good questions, so don't hesitate to ask."
During the first few tries, students write down quite a list (4-5) questions and observations -- some of which focus on even more irrelevancies. But when they realize that they are evaluated not on the quantity, but on the quality of their questions, they buckle down, examine the problem very carefully, and come up with a couple of insightful questions. At first, it takes some students some time to realize what a "good question" is -- though quite a number catch on almost immediately, and enjoy spotting errors in the problem. To help the former formulate "good questions", I lead a class discussion afterwards. (During the first trial, the class was given ten minutes to attempt to solve their version and to generate questions -- but in subsequent trials, I discovered that five minutes per problem was usually more than enough, unless the problem was especially complicated. The amount of time allotted would vary with the level of understanding of the class, and the amount of difficulty of the problem.)
Here are some insightful observations:
Deletion version: "What do you mean by 'a few'?"; How few is 'a few'?"
Contradiction version: "There are two different facts given. Which one is correct?" "There seems to be a typographical error here -- shouldn't 1% be 10%?"
Salient irrelevancy version: "Why did you include the fact that 25% are scared to die? I don't think we are supposed to use this." "This seems to be a trick question -- there is one fact totally unrelated to the problem."
Subtle irrelevancy version: "I am not too sure about this, but what is the use of knowing that 1% are found guilty?" "Fine, 1% of the suspects are found guilty, but that's just in that group. I think we should still go with the 90% and 95%. I hope my solution is correct."
After two weeks (2 to 3 exercises for each of the 6 class sessions) of intense question and answer, the class learns to pay careful attention to the problems. (The subtle irrelevancy versions poses the most challenge, but before long, the majority of the class becomes adept at understanding and solving them.) Sometimes I even vary the method a little -- I assign them problems in the text, and tell them to generate their own versions of the originals. During the next class period, they exchange problems with each other, and earn points based on their solution, questions, and observations. Most of the students come to like this activity very much, and spend quite some time writing their versions, especially on salient and subtle irrelevancies.
As an extension, I sometimes ask the class to search for probability-related concepts in the news: newspapers, magazines, TV shows, journals -- and subject these to intense scrutiny. They revel in this activity, especially when they discover statistical errors -- deletions of important data are especially prevalent in women's magazines on health and pop psychology.
How about rewording? Aside from doing the anomalous-information activities above, I trained the students to reword the problem. They can add sentences, simplify the words, use tables, and so on.
Here is a good rewording of the original truth serum problem:
"A truth serum given to a suspect is known to be 90% reliable when the person is guilty and 99% reliable when the person is innocent. This means that 90% of the guilty ones are judged guilty by the serum, and 10% of the guilty ones are judged innocent. Also, this means that 99% of the innocent ones are judged innocent, and 1% of the innocent ones are judged guilty. The suspect was selected from a group of suspects of which only 5% have ever committed a crime--this means that 5% are guilty. If the serum indicates that the suspect is guilty, what is the probability that he is innocent?"
When students take the time to reword a problem, they understand it more -- and find it more meaningful. Add to this their powers of analysis honed by the preceding activity -- their understanding of the salient pieces of information needed to solve the problem becomes even greater.
Do I have hard data to support this claim? When I introduced this method in my third year of teaching probability, the students' mean grades went from a C to a C+. (I have done some statistical analyses comparing the different versions and approaches, and am now working on a paper expounding on the results.) Aside from grades, though, the benefits are more subtle -- students' faces light up after a particularly good rewording (done by themselves or by classmates); discussion of problems (whether those in the text or those generated by the class) become more animated; they pay more attention to my subsequent lectures on discrete and continuous probability distributions, as well as other statistical concepts, making my job a lot easier in the long run.
The only apparent shortcoming is the initial amount of time and effort spent in training students to pay careful attention to the problem, analyze anomalous information, reword the problem -- a period of two to three weeks on the average. I realize that in an already tight curriculum, this activity may appear to be an additional burden. But we have always managed to finish what is required in one semester (18 weeks), since subsequent lessons (chock-full of word problems) are more easily absorbed.
One time a student came to me, beaming with pride. In her finance class, she was able to expose a contradiction in her professor's reasoning -- through the use of a timely, well placed question. "He told me it was a good question, and that he would look into it. In our next class, he apologized for his mistake. Now I realize that the art of good questioning can be used outside of our probability class!"
Part of this project was supported by a grant given to the author by the Ateneo de Manila University, School of Arts and Sciences. (The grant was from the Ayala Foundation and Bank of America Professorial Chairs.)
If you have any questions, please feel free to e-mail me at email@example.com or firstname.lastname@example.org
 Davis-Dorsey, J., Ross, S.M., & Morrison, G.R. (1991). The role of rewording and context personalization in the solving of mathematical word problems. Journal of Educational Psychology 83(1), 61-68.
 Graessser, A. & McMahen, C.L. (1993). Anomalous information triggers questions when adults solve quantitative problems and comprehend stories. Journal of Educational Psychology, 85(1), 136-151.
 King, A. (1989). Effects of self-questioning training on college students' comprehension of lectures. Contemporary Educational Psychology, 14, 366-381.
 Walpole, R.E. & Myers, R. H. (1995). Probability and Statistics for Engineers and Scientists. 4th ed. New York: Macmillan Publishing Co.
Queena N. Lee-Chua graduated with a B.S. Mathematics, summa cum laude, from the Ateneo de Manila University in 1987, and earned her doctorate in clinical psychology in 1995. She is currently teaching in both the mathematics and psychology departments. Every week she writes a math and science column for a national daily, and hosts a math children's show. Research interests include psychology of math, cognition in math teaching and learning, history of math.