Students and Teachers Don’t Reason Well
this essay by developing the notion that all human action presupposes
the use of humanly created logical systems that model, abridge,
and summarize the features of the world about us, and that abstract
inferential systems, and the reasoning they make possible, are as
natural to us as a species as swimming is to a dolphin or flying
is to a bird. As Paul puts it, we are continually “making
inferences within a system we have created — about what is
going on in our lives.”
according to Paul, to reason well we must do more than simply engage
in it. We must become aware of that engagement and use our knowledge
of the nature of that engagement to improve it. Paul compares the
good reasoner to the good ballet dancer, the good chess and tennis
players. All three must explicitly study the principles and practice
the moves involved (with explicit standards of performance in mind).
what good reasoning requires, Paul presents evidence to show that
most students are not good at it. What is more, he presents evidence
to suggest that most teachers are not good at it either —
at least not at assessing it when students are called upon to use
it in their work. One of the major reasons, combining with ignorance
of what reasoning requires, is a systematic confusion between intelligent
subjectivity (wit, articulateness, cleverness without substance),
and reasoned objectivity (careful, disciplined, reasoning about
an issue), between subjective opinion (however “bright”),
and reasoned judgment (however mundane).
this problem with an analysis of a major mistake in a California
Department of Education statewide assessment of reasoned evaluation
in writing. He follows up this documentation of a mistake on the
part of testing experts with the same mistake made by teachers.
He then briefly explicates a model for the analysis and assessment
of reasoning (based on the logic of the question at issue) complete
with a series of samples of student reasoning, all duly analyzed
for the reader.
the paper with a brief argument to the effect that “the logical
structures implicit in an educated person’s mind are highly
systematized.” In contrast he argues:
the logical structures by which a mind figures out the world are
confused, a jumble, a hodgepodge, a mere conglomeration, then that
figuring out is radically defective.... Then the mind begins it
knows not where, takes things for granted without analysis or questioning,
leaps to conclusions without sufficient evidence... meanders without
a consciousness of its point of view.... Then the mind wanders into
its own prejudices and biases, its own egocentricity and socio-centricity.
Then the mind is not able to discipline itself by a close analysis
of the question at issue and ignores the demands that the logic
of that question puts on it and us as rational, logic-creating,
The Ability to Reason: A Defining Feature of Humans
Your capacity to reason
is at the heart of all disciplined thinking. It explains how we
alone of all the creatures of the earth have been able to develop
full-fledged academic disciplines: biology, physics, botany, zoology,
chemistry, geography, history, psychology, sociology, etc. We can
go beyond immediate, instinctive reactions to reflective, reasoned
responses precisely because we are able to develop small-scale and
large-scale systems in which to intellectually operate and act.
These systems enable us to mentally manipulate our possible responses
to situations — to formulate them explicitly, to hold them
at intellectual arm’s length, to analyze and critique them,
and to decide what their implications are for us. Let me explain.
We understand the various
particulars of everyday life by constructing abstract models or
systems that abridge and summarize their features. In simplest form,
we call these models or systems ideas. For example, our abstract
concept of a bird is a model or system for thinking about actual
birds in order to make sense of their behavior — in contrast
to the behavior, say, of cats, dogs, turtles, beetles, and people.
As we construct these abstract systems or models, we are enabled
to use the reasoning power of our minds to go beyond a bare unconceptualized
noticing of things to the making of inward interpretations of them,
and hence derivations from them. In short, our concepts provide
our minds with systems in which to experience and think; our minds
operate (reason) within them to invest the world we experience with
meanings rich in implications and consequences. Much of this is
done, of course, quite automatically and subconsciously.
I can reason to any number
of conclusions as the result of my having one simple model for a
thing. For example, if I recognize a creature to be a dog, I can
quickly infer it will:
- bark rather than
meow or chirp
- wag its tail when
- growl when irritated
- be unable to fly
- have no feathers
- be unable to live
- be carnivorous
- need oxygen
- have teeth
- have paws rather
than feet, etc.
This word (‘dog’)
is part of a much larger logical map upon which our minds can move
in virtue of our capacity to reason. As we act bodily in the world,
we act intellectually in our minds. These intellectual moves guide
our actions in the world. Without these maps and the capacity to
locate particulars on them, we would either thrash about aimlessly
or be paralyzed by the bewildering mystery of things and events
before us. In every situation in our lives we “construct”
a response that results from how we are modeling the situation in
Hence, put us in any
situation and we start to give it meaning, to figure it out with
the logical structures we have at our disposal. So quickly and automatically
do we make inferences — as the result of the way we are modeling
the situation in our minds — that we do not typically notice
For example, we see dark
clouds and infer rain. We hear the door slam and infer someone has
arrived. We see a frowning face and infer the person is angry. Our
friend is late and we infer she is being inconsiderate. We meet
a tall boy and infer he is good at basketball, an Asian and infer
he will be good at math. We read a book, and infer what the various
sentences and paragraphs, indeed what the whole book, is saying.
We listen to what people say, and make a continual series of inferences
as to what they mean. As we write we make inferences as to what
others will make of what we are writing. We make inferences as to
the clarity of what we are saying, as to what needs further explanation,
as to what needs exemplification or illustration. We could not do
this without “logical structures” by means of which
to draw our inferences.
Many of our inferences
are justified and reasonable. But, of course, many are not. One
of the most important critical thinking skills is the skill of noticing
and reconstructing the inferences we make, so that the various ways
in which we inferentially shape our experiences become more and
more apparent to us. This skill, this sensitivity or ability, enables
us to separate our experiences into analyzed parts. We learn to
distinguish the raw data of our experience from our interpretations
of those data (in other words, from the inferences we are making
Eventually we realize
that the inferences we make are heavily influenced by our point
of view and the assumptions we have made. This puts us in the position
of being able to broaden the scope of our outlook, to see situations
from more than one point of view, to become more open-minded. This
requires that we recognize our point of view as a “logical
system” that guides our inferences, a system that we can exchange
for another (an alternative point of view), depending on our assumptions.
Often, then, different
people make different inferences because they bring to situations
a different point of view. They see the data differently. Or, to
put it another way, they have different assumptions about what they
see. For example, if two people see a man lying in a gutter, one
might infer, “There’s a drunken bum.” The other
might infer, “There’s a man in need of help.”
These inferences are based on different assumptions about the conditions
under which people end up in gutters and these assumptions are connected
to the point of view about people that each has formed. The first
person assumes: “Only drunks are to be found in gutters.”
The second person assumes: “People lying in the gutter are
in need of help.” The first person may have developed the
point of view that people are fundamentally responsible for what
happens to them and ought to be able to take care of themselves.
The second may have developed the point of view that the problems
people have are often caused by forces and events beyond their control.
The two are modeling the situation differently. They are using a
different system for experiencing it.
In any case, if we want
our students to become good reasoners, we must become concerned
to help them begin to notice the inferences they are making, the
assumptions they are basing those inferences on, and the point of
view about the world they are taking — hence the systems in
which they are thinking. To help our students do this, we need to
give them clear examples of simple cases, and lots and lots of practice
analyzing and reconstructing them. For example, we could display
the above inferences in the following way:
Situation: “A man is lying in the gutter.”
Assumption: “Only bums lie in gutters.”
Inference: “That man’s a bum.”
Situation: “A man is lying in the gutter.”
Assumption: “Anyone lying in the gutter is
in need of help.”
Inference: “That man is in need of help.”
Our goal of sensitizing
students to the inferences they make and to the assumptions that
underlie their thinking enables them to begin to gain command over
their thinking (the way they are using logical structures to model
the world). Of course, it may seem odd to put any effort into making
explicit such obvious examples. In the harder instances, however,
the value of the explication becomes more evident. In any case,
because all human thinking is inferential in nature, and all inferences
are embedded in a system, we cannot gain command of our thinking
unless we can recognize, one way or another, the inferences embedded
in it and the assumptions that underlie it.
Consider the way in which
we plan and think our way through everyday events. We think of ourselves
as washing up, eating our breakfast, getting ready for work, arriving
on time, sitting down at our desks, making plans for lunch, paying
bills, engaging in small talk, etc. Another way to put this is to
say that we are continually interpreting our actions, giving them
meanings — making inferences within a system we have created
— about what is going on in our lives.
And this is to say that
we must choose among a variety of possible systems for thinking
about things. Again, consider some simple cases. As I am sitting
in my easy chair, am I “relaxing” or “wasting
time”? Am I being “determined” or “stubborn”,
or worse, “pig-headed”? Did I “join” the
conversation or “butt in”? Is Jack “laughing with
me” or “laughing at me”? Am I “helping him”
or “being taken advantage of”? Every time I interpret
my actions within one of these systems that each word in the language
represents, every time I give them a meaning, I make one or more
inferences on the basis of one or more assumptions within some point
As humans we continually
make assumptions about ourselves, our jobs, our mates, our children,
about the world in general. We take some things for granted, simply
because we can’t always be questioning everything. Sometimes
we take the wrong things for granted. For example, I run off to
the store (assuming that I have enough money with me) and arrive
to find that I have left my money at home. I assume that I have
enough gas in the car only to find that I have run out. I assume
that an item marked down in price is a good buy only to find that
it was “marked up” before it was “marked down”.
I assume that it will not, or that it will, rain. I assume that
my car will start when I turn the key and press the starter. I assume
that I mean well in my dealings with others. We make hundreds of
assumptions, use hundreds of concepts, make hundreds of inferences,
without noticing that we are doing so. Most of them are quite sound
and justifiable. Some however are not.
The question then becomes:
“How can we teach our students to begin to recognize the inferences
they are making, the assumptions they are basing those inferences
on, and the point of view, the perspective on the world that they
are beginning to form?” That is, “How can we help students
to recognize how they are reasoning about the world?”
Our Students Are Not Learning to Reason Well
Though we are “logic-creating”
and “logic-using” animals, we typically operate with
little awareness of this fact. We create and apply logical systems
without knowing that we are doing so. Our intellectual modeling
of the world is done sub rosa, without mindfulness. It is small
wonder, then, that we often reason poorly.
Imagine a ballet dancer improving her ballet without knowing that
she is a dancer or how and when she is dancing. Imagine a chess
player who does not know she is playing chess. Or a tennis player
who does not know she is playing tennis. We can hardly imagine people
developing these physical and intellectual abilities without high
consciousness of how and what they are doing in the doing of it.
Yet we expect students to develop the ability to reason well without
any mindfulness of the nature of reasoning, the elements of reasoning,
or the criteria for assessing reasoning. We expect students to become
good reasoners, in other words, without any knowledge of the logic
of reasoning. Not surprisingly our approach doesn’t work.
Most students are very poor reasoners.
WHAT DOES RESEARCH ON LEARNING AND TEACHING TELL US?
By any measure whatsoever,
most students are not learning to reason well. A recent summary
of research by Mary Kennedy regarding student learning and instruction
at the K–12 level documents serious reasoning deficiencies
on the part of students. (See figure next page.)
State-Wide Test Fiasco: Teachers and Testers Who Don’t Understand
Before teachers will
be able to help students to reason well, it is essential that they
learn what reasoning is and how to assess it. A recent statewide
test in California demonstrated that many teachers, and even some
educational testing experts, have serious misunderstandings about
the nature of reasoning and how to assess it.
The student essay below
(figure 2) should have been graded at the lower rather than the
higher end of the continuum of eight levels: “minimal evidence
of achievement” or, at best, “limited evidence of achievement”
rather than the highest grade of “exceptional achievement”.
For though the essay may have “flair and sparkle” (as
one teacher expressed it), it is a poor example of evaluative reasoning,
since it systematically confuses the objective goal of reasoned
evaluation with the very different goal of explaining subjective
preference, an important distinction in critical thinking which
the teacher-evaluators apparently missed entirely.
First of all, the instructions themselves are confused. They begin
with a clear requirement of “objective” evaluation:
asked to write an evaluative essay, make judgments about the worth
of a book, television program, or type of music and then support
their judgments with reasons and evidence. Students must consider
possible criteria on which to base an evaluation, analyze their
subject in the light of the criteria, and select evidence that clearly
supports their judgments.”
Unfortunately, this request
for reasoned evaluation is blended in the second half of the instruction
with what might possibly be taken, with a little stretching and
selective reading, as a request for the expression of a “subjective”
Each student was assigned one of the following evaluative tasks:
to write a letter to a favorite author telling why they especially
liked one of the author’s books, to explain why they enjoyed
one television program more than any others, or to justify their
preference for a particular type of music. The tasks made clear
that students must argue convincingly for their preferences and
not just offer unsupported opinions.
Let’s look closely at this confusion. In the first place,
there is still an emphasis on objective evaluation (“The tasks
made clear that students must argue convincingly for their preferences
and not just offer unsupported opinions”) while the task itself
is defined as the justification of a “preference”.
Now most people prefer
books, television programs, and types of music for fundamentally
subjective, not objective, reasons. They like a particular book,
television program, or song for no reason other than that they like
it, that is, because they enjoy it or find pleasure in it or are
interested or absorbed or excited or amused by it. Their reasons
for liking what they like are not the result of an objective evaluation.
They have no relation to the objective quality of what is judged.
They are about the personal responses of the experiencer, not about
the objective qualities of that which is experienced.
Most people, to take
the point a step further, do not have “evidence” —
other than the stuff of their subjective reactions — to justify
their preferences. They prefer because of the way they feel not
because of the way they reason. To choose because of these subjective
states of feeling is precisely to lack criteria of evaluation or
evidence that bears upon objective assessment. When challenged to
support subjective preferences, people usually can do little more
than repeat their subjective reactions (“I find it boring,
amusing, exciting, dull, interesting, etc.”) or rationalize
them (“I find it exciting because it has a lot of action in
A reasoned evaluation
of a book, a program, or a type of music requires more than this;
it requires some knowledge of the qualities of what we are evaluating
and of the criteria appropriate to the evaluation of those qualities.
One needs to be well informed about books, about programs, about
music if one is to claim to be in a position to objectively evaluate
them. If one is not well informed, one is unable to render a justified
evaluative judgment, though one can always subjectively react and
freely express one’s subjective reactions as (mere) personal
preferences. This is what the student (graded as having written
an objective evaluation of “exceptional achievement”)
actually does. But his evaluators, not having this distinction clear
in their own minds, completely miss the difference.
The sample student essay
can, for analytic purposes, be divided into three parts. We shall
comment briefly on each in turn. The first segment of the essay
is an account of a highly emotional exchange between the student
and his mother:
“Well, you’re getting to the age when you have to learn
to be responsible!” my mother yelled out. “Yes, but
I can’t be available all the time to do my appointed chores!
I’m only thirteen! I want to be with my friends, to have fun!
I don’t think that it is fair for me to baby-sit while you
run your little errands!” I snapped back. I sprinted upstairs
to my room before my mother could start another sentence.
It is clear that in this segment there is no analysis, no setting
out of alternative criteria, no clarification of the question at
issue, no hint at reasoning or reasoned evaluation.
In the second part, the
student makes a sweeping claim about a purported causal relationship
between listening to rock music and his asserted, but unsupported,
ability to control his emotions. He does not consider “possible
criteria on which to base an evaluation”. He does not present
any evidence, though he does cite two examples, one where a song
prompts him to punch his pillow and one where another song prompts
him to stop. This gives little credence to the notion that rock
music leads to his “controlling” his emotions. If anything,
his examples seem to imply that, rather than learning control from,
he is learning to be controlled by, the music he listens to. His
major claim that “Without this music, I might have turned
out to be a violent and grumpy person” is without reasoned
or evidentiary support. He merely brashly asserts that it is true:
I turned on my radio
and “Shout” was playing. I noted how true the song was
and I threw some punches at my pillow. The song ended and “Control”,
by Janet Jackson came on. I stopped beating my pillow. I suddenly
felt at peace with myself. The song had slowed me down. I pondered
briefly over all the songs that had helped me to control my feelings.
The list was endless. So is my devotion to rock music and pop rock.
These songs help me to express my feelings, they make me wind down,
and above all they make me feel good. Without this music, I might
have turned out to be a violent and grumpy person.
In the third, and final,
section of the essay the student closes his remarks with a series
of subjective, unsupported, even irrelevant statements:
Some of my favorite songs
are by Howard Jones, Pet Shop Boys, and Madonna. I especially like
songs that have a message in them, such as “Stand by Me”,
by Ben E. King. This song tells me to stand by the people I love
and to not question them in time of need. Basically this song is
telling me to believe in my friends, because they are my friends.
My favorite type of music
is rock and pop rock. Without them, there is no way that I could
survive mentally. They are with me in times of trouble, and best
of all, they are only a step away.
If this is reasoning,
it is very bad reasoning: “Believe in your friends because
they are your friends”, “If you feel you cannot survive
without rock music, then it follows that you can’t.”
Of course, a more appropriate interpretation of what is going on
is that the student is not reasoning at all but merely asserting
his subjective opinions.
Consider, the student
doesn’t examine alternative criteria on which to base an evaluation
of music. He doesn’t analyze rock music in the light of evaluative
criteria. He doesn’t provide evidence that clearly supports
his judgment. His writing is vague where it needs to be precise,
logically rambling where it needs to be critically reasoned. We
don’t really know what he means by songs “controlling”
We are not provided with
any evidence on the basis of which we could assess whether there
is any truth in his sweeping claims about himself, for example,
that he could not survive mentally without rock music. Indeed, common
sense experience strongly suggests, we believe, that the student
is simply deluding himself on this point, or, alternatively, engaging
in unbridled hyperbole.
When a blatantly weak
essay such as this is disseminated nationally as an example of “exceptional
achievement” in the writing of a reasoned evaluative essay,
then it is clear that there are large numbers of educators who are
not clear about the assessment of reasoning. Remember, the California
Assessment Program of the California State Department of Education
is the second largest assessment unit in the country. (I should
add that Dale Carlson, the head of CAP, is now putting a major effort
into rectifying this problem.)
THE MANY WAYS TEACHERS MIS-ASSESS REASONING
If many teachers take
bad reasoning to be good, do they also take good reasoning to be
bad? Unfortunately the answer appears to be, “Yes.”
This became apparent in a Center for Critical Thinking research
project in which teachers were provided with a well-reasoned response
to the California prompt, in addition to the poorly reasoned one.
The participants were teachers enrolled in critical thinking workshops.
They were given the two essays to assess after receiving a morning’s
instruction on critical thinking. What is significant is the myriad
of confusions and misunderstandings about the assessment of reasoning
that emerged and the inconsistencies in both grading and in justifying
Here is the “well-reasoned
response” they were asked to assess alongside the poorly reasoned
“Rock Around the Clock”.
This second essay (next
page) was written by one of the research staff members of the Center
who made sure that it was responsive to the directions and displayed
all of the critical thinking abilities called for:
- it distinguished
mere subjective preference from well-reasoned assessment,
- it was responsive
to the logic of the question at issue,
- it formulated and
discussed alternative relevant criteria,
- it distinguished
having evidence relevant to a question from lacking such evidence,
- it displayed intellectual
- it displayed intellectual
- it drew only those
conclusions the evidence warranted.
The results highlighted
the problem. On one occasion 81 teachers and administrators assessed
the two essays. The poorly reasoned essay was given an average score
of 5.4 (out of 8) while the well-reasoned essay was given an average
score of 3.9. Forty-nine of the teachers gave the poorly-reasoned
essay a 6, 7, or 8, while only 18 teachers gave the well-reasoned
essay a 6, 7, or 8.
Even more illuminating
than the raw scores were the reasons given by the teachers and administrators.
Multiple confusions surfaced, as I suggested above, about the nature
of reasoning and the appropriate way to assess it. Let’s look
at some of the responses. Try to imagine students actually receiving
these grades along with the often mistaken, confused, or unintelligible
I have divided teacher
assessments for convenience into two groups. The first consists
of those teachers who grade the poorly reasoned essay higher than
the well-reasoned essay. The second consists of those teachers who
grade the poorly reasoned essay lower than the well-reasoned essay.
Reading the teachers’ justifications for their grades reveals
a great deal of misunderstanding of the nature of reasoning. [First
Essay: “Rock Around the Clock” (the poorly reasoned
essay) Second Essay: “Can I Prove Rock Music is Better?”
(the well-reasoned essay)]
FIRST GROUP OF TEACHERS
- The following teachers
give a high grade to the poorly reasoned essay and a low grade
to the well-reasoned essay. In virtually every case, the teachers
reveal no awareness of the importance of intellectual humility,
wherein one does not claim to justify a conclusion when one lacks
the evidence to do so; instead, one gives good reasons for suspending
- 1) A Physical Education
Teacher: [#1] “The first essay better fulfills the criteria
for the assignment because the writer justifies (his or her) preference
for a particular type of music. I think I would give it a 7 though
because it was kind of confusing how the writer got on the subject.
[#2] “The second essay did not justify a preference for
any particular type of music. So the writer did not meet the criteria
for the assignment. Strangely enough it was easier to read but
possibly because the way the writer feels is how I feel about
music in general. I think the essay deserves a ‘0’.”
- An English Teacher:
[#1] “I would give this essay a 7 because he/she gave experience
from his/her life to support their opinion — gave reasons
and evidence by example.
[#2] “I would give this essay a grade of 2 because he/she
did not prove a point — merely rambled from one thing to
another searching for a reason.”
- A Math Teacher: [#1]
“I would give the first essay a 5 because it did not support
the judgment well but did make many references.
[#2] “I would give the second essay a 3 because it is not
very evaluative! It did analyze the subject but provided no real
support of any judgment.”
- A Math Teacher: [#1]
“I would give this paper a grade of 7 because criteria were
evident, analysis was good and it had lots of supporting evidence.
[#2] “I would give this paper a 3 because criteria are given
but nothing was analyzed and no supporting evidence.”
- Freshman Studies
Teacher: [#1] “I would give ‘Rock Around the Clock’
a grade of 6 because: a) a more flowing style of writing than
a series of loosely related points, b) a personal approach, c)
specific information as to records and effects of the songs, d)
valid and accurate comparisons, e) personalization, f) availability,
g) a well-supported point of view, and h) R&R as an avoidance
[#2] “I would give ‘Can I Prove Rock Music is Better?’
a 3 because
a) statement of problem OK, b) no exploration about ‘Why
we like it’,
c) discusses what it is about, not why we listen. Do we listen
to the words or music?, d) the idea of ‘better performances’
not followed through on, and e) How do they know they are like
- A Math Teacher: [#1]
“The first essay: grade 6. The writer has set up some criteria
for his choice, the music gives him a calming influence.... Since
the writer is given the opportunity to set his own criteria, this
will suffice. He gives examples to justify his conclusions.
[#2] “The second essay: grade 3. An attempt is made to give
reasons for supporting the music but no conclusions are made.
The writer cannot make an argument for his case in any area. It
is difficult, as the writer has said, to justify choice or preference,
but since one can choose one’s own criteria it would seem
any position well-argued and justified would fulfill the assignment.
The author did not succeed in doing that.”
- Subject Taught Not
Identified: [#1] “‘Rock Around the Clock’ Score:
6. This student does not give any clear criteria to start off
as to possible criteria to base their evaluation on. This student
based their evaluation on how it made them feel or respond. It
was based on reactions — not facts to choose music by, but
at least this student used something to justify their preference.
[#2] “‘Can I Prove Rock Music Is Better?’ Score:
2 Too vague — never really makes a decision about their
preference of music. This student talks about possible criteria
but never really says anything about it. Shows no support to justify
- Former English Teacher:
[#1] “I would give this essay a grade of 8 because: a) essay
cites specific examples, b) catchy opening, c) the criteria used
was based on student’s personal experience, d) student was
asked to justify their preference. I think she did.
[#2] “I would give this essay a grade of 2 because: a) very
b) few, if any, concrete examples, c) essay is not personalized
to any extent, d) no specific conclusions drawn.”
- Special Ed. Teacher:
[#1] “Point total: 7. This essay listed three criteria on
which to base a judgment. It gave examples of each — maybe
better examples could be found. The writer attempted to analyze
a basically subjective issue in concrete terms — what the
songs do for them: not objective, but a fairly concrete assessment
of music’s subjectivity.
[#2] “Point total: 0. This essay did not seriously attempt
to answer the issue at hand. Instead it concluded, quite lamely,
that no objective statement of worth could be made. While this
may be accurate in the broadest sense, no effort was made to justify
- English Teacher:
[#1] “I would give this essay a 7 because the author is
not afraid to take a stand. Although the ‘proof’ is
emotionally based, that was the direction of his/her argument.
[#2] “I would give this essay a 3 because the writer was
not able to take a position. He/she beats around the bush and
asks the reader to make the decision when that was the assignment
to the writer. The insecurity and negative attitude runs through
the entire paper.”
SECOND GROUP OF TEACHERS
The following teachers
give a low grade to the poorly reasoned essay and a high or higher
grade to the well-reasoned essay. In some cases the teachers revealed
some awareness of the importance of intellectual humility. Some
are, however, confused or mistaken in part about reasoning and its
assessment. For most, thankfully, this confusion is conjoined with
some insight into reasoning. For some few others, the fact that
they graded the poorly-reasoned essay lower is not based on insight
but chance. This is apparent from some of the reasons they give.
- A Library-Media Teacher:
[#1] “Grade: 3 or 4. Reasons: My first thought that it wasn’t
a typical essay but rather starts out with a rather clever, attention-getting
device. In that sense, the student did catch my attention —
and also confused me somewhat. That is, it doesn’t start
out as a typical essay. The student is a good writer in that their
word choices make sense and there are supporting reasons for why
they chose rock music and pop music.... Now that I read this again,
I can see that really the writer has only supplied one reason
for their selection: the control/expression of feelings. Well,
it’s the same old problem in grading a paper, i.e., the
student writes well but hasn’t followed the criteria strictly.
[#2] “Grade: 7. Reasons: Just a first critical response
before I re-read it. It strikes me as thoughtful and honest (which
always impresses me). Now I’ll see how it fits the criteria.
The writer states he needs good reasons for his judgment. I don’t
think that ‘good’ is the word he wants.... Why do
we like what we like? That’s a provocative question!...
A quickie, yes, I think they’ve fulfilled most of the criteria,
just not in the usual fashion. Also, it’s an essay (as I
- A Special Ed. Teacher:
[#1] “The student in this essay never really makes a statement
that involves an evaluation of a judgment made concerning a type
of music, except to say ‘My favorite type of music is rock
and pop rock. Without them there is no way I could survive mentally.’
He does try to show what he means by this statement when he offers
examples of music that affect his mood. He lacks a clear evaluation
or supportive evidence toward the topic. I think his statement
about surviving mentally is a bit much. I give it a 4.
[#2] “This student doesn’t know what he thinks and
he lets you know it continually. His closing paragraph summarizes
what he is trying to put down in the essay and it is the most
straightforward part of the essay. His title doesn’t quite
jibe with the rest of the essay. He was supposed to prove rock
music is better, but what he really talked about was whether there
was any justification for why people like rock music. I give it
- A Social Studies
Teacher: [#1] “I would give essay one a grade of 6. Essay
number one lists reasons for liking rock music, but it is very
superficial in analyzing them in the light of the criteria. It
really does not approach the subject in a way that logically lists
possible criteria as a basis for analysis and then applies the
criteria to the music. The essay is generally Bull Shit with only
a general connection to the instructions.
[#2] “I would give essay #2 an 8 because the possible criteria
for analyzing the issue are covered....”
- An English Teacher:
[#1] “Score: 3. The writer in essay one has discussed how
he/she feels about rock and pop music, but generalities are given
and his/her statements aren’t supported with evidence. The
assignment is to ‘justify’ preference, not discuss
that it makes him/her ‘feel good’ period. No criteria
have been established, so the essay just rambles on about ‘feelings’
and not much else. Reasons and evidence are lacking.
[#2] “Score: 5. This essay does a little bit better in attempting
an argument. The essay establishes two ‘criteria’
on which to base his/her essay.... Examples of ‘answers’
in paragraph 3 are needed as evidence.... Paragraph 4 isn’t
developed. Needs reasons and evidence/ examples. Weak Conclusion.”
- A Physical Education
Teacher: [#1] “I would grade the essay 0. The essay does
not show their judgment about worth with reason and evidence as
asked in the directions. There are no criteria for evaluation,
analysis with criteria or evidence that clearly supports the judgments.
[#2] “I would grade the essay 5. The essay attempts to set
up criteria for evaluation, yet not as completely as it could
have been done. There was an attempt to analyze the subject with
the criteria, but not complete. There was no evidence to clearly
support the judgment.”
- A Second Grade Teacher:
[#1] “The first essay should have a 3 because the stated
criterion is subjective. The conclusion comes down to, ‘I
like it because I like it.’
[#2] “The second essay would have a 6 because there was
a search for good criteria and no evidence was found to support
the good criteria.”
- A Counselor: [#1]
“I would give this essay a 1 because the student did select
a topic to evaluate which fit the directions. However, she reported
her subjective taste (how some songs have affected her, which
songs she likes) rather than evaluating ‘rock music’.
[#2] “I would give this essay a 7 because: a) she selects
an appropriate topic, 2) she considered what criteria would be
appropriate to evaluate rock music, c) she made judgments based
on the criteria she listed, 4) her conclusion was based on her
criteria/judgment. However, she might have considered/used other
- A Sixth Grade Language
Arts Teacher: [#1] “A grade of 1. There was no evaluation,
went strictly by senses.
[#2] “A grade of 8. The writer did a good job on a subject
that is a matter of preference no matter how you look at it! He
tried to objectively judge rock music, but in the end... ‘We
like it just because we like it.’”
- A First Grade Teacher:
[#1] “I would give ‘Rock Around the Clock’ a
4 because the writer did give some facts for liking rock music
but wrote mostly from emotion without questioning if her facts
were sound. For example, ‘believe in my friends because
they are my friends’.
[#2] “I would give ‘Can I Prove Rock Music is Better?’
a 7. The writer stated the purpose, criteria, facts, and gave
a conclusion. The writer considered more than just feeling. More
facts for liking rock music are needed.”
to the Analysis and Evaluation of Reasoning
There are two obstacles
that stand in the way of fostering sound reasoning K–12: 1)
teachers must learn how to devise assignments that require reasoning,
and 2) teachers must learn how to analyze and evaluate reasoning
objectively. This process will not happen overnight, but the sooner
it begins, the sooner it can be achieved.
We will shortly take a look at three assignments that call for reasoning
as well as at three examples of student work for each of those assignments:
student work with no reasoning in it, student work with poor reasoning
in it, and student work with good reasoning in it. In each case,
we will provide a brief commentary to help make clear what one should
look for in the reasoning. But first we will provide a brief overview
of what is involved, in general, in the analysis and evaluation
WHAT IS INVOLVED IN ANALYZING AND EVALUATING REASONING?
The fundamental criterion
to use in analyzing and evaluating reasoning comes from an analysis
of the purpose of the reasoner and the logic of the question or
questions raised. For example, if a person raises the question,
say, as to whether democracy is failing in the USA (in the light
of the dwindling number of people who vote and the growing power
of vested interest groups with significant money to expend on campaign
contributions), we can establish general criteria for assessing
the reasoning by spelling out what in general one would have to
do to settle the question. Those criteria would include such matters
as the following:
- An Analysis of the
Concept of the Ends of Democracy. What would it be for democracy
to succeed? What would it be for it to fail? What do we take the
fundamental objective of democracy to be? For democracy to succeed
is it enough that it simply ensures the right of the people at
large to vote or must it also serve the well being of the people
- Collection of the
Facts About the Numbers of People Not Voting. What is the actual
number of people not voting? Is it growing? By what percentage?
- An Interpretation
of the Significance of the Facts Collected in #2. What are the
reasons why growing numbers of people are not voting? What are
the implications of those facts?
- Collection of Facts
About the Number of Vested Interest Groups Influencing Elections.
How many vested interested groups are influencing elections today
in comparison to the past? What is the nature and extent of their
influence in money spent?
- An Interpretation
of the Significance of the Facts Collected in #4. What is the
significance of the growing influence of vested interest groups
on election outcomes? What is gained and lost by means of that
- Synthesis of Numbers
1 through 5. What is the overall significance of what we have
found out in 1 through 5? What does it all add up to? What exactly
are we gaining and losing as a result of the growing influence
of vested interest groups and diminished numbers of voters? In
attempting to put everything together we would want to see reflection
on this issue from more than one point of view. We would want
to assess how the reasoner responds to reasonable objections from
other points of view.
These are some of the
considerations relevant to reasoning well about the issue. A rational
analysis of someone’s response to this issue would involve,
then, checking to see if the above considerations were reasonably
addressed, to see if the reasoner had done a plausible job in analyzing
the functions of democracy, collecting relevant facts and information,
interpreting those facts, and putting everything together, with
a sensitivity to more than one point of view, into one coherent
line of reasoning.
Many of the teachers
assessing the reasoning of the essays on rock music above failed
to analyze or review the logic of the question at issue. Instead
they read the essays impressionistically, allowing the grade they
gave to be determined more by whether their impressions were positive
or negative than by any close analysis of the degree to which the
student responded adequately to the demands inherent in the precise
question at issue.
It is the logic of the
question at issue that is the “system for thinking”
that should guide our reasoning. If we do not develop skill in explicating
that logic, our reasoning is apt to become impressionistic, guided
by our prejudices and biases, by our egocentrism and ethnocentrism,
rather than disciplined by rational considerations.
Examples of Student Reasoning
What follows below are
three assignments designed to call for reasoning on the part of
the students, along with three examples of student “reasoning”
in response to those assignments. Two of the assignments are in
history and the other in literature. The three issues the students
are asked to develop their reasoning on involve: reasoning about
the character of the American people, reasoning about the meaning
of a poem, and reasoning about the comparative importance of inventions.
It would be useful if you thought a little about your own assessment
of the students’ reasoning before you looked at ours’.
You could then compare the two.
AMERICAN HISTORY: REASONING ABOUT THE AMERICAN CHARACTER
Question at Issue: “Are
the Americans you know capable of the kind of mass hysteria which
occurred in 1919 and is described in a textbook as the ‘Red
One of the most important reasons to write our history is to discover
who we are and who we are not, how we can develop ourselves, what
faults we have to watch out for, and what strengths we can build
upon. Read the passage in your textbook on the “Red Scare”.
Then write a couple of paragraphs in which you try to figure out
whether the Americans you know are “capable” or “not
capable” of reacting as many Americans did in 1919. (See textbook,
p. 731.) Be sure you show us your reasoning. Support and explain
why you think as you do.
I know are not like the people who lived in 1919. They obey the
law and, though they might make some mistakes or do some things
they ought not to, they would never hurt someone who was innocent.
Most of the people I know go to church and believe in God. They
are good Christians. They read the Bible. They try to raise their
children to be good and avoid evil. They are kind people. So I don’t
believe that what happened in 1919 could ever happen again. It won’t
happen in my neighborhood.
on the Student’s Reasoning
There is very little reasoning in this student’s work and,
on the whole, what there is seems uncritical and self-serving: in
essence, “My friends are good. Therefore they wouldn’t
do anything bad.” There are obvious objections to this reasoning.
Presumably, most of the people in 1919 also went to church and believed
in God. Presumably, they too would have thought themselves to be
good Christians. Presumably, their friends thought of them as kind
and as trying to raise their children to be good and to avoid evil.
As a result, the student has not really responded to the logic of
the question which implicitly requires that we think about mass
hysteria, how it occurs, and how it influences otherwise morally
sensitive people to behave in a morally insensitive way.
Certainly there are always people who go overboard.
That is human nature. And it is unreasonable to think that we will
ever abandon human nature. The American people rightly recognized
the threat that communism posed to our way of life and fought against
it. After all, if we had defeated it then we would not have to have
fought the Cold War and spent so much money and resources to defeat
the communists after WW II. So what is the lesson? Watch out for
human nature. Don’t go overboard. But on the other hand, don’t
forget who your enemies are and don’t give up the fight against
them just because some people punish them too severely or go to
on the Student’s Reasoning
There is more reasoning in this student’s work, but still
not very good reasoning: in essence, “It is human nature for
some people to lose control. So (by implication) some of us might
do so, but whether or not some of us might act as some people in
1919 did, the people in 1919 were right to fight against communists”.
This reasoning is weak because it largely ignores the issue raised.
The question at issue is not whether it was right for the people
in 1919 to oppose communism, such as it was, in the USA at the time.
The question is rather how it came to pass that, as we expressed
above, otherwise morally sensitive people came to behave in a morally
insensitive way. The student didn’t take this question seriously.
hard to answer the question as to what anyone is capable of. Perhaps
what we are capable of is largely a result of the circumstances
we are under. If we assume that all humans share human nature and
that because of human nature we are capable of acting out of intense
fear or insecurity or hate, then a lot depends upon whether something
or someone is able to stir those things up in us. Perhaps, of course,
there is a way to raise people so that they have so much good character
that even when someone tries to stir up the “worst”
in them, they do not give in, they resist the temptation to let
their worst side take control of them. The question could then be
asked whether I and my friends and neighbors are in the first or
the second group. Since we have never been “tested”
in a crisis situation, since we have never felt deeply threatened,
I don’t think I can honesty say we would pass the test.
I don’t know whether
we would act like a “Charles Evans Hughes” or a “Billy
Sunday”. It’s a scary thought.
on the Student’s Reasoning
This is better reasoning than in either of the two passages above:
in essence, “Everyone has a worse and a better side. Everyone’s
worse side can be appealed to. Whether you have the “character”
to withstand an appeal to your worse cannot be known until you are
“tested”. My friends and I have not been tested. Therefore,
we cannot know whether we have the character to withstand such an
appeal. Therefore, we don’t know whether we would or would
not act as many did in 1919.”
ENGLISH: INTERPRETING POEMS
Issue: What is John Donne saying in his poem “Death
Be Not Proud”?
Directions: Carefully read the poem below, trying
to figure out what the poet is saying. Be careful to explain what
your interpretation is and what exactly it is based on. Show us
your reasoning. Make sure your interpretation is consistent with
(all of) what the poem says.
Be Not Proud
(John Donne 1572–1631)
not proud, though some have called thee
Mighty and dreadful, for, thou art not soe,
For, those, whom thou think’st, thou dost overthrow,
Die not, poore death, nor yet canst thou kill mee.
Much pleasure, then from thee, much more must flow,
And soonest our best men with thee doe goe,
Rest of their bones, and soules deliverie.
Thou art slave to Fate, Chance, kings, and desperate men,
And dost with poyson, warre, and sicknesse dwell,
And poppie, or charmes can make us sleepe as well,
And better then thy stroake; why swell’st thou then?
One short sleepe past, wee wake eternally,
And death shall be no more; death, thou shalt die.
I don’t like this poem. It is boring and confusing. The guy
does not spell correctly. He talks a lot about death but he does
not say anything. I don’t see why he thinks death is mighty
or why he thinks it can’t kill him. He says a lot of confusing
things. At one time he says it gives pleasure and then talks about
bones resting, which makes no sense. Then he talks about flowers
and sleeping. Finally he says that death shall be no more and that
it shall die. I don’t get it. Why doesn’t he just say
what he wants to say? This is a terrible poem. Why do we have to
read such stupid stuff?
on the Student’s Reasoning
This student provides us with virtually no reasoning at all. Rather
than attempt to figure out what the poet is saying by closely reading
what is said, the student rejects the poem, dismisses it emotionally.
The result is that the student flagrantly mis-reads the poem and
blames his mis-reading on the poem itself and the poet. The student
needs to be introduced to the concept of critical reading in which
the reader uses the text as evidence to use in interpreting the
Mr. Donne says that death should not be proud. It is not mighty
or dreadful. He says this because death is like sleep and when you
go to sleep you rest. Therefore, because it is restful even the
best people sleep, even slaves. And sleeping is better than being
poisoned or being sick. Finally, he says that we only sleep a while
and then we awake. And then death is gone. In fact, it is dead.
He thinks this is good.
on the Student’s Reasoning
There is more reasoning in this student’s work but most of
it ignores the evidence of what the poem says. The poem does not
say or imply, for example, that “because it [death] is restful
even the best people sleep, even slaves”. The poem does not
say or imply that “sleeping is better than being poisoned
or being sick”. Finally, it is clear that the student is not
getting the major point of the poem, namely, that because of the
promised resurrection, last judgment, and eternal life in heaven
or hell, there is a sense in which “death” is not real
and lasting, but only something that will “die”. Like
the first student, this student also needs to be introduced to the
concept of critical reading in which the reader uses the text as
evidence in interpreting meaning.
It is clear that Donne believes in God or at least in an afterlife.
This is implied in the first four lines which I interpret as saying
something like this: “Don’t think you’re so powerful
because no one really dies but only appears to die” (People
who “die” are really just awaiting their resurrection).
This interpretation is supported in the next line which implies
that what we call death is really a kind of “sleepe”
and is not, therefore, very bad. In fact, as he says sleep often
gives us “pleasure”. The next lines make a different
kind of point but still are a criticism of the view that death is
“mighty” and “dreadful”. Death, he says,
is not able to control “Fate, Chance, kings, and desperate
men”. Furthermore, not only is it not able to control these
other forces, it can’t even get away from such unpleasant
associates as “poyson, warre, and sicknesse”. Finally,
he reasons, narcotics makes us sleep as well as death does and when
everyone is resurrected for final judgment (which I infer is what
he means) then death itself will be gone forever, and therefore
on the Student’s Reasoning
Finally, we have a student who illustrates the process of critical
reading, carefully reasoning her way through the poem, using the
words of the poem to carefully back up her interpretation.
HISTORY: REASONING ABOUT THE SIGNIFICANCE OF INVENTIONS
Question at Issue:
“Of two inventions discussed in your textbook, which was the
most important and why?”
The textbook for the course describes a number of important inventions,
including those of Gutenberg, Edison, and George Washington Carver.
Take two inventions, either from those mentioned in the book or
some other inventions you know of, and compare their importance.
Defend your answer by giving reasons in favor of your judgment.
An invention that is very important is the printing
press. Johann Gutenberg, who was a man that lived in Germany, invented
it. He invented the printing press in the Fifteenth Century. The
first book ever printed by Gutenberg was the Bible. But he soon
printed many other books as well. The first printing press worked
by using movable type.
Another important invention
mentioned in the textbook was the dehydration of foods. This was
invented by George Washington Carver. When you dehydrate foods you
take the water out of them. George Washington Carver wanted many
people to use his inventions, so he did not take out any patents
on them. He made many other inventions besides dehydration. He even
thought of more than 300 uses for the peanut, including facial cream,
shoe polish, and ice cream.
Both inventions are very
important. Many people read books that are printed on a printing
press. Many people eat food that has been dehydrated. But to me
the printing press was more important than dehydration.
on the Student’s Reasoning
The student does not provide any reasoning to support
his conclusion. He discusses no criteria for assessing inventions
for their importance, nor any evidence to support one or the other
with respect to those criteria. Most of the factual detail is irrelevant
to the issue.
The first sound I hear
in the morning is my alarm clock going off. It’s an invention
I truly hate.
R-r-r-r-ring.It is not
a pretty sound, and as soon as I hear it I feel myself getting angry.
If only I didn’t have to get up so early! All my muscles cry
out that I want to sleep! Most mornings when I hear that sound,
I even cover my ears with my pillow in the hope that I won’t
hear it going off.
It is an old-fashioned
wind-up alarm clock that loses ten minutes a day. It is not a digital
alarm clock because all the digital alarm clocks I’ve ever
tried have alarms that are too soft to awaken a really sound sleeper.
And believe me I am a very sound sleeper.
R-r-r-r-ring. But no
matter what I do, or how I feel, I end up wide-awake and out of
bed and getting dressed for school.
Once I am awake I look
at my other clock, the one that is hanging on the wall over my dresser.
It is a great invention too. It’s a digital clock that keeps
perfect time. It has a red LED display and it glows in the dark.
It has an emergency battery backup, so that even if the electricity
cuts out in the night, my wall clock never loses a second.
Which of the two inventions
is more important? That’s the question I ask myself as I head
off for school. And then the answer comes to me. No matter how perfectly
the digital wall clock keeps time, without the alarm clock I wouldn’t
be awake to see it. So without doubt the alarm clock wins the prize
as most important.
on the Student’s Reasoning
The student provides some reasoning but when considered closely
it is apparent that the reasoning is absurd. The notion that without
the alarm clock people would never wake up is ridiculous. What does
this student think happened before the alarm clock was invented?
Furthermore, does she really think that loud alarms cannot be built
into digital clocks? Once again, the student has not learned to
think about the logic of the question at issue. Therefore, the student
gives no time to reflecting on the general criteria by means of
which we might assess the social worth of inventions by relating
that worth to the most basic human values, like the preservation
of life, the minimization of pain and suffering, the development
of a more just society, and so forth. It is only in terms of the
concepts of basic human values that criteria can be generated that
give a solid logic to the question and hence a means to assess the
reasoning which purports to settle the question.
Two inventions mentioned in the book are television and the dehydration
of food. Each is important in different ways. The television set,
for example, affects many people’s lives. I watch television
almost every night and so do all of my friends. But it’s not
just me and my friends. The same is true for people all across the
country, and in most foreign countries as well. Television allows
more people to be entertained than was ever possible before. We
witness world news, nature programs, comedies and many other programs.
Television lets us see much of what is going on in the world.
Dehydration of foods
is important in a very different way. The main effects of dehydration
are that it allows food to be kept for a long time without spoiling,
and to be shipped for a lower cost. I don’t know how many
people in the world today use dehydrated foods, but I’m pretty
sure that it’s far smaller than the number of people who enjoy
TV. So that seems to show that TV is more important.
And yet I don’t feel right saying that one invention is more
important than another simply because it has affected more people.
If dehydration is used more than it is now, it could help cut down
on the number of people who are starving in the world. Saving just
a few people from dying of starvation is more important than taking
a lot of people and entertaining them.
Commentary on the Student’s Reasoning
The student provides
some reasoning, which might at first appear absurd, but on reflection
makes good sense. This student is thinking about the logic of the
question at issue and hence is reflecting on the general criteria
by means of which we might assess the social worth of inventions
by relating that worth to the most basic human values: like the
quality or preservation of life, the minimization of pain and suffering,
the development of a more just society, and so forth. To say that
this student’s reasoning is better than the first two students
— because she does respond to the logic of the question at
issue — does not mean that her reasoning is perfect, for perhaps
there are yet further considerations that might be mentioned about
the effects of television which might persuade us that television
itself is making so large a contribution to the quality or preservation
of human life that it is indeed more important than food dehydration.
We may know the basic logic of a question without knowing whether
we yet have the best answer to that question, the answer that best
fulfills its logic.
The whole of this book is concerned with the process of developing
students who reason through what they are learning so as to grasp
the logic of it, students who know clearly the difference between
coming to terms with the logic of something and merely rotely memorizing
it. But reasoning is not a matter to be learned once and for all.
It is a matter of life-long learning, a matter of bringing insightful
mindfulness into the fabric of our thinking and our action. For
the teacher, it is a matter of learning how to design instruction
so that students take command of the logic of their own thinking
while they are thinking and through that insightful grasp, improve
We figure things out
better if we can monitor what we are doing, intellectually, in trying
to figure them out, so that we go beyond simply using logical structures,
so that we go beyond simply making logical moves, so that we start
to intentionally, deliberately, and willfully examine and take apart
the logical structures we are using, so that we designedly, purposively,
and alertly assess our use of the structures in everyday situations,
and, of course, so that we do these things well: clearly, accurately,
To understand logical
structures is to integrate them, to establish logical connections
between them, to make it possible for the mind to make an extended
series of nuanced inferences, deductions, and derivations. “This
is so, therefore that also is so, and that, and that.” The
logical structures implicit in an educated person’s mind are
highly systematized. The well-educated person is able to reason
quite directly and deliberately, to begin somewhere, know where
one is beginning, and then reason with awareness from that point
to other points, all with a given question in mind, with specific
evidence in mind, with specific reasons to advance, with specific
conclusions to support, with consciousness of one’s point
of view and of contrasting points of view. The good reasoner is
always reasoning within a system that disciplines and restrains
When the logical structures
by which a mind figures out the world are confused, a jumble, a
hodgepodge, a mere conglomeration, then that figuring out is radically
defective, typically in any of a variety of ways: incomplete, inaccurate,
distorted, muddled, inexact, superficial, rigid, inconsistent, and
unproductive. Then the mind begins it knows not where, takes things
for granted without analysis or questioning, leaps to conclusions
without sufficient evidence to back them up, meanders without a
consciousness of its point of view or of alternative points of view.
Then the mind wanders into its own prejudices and biases, its own
egocentricity and socio-centricity. Then the mind is not able to
discipline itself by a close analysis of the question at issue and
ignores the demands that the logic of that question puts on it and
us as rational, logic-creating, logic-using animals.
Paul, R. (1993). Critical Thinking: What Every Student Needs to
Survive in A Rapidly Changing World, Dillon Beach, CA: Foundation
For Critical Thinking).