Subject: Re: Eight Features of an Ideal Intro Stat Course
(Response to comments by Jan de Leeuw)
To: EdStat-L and sci.stat.edu
From: Donald B. Macnaughton <donmac@matstat.com>
Date: Sunday October 31, 1999
Cc: Jan de Leeuw <deleeuw@stat.ucla.edu>
Quoting a 99/5/2 post of mine, Jan de Leeuw writes (on 99/5/6)
> This is a very interesting discussion and Don's position is, as
> always, well-argued and convincing. But
>
> -- "the external world" is ill-defined. How about introspec-
> tion ? How about flying saucers ? Or, to put it differently,
> what ARE data about a non-external world ?
These are excellent questions. I begin with a definition:
The EXTERNAL WORLD is what is "out there" -- what we see
when we look out the windows in our heads and what we
sense through our other senses.
I use the concept of 'external world' as a way of indicating the
broadness of the applications of statistics -- we can use the
field of statistics almost anywhere "external" to ourselves. (To
use statistics in some area we need only find reasonable ways to
measure the relevant variables in the area.)
However, until I received Jan's post I believed that we could not
obtain reliable measurements of variables in the INTERNAL world
because that world is often (as a fact of human nature) ruled by
prejudices -- it is subjective. Thus I concluded that we could
NOT use the field of statistics in the world of our inner experi-
ence.
However, after considering Jan's question about introspection, I
now think my conclusion may be incorrect. It is indeed an
EMPIRICAL question whether the internal world of one's own per-
sonal experience can also be subjected to statistical analysis,
just like the external world. I will be pleased if researchers
show that the field of statistics is also applicable in the in-
ternal world of one's private experience.
* * *
In asking about flying saucers, I think Jan is wondering about
things that do not "really" exist. That is, I think he probably
believes (as I believe) that flying saucers (vehicles used by ex-
traterrestrials visiting the earth) do not really exist (except
in people's imaginations). If Jan is asking how such things re-
late to the external world I refer to, my answer is that this is
a matter of choosing a point of view. I take the point of view
that things (entities) that do not exist (like flying saucers)
are still a part of the external world, because people talk about
such things. The fact that these things are (or may be) imagi-
nary need not somehow remove them (in their imaginary existence)
from being a part of external reality.
* * *
In my 99/5/2 post, my main reference to the concept of 'external
world' was in the definition I proposed of "empirical research",
which I gave as follows:
EMPIRICAL RESEARCH is any research in which data are
gathered from the external world and then conclusions are
drawn from the data about the external world.
Jan's questions have shown me that the concept of 'external
world' is an unnecessary restriction in the definition. Thus I
now propose the following new definition:
EMPIRICAL RESEARCH is any research in which data are
gathered from some area of experience and then conclu-
sions are drawn from the data about the area of experi-
ence.
The new definition encompasses both the external and internal
worlds and is thus broader than the original definition. I thank
Jan for motivating the new definition.
* * *
Note that the new definition does not explicitly refer to the
concept of 'data analysis'. However, this concept is certainly
implicit in the definition because the conclusions we draw from
data in empirical research can only (sensibly) be drawn on the
basis of some form of examination or "analysis" of the data.
Diagrammatically
empirical research = data gathering, which is followed by
data analysis, which is followed by
conclusion drawing.
I omit a direct reference to the concept of 'data analysis' in
the definition because I wish to emphasize the goal of empirical
research, which is not data analysis, but is instead (correct)
conclusion drawing.
(I discuss three types of conclusion drawing in the application
of the field of statistics in appendix A.
> -- I suggest "Statistics develops and evaluates techniques for
> data analysis".
Although Jan does not say so explicitly, I believe he is propos-
ing a definition of the role of the field of statistics. Jan's
definition is worth setting apart:
Statistics develops and evaluates techniques for data
analysis.
Contrast Jan's definition of the role of statistics with mine,
which is
Statistics provides efficient general techniques to help
empirical researchers study variables and relationships
between variables, mainly as a means to predict and con-
trol the values of variables.
Note that the concept of 'data analysis', which is central
in Jan's definition, is implicit in my definition because (as I
discuss above) the concept is implicit in my prior definition of
empirical research.
I believe that Jan's definition and my definition of the role of
statistics are mutually fully consistent. However, my definition
is both more explicit and more limiting. That is, Jan's defini-
tion says that statistics supports "data analysis". My defini-
tion also says (through my definition of "empirical research")
that statistics supports "data analysis", but it also says where,
how, and why statistics supports data analysis.
The definition says WHERE statistics supports data analysis -- in
empirical research. I believe we should emphasize where statis-
tics supports data analysis because people sometimes forget that
the field of statistics is tightly tied (both generally and spe-
cifically) to empirical research projects -- for most statisti-
cians the field of statistics is not an end in itself.
The definition says HOW statistics supports data analysis -- by
providing efficient general techniques for studying variables and
relationships between variables.
The definition says WHY statistics supports data analysis --
(mainly) to enable us (i.e., society) to accurately predict and
control the values of variables.
The "where", "how", and "why" ideas in my definition impose extra
limitations on the role of statistics that are not present in
Jan's definition. However, many readers will agree that these
limitations are not unduly limiting because statistics does not
(in a practical sense) do much beyond providing techniques for
studying variables and relationships between or among variables
in empirical research, and such study is mainly useful as a means
to accurately predict and control the values of variables.
(One can also reasonably argue that, in addition to providing the
ability to predict and control, the field of statistics also en-
ables us to "explain" and "understand" the area of experience we
are studying. I view the relationship between the first pair of
concepts ['prediction' and 'control'] and the second ['explana-
tion' and 'understanding'] as follows: In almost every area of
serious human endeavor the most widely accepted TEST of a good
explanation or of good understanding is whether the explanation
or understanding leads to correct prediction or correct control.
Therefore, although explanation and understanding are important,
they are subordinate to prediction and control. I develop this
view in two earlier posts [1996a, 1996b]. I discuss the action-
prescriptive role of the field of statistics in appendix A.)
I include the "where", "how", and "why" ideas in the definition
because I believe these ideas deserve strong emphasis. I believe
that simply saying the role of statistics is to support "data
analysis" is too vague -- outsiders cannot easily see value in
doing "data analysis". Instead, to an outsider, doing data
analysis sounds like something medieval monks might do with great
rigor in an isolated monastery in the mountains, but with no
known practical value.
Most importantly, if we emphasize WHY statistics supports data
analysis (i.e., to enable accurate prediction and control), we
pique the curiosity of outsiders because almost everyone would
like an easy path to accurate prediction and control. Most stat-
isticians and many empirical researchers agree that proper use of
statistics is by far the easiest and best path to accurate pre-
diction and control in any area of empirical research.
> This [i.e., Jan's proposed definition of the role of statis-
> tics] avoids "empirical", "external", and "science", because
> defining these is metaphysics.
I believe Jan is here using the term "metaphysics" in the pejora-
tive sense of "building conceptual structures on slippery elusive
concepts at the bottom". I fully agree with the importance of
eliminating this type of metaphysics.
Jan identifies three possible slippery elusive concepts -- 'em-
pirical', 'external', and 'science'. Can we omit these three
concepts from the definition of the role of statistics?
Regarding the concept of 'empirical', recall that my definition
of the role of statistics refers to the concept of 'empirical re-
search'. Since the concept of 'empirical research' is central in
the present discussion, let us focus not on the concept of 'em-
pirical' but on the concept of 'empirical research'.
My definition above of "empirical research" states that empirical
research is data gathering, (data analysis,) and conclusion draw-
ing. The main concepts in this definition are the concept of
'data' and the concept of 'conclusion drawing'. To determine
whether the concept of 'empirical research' is slippery or elu-
sive let us consider its roots and ask whether the concepts of
'data' and 'conclusion drawing' are slippery or elusive.
Consider the concept of 'data' (which Jan uses directly and which
I use indirectly in defining the role of statistics). I define
"data" as "a set of measured values of properties of entities or,
equivalently, a set of values of variables". This concept ap-
pears to be a straightforward (not slippery or elusive) concept.
Consider the concept of 'conclusion-drawing'. The act of drawing
a conclusion is equivalent to making a (preferably correct)
statement about some area of experience. Thus this concept
(which I discuss in appendix A) also appears to be a straightfor-
ward concept.
In view of the preceding points, I suggest that the concept of
'empirical research' in my definition of the role of statistics
is not particularly slippery or elusive. And the term "empiri-
cal" is merely a useful modifier that conveys the important ideas
of 'data-based' and 'observation-based'.
Finally, as I discuss above, I believe it is helpful to include
the concept of 'empirical research' in the definition of the role
of statistics because I believe students understand our field
better if we emphasize where it plays its role.
* * *
Consideration of Jan's second concept ('external') has led me to
realize (as I discuss above) that it is unnecessary to refer to
the concept of the external world in the definition of "empirical
research". Thus I fully agree with Jan that this concept need
not be (explicitly or implicitly) present in the definition of
the role of statistics.
I also agree that we should not use Jan's third concept ('sci-
ence') in the definition of the role of statistics. The field of
statistics has many applications outside of what is traditionally
viewed as "science", such as in business, industry, law, and gov-
ernment.
> It [i.e., Jan's proposed definition of the role of statistics]
> also avoids "inference" and "modeling" and "quantitative", be-
> cause these are irrelevant for the definition.
I believe we should include the concept of 'inference' in the
definition of the role of statistics. (I include the concept in
my definition of the role by including the phrase "conclusions
are drawn from the data" in my conceptually prior definition of
"empirical research".) We should include the concept of 'infer-
ence' in the definition because the data analysis we do in sta-
tistics is not an end in itself, but is instead a MEANS to draw-
ing valid conclusions (inferences) about the particular area of
experience we are using statistics to study.
> Inference is the responsibility of the scientist, statisticians
> just develop tools which may help the scientist doing the job.
I agree that the final responsibility for correct statistical in-
ference rests with the scientist or empirical researcher who uses
statistics. However, statisticians must be experts in the
GENERAL ideas of statistical inference so that we can teach users
of statistics to draw valid inferences from data. Thus inference
(in the GENERAL sense) is very much the responsibility of statis-
ticians.
I suspect Jan wrote his sentence above to be provocative. He may
agree that saying "statisticians just develop tools which may
help" minimizes the fundamental role our field plays (or can
play) across almost all empirical research. Such minimization
may suggest to outsiders that our field is obscure and uninter-
esting.
Instead of minimization, if statisticians politely trumpet the
vital role our field plays (in providing optimal methods for ac-
curate prediction and control), I believe we can lead the field
to receive the very broad recognition it deserves.
* * *
The second concept Jan says is irrelevant for the definition of
the role of statistics is the concept of 'modeling'. By "model-
ing" Jan may mean "building statistical model equations of rela-
tionships between variables". I agree that it is unnecessary to
refer to this sense of the word "modeling" in the definition of
the role of statistics.
On the other hand, by "modeling" Jan may mean "studying relation-
ships between variables" WITHOUT any reference to statistical
model equations. If Jan is using the word "modeling" in this
sense, he is suggesting that the idea of studying relationships
between variables is irrelevant for the definition of (the role
of) statistics.
My definition of the role of statistics explicitly refers to the
idea of studying relationships between variables. I believe this
idea is relevant for the definition because my experience sug-
gests that
- almost all empirical research projects can be usefully viewed
as studying relationships between or among variables and
- almost all the commonly used statistical procedures can be use-
fully unified as methods for studying relationships between or
among variables.
I discuss these points further in a paper (1999a).
* * *
The third concept Jan says is irrelevant for the definition of
the role of statistics is the concept of 'quantitative', and I
agree.
> Although, in the case of hypothesis testing for instance, they
> [statisticians] may have hindered more than they have helped.
I agree that there is much confusion about hypothesis testing --
so much confusion that some empirical researchers eschew hypothe-
sis testing and instead use confidence intervals to perform what
is usually the same function.
I suggest that many instances of hypothesis testing can be best
viewed as testing for evidence of the existence of a relationship
between variables (or testing for evidence of the existence of an
extension to an already known relationship between variables).
This point of view helps empirical researchers to better under-
stand the usefulness of hypothesis testing.
Testing hypotheses about the existence of relationships between
variables is important because (to avoid embarrassing and costly
errors) we must first verify that there is clear evidence of a
relationship between the relevant variables before attempting to
use information about a putative relationship between variables
for prediction or control. Computing p-values and verifying that
they are low enough is an objective and standardized aid in
checking for proper evidence of the existence of a relationship
between variables. I discuss these ideas further in two papers
(1997, sec. 9; 1998, sec. 5).
> Modeling (in statistics) is one way (and a controversial one at
> that) to develop data analysis techniques.
I agree. (I believe that most [all?] statistical models are
models of relationships between variables.)
> -- the distinction between univariate frequency distributions
> and "relations between variables" is better formulated, I
> think, as the distinction between univariate and multivariate
> frequency distributions or between unconditional and condi-
> tional distributions. This makes most of the distinction go
> away.
The two formulations Jan proposes are more elegant and more com-
pact than the formulation I propose. Jan's formulations may be
very useful in advanced statistics courses.
However, I am interested in finding the best formulation of the
concepts for the introductory statistics course for students who
are not majoring in statistics. Here I think Jan's formulations
are more complicated than necessary because they require that we
use the concept of 'distribution' (either 'multivariate distribu-
tion' or 'conditional distribution'), which is a difficult con-
cept for beginners to grasp.
Contrast Jan's formulations with the formulation I propose, which
makes no reference to the concept of 'distribution'. We can ini-
tially characterize relationships between variables in terms of
(the values of) one variable "depending" on (the values of) an-
other variable, or the values of one variable varying "in step"
with the values of another variable. (We can reinforce the char-
acterizations by carefully discussing illustrative practical ex-
amples of relationships.) Later, we can formally characterize
the concept in terms of conditional expected value (still without
having to appeal to the concept of 'distribution') as I discuss
in a paper and in an earlier post (1997, sec. 7.10, 1999b).
I suggest that if two or more consistent formulations are avail-
able upon which to build an approach to the introductory statis-
tics course, we should use the formulation that is easiest for
beginners to understand. I believe the formulation that distin-
guishes between univariate distributions and relationships be-
tween variables allows the teacher to introduce the concept of
'relationship between variables' very early in the introductory
course. I further believe that a careful discussion of this
easy-to-understand concept gives students a fundamental grasp of
the important practical value of the field of statistics (in fa-
cilitating accurate prediction and control). Thus I recommend
that teachers use the formulation that distinguishes between re-
lationships between variables and univariate distributions in-
stead of Jan's (correct, more elegant, but harder to understand)
formulations that make the distinction go away.
> The constructions that make studying a univariate distribution
> a limiting case of studying a relationship are not very con-
> vincing to me. Another construction (instead of zero predic-
> tors) is one predictor which only takes a single value. Dis-
> tribution of IQ in Arkansas is distribution of IQ in the 50
> states, conditional on Arkansas.
Here Jan is referring to the idea that univariate distributions
are simply a special (degenerate) case of relationships between
variables -- the case in which, as usual, there is one response
variable, but the number of predictor variables, instead of being
one or more, is reduced to zero. Jan's construction (with a sin-
gle constant-valued predictor variable) is another valid way of
showing that univariate distributions are (rigorously) a special
case of relationships between variables.
> -- It simply will not do to minimize the importance of univari-
> ate summaries. My guess is (unlike Don's) that 96% of the sta-
> tistics published (in newspapers, the gray literature, govern-
> ment reports) are univariate frequency distributions or their
> measures of central tendency and dispersion. Saying that this
> is not science will not make it go away.
To follow up on Jan's point I studied each article and letter in
the first section the May 3, 1999 issue of The Toronto Star news-
paper, and I identified all the statements or graphics that in
any way refer to the values of variables. I classified each of
these statements and graphics into one of the following four
categories:
1. Statements about the value of a variable in a single entity.
(For example, a statement might say a particular person is 36
years old, or a particular concert was attended by approxi-
mately 25,000 people, or a particular town is "small".)
2. Statements about a univariate distribution of the values of a
variable (including statements about the central tendency or
the dispersion of a univariate distribution). (For example, a
statement might say that the average age of members of a par-
ticular social group is 34 years.)
3. Statements about a relationship between variables. (For exam-
ple, a statement might say that the average temperature of the
earth has been increasing over the last 100 years.)
4. Statements that do not fit into any of the preceding three
categories.
Of the 1237 statements I found that refer to the values of vari-
ables 85% are statements of the value of a variable in a single
entity, 2% are statements of univariate distributions, and 13%
are statements of relationships between variables. (No state-
ments fall into the fourth category.) I further discuss my find-
ings in appendix B.
Jan estimates that 96% of the statistics published in newspapers
are reports of univariate distributions or their measures of cen-
tral tendency and dispersion. My informal survey of articles in
The Toronto Star suggests that Jan's estimate may be high.
However, my estimate in my 99/5/2 post that 96% of empirical re-
search projects study relationships between variables, and Jan's
estimate that 96% of statistics in newspapers, the gray litera-
ture, and government reports are related to univariate distribu-
tions, and my informal survey of The Toronto Star are NOT par-
ticularly reliable sources of information. It would be much bet-
ter if someone (an empirical researcher) could present properly-
sampled survey information about the relative frequency with
which various types of statistical statements appear in various
areas of interest. As I suggest in the 99/5/2 post, I hope an
interested reader will perform a proper survey.
Finally, to the extent that the general public PREFERS reports of
univariate distributions over reports of relationships between
variables, I believe we have failed as statistics teachers to
educate the general public. We have failed because a report of a
relationship between a response variable Y and one or more rele-
vant predictor variables is invariably more informative and more
useful than a report of the univariate distribution of Y in iso-
lation.
> Also remember that density estimation, robust estimation, etc.
> are important areas of statistical activity.
With respect to empirical density estimation, assuming we are
studying a UNIVARIATE density, this is clearly NOT (in my every-
day language) a study of a relationship between variables. I be-
lieve this activity is subsumed under the roughly 2% of empirical
research projects that study univariate distributions that I re-
fer to in the 99/5/2 post. Again though, this is an estimate on
my part, which should be corroborated or corrected through a
proper survey.
With respect to empirical MULTIVARIATE density estimation, in
this case two or more variables are always involved. Thus it is
easy to view this activity as simply another way of studying the
relationship between (or, better, among) these variables.
With respect to robust estimation, if a research project uses ro-
bust estimation to estimate the values of the parameters of a
univariate distribution, the research project is clearly studying
a univariate distribution and not studying a relationship between
variables. I believe this use of robust estimation is subsumed
under the 2% of empirical research projects that study univariate
distributions. On the other hand, if a research project uses ro-
bust estimation to estimate the parameters of a model, it is al-
most certainly studying a relationship between variables because
(as I suggest above) most (all?) statistical models are models of
relationships between variables.
> -- Emphasizing that statisticians develop and evaluate data
> analysis techniques is consistent with the Box idea that sta-
> tistics is a service activity, serving the sciences, but also
> policy, government, sports, the law, and the public.
I fully agree with Jan's statement of George Box's important idea
that statistics is a service activity serving the areas mentioned
(and other areas, such as business and industry).
Jan recommends emphasizing "data analysis techniques". However,
as I discuss above, I believe the concept of 'data analysis' is
cryptic and diffuse -- outsiders cannot easily see any practical
service or practical value in "data analysis".
In contrast, I recommend that teachers emphasize the easy-to-
understand unifying concept of (accurate) prediction and control
on the basis of relationships between variables. I believe that
emphasizing prediction and control
- impresses students with the significant practical value of our
field and
- makes our field substantially easier for students to under-
stand.
-------------------------------------------------------
Donald B. Macnaughton MatStat Research Consulting Inc
donmac@matstat.com Toronto, Canada
-------------------------------------------------------
APPENDIX A: THREE TYPES OF CONCLUSIONS IN STATISTICAL ANALYSIS
OF EMPIRICAL RESEARCH RESULTS
In the new definition I propose above of "empirical research" I
say that empirical research involves "drawing conclusions" from
data. The conclusions we draw from data in empirical research
can be broken into three types. We can draw
- conclusions about the values of variables or about the values
of parameters in entities of some type (and more generally con-
clusions about univariate distributions of the values of vari-
ables) or
- conclusions about relationships between or among variables in
entities of some type (and how the relationships can be used
for accurate prediction or control) or
- conclusions about the best course of action to take in some en-
deavor.
The first type of conclusion -- about the values of variables or
the values of parameters -- is the simplest. For example, if a
researcher has made several empirical estimates of the speed of
light, he or she can use statistical methods to analyze these es-
timates and determine a "best" estimate of the speed, and also
determine an estimate of the precision of the estimate.
The second type of conclusion -- about relationships between
variables -- is, I believe, the most important type. The field
of statistics can help researchers to determine whether a rela-
tionship exists between variables and, when a relationship is
found, can help to accurately predict and possibly control the
values of the response variable in new entities from the popula-
tion on the basis of the relationship.
The third type of conclusion -- about the best course of action
-- is clearly important because almost every individual and al-
most every organization is regularly faced with making important,
but difficult, conclusions or decisions about courses of action.
Thus the statistical methods for making conclusions or decisions
about the best course of action are an important subset of the
methods of statistics.
* * *
I place the three types of conclusions in two main groups. I
call the methods to make the first and second types of conclu-
sions the "descriptive" aspect of statistics because these con-
clusions DESCRIBE a state of the world. That is, they DESCRIBE
values of variables or parameters, or they DESCRIBE relationships
between variables, or they DESCRIBE methods for predicting and
controlling on the basis of such relationships.
I call the methods to make the third type of conclusion the
"action-prescriptive" aspect of statistics because these conclu-
sions prescribe a course of action.
Conclusions in the descriptive aspect of statistics often involve
statistical tests of hypotheses. For example, a researcher may
perform a research project and conclude from the data that a re-
lationship exists between one or more of the predictor variables
and the response variable. The researcher will typically support
this conclusion by reporting a test of the hypothesis that the
relationship exists and will note that the p-value yielded by the
test is low enough that we cannot reasonably explain away the
evidence of the relationship in the data as being merely due to
chance.
Conclusions in the action-prescriptive aspect of statistics are
usually based on conclusions provided by the descriptive aspect.
That is, most (all?) conclusions made using the action-prescrip-
tive methods of statistics are based on knowledge about the val-
ues of relevant variables and parameters or (more likely) are
based on knowledge about relevant relationships between vari-
ables. However, in addition, the action-prescriptive aspect of
statistics takes account of a set of VALUES and GOALS. (Here the
word "values" denotes principles or qualities that are socially
or commercially valuable as opposed to denoting the values of a
variable or parameter.) The values and goals are used together
with knowledge of variables and relationships between variables
to identify (possibly with the help of "objective functions" or
"utility functions") the conclusion (i.e., the decision about a
course of action) that optimizes the achievement of the values
and goals.
Clearly, descriptive conclusions and action-prescriptive conclu-
sions are made using different sets of underlying methods.
Therefore, in discussing the use of statistics for drawing con-
clusions or making decisions it is important to identify the type
of conclusions or decisions one is discussing.
* * *
As a practical matter, decision-makers often find it unnecessary
to use action-prescriptive concepts to make a decision about a
course of action on the basis of the results of an empirical re-
search project because the appropriate course of action is obvi-
ous. For example, if a medical research project shows that a
particular drug lessens the severity of a disease in people, the
action-prescriptive aspect of statistics cannot generally improve
on the recommendation that doctors should prescribe the lowest
(safe) dose of the drug that yields a "full" response in most
people (possibly subject to corrections related to ancillary
variables such as people's ages or body weights).
As a second practical matter, the concepts of the action-pre-
scriptive aspect of statistics are often unusable in real-life
situations in which we must decide a course of action. This is
because the necessary objective functions (or other pieces of in-
formation needed to use precise action-prescriptive concepts) are
often unavailable. (This problem can sometimes be bypassed by
working with a range of possible objective functions.)
As a third practical matter, descriptive conclusions and action-
prescriptive conclusions are often widely separated. That is,
empirical researchers generally provide descriptive conclusions
about the values of variables or about the values of parameters
or descriptive conclusions about relationships between variables.
But (apart from possibly SUGGESTING a course of action) the em-
pirical researchers who provide the descriptive conclusions play
little or no direct role in MAKING DECISIONS about a course of
action on the basis of the conclusions. This is because empiri-
cal researchers typically place the information they discover at
other people's disposal. (Researchers often place the informa-
tion in the public domain in the form of a journal article.)
Then the researchers go back to doing what they do best -- em-
pirical research -- and other people (people who make policy)
make the relevant decisions about courses of action.
(An [infrequently occurring] exception to the point in the pre-
ceding paragraph occurs when we directly use methods from the ac-
tion-prescriptive aspect of statistics in conducting empirical
research. This occurs in so-called "sequential" statistical
analysis in which at each main step in the analysis we make a
formal decision as to what action to take next. [Usually one
possible action is to gather more data.] Each decision is made
on the basis of the data accumulated to that point.)
Since
- the action-prescriptive aspect of statistics is not often used
to interpret the results of empirical research projects and
- when the action-prescriptive aspect IS used it is usually
widely separated from the descriptive aspect
therefore, it is possible and reasonable to separately teach stu-
dents about the descriptive and action-prescriptive aspects of
statistics.
Since the action-prescriptive aspect of statistics relies on con-
clusions provided by the descriptive aspect, but not vice versa,
it is reasonable to teach the descriptive aspect of statistics to
students before teaching them the action-prescriptive aspect.
Finally, the descriptive aspect of statistics is much broader
than the action-prescriptive aspect, perhaps at least in the ra-
tio of ten to one in terms of, say, the amount of associated lit-
erature published.
The above considerations lead me to recommend that introductory
teachers focus on the descriptive aspect of statistics. That is,
I recommend that teachers concentrate on how to draw conclusions
about variables and parameters and (especially) how to draw con-
clusions about relationships between variables. I recommend that
discussion of how to use statistical techniques to draw conclu-
sions (or make decisions) about correct courses of action be
postponed until a later course.
My sense of the distinction between the descriptive and action-
prescriptive aspects of statistics grew from an insightful arti-
cle about p-values by Ware, Mosteller, Delgado, Donnelly, and
Ingelfinger (1992).
APPENDIX B: EXAMINATION OF THE MAY 3 ISSUE OF THE TORONTO STAR
FOR STATEMENTS ABOUT VARIABLES
The Toronto Star is a broadsheet newspaper with a weekday circu-
lation of about 450,000. I chose to work with the May 3, 1999
Metro Edition of the Star "at random" in the sense that this was
the issue at hand when I began work.
The first section of the May 3 Star contains 20 pages, with
roughly 45% of the material being advertising. It contains 44
news stories (some only a paragraph or two long) in the catego-
ries of world news, Canada news, and Ontario news. (Most Toronto
news appears in another section.) The first section also con-
tains 10 letters to the editor, 6 opinion articles, and 4 edito-
rials.
I carefully scanned all the above-mentioned non-advertising mate-
rial in the section, circling each statement of the simple value
of a variable, writing "D" in the margin for each statement of
some aspect of a univariate distribution, writing "R" in the mar-
gin for each statement of a relationship between variables, and
writing "N" in the margin for each statement that pertains to
variables but that falls into none of the preceding three groups.
If I detected that the same statement was made more than once
(e.g., in the text and in a graphic), I counted the statement
only once. The following table shows my counts of the four types
of statements:
Frequency of Occurrence
of Different Types of Statements
Pertaining to Variables
in the Non-Advertising Material
in the First Section of the
Metro Edition of the
May 3, 1999 Toronto Star
---------------------------------
Statement
Type Count Percent
---------------------------------
1. Value of 1051 85
a variable
2. Univariate 30 2
distribution
3. Relationship 156 13
between
variables
4. Other 0 0
___ ___
TOTALS 1237 100
---------------------------------
Let me describe some of my classifications. The first news story
on the front page of the May 3 Star is titled "Release of PoWs
fails to sway NATO". The first sentence in this story says
NATO will continue bombing Yugoslavia despite the release
of three U.S. prisoners of war.
I interpreted this (unhappy) sentence as stating a relationship
between variables or, actually, a NON-relationship between vari-
ables. The response variable can be viewed as "bombing intensity
of Yugoslavia" and the predictor variable can be viewed as "num-
ber of NATO prisoners released". The sentence says there will be
no relationship between the release of the three prisoners and
the bombing intensity.
The second sentence fills in details of the first sentence and
introduces a new relationship between variables as follows:
"We are not only not going to stop the bombing, we're go-
ing to intensify the bombing," U.S. Defence Secretary
William Cohen said yesterday.
This sentence indicates that there will be a relationship between
the intensity of the bombing and time -- the intensity will in-
crease in the immediately following time.
The second sentence also states the value of the variable that
indicates the time of Cohen's statement: "yesterday".
The third sentence says
Still, diplomatic efforts to end the conflict in the
Balkans appeared to gain some strength yesterday.
This sentence is another statement of a relationship between
variables. The response variable is "strength of diplomatic ef-
forts to end the conflict in the Balkans" and the predictor vari-
able is time. The sentence is saying that the strength of the
efforts appears to have increased in recent time (i.e., over the
course of "yesterday").
The fourth sentence indicates how the diplomatic efforts to end
the conflict appeared to gain strength as follows:
The White House announced that U.S. President Bill
Clinton would meet today with Russian envoy Viktor
Chernomyrdin, who will brief him on Russian efforts to
resolve the crisis.
Part of the fourth sentence states the value of a variable -- the
(anticipated) value of the "when" property of the meeting, which
is "today".
The second story on the front page is titled "Rental housing cri-
sis predicted" and begins with the following sentence:
Toronto and other Ontario cities face a housing crisis as
demand for affordable apartments grows at a much faster
rate than construction, a comprehensive new report says.
This sentence is a somewhat complicated statement of a (extrapo-
lated) relationship between variables. One way of viewing the
relationship is to say that the predictor variable is "time" and
the two response variables are "demand for affordable apartments"
and "rate of construction of affordable apartments".
A third story on the front page is titled "Immigrants feel wel-
come" and begins with the following sentence:
Toronto is a place that welcomes newcomers, more so than
many cities around the world, say participants in a poll
commissioned for The Star's Beyond 2000 project.
This is a statement of a relationship between variables. The re-
sponse variable could be labeled "amount of welcome extended to
newcomers" and the predictor variable is the nominal-level vari-
able "city" (e.g., Toronto, New York, Tokyo).
* * *
Although relationships between variables dominate the important
lead sentences of the three stories I discuss above, relation-
ships between variables are only peripheral in the other two sto-
ries on the front page. One of these stories is about the
Toronto Maple Leafs hockey team winning a series of playoff
games. This story contains many statements of the values of
variables, but I found no statements of relationships between
variables.
To illustrate some statements of values of variables, here are
the third and fourth sentences from the hockey story, with the
values of variables between asterisks:
*Thirty-two years to the day* after winning their *last*
Stanley Cup, the Leafs moved on to the *second* round of
the *'99* playoffs *last night* by finishing off the
Philadelphia Flyers. The *1-0* victory came on *winger*
*Sergei Berezin's* goal with *one* minute to play in
regulation time.
(The final score of the game can be viewed as a vector-valued
variable with two components.)
Note the broad view I take of what are statements of the values
of variables. That is, as well as values that denote numbers, I
view all words or phrases that function as adjectives or adverbs
and all words or phrases that state a time or date as stating
values of variables. I take this point of view because these
differing linguistic operations all seem to be performing the
same conceptual function -- that of stating the value of some
property of an entity or, equivalently, that of stating the value
of some variable. (In the case of an adjective, the entity is
denoted by the NOUN the adjective modifies. In the case of an
adverb, the entity is the PROCESSES the adverb helps to describe.
In the case of a point in time or a duration, the entity is the
EVENT the time refers to.)
The final story on the front page is about a shooting in Alberta.
This story contains many statements of the values of variables,
but only one statement of a relationships between variables.
* * *
As I note above, I found only 30 statements of univariate distri-
butions, including statements of means, standard deviations, or
other measures of location or dispersion. Two of these occur in
one sentence in the hockey story on the front page. The sentence
quotes the player who scored the only goal in the game as fol-
lows:
"It's the biggest goal I've ever scored in the NHL," said
Berezin, who led the team in goals this season.
This sentence identifies a property that Berezin associates with
each of the goals he has scored in the NHL -- the goal's bigness
(i.e., its importance) -- and the sentence says that his latest
goal has the maximum value in the distribution of the values of
that property.
The same sentence (implicitly) refers to the distribution of the
number of goals scored by players on Berezin's team and says that
Berezin is the player who is associated with the maximum value of
that distribution.
Here are some other examples of statements of univariate distri-
butions:
Testicular cancer tends to hit men in their 20s (page
A3).
The statement says that the age distribution of the occurrence of
testicular cancer peaks strongly in the 20s.
[Curt] Newport used computer models and state-of-the-art
underwater equipment to search an area near Grand Bahama
in water that averaged almost 4,700 meters deep (page
A3).
The statement gives the average of the univariate distribution of
water depths in the area searched.
Mireya Moscoso ... won election yesterday to become the
president [of Panama] .... She beast Martin Torrijos
.... Torrijos conceded defeat with 59.4 per cent of bal-
lots counted, showing Moscoso with 44 per cent of the
vote to his 38 per cent. ... A third candidate ...
Alberto Vallarino ... held 17 per cent of the vote (page
A10).
This passage reports the univariate distribution of the votes
among the three candidates. (One could also reasonably view the
passage as stating a relationship between two variables: "candi-
date" [i.e., Moscoso, Torrijos, or Vallarino] and "percent of the
vote won".)
* * *
I found that classifying the text in the Star was difficult be-
cause I often found that rereading a passage suggested another
(usually deeper) interpretation, or suggested an additional
statement of one of the types. Thus it seems likely that another
reader performing the classification would produce a somewhat
different classification from mine.
However, I believe one could develop a set of straightforward
rules for classifying statements that refer to variables. These
rules would lead to strong agreement among trained classifiers
about statements in passages of text. I suspect that these rules
would lead to finding proportions of the three types of state-
ments about variables that are close to the proportions I give in
the table above.
* * *
I also studied the sports section in the May 3 Star. The sports
section is a special case because it is full of statistics.
Since many newspaper sports statistics are presented in tables,
and since the row and column (and other) dimensions of a table
each suggest a different variable, it is clear that tables of
newspaper sports statistics reflect relationships between vari-
ables.
Interestingly however, tables of newspaper sports statistics do
not generally reflect relationships between variables in
POPULATIONS. This is because the tables are generally not rea-
sonably viewed as representing SAMPLES that reflect some wider
population of entities. Instead, the tables are generally de-
tailed cross-classified reports of the values of properties of
individual entities (typically teams, players, or horses in the
Star), and interest is focused on these specific entities, rather
than on some population they represent.
Thus (unless the tables themselves are sampled over different is-
sues of a newspaper) the relationships between variables re-
flected in tables of newspaper sports statistics cannot generally
be used for predicting or controlling the values of variables in
other entities in a population, as is generally the case in stan-
dard statistical analysis.
Although tables of newspaper sports statistics are generally not
(directly) useful for predicting and controlling the values of
variables in other entities in a population, such tables are of
definite interest in predicting and controlling the values of the
variables FOR THE SAME INDIVIDUAL ENTITIES that are reported in a
table AT A LATER TIME. For example, if a table of sports statis-
tics shows teams as rows and if the statistics in the columns of
the table indicate that one team is presently doing particularly
well relative to the others, we can reasonably use the table
(possibly with other relevant consistent information) to predict
that this particular team will do well in its next game. If one
is a betting person, this prediction might then form the basis of
a bet.
Thus the use of statistics in sports is often different from the
standard use of statistics in empirical research
REFERENCES
Macnaughton, D. B. 1996a. "EPR approach to intro stat: Relation-
ships between variables (response to comments by Brian
Griffin)." Published in sci.stat.edu and EdStat-L on July 18,
1996. Available at http://www.matstat.com/teach/p0012.htm
Macnaughton, D. B. 1996b. "EPR approach to intro stat: Relation-
ships between variables (response to comments by George
Zeliger)." Published in sci.stat.edu and EdStat-L on July 30,
1996. Available at http://www.matstat.com/teach/p0016.htm
Macnaughton, D. B. 1997. "The entity-property-relationship ap-
proach to statistics: An introduction for students." Avail-
able at http://www.matstat.com/teach/
Macnaughton, D. B. 1998. "Eight features of an ideal introductory
statistics course." Available at
http://www.matstat.com/teach/
Macnaughton, D. B. 1999a. "The introductory statistics course:
The entity-property-relationship approach." Available at
http://www.matstat.com/teach/
Macnaughton, D. B. 1999b. "Re: Eight features of an ideal intro
stat course (response to comments by Herman Rubin)." Published
in sci.stat.edu and EdStat-L on May 16, 1999. Available at
http://www.matstat.com/teach/p0041.htm
Ware, J. H., Mosteller, F., Delgado, F., Donnelly, M. S., and
Ingelfinger, J. A. 1992. "P values." In _Medical uses of sta-
tistics_ 2nd ed. edited by J. C. Bailar III and F. Mosteller,
181-200. Boston: NEJM (New England Journal of Medicine) Books.
Home page for Donald Macnaughton's papers about introductory statistics