Statistics is the science of making effective use
of numerical data relating to groups of individuals or experiments.
It deals with all aspects of this, including not only the
collection, analysis and interpretation of such data, but also the
planning of the collection of data, in terms of the design of
surveys and experiments.
A
statistician is someone who is
particularly versed in the ways of thinking necessary for the
successful application of statistical analysis. Often such people
have gained this experience after starting work in any of a
list of
fields of application of statistics. There is also a discipline
called
mathematical
statistics, which is concerned with the theoretical basis
of the subject.
The word
statistics can either be singular or plural. In
its singular form,
statistics refers to the mathematical
science discussed in this article. In its plural form,
statistics is the plural of the word
statistic, which refers to a quantity (such
as a
mean) calculated from a set of data.
Scope
Statistics is considered by some to be a
mathematical science pertaining to the
collection, analysis, interpretation or explanation, and
presentation of
data, while others consider it
to be a branch of
mathematics concerned
with collecting and interpreting
data. Because
of its empirical roots and its focus on applications, statistics is
usually considered to be a distinct mathematical science rather
than a branch of mathematics.
Statisticians improve the quality of data with the
design of experiments and
survey sampling. Statistics also provides
tools for prediction and forecasting using data and
statistical models. Statistics is
applicable to a wide variety of
academic disciplines, including
natural and
social
sciences, government, and business.
Statistical methods can be used to summarize or describe a
collection of data; this is called
descriptive statistics. This is
useful in research, when communicating the results of experiments.
In addition, patterns in the data may be
modeled in a way that accounts for
randomness and uncertainty in the
observations, and are then used to draw inferences about the
process or population being studied; this is called
inferential statistics.
Inference is a vital element of scientific advance, since it
provides a prediction (based in data) for where a theory logically
leads. To further prove the guiding theory, these predictions are
tested as well, as part of the
scientific method. If the inference holds
true, then the descriptive statistics of the new data increase the
soundness of that hypothesis. Descriptive statistics and
inferential statistics (a.k.a., predictive statistics) together
comprise
applied statistics.
History
Some scholars pinpoint the origin of statistics to 1663, with the
publication of
Natural and Political Observations upon the
Bills of Mortality by
John Graunt.
Early applications of statistical thinking revolved around the
needs of states to base policy on demographic and economic data,
hence its
stat-
etymology. The scope of the discipline of statistics broadened
in the early 19th century to include the collection and analysis of
data in general. Today, statistics is widely employed in
government, business, and the natural and social sciences.
Its mathematical foundations were laid in the 17th century with the
development of
probability theory
by
Blaise Pascal and
Pierre de Fermat. Probability theory arose
from the study of games of chance. The
method of least squares was first
described by
Carl Friedrich
Gauss around 1794. The use of modern
computers has expedited large-scale statistical
computation, and has also made possible new methods that are
impractical to perform manually.
The American Statistical Association has ranked
Deming,
Fisher, and
Rao as the
greatest statisticians of all time.
Overview
In applying statistics to a scientific, industrial, or societal
problem, it is necessary to begin with a
population or process to be studied.
Populations can be diverse topics such as "all persons living in a
country" or "every atom composing a crystal". A population can also
be composed of observations of a process at various times, with the
data from each observation serving as a different member of the
overall group. Data collected about this kind of "population"
constitutes what is called a
time
series.
For practical reasons, a chosen subset of the population called a
sample is studied — as
opposed to compiling data about the entire group (an operation
called
census). Once a sample that is
representative of the population is determined, data is collected
for the sample members in an observational or
experimental setting. This data can then be
subjected to statistical analysis, serving two related purposes:
description and inference.
The concept of correlation is particularly noteworthy for the
potential confusion it can cause. Statistical analysis of a
data set often reveals that two variables
(properties) of the population under consideration tend to vary
together, as if they are connected. For example, a study of annual
income that also looks at age of death might find that poor people
tend to have shorter lives than affluent people. The two variables
are said to be correlated; however, they may or may not be the
cause of one another. The correlation phenomena could be caused by
a third, previously unconsidered phenomenon, called a lurking
variable or
confounding
variable. For this reason, there is no way to immediately infer
the existence of a causal relationship between the two variables.
(See
Correlation
does not imply causation.)
For a sample to be used as a guide to an entire population, it is
important that it is truly a representative of that overall
population. Representative sampling assured, inferences and
conclusions can be safely extended from the sample to the
population as a whole. A major problem lies in determining the
extent to which the sample chosen is actually representative.
Statistics offers methods to estimate and correct for any random
trending within the sample and data collection procedures. There
are also methods for designing experiments that can lessen these
issues at the outset of a study, strengthening its capability to
discern truths about the population. Statisticians describe
stronger methods as more "robust".(See
experimental design.)
The fundamental mathematical concept employed in understanding
potential randomness is
probability.
Mathematical statistics
(also called
statistical theory)
is the branch of
applied
mathematics that uses probability theory and
analysis to examine the theoretical
basis of statistics. The use of any statistical method is valid
only when the system or population under consideration satisfies
the basic mathematical assumptions of the method.
Misuse of statistics can
produce subtle, but serious errors in description and
interpretation — subtle in the sense that even experienced
professionals make such errors, and serious in the sense that they
can lead to devastating decision errors. For instance, social
policy, medical practice, and the reliability of structures like
bridges all rely on the proper use of statistics.Even when
statistics are correctly applied, the results can be difficult to
interpret for those lacking expertise. The
statistical significance of a trend
in the data - which measures the extent to which a trend could be
caused by random variation in the sample - may or may not agree
with an intuitive sense of its significance. The set of basic
statistical skills (and skepticism) that people need to deal with
information in their everyday lives properly is referred to as
statistical literacy.
Statistical methods
Experimental and observational studies
A common goal for a statistical research project is to investigate
causality, and in particular to draw a
conclusion on the effect of changes in the values of predictors or
independent variables on
dependent variables or response.
There are two major types of causal statistical studies:
experimental studies and observational studies. In both types of
studies, the effect of differences of an independent variable (or
variables) on the behavior of the dependent variable are observed.
The difference between the two types lies in how the study is
actually conducted. Each can be very effective.
An experimental study involves taking measurements of the system
under study, manipulating the system, and then taking additional
measurements using the same procedure to determine if the
manipulation has modified the values of the measurements. In
contrast, an observational study does not involve experimental
manipulation. Instead, data are gathered and correlations between
predictors and response are investigated.
An example of an experimental study is the famous
Hawthorne study, which attempted to test
changes to the working environment at the Hawthorne plant of the
Western Electric Company. The researchers were interested in
determining whether increased illumination would increase the
productivity of the
assembly line
workers. The researchers first measured the productivity in the
plant, then modified the illumination in an area of the plant and
checked if the changes in illumination affected productivity. It
turned out that productivity indeed improved (under the
experimental conditions). However, the study is heavily criticized
today for errors in experimental procedures, specifically for the
lack of a
control group and
blindness. The
Hawthorne effect refers to finding that an
outcome (in this case, worker productivity) changed due to
observation itself. Those in the Hawthorne study became more
productive not because the lighting was changed but because they
were being observed.
An example of an observational study is one that explores the
correlation between smoking and lung cancer. This type of study
typically uses a survey to collect observations about the area of
interest and then performs statistical analysis. In this case, the
researchers would collect observations of both smokers and
non-smokers, perhaps through a
case-control study, and then look for the
number of cases of lung cancer in each group.
The basic steps of an experiment are:
- Planning the research, including determining information
sources, research subject selection, and ethical considerations for the proposed research and
method.
- Design of experiments,
concentrating on the system model and the interaction of
independent and dependent variables.
- Summarizing a collection of
observations to feature their commonality by suppressing
details. (Descriptive
statistics)
- Reaching consensus about what the observations tell about the world
being observed. (Statistical
inference)
- Documenting / presenting the results of the study.
Levels of measurement
There are four types of measurements or
levels of measurement or measurement
scales used in statistics:
- nominal,
- ordinal,
- interval, and
- ratio.
They have different degrees of usefulness in statistical
research. Ratio measurements have both a zero value
defined and the distances between different measurements defined;
they provide the greatest flexibility in statistical methods that
can be used for analyzing the data. Interval measurements have
meaningful distances between measurements defined, but have no
meaningful zero value defined (as in the case with IQ measurements
or with temperature measurements in
Fahrenheit). Ordinal measurements have imprecise
differences between consecutive values, but have a meaningful order
to those values. Nominal measurements have no meaningful rank order
among values.
Since variables conforming only to nominal or ordinal measurements
cannot be reasonably measured numerically, sometimes they are
called together as categorical variables, whereas ratio and
interval measurements are grouped together as quantitative or
continuous variables due to their numerical nature.
Key terms used in statistics
Null hypothesis
Interpretation of statistical information can often involve the
development of a
null hypothesis in
that the assumption is that whatever is proposed as a cause has no
effect on the variable being measured.
The best illustration for a novice in the predicament encountered
by a jury trial. The null and alternative hypotheses are:
Ho: defendant is innocent and H1: defendant is guilty
The indictment comes because of suspicion of the guilt. The Ho
(status quo) stands in opposition to H1 and is maintained unless H1
is supported by evidence "beyond a reasonable doubt". However,
"failure to reject Ho" in this case does not imply innocence, but
merely that the evidence was insufficient to convict. So the jury
does not necessarily
accept Ho but
fails to
reject Ho.
Error
Working from a
null hypothesis two
basic forms of error are recognised:
- Type I errors where
the null hypothesis is falsely rejected giving a "false
positive".
- Type II errors where
the null hypothesis fails to be rejected and an actual difference
between populations is missed.
Confidence intervals
Most studies will only sample part of a population and then the
result is used to interpret the null hypothesis in the context of
the whole population. For various reasons any values obtained or
derived from the sample will only be an approximation of the true
value.
Confidence intervals
allow statisticians to express how closely the answer derived from
the sample data matches the true value in the whole population.
Often they are expressed as 95% confidence limits so that there is
a 95% chance of the whole population value lying between the two
limits. If these intervals span a value (such as zero) where the
null hypothesis would be confirmed then this can indicate that any
observed value has been seen by chance. (For example a drug that
gives a mean increase in heart rate of 2 beats per minute but has
95% confidence intervals of -5 to 9 for its increase may well have
no effect whatsoever.)
Significance
Statistics rarely give a simple Yes/No type answer to the question
asked of them. Interpretation often comes down to the level of
statistical significance applied to the numbers and often refer to
the probability of a value accurately rejecting the null hypothesis
(sometimes referred to as the
p-value).
When interpreting an academic paper reference to the significance
of a result when referring to the statistical significance does not
necessarily mean that the overall result means anything in real
world terms. (For example in a large study of a drug it may be
shown that the drug has a statisically significant but very small
beneficial effect such that the drug will be unlikely to help
anyone given it in a noticeable way.)
Examples
Some well-known statistical
tests and
procedures are:
Specialized disciplines
Some fields of inquiry use applied statistics so extensively that
they have
specialized
terminology. These disciplines include:
In addition, there are particular types of statistical analysis
that have also developed their own specialised terminology and
methodology:
Statistics form a key basis tool in business and manufacturing as
well. It is used to understand measurement systems variability,
control processes (as in
statistical process control or
SPC), for summarizing data, and to make data-driven decisions. In
these roles, it is a key tool, and perhaps the only reliable
tool.
Statistical computing
The rapid and sustained increases in computing power starting from
the second half of the 20th century have had a substantial impact
on the practice of statistical science. Early statistical models
were almost always from the class of
linear
models, but powerful computers, coupled with suitable numerical
algorithms, caused an increased interest
in
nonlinear models (such as
neural networks) as well as the
creation of new types, such as
generalized linear models and
multilevel models.
Increased computing power has also led to the growing popularity of
computationally-intensive methods based on
resampling, such as permutation
tests and the
bootstrap,
while techniques such as
Gibbs
sampling have made use of Bayesian models more feasible. The
computer revolution has implications for the future of statistics
with new emphasis on "experimental" and "empirical" statistics. A
large number of both general and special purpose
statistical software are now
available.
Misuse
There is a general perception that statistical knowledge is
all-too-frequently intentionally
misused by finding ways to interpret
only the data that are favorable to the presenter. A famous saying
attributed to
Benjamin Disraeli
is, "
There are three
kinds of lies: lies, damned lies, and statistics." Harvard
President
Lawrence Lowell wrote in
1909 that statistics,
"...like veal pies, are good if you know
the person that made them, and are sure of the
ingredients."
If various studies appear to contradict one another, then the
public may come to distrust such studies. For example, one study
may suggest that a given diet or activity raises
blood pressure, while another may suggest
that it lowers blood pressure. The discrepancy can arise from
subtle variations in experimental design, such as differences in
the patient groups or research protocols, which are not easily
understood by the non-expert. (Media reports usually omit this
vital contextual information entirely, because of its
complexity.)
By choosing (or rejecting, or modifying) a certain sample, results
can be manipulated. Such manipulations need not be malicious or
devious; they can arise from unintentional biases of the
researcher. The graphs used to summarize data can also be
misleading.
Deeper criticisms come from the fact that the hypothesis testing
approach, widely used and in many cases required by law or
regulation, forces one hypothesis (the
null hypothesis) to be "favored," and can
also seem to exaggerate the importance of minor differences in
large studies. A difference that is highly statistically
significant can still be of no practical significance. (See
criticism of hypothesis
testing and
controversy
over the null hypothesis.)
One response is by giving a greater emphasis on the
p-value than simply reporting whether a
hypothesis is rejected at the given level of significance. The
p-value, however, does not indicate the size of the
effect. Another increasingly common approach is to report
confidence intervals. Although these are
produced from the same calculations as those of hypothesis tests or
p-values, they describe both the size of the effect and
the uncertainty surrounding it.
Statistics applied to mathematics or the arts
Traditionally, statistics was concerned with drawing inferences
using a semi-standardized methodology that was "required learning"
in most sciences. This has changed with use of statistics in
non-inferential contexts. What was once considered a dry subject,
taken in many fields as a degree-requirement, is now viewed
enthusiastically. Initially derided by some mathematical purists,
it is now considered essential methodology in certain areas.
- In number theory, scatter plots of data generated by a
distribution function may be transformed with familiar tools used
in statistics to reveal underlying patterns, which may then lead to
hypotheses.
- Methods of statistics including predictive methods in forecasting, are combined with chaos theory and fractal geometry to create video works that
are considered to have great beauty.
- The process art of Jackson Pollock relied on artistic
experiments whereby underlying distributions in nature were
artistically revealed. With the advent of computers, methods of
statistics were applied to formalize such distribution driven
natural processes, in order to make and analyze moving video
art.
- Methods of statistics may be used predicatively in performance art, as in a card trick based on
a Markov process that only works some
of the time, the occasion of which can be predicted using
statistical methodology.
- Statistics is used to predicatively create art, as in
applications of statistical
mechanics with the statistical or stochastic music invented by Iannis Xenakis, where the music is
performance-specific. Though this type of artistry does not always
come out as expected, it does behave within a range predictable
using statistics.
See also
Related disciplines
Notes
- Dodge, Y. (2003) The Oxford Dictionary of Statistical
Terms, OUP. ISBN 0199206139
- Moses, Lincoln E. Think and Explain with statistics,
pp. 1 - 3. Addison-Wesley, 1986.
- Hays, William Lee, Statistics for the social sciences,
Holt, Rinehart and Winston, 1973, p.xii, ISBN 978-0030779459
- Statistics at Encyclopedia of Mathematics
- Anderson, , D.R.; Sweeney, D.J.; Williams, T.A..
Statistics: Concepts and Applications, pp. 5 - 9. West
Publishing Company, 1986.
- Willcox, Walter (1938) The
Founder of Statistics. Review of the International Statistical
Institute 5(4):321-328.
References
External links
Online non-commercial textbooks
- "A New View of Statistics", by Will G. Hopkins, AUT University
- "NIST/SEMATECH e-Handbook of Statistical Methods", by
U.S.
National
Institute of Standards and Technology and SEMATECH
- "Online Statistics: An Interactive Multimedia Course of
Study", by David Lane, Joan Lu, Camille Peres, Emily Zitek,
et al.
- "The Little Handbook of Statistical Practice", by
Gerard
E. Dallal, Tufts University
- "StatSoft Electronic Textbook", by StatSoft
Other non-commercial resources
Multimedia