A data set (or dataset) is a collection of data. In the case of tabular data, a data set corresponds to one or more database tables, where every column of a table represents a particular variable, and each row corresponds to a given record of the data set in question. The data set lists values for each of the variables, such as for example height and weight of an object, for each member of the data set. Data sets can also consist of a collection of documents or files.
In the open data discipline, a data set is a unit used to measure the amount of information released in a public open data repository. The European data.europa.eu portal aggregates more than a million data sets.
Properties
Several characteristics define a data set's structure and properties. These include the number and types of the attributes or variables, and various statistical measures applicable to them, such as standard deviation and
kurtosis.
The values may be numbers, such as or , for example representing a person's height in centimeters, but may also be nominal data (i.e., not consisting of Number values), for example representing a person's ethnicity. More generally, values may be of any of the kinds described as a level of measurement. For each variable, the values are normally all of the same kind. Missing values may exist, which must be indicated somehow.
In statistics, data sets usually come from actual observations obtained by sampling a statistical population, and each row corresponds to the observations on one element of that population. Data sets may further be generated by algorithms for the purpose of testing certain kinds of software. Some modern statistical analysis software such as SPSS still present their data in the classical data set fashion. If data is missing or suspicious an imputation method may be used to complete a data set.
Applications and use cases
Data sets are widely used across various fields to support data analysis, research, and decision-making. In the sciences, data sets provide the empirical foundation for studies in disciplines such as
biology,
physics, and
social science, enabling discoveries in medicine, environmental science, and social research. In
machine learning and artificial intelligence, data sets are essential for training, validating, and testing algorithms for tasks such as image recognition, natural language processing, and predictive modeling. Governments and organizations publish open data sets to promote transparency, inform policy-making, and facilitate urban and social planning. The business sector uses data sets for market analysis, customer segmentation, and operational improvements. Additionally, healthcare relies on data sets for clinical research and improving patient outcomes. These varied applications demonstrate the critical role data sets play in enabling evidence-based insights and driving technological progress.
Classics
Several classic data sets have been used extensively in the
statistical literature:
-
Iris flower data set – Multivariate data set introduced by Ronald Fisher (1936).
-
MNIST database – Images of handwritten digits commonly used to test classification, clustering, and image processing algorithms
-
Categorical data analysis – Data sets used in the book, An Introduction to Categorical Data Analysis, provided online by UCLA Advanced Research Computing.
-
Robust statistics – Data sets used in Robust Regression and Outlier Detection (Peter Rousseeuw and Leroy, 1968). Provided online at the University of Cologne.
-
Time series – Data used in Chatfield's book, The Analysis of Time Series, are provided on-line by StatLib.
-
Extreme values – Data used in the book, An Introduction to the Statistical Modeling of Extreme Values are a snapshot of the data as it was provided on-line by Stuart Coles, the book's author.
-
Bayesian Data Analysis – Data used in the book are provided on-line ( archive link) by Andrew Gelman, one of the book's authors.
-
The Bupa liver data – Used in several papers in the machine learning (data mining) literature.
-
Anscombe's quartet – Small data set illustrating the importance of graphing the data to avoid statistical fallacies.
Example
Loading data sets using Python:
$ pip install datasets
from datasets import load_dataset
dataset = load_dataset(NAME OF DATASET)
See also
-
List of datasets for machine-learning research
-
List of datasets in computer vision and image processing
-
Data blending
-
Data (computer science)
-
Sampling
-
Data store
-
Interoperability
-
Data collection system
External links
