Probability and Statistics for Data Science

Probability provides the theoretical foundation needed to make statistical inferences, while statistics applies those theories to analyze and make sense of real-world data. These concepts form the backbone of data analysis, machine-learning algorithms, and their interpretation.

This article walks through the essential concepts and shows how to implement several of them with Python.

Descriptive statistics

Descriptive statistics summarize the main features of a dataset, providing a quick overview of a sample and its measures.

Measures of central tendency

These metrics represent the center point or typical value of data:

• Mean: the average of the data.
• Median: the middle value in a sorted list.
• Mode: the most frequently occurring value.

Measures of spread

These metrics indicate how dispersed the data points are:

• Range: the difference between the highest and lowest values.
• Variance: a measure of how far each number is from the mean.
• Standard deviation: the square root of the variance.

Skewness and kurtosis

These measures describe the shape of a distribution:

• Skewness: the asymmetry of the probability distribution.
• Kurtosis: the “tailedness” of the distribution.

Here is one way to calculate these descriptive statistics:

import numpy as np
from scipy import stats
 
np.random.seed(0)
data = np.random.normal(50, 15, 100)
 
mean = np.mean(data)
median = np.median(data)
range_data = np.ptp(data)
variance = np.var(data)
std_dev = np.std(data)
skewness = stats.skew(data)
kurtosis = stats.kurtosis(data)
 
print(mean, median, range_data, variance, std_dev, skewness, kurtosis)

50.89712023301727
51.41144179156997
72.34116659732528
228.56098932335956
15.118233670748696
0.005171839713550985
-0.37835455663313455

We can also visualize the same data:

import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
 
plt.figure(figsize=(18, 6))
 
plt.subplot(1, 3, 1)
sns.histplot(data, kde=True)
plt.title('Histogram with Kernel Density Estimate')
plt.axvline(np.mean(data), color='r', linestyle='--')
plt.axvline(np.median(data), color='g', linestyle='-')
 
plt.subplot(1, 3, 2)
sns.boxplot(x=data)
plt.title('Box Plot')
 
plt.subplot(1, 3, 3)
sns.violinplot(x=data)
plt.title('Violin Plot')
 
plt.show()

The histogram shows the distribution and its center. The box plot summarizes the median, quartiles, range, and possible outliers. The violin plot adds a density estimate, making the shape of the distribution easier to inspect.

Probability

Probability measures the likelihood that an event will occur. Key concepts include:

• Probability rules: including the addition and multiplication rules.
• Conditional probability: the probability of one event given that another has occurred.
• Discrete distributions: such as Binomial and Poisson.
• Continuous distributions: such as Normal and Uniform.

Consider a fair six-sided die. Every outcome has the same probability:

import numpy as np
import pandas as pd
 
die_rolls = np.arange(1, 7)
probabilities = np.full(6, 1 / 6)
 
distribution = pd.DataFrame({
    'Outcome': die_rolls,
    'Probability': probabilities,
})
 
print(distribution)

   Outcome  Probability
0        1     0.166667
1        2     0.166667
2        3     0.166667
3        4     0.166667
4        5     0.166667
5        6     0.166667

Each outcome has a probability of $1/6$.

Inferential statistics

Inferential statistics let us make predictions or draw conclusions about a population from a sample.

Sampling

• Random sampling: every member has an equal chance of being selected.
• Sampling distribution: the distribution of a statistic over many samples.

Hypothesis testing

• Null hypothesis ($H_0$): a statement of no effect, difference, or relationship in a population.
• Alternative hypothesis ($H_1$): a competing statement that represents the effect or relationship being investigated.

The evidence in a sample is used to decide whether the null hypothesis should be rejected under a defined significance threshold.

Correlation and regression

Correlation measures the strength and direction of a relationship between two variables. Regression estimates how a dependent variable changes with one or more independent variables.

The following example computes a correlation and fits a simple linear-regression model:

import numpy as np
from scipy import stats
from sklearn.linear_model import LinearRegression
 
np.random.seed(0)
data = np.random.normal(50, 15, 100)
 
np.random.seed(1)
data2 = np.random.normal(30, 10, 100)
 
correlation, _ = stats.pearsonr(data, data2)
 
X = data.reshape(-1, 1)
Y = data2.reshape(-1, 1)
 
model = LinearRegression()
model.fit(X, Y)
 
slope = model.coef_[0]
intercept = model.intercept_
 
print(correlation, slope, intercept)

0.14939503462531986
[0.08746918]
[26.15389939]

The correlation coefficient of approximately 0.149 suggests a weak positive linear relationship. The fitted regression equation is approximately $Y = 0.087X + 26.154$.

Bayesian statistics

Bayesian statistics updates a probability estimate as evidence becomes available. It combines prior beliefs with the likelihood of observed data.

Bayes’ theorem states:

Where:

• $P(A|B)$ is the posterior probability.
• $P(B|A)$ is the likelihood.
• $P(A)$ is the prior probability.
• $P(B)$ is the marginal likelihood or evidence.

from scipy.stats import binom
 
prior = 0.5
likelihood = binom.pmf(7, 10, 0.5)
marginal_likelihood = 0.5
posterior = (likelihood * prior) / marginal_likelihood
 
print(posterior)

0.11718749999999999

Summary

Probability gives us a language for uncertainty. Statistics applies that language to observed data so we can summarize samples, test claims, model relationships, and update beliefs. Together, they form a core foundation for practical data science.