Bernoulli Distribution

The Bernoulli Distribution models a single binary outcome (0/1, false/true) with success probability p and failure probability 1−p. It is fundamental in AI/ML because many labels, events, and decisions are binary, and it underpins likelihoods used in logistic regression and binary cross-entropy loss. Example: predicting whether an email is spam (1) or not spam (0) for a given message.

Imagine a light switch: it’s either off or on—no in-between. The Bernoulli distribution is the math version of that kind of situation, where there are only two possible outcomes, often called “failure” (0) or “success” (1). The only thing you need to describe it is the chance of success, usually written as p. For example: the probability a coin flip is heads, whether an email is spam or not, or whether a customer clicks an ad. In AI and machine learning, it’s commonly used to model yes/no data and binary predictions.

import numpy as np  # numerical computing (random sampling, vectorized ops)
import pandas as pd  # tabular summaries
from scipy.stats import bernoulli  # Bernoulli distribution utilities

# Reproducibility for sampling-based workflows
rng = np.random.default_rng(42)

# Scenario: email campaign — each email either converts (1) or not (0)
p_convert = 0.08  # assumed conversion probability for a single email
n_emails = 5000   # number of sent emails (independent trials)

# *** KEY LINE ***: draw Bernoulli samples (0/1 outcomes) for each email
conversions = bernoulli.rvs(p_convert, size=n_emails, random_state=rng)  # *** KEY LINE ***

# Summarize outcomes in a way you'd feed into downstream ML/monitoring
df = pd.DataFrame({"converted": conversions})
empirical_rate = df["converted"].mean()  # observed conversion rate from samples

# Compute log-likelihood of the observed 0/1 data under the assumed p (useful in model fitting)
log_lik = bernoulli.logpmf(df["converted"].to_numpy(), p_convert).sum()

# Compare empirical vs theoretical moments (sanity checks in simulation pipelines)
theoretical_mean = bernoulli.mean(p_convert)
theoretical_var = bernoulli.var(p_convert)

print(f"Empirical conversion rate: {empirical_rate:.4f}")
print(f"Theoretical mean/var:     {theoretical_mean:.4f} / {theoretical_var:.6f}")
print(f"Total log-likelihood:     {log_lik:.2f}")

• The vector conversions contains simulated 0/1 outcomes; its mean is the observed conversion rate you’d typically monitor or use as a label prevalence check.
• The key call to scipy.stats.bernoulli.rvs uses size to generate many independent trials at once; pass a seeded RNG for reproducibility.
• scipy.stats.bernoulli.logpmf provides per-observation log-probabilities; summing them is a common building block in maximum-likelihood estimation and probabilistic model evaluation.
• A common mistake is mixing up p_convert (probability of 1) with the probability of 0; in Bernoulli, P(0)=1-p.

set.seed(42)  # make simulation reproducible

# Scenario: email campaign — each email either converts (1) or not (0)
p_convert <- 0.08  # assumed conversion probability for a single email
n_emails <- 5000   # number of sent emails (independent trials)

# *** KEY LINE ***: draw Bernoulli samples via Binomial(size=1)
conversions <- rbinom(n = n_emails, size = 1, prob = p_convert)  # *** KEY LINE ***

# Summarize outcomes (often used to validate data generation or label distribution)
empirical_rate <- mean(conversions)

# Compute total log-likelihood under the assumed p (useful for fitting/diagnostics)
log_lik <- sum(dbinom(x = conversions, size = 1, prob = p_convert, log = TRUE))

# Compare empirical vs theoretical moments (sanity checks)
theoretical_mean <- p_convert
theoretical_var <- p_convert * (1 - p_convert)

cat(sprintf("Empirical conversion rate: %.4f\n", empirical_rate))
cat(sprintf("Theoretical mean/var:     %.4f / %.6f\n", theoretical_mean, theoretical_var))
cat(sprintf("Total log-likelihood:     %.2f\n", log_lik))

• conversions is a 0/1 vector; its mean is the observed conversion rate, commonly checked before training classifiers on binary labels.
• The key line uses rbinom with size = 1 to generate Bernoulli outcomes; this is the standard R idiom for Bernoulli sampling.
• dbinom with log = TRUE yields log-probabilities; summing them gives a log-likelihood used in estimation and model comparison workflows.
• A common pitfall is forgetting size = 1; larger size changes the data to counts (binomial), not binary outcomes.