13 probability distributions that must be mastered in deep learning

1. Overview of probability distribution

• Conjugate means that it has the relationship of a conjugate distribution.
• In Bayesian probability theory, if the posterior distribution p(θx) and the prior probability distribution p(θ) are in the same probability distribution family, the prior and posterior are called conjugate distributions, The prior is called the conjugate prior of the likelihood function. Conjugate prior Wikipedia is here (https://en.wikipedia.org/wiki/Conjugate_prior).
• Multiple classification means that the random variance is greater than 2.
• n times means that we also consider the prior probability p(x).
• To learn more about probability, I recommend reading [pattern recognition and machine learning, Bishop 2006].

2. Distribution probability and characteristics

1. Uniform distribution (continuous)

Code: https://github.com/graykode/distribution-is- all-you-need/blob/master/uniform.py

The uniform distribution has the same probability value on [a, b] and is a simple probability distribution.

2. Bernoulli distribution (discrete)

Code: https://github.com/graykode/distribution-is-all-you- need/blob/master/bernoulli.py

• The prior probability p(x) does not consider the Bernoulli distribution. Therefore, if we optimize for maximum likelihood, we can easily become overfitted.
• Use binary cross-entropy to classify binomial classification. Its form is the same as the negative logarithm of the Bernoulli distribution. Picture

3. Binomial distribution (discrete)

Code: https://github.com/graykode/distribution-is-all-you-need/blob/master/ binomial.py

• The binomial distribution with parameters n and p is a discrete probability distribution of the number of successes in a series of n independent experiments.
• The binomial distribution is a distribution that takes into account the prior probability by specifying the quantity to be picked in advance.

4. Multi-Bernoulli distribution, categorical distribution (discrete)

Code: https://github.com/graykode/distribution -is-all-you-need/blob/master/categorical.py

• Multiple Bernoulli is called a categorical distribution.
• Cross entropy has the same form as the multi-Bernoulli distribution taking the negative logarithm.

5. Polynomial distribution (discrete)

Code: https://github.com/graykode/distribution-is-all-you -need/blob/master/multinomial.py

The relationship between polynomial distribution and categorical distribution is the same as the relationship between Bernoul distribution and binomial distribution.

6.β distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you-need/ blob/master/beta.py

• The beta distribution is conjugate to the binomial distribution and the Bernoulli distribution.
• Using conjugation, the posterior distribution can be more easily obtained by using the known prior distribution.
• When the β distribution satisfies the special case (α=1, β=1), the uniform distribution is the same.

7.Dirichlet distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you -need/blob/master/dirichlet.py

• dirichlet distribution and polynomial distribution are conjugate.
• If k=2, it is beta distribution.

8. Gamma distribution (continuous)

Code: https://github.com/graykode/distribution-is-all- you-need/blob/master/gamma.py

• If gamma(a,1)/gamma(a,1) gamma(b,1) is the same as beta(a,b), then The gamma distribution is the beta distribution.
• The exponential distribution and the chi-square distribution are special cases of the gamma distribution.

9. Exponential distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you -need/blob/master/exponential.py

The exponential distribution is a special case of the γ distribution when α is 1.

10. Gaussian distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you-need/ blob/master/gaussian.py

The Gaussian distribution is a very common continuous probability distribution.

11. Normal distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you-need /blob/master/normal.py

The normal distribution is the standard Gaussian distribution, with a mean of 0 and a standard deviation of 1.

12. Chi-square distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you-need /blob/master/chi-squared.py

• The chi-square distribution with k degrees of freedom is the distribution of the sum of the squares of k independent standard normal random variables.
• Chi-square distribution is a special case of beta distribution

13.t distribution (continuous)

Code: https:/ /github.com/graykode/distribution-is-all-you-need/blob/master/student-t.py

t The distribution is a symmetrical bell-shaped distribution, similar to the normal distribution, but with a longer tail Heavy, which means it's more likely to produce values ​​well below average.

via：https://github.com/graykode/distribution-is-all-you-needa

The above is the detailed content of 13 probability distributions that must be mastered in deep learning. For more information, please follow other related articles on the PHP Chinese website!