HW4#

Task 4.1 (0.5 points)#

A bridge player had no ace in three consecutive hands. What is the probability of this event? Did he have reason to complain of ill luck? A hand consists of \(13\) random cards drawn out of \(52\).

YOUR SOLUTION HERE#

Task 4.2 (2 points)#

Let \(x_1, \dots, x_n \sim U[0,1]\). Sort them in the nondecreasing order:

\[ x_{(1)} \leqslant x_{(2)} \leqslant \ldots \leqslant x_{(k)} \leqslant\ldots\leqslant x_{(n)}. \]

The \(k\)-th element \(x_{(k)}\) in this sequence is called \(k\)-th order statistics. Show that \(x_{(k)} \sim \mathrm{Beta}(k, n+1 -k)\). Do it in two ways:

  • experimentally: generate a random matrix of shape \(N \times n\) (try \(N = 1000, 10000, 100000\)), calculate \(k\)-th order statistics for each row and plot historgram of these values along with pdf of beta distribution. (1 point)

  • theoretically: present a mathematical proof of this statement (1 point)

Warning

To get full points for the experimental part you should avoid pythonic loops when generating random matrix. The only place where for loop is allowed is iterating over several values of \(N\).

YOUR SOLUTION HERE#

Your historgram for a paticular value of \(N\) should look like this:

from scipy.stats import beta, gamma
import numpy as np
import matplotlib.pyplot as plt

%config InlineBackend.figure_formats = ['svg']

def beta_hist(a, b, N=10000):
    xs = np.linspace(0.001, 0.999, num=1000)
    plt.hist(beta(a, b).rvs(size=N), bins=100, color='b', density=True)
    plt.plot(xs, beta(a, b).pdf(xs), c='r', lw=2, ls="--")
    plt.grid(ls=":")
    
beta_hist(3, 5)
../_images/5b53f699740791539686074b9fdbfb72c28cd9901e07e735c907eb2db487c189.svg

Task 4.3 (1 point)#

Calculate entropy of \(\mathcal N(\mu, \sigma^2)\). Plot the graph of this entropy as function of \(\sigma\). What is the limit of this function as \(\sigma \to +0\)? \(\sigma \to +\infty\)?

YOUR SOLUTION HERE#

Task 4.4 (1 point)#

Find KL divergence between \(p\sim \mathrm{Geom}(s)\) and \(q\sim \mathrm{Geom}(t)\), \(0 < s, t < 1\). Is \(\mathbb{KL}(p, q) = 0\) when \(s = t\)? Does equality \(\mathbb{KL}(p, q) = \mathbb{KL}(q, p)\) hold? Plot the graphs of \(\mathbb{KL}(p, q)\) as functions of \(s\) for several fixed values of \(t\).

YOUR SOLUTION HERE#