Solutions To Problems of Chapter 2

1 Solutions To Problems of Chapter 2

2.1. Derive the mean and variance for the binomial distribution.

Solution: For the mean value we have that,

E[x] = n X k=0 kn!(nk)!k!p k (1p) nk = n X k=1 n!(nk)!(k1)!p k (1p) nk =np n X k=1 (n1)!

(n1)(k1)

!(k1)!p k1 (1p) (n1)(k1) =np n1 X l=0 (n1)!

(n1)l

!l!p l (1p) (n1)l =np(p+ 1p) n1

=np:(1)

where the formula for the binomial expansion has been employed. For the variance we have,

2 x= n X k=0 (knp) 2 n!(nk)!k!p k (1p) nk = n X k=0 k 2 n!(nk)!k!p k (1p) nk + n X k=0 (np) 2 n!(nk)!k!p k (1p) nk

2np n X k=0 k n!(nk)!k!p k (1p) nk ; (2) or

2 x= n X k=0 k 2 n!(nk)!k!p k (1p) nk

• (np)

2 2(np) 2 ; (3) No Solution Problems for Chapter 1 Machine Learning A Bayesian and Optimization Perspective, 2e Sergios Theodoridis Solution Manual 1 / 4

2 However, n X k=0 k 2 n!(nk)!k!p k (1p) nk = np n X k=1 k (n1)!

(n1)(k1)

!(k1)!p k1 (1p) (n1)(k1) = np n1 X l=0 (l+ 1) (n1)!

(n1)l

!l!p l (1p) (n1)l = np+np(n1)p;(4) which nally proves the result.

2.2.

Solution: For the mean we have

=E[x] = Z b a 1 ba xdx = 1 ba b 2

b a = b+a 2

:(5)

For the variance, we have

2 x= 1 ba Z b a (x) 2 dx= 1 ba Z b a y 2 dy = 1 ba y 3 3

b a = 1 12 (ba) 2

:(6)

2.3.

Solution: Without harming generality, we assume that=0, in order to

simplify the discussion. We have that 1 (2) l=2 jj 1=2 Z +1 1 xexp

1 2 x T

1 x

dx; (7) which due to the symmetry of the exponential results inE[x] =0.For the covariance we have that Z +1 1 exp

1 2 x T

1 x

dx= (2) l=2 jj 1=2

: (8) 2 / 4

3 Following similar arguments as for the univariate case given in the text, we are going to take the derivative on both sides with respect to matrix . Recall from linear algebra the following formulas.@tracefAX 1 Bg @X =(X 1 BAX 1 ) T ; @jX k j @X =kjX k jX T

:

Hence, taking the derivatives of both sides in (8) with respect towe obtain, 1 2 Z +1 1 exp

1 2 x T

1 x

1 xx T

1

T dx= 1 2 (2) l=2 jj 1=2

T ; (9) which then readily gives the result.

2.4.aandbare given by E[x] = a a+b ; and

2 x= ab (a+b) 2 (a+b+ 1)

:

Hint: Use the property (a+ 1) =a(a).

Solution: We know that

Beta(xja; b) = (a+b) (a)(b) x a1 (1x) b1

:

Hence E[x] = (a+b) (a)(b) Z 1

xx a1 (1x) b1 dx= (a+b) (a)(b) (a+ 1)(b) (a+ 1 +b) ; which, using the property (a+ 1) =a(a), results in E[x] = a a+b

:(10)

For the variance we have E[(xE[x]) 2 ] = (a+b) (a)(b) Z 1

x a a+b

2 x a1 (1x) b1 dx;(11) or

2 x= (a+b) (a)(b) Z 1

x a+1 (1x) b1 dx + a 2 (a+b) 2 (a+b) (a)(b) Z 1

x a1 (1x) b1 dx 2 a a+b (a+b) (a)(b) Z 1

x a (1x) b1 dx; (12) 3 / 4

4 and following a similar path as the one adopted for the mean, it is a matter of simple algebra to show that

2 x= ab (a+b) 2 (a+b+ 1)

:

2.5.etersa; bis given by (a+b) (a)(b)

:

Solution: The beta distribution is given by

Beta(xja; b) =Cx a1 (1x) b1

;0x1: (13)

Hence C 1 = Z 1

x a1 (1x) b1

dx:(14)

Let x= sin 2

)dx= 2 sincosd: (15)

Hence C 1 = 2 Z 2

(sin) 2a1 (cos) 2b1

d: (16)

Recall the denition of the gamma function (a) = Z 1

x a1 e x dx; and set x=y 2 )dx= 2ydy; hence (a) = 2 Z 1

y 2a1 e y 2

dy:(17)

Thus (a)(b) = 4 Z 1

Z 1

x 2a1 y 2b1 e (x 2 +y 2 )

dxdy: (18)

Let

x=rsin; y=rcos)dxdy=rdrd:

Hence (a)(b) = 4 Z 2

Z 1

r 2(a+b)1 e r 2 (sin) 2a1 (cos) 2a1

drd:(19)

• / 4