Show Menu
Cheatography
  1. Home >
  2. Education >

Probability - Midterm Cheat Sheet (DRAFT) by

Covering counting principles, probability principles, Bayes theorem

This is a draft cheat sheet. It is a work in progress and is not finished yet.

Special Distri­butions (Discrete RVs)

E and Var
NAME
RX
PMF
p & p(1-p)
Bernou­lli(p)
{0,1}
p for x=1, 1-p for x=0
1/p and (1-p)/p2
Geomet­ric(p)
Z+
p(1-p)(k-1) for k C Z+
np and np(1-p)
Binomi­al(n,p)
{0,1,..,n}
nCk . pk . (1-p)(n-k) for k = 0 to n
m/p and (m.(1-­p))/p2
Pascal­(m,p)
{m,m+1­,m+­2...}
(k-1)C(m-1) . pm . (1-p)(k-m) for k = m, m+1, m+2, m+3, ...
np and ((b+r-­n)/­(b+­r-1­)).n­p(1-p)
Hyperg­eom­etr­ic(­b,r,k)
{max(0­,k-r), max(0,­k-r)+1, ..., min(k,b)}
(bCx . rC(k-x))/((b+r)Ck) V x C RX
Both equal to lambda
Poisso­n(l­ambda)
Z+
(e-lambda . lambdak)/k! for k C RX

Continuous RVs, PDFs and Mixed RVs

RV X with CDF FX(x) is continuous if FX(x) is a continuous function V x C R
PMF doesn't work for CRVs, since V x C R, PX(x) = 0. Instead, PDFs are used.
PDF = fX(x) = dFX(x)/dx (if FX(x) is differ­ent­iable at x) >=0 V x C R.
P(a<X<=b) = integral from a to b (fX(u).du) and integral from -inf to +inf (fX(u) . du) = 1
EX = integral from -inf to +inf (x . fX(x) . dx) and E[g(X)] = integral from -inf to +inf (g(x) . fX(x) . dx)
Var(X) = integral from -inf to +inf (x2 . fX(x) . dx - mu2X)
If g: R-> R is strictly monotonic and differ­ent­iable, then PDF of Y=g(X) is fY(y) = fX(x1) . |dx1/dy| where g(x1)=y and 0 if g(x) = y has no solution

Joint Distri­but­ions: RVs >= 2

Joint PMF of X and Y = PXY(x,y) = P(X=x, Y=y) = P((X=x) and (Y=y)) and Joint range = RXY = {(x,y)| PXY(x,y) > 0} and summing up PXY over all (x,y) pairs will result in 1
Marginal PMF of X = PX(x) = sum over all yj C RY (PXY(x, yj)) for any x C RX. Similarly, Marginal PMF of Y = PY(y) = sum over all xi C RX (PXY(xi, y)) for any y C RY
To show indepe­ndence between X and Y, prove P(X = x, Y=y) = P(X=x) . P(Y=y) for all x-y pairs. Similarly, for condit­ional indepe­ndence, show that P(Y=y|X=x) = P(Y=y) for all x-y pairs
Joint CDF = FXY(x,y) = P(X<=x, Y<=y) and Marginal CDF for X = FX(x) = limit y to inf(FXY(x,y)) for any x and Marginal CDF for Y = FY(y) = limit x to inf (FXY(x,y)) for any y
Condit­ional expect­ation: E[X|Y=yj] = sum over all xi C RX (xi . PX|Y(xi|yj))
NOTE: FXY(inf,inf) = 1, FXY(-inf,y) = 0 for any y and FXY(x,-inf) = 0 for any x
P(x1 < X <= x2, y1 < Y <= y2) = FXY(x1,y1) + FXY(x2,y2) - FXY(x2,y1) - FXY(x1,y2)
Condit­ional PMF given event A = PX|A(xi) = P(X=xi|A) = P(X=xi and A)/P(A) for any xi C RX and Condit­ional CDF = FX|A(x) = P(X <= x | A)
Given RVs X and Y, PX|Y(xi, yj) = PXY(xi,yj)/PY(yj). Similarly for Y|X
E[X + Y] = E[X] + E[Y] - indepe­ndence not required
E[X . Y] = E[X] . E[Y] - indepe­ndence IS required
 

Problem Solving Techniques

* CARD PROBLEMS:
Number of ways to pick k suits = 4Ck with k=1,2,3,4
* n BALLS, r BINS:
- Distin­gui­shable balls: each ball can go into any 1 of r bins. The # of distinct perms would be rPn = rn
- Indist­ing­uis­hable balls: there will be 2 cases:
* No empty bins. Occupancy vector is x1+...+xr=n where every x is >= 1. There can be n-1 possible locations for bin dividers from which we can choose r-1 to keep >= 1 ball in each bin. # of possible arrang­ements = (n-1)C(r-1).
* Bin may have 0 balls. Then the occupancy vector would be y1+...+yr=n+r and the # of arrang­ements will be (n+r-1)C(r-1)
* COMMITTEE SELECTION: Solve using product rule/h­ype­rge­ometric approach.
* HAT MATCHING PROBLEM:
Probab­ility of k men drawing their own hats (over all k-tuples) = (nCk(n-k)!)/n! = 1/k!
# of derang­ements = n![1-1­/1!­+1/­2!-­1/3­!+...+(-1)n/n!]
P(k matches) = [1/2! - 1/3! + 1/4! -...+(-1)(n-k)/(n-k)­!]/k!
* DRAWING THE ONLY SPECIAL BALL FROM n BALLS IN k TRIALS:
Total # of outcomes = nCk = [1 + (n-1)]Ck = 1C0(n-1)Ck + 1C1(n-1)C(k-1) , with term #1 denoting no special ball, and term #2 denoting the special ball
*Total # of roundtable arrang­ements with k people = k!/k = (k-1)!
* SYSTEM RELIAB­ILITY ANALYSIS:
P(fail)=p, P(succ­ess­)=1-p
For parallel config, 2n-1 successes and 1 failure, P(fail)=pn
For series config, 2n-1 failures and 1 success, P(success) = (1-p)n
For series connec­tions, take inters­ection, and for parallel connec­tions take union
* PMF FOR SUM, DIFF, MAX, MIN OF 4-SIDED DICE:
Uniform PMF = PXY(x,y) = 1/16
For each (x,y) point in the Cartesian coordinate diagram, calculate the diff/sum label or min/max label.
Write down tables for Joint, Marginal and Condit­ional PMFs
Headers are: x y PXY(x,y) x PX(x) y PY(y) x y PY|X(y,x). First 3 for joint, next 4 for marginal, the remaining for condit­ional
For marginal, plot PMF on y-axis and RV value on x-axis.
For joint, plot y on y-axis and x on x-axis

Facts for PMFs and RV Distri­butions

0≤PX(x)≤1 V x and Sum over all x C Rx (PX(x)) = 1
For any set A⊂RX, P(X∈A) = ∑x∈APX(x)
RVs X and Y are indepe­ndent if P(X=x, Y=y) = P(X=x) * P(Y=y), V x,y The first formula can be extended to n times.
P(Y=y|X=x) = P(Y=y), V x,y if X & Y are indepe­ndent
If X1,...,Xn are indepe­ndent Bernou­lli(p) RVs, then X=X1+X2+...+Xn has Binomi­al(n,p) distri­bution, and Pascal (1,p) = Geometric (p)
For distri­butions using parameter p, 0<p­<1
If X is of Binomial (n, p = lambda/n), with fixed lambda > 0. Then, for any k C Z, limn->infPX(k) = (e-lambda.lambdak)/k!
* SPECIAL DISTRI­BUT­IONS:
TYPE, PDF & E[X] AND VAR(X)
Uniform(a, b) || 1/(b-a) if a<x­<b || (a+b)/2 and (b-a)2/12
Expone­nti­al(­lambda) || lambda . e(-lambda . x) || 1/lambda and 1/(lambda)2
Normal­/Ga­ussian, ie: N(0,1) || (1/sqrt(2 . pi)) . exp(-x2/2), V x C R || 0 and 1
Gamma (alpha, lambda) || (lambdaalpha . x(alpha -1) . e(-lambda . x))/(alpha -1)! for x>0 || alpha/­lambda and EX/lambda
CDF: FX(x) = P(X <= x) V x C R and P(a < X <= b) = FX(b) - FX(a)
 

Counting Princi­ples, n-nomial Expansions

Permut­ations of n distinct objs. take n w/ r groups of indistinct objs. = (n!)/(n1! ... nr!)
nPr = nr and nCr=(n+r-1)Cr : for perms and combs where k objs are taken at a time
(a+b)n = Sum over k (nCkakb(n-k)) where k=0,...,n
Binomial coeff. identity: nCk = (n-1)C(k-1) + (n-1)Ck where first term maps to A and second to AC
nCm = nCn-m
Sum over r (nCr(-1)r(1)(n+r) is 0 where r=0,...n
Sum over r ((nCr)2) = 2nCn where r=0,...,n
Sum over s (sCm) = ^(n+1)­C~(m+1) where s=m,..,n
Hyperg­eom­etric expansion: (n+m)Cr = nC0mCr + nC1mC(r-1) + ... + nCrmC0 a CE and ME enumer­ation
n! = (n/e)n x root(2n x pi) - Stirling's approx. for n!
Trinomial expansion: (a+b+c)n = sum over i, j, k (C'aibjck) where i, j, k=0,...n and i+j+k=n and C'=n!/­(i!­j!k!)
In n-nomial expansion (a1 + ... + ar)n , the # of terms in the sum is rCn = (r+n-1)Cn = (r+n-1)C(r-1)

Expect­ation, Variance, RV Functions

Expected value of X, ie: EX/E[X]/muX = sum over all xk C RX (xk . P(X = xk)). It is linear
E[aX + b] = aE[X] + b, V a,b C R
E[X1 + ... + Xn] = E[X1] + ... + E[Xn]
If X is an RV and Y=g(X), then Y is also an RV.
RY = {g(x) | x C RX} and PY(y) = sum over all x:g(x)=y (PX(x))
E[g(X)] = sum over all xk C RX (g(xk) . PX(xk)) (LOTUS)
Var(X) = E[(X - muX)2] = sum over all xk C RX ((xk - muX)2.PX(xk))
SD(X)/­sigmaX = sqrt(V­ar(X))
Covariance = Cov(X,Y) = E[XY] - E[X].E[Y], which will be 0 if X & Y are indepe­ndent
Var(X) = Cov (X,X) = E[X2] - (E[X])2
Var(aX + b) = a2Var(X), and if X = X1 + ... + Xn, then Var(X) = Var(X1) + ... + Var(Xn)
Var(aX + bY) = a2Var(X) + b2Var(Y) + 2abCov­(X,Y)
Var(total up to Xn) = sum of all Var(Xi) if Xi is mutually indepe­ndent for i = 1...n. Summing up over the same conditions for expected values holds true, regardless of indepe­ndence or not
Correl­ation coeffi­cient = Cov(X,­Y)/­(SD(X) . SD(Y)) - ranges between -1 and 1 (inclusive for both limits)
Z-stan­dar­dized transf­orm­ation: Z=(X - muX)/SD(X) - zero mean and unit variance