Lecture # 22: Questions/Answers

There will be 5 simple questions asked during the lecture, everyone can answer to get '1 (one) Absolute Mark'. (considering only first answer will be accepted) Problem 1: There is a '22 Cricket Players' get together, in how many ways we can make 2 teams to play a Game? Answer: Explaination: There are two sets …

P&S Lecture# 21: Statistics

In this lecture we start the second main part of our course, namely, statistics. In simple terms, the purpose of using statistics is to make reasonably accurate estimates for parameters of an unknown set of measurements. Here we will define some basic definitions used in statistics: Population: The complete set of measurements in which we are …

Quiz 8 Solution

Question: Toss a coin  times, Find $latex P(H \geq \frac{3n}{4})$ by Markov’s inequality Chebyshev Chernoff   Solution: 1- by Markov's Inequality, we have that $latex P(H \geq \frac{3n}{4})  \leq \frac {E[X]}{3n/4}$ $latex = \frac {1/2}{3n/4}$ $latex = \frac {2}{3}$   2- Applying the Chebyshev inequality we get that $latex P(X \geq \frac {3n}{4}) = P(X - \frac {n}{2} …

Quiz 6 Solution

Lecture # 20: Entropy(Conditional and Joint)

In this lecture we are going to discuss conditional and joint entropy of random variables. First we'll discuss conditional entropy. Lets look into a fact about conditional entropy. For X,Y random variables we have to prove the following fact. To prove this fact we'll use the following tools. Law of Total Probability Jensen's Inequality Log …

Assignment # 10

Solve the following problems carefully. As there is no grade for this assignment you don’t have to submit it. However questions from this homework may appear on one or more graded quizzes/exams.   Q1. Prove that $latex H(X)$ is maximized when all values of $latex X$ are equally likely.   Q2. Prove that $latex H(X)$ is minimized …

Lecture No. 19: Entropy

Before starting off with the mentioned topic, a few highlights: The lecture started off lightly with a few jokes about the quiz, about how most of us had done the entire assignment except that question. Talk about tough luck, eh? Pro Tip: Sir will always be biased towards the questions that he’s mentioned himself during …

P&S Lecture # 18 – Inequality Examples

if $latex f(x)$ is a convex function i.e $latex f'' \geq 0$ Then Jensen's Inequality states: $latex f(E(x)) \leq E(f(x)$ $latex x=a$ with $latex p=0.5$ and $latex x=b$ with $latex p=0.5$ $latex E(x)=(a+b)/2$ Example: if $latex f(x)=x^2$ , then $latex E(x)^2 \leq E(x^2)$ which is always true as $latex E(x^2)-E(x)^2=Var(x) \geq 0$ Exercise#1:  Prove: $latex F(X) \leq …

Homework 9

Solve the following problems carefully. As there is no grade for this assignment you don’t have to submit it. However questions from this homework may appear on one or more graded quizzes/exams. Q#1.  Let X be a non negative random variable.Prove that $latex E(X) \leq (E(X^2))^{\frac{1}{2}} \leq (E(X^3))^{\frac{1}{3}} \leq (E(X^4))^{\frac{1}{4}} \leq ...$ Q#2. Let $latex X_1, …

P&S Lecture # 17 – Convergence

Series Convergence: Let's say that we have a series of real numbers as, $latex a_1, a_2, a_3, . . .$ $latex a_i = 1 - \frac{1}{i}$ $latex 0, \frac{1}{2}, \frac{2}{3}, \frac{3}{4}, \frac{4}{5}, . . . $ When $latex \lim\limits_{i \to \infty} a_i \to 1 $ It gives us a sequence that converges to 1. When talking …