**Important Notice**

Before taking this course, a student is supposed to be very good in probability theory (UG level). To brush up your concepts,

I am providing link to some lecture notes, video lectures and presentations.

**Lecture Series on Probability and Random Variables by Prof. M. Chakraborty, Department of Electronics and Electrical Communication Engineering, I.I.T.,Kharagpur.**(Watch at least first 9 lectures).**A revision lecture on probability theory by MIT OpenCourseWare****Lecture notes for the course of****Probabilistic Systems Analysis and Applied Probability at MIT**

## Information theory and coding

**Course No.**

**:**

**Instructor**

Shahid M Shah, email id: FIRST NAME AT ece.iisc.ernet.in, FIRST NAME DOT nit AT gmail.com

**Credits**

4

**Location**

Room No. 426, Reva University

**Lecture Hours**

Monday : 11:40 AM to 12:35 PM

Tuesday : 2:25 PM to 3:20 PM

Wednesday : 2:25 PM to 3:20 PM

Friday : 3:20 PM to 4:15 PM

**Announcement: Solution to HW 1 available, see below!**

**Homeworks**

1)

**Home work 1**. Deadline Monday morning, drop it in my office @ Wireless research Lab (Ground floor). If you are not able to give before 8:30, the during first break i.e., 10:20AM handover to me. Click here to open the homework

*Solution to HW 1 available, click here to download*

2)

**Homework 2**. Deadline Monday morning (24th august). Submit at most by 10:15AM. Click here to open the Homework.

3)

**Homework 3**. Deadline Monday morning (28th september). Click here to open the HW.

*Note: There will be homeworks for every week. You will be assigned homework on every friday and you have to submit that on monday morning without fail.*Click here for the solution of problem 4 in HW 3

4)

**Homework 4**. Deadline Monday morning (25th October). Click here to open the HW.

*Note: There will be homeworks for every week. You will be assigned homework on every friday and you have to submit that on monday morning without fail.*Click here for the solution of problem 4 in HW 3

5)

**Homework 5**. Deadline Monday morning (02 November). Click here to open the HW.

*Note: There will be homeworks for every week. You will be assigned homework on every friday and you have to submit that on monday morning without fail.*Click here for the solution of problem 4 in HW 3

**Internal exam papers:**

First internal question paper

second internal question paper

third internal question paper

**Course Syllabus**

*Motivating with examples like coin tossing, drawing a cube etc, Concept of probability space, Discrete Random variable,*

**Revision of Probability theory**:Continuous random variable, Cumulative distribution function, Probability density function, Independence of random variable, Expectation of a random variable,

Some basic results on sequence of random variables, Law of large numbers, Gaussian random variable, Central limit theorem.

*Motivating about measure of information with examples, Concept of Entropy, Mutual information, Some properties of Entropy and mutual information, Differential entropy*

**Measures of Information**:*If possible then introduce the concept of typical sets in simple way*

**Source Coding**:*,*Source coding theorem

**Channel Coding:**Definition of Channel capacity, Shannon's Channel capacity theorem for Discrete memoryless channel, Deriving Channel capacity of Binary Symmetric Channel, Various properties and relations of mutual information and entropy, Differential entropy, Additive White Gaussian Noise Channel (AWGN), Sphere packing argument to find capacity of AWGN channel*Error Control codes: Galios field, Linear codes, Syndrome decoding, Cyclic codes, Reed Muller codes, BCH Codes, Convolutional codes, Introduction to LDPC codes.*

**Prerequisites**

Probability Theory/Random Process, Digital Communication, Working Brain :)

**Course Grade**

T.B.D

**References**

1) Digital Communication by Simon Haykins.

*Click here to download*

2) Elements of Information theory by Cover and Thomas, 1st edition