Important Notice
Before taking this course, a student is supposed to be very good in probability theory (UG level). To brush up your concepts,
I am providing link to some lecture notes, video lectures and presentations.
Before taking this course, a student is supposed to be very good in probability theory (UG level). To brush up your concepts,
I am providing link to some lecture notes, video lectures and presentations.
- Lecture Series on Probability and Random Variables by Prof. M. Chakraborty, Department of Electronics and Electrical Communication Engineering, I.I.T.,Kharagpur. (Watch at least first 9 lectures).
- A revision lecture on probability theory by MIT OpenCourseWare
- Lecture notes for the course of Probabilistic Systems Analysis and Applied Probability at MIT
Information theory and coding
Course No.:
Instructor
Shahid M Shah, email id: FIRST NAME AT ece.iisc.ernet.in, FIRST NAME DOT nit AT gmail.com
Credits
4
Location
Room No. 426, Reva University
Lecture Hours
Monday : 11:40 AM to 12:35 PM
Tuesday : 2:25 PM to 3:20 PM
Wednesday : 2:25 PM to 3:20 PM
Friday : 3:20 PM to 4:15 PM
Announcement: Solution to HW 1 available, see below!
Homeworks
1) Home work 1. Deadline Monday morning, drop it in my office @ Wireless research Lab (Ground floor). If you are not able to give before 8:30, the during first break i.e., 10:20AM handover to me. Click here to open the homework
Solution to HW 1 available, click here to download
2) Homework 2. Deadline Monday morning (24th august). Submit at most by 10:15AM. Click here to open the Homework.
3) Homework 3. Deadline Monday morning (28th september). Click here to open the HW.
Note: There will be homeworks for every week. You will be assigned homework on every friday and you have to submit that on monday morning without fail. Click here for the solution of problem 4 in HW 3
4) Homework 4. Deadline Monday morning (25th October). Click here to open the HW.
Note: There will be homeworks for every week. You will be assigned homework on every friday and you have to submit that on monday morning without fail. Click here for the solution of problem 4 in HW 3
5) Homework 5. Deadline Monday morning (02 November). Click here to open the HW.
Note: There will be homeworks for every week. You will be assigned homework on every friday and you have to submit that on monday morning without fail. Click here for the solution of problem 4 in HW 3
Internal exam papers:
First internal question paper
second internal question paper
third internal question paper
Course Syllabus
Revision of Probability theory: Motivating with examples like coin tossing, drawing a cube etc, Concept of probability space, Discrete Random variable,
Continuous random variable, Cumulative distribution function, Probability density function, Independence of random variable, Expectation of a random variable,
Some basic results on sequence of random variables, Law of large numbers, Gaussian random variable, Central limit theorem.
Measures of Information: Motivating about measure of information with examples, Concept of Entropy, Mutual information, Some properties of Entropy and mutual information, Differential entropy
Source Coding: If possible then introduce the concept of typical sets in simple way, Source coding theorem
Channel Coding: Definition of Channel capacity, Shannon's Channel capacity theorem for Discrete memoryless channel, Deriving Channel capacity of Binary Symmetric Channel, Various properties and relations of mutual information and entropy, Differential entropy, Additive White Gaussian Noise Channel (AWGN), Sphere packing argument to find capacity of AWGN channel
Error Control codes: Galios field, Linear codes, Syndrome decoding, Cyclic codes, Reed Muller codes, BCH Codes, Convolutional codes, Introduction to LDPC codes.
Prerequisites
Probability Theory/Random Process, Digital Communication, Working Brain :)
Course Grade
T.B.D
References
1) Digital Communication by Simon Haykins. Click here to download
2) Elements of Information theory by Cover and Thomas, 1st edition