ECE 563 - Information Theory

Semesters Offered

TitleRubricSectionCRNTypeTimesDaysLocationInstructor
Information TheoryECE563A37140DIS1230 - 1350 T R  2015 ECE Building Pramod Viswanath

Official Description

Mathematical models for channels and sources; entropy, information, data compression, channel capacity, Shannon's theorems, and rate-distortion theory. Course Information: Prerequisite: One of ECE 534, MATH 464, MATH 564.

Prerequisites

Credit in ECE 534 or MATH 464 or MATH 564

Subject Area

Communications

Course Directors

Description

Mathematical models for channels and sources; entropy, information, data compression, channel capacity, Shannon's theorems, rate-distortion theory.

Notes

Same as: CS 578 and STAT 563

Topics

  • Entropy, relative entropy, mutual information
  • Asymptotic equipartition property
  • Entropy rates of a stochastic process
  • Lossless data compression (Huffman, Ziv-Lempel, Arithmetic, Shannon-Fano codes): Kraft inequality, Shannon's source coding theorem
  • Channel capacity: jointly typical sequences, Fano's inequality, Shannon's channel coding theorem and its converse
  • Differential entropy
  • Gaussian channels
  • Rate-distortion theory: Shannon's source coding theorem relative to a fidelity criterion

Detailed Description and Outline

Topics:

  • Entropy, relative entropy, mutual information
  • Asymptotic equipartition property
  • Entropy rates of a stochastic process
  • Lossless data compression (Huffman, Ziv-Lempel, Arithmetic, Shannon-Fano codes): Kraft inequality, Shannon's source coding theorem
  • Channel capacity: jointly typical sequences, Fano's inequality, Shannon's channel coding theorem and its converse
  • Differential entropy
  • Gaussian channels
  • Rate-distortion theory: Shannon's source coding theorem relative to a fidelity criterion

Same as: CS 578 and STAT 563

Texts

T. Cover and J. Thomas, Elements of Information Theory, Wiley, 1991.

Last updated

2/13/2013