| Web Page |
http://courses.ece.uiuc.edu/ece563 |
| Subject Area |
Communications |
| Course Prerequisites |
Credit in MATH 464 or MATH 564 or ECE 534 |
| Course Directors |
Richard E Blahut
|
| Description |
Mathematical models for channels and sources; entropy, information, data compression, channel capacity, Shannon's theorems, rate-distortion theory. |
| Notes |
Same as: CS 578 and STAT 563 |
| Credit |
4 hours |
| Topics |
- Entropy, relative entropy, mutual information
- Asymptotic equipartition property
- Entropy rates of a stochastic process
- Lossless data compression (Huffman, Ziv-Lempel, Arithmetic, Shannon-Fano codes): Kraft inequality, Shannon's source coding theorem
- Channel capacity: jointly typical sequences, Fano's inequality, Shannon's channel coding theorem and its converse
- Differential entropy
- Gaussian channels
- Rate-distortion theory: Shannon's source coding theorem relative to a fidelity criterion
|
| Course Prerequisites |
One of MATH 464, MATH 564, or ECE 534. |
| Texts |
T. Cover and J. Thomas, Elements of Information Theory, Wiley, 1991.
|
