Information Theory and Most Efficient Codings for Communication or Memory Devices

Abstract

Shannon's theorem about the capacity of a channel is discussed, and it is shown that the most efficient coding is the one yielding the most probable distribution of the code symbols. A specific rule is obtained for this most probable distribution. Most efficient coding is essential for communication channels or memory devices in large scale computers.