Past exam question: 2073 Shrawan, Q9(a)
Open in the Padhaiko appQuestion
What do you mean by entropy?
Solution
Definition of Entropy: In information theory (formulated by Claude Shannon), entropy is the statistical measure of the average uncertainty, information content, or surprise associated with the output of a discrete memoryless information source :
where is the probability of occurrence of source symbol . Entropy establishes the fundamental theoretical minimum average codeword length required to encode source symbols losslessly without information distortion ().
Answer:
Entropy is the measure of the average uncertainty or information content produced per symbol by an information source. It sets the absolute theoretical lower limit on lossless data compression.