Past exam question: 2071 Shrawan, Q9(a)
Open in the Padhaiko appQuestion
Define Information with example.
Solution
Definition of Information: In information theory (formulated by Claude Shannon), information is a quantitative measure of the uncertainty or surprise associated with the occurrence of an event or message :
where is the probability of occurrence of symbol .
- An event that is certain () conveys zero information: .
- A rare event () conveys high information (high surprise value).
Example: Consider the flip of a fair coin:
- Probability of Heads: .
- Information conveyed upon observing Heads:
Answer:
Self-information quantifies the uncertainty resolved by observing an event of probability : bits (e.g., a fair coin toss with conveys ).