Maths encyclopedia and lessons  
Search

Mathematics Encyclopedia and Lessons

 
     
 

Lessons

Popular
Subjects

algebra
arithmetic
calculus
equations
geometry
differential equations
trigonometry
number theory
probability theory
more
 

References

applied mathematics
mathematical games
mathematicians
more
 
 

Conditional entropy

The conditional entropy is an entropy measure used in information theory. The conditional entropy measures how much entropy a random variable Y has remaining if we have already learned completely the value of a second random variable X. It is referred to as the entropy of Y conditional on X, and is written H(Y | X). Like other entropies, the conditional entropy is measured in bits.

Given random variables X and Y with entropies H(X) and H(Y), and with a joint entropy H(X,Y), the conditional entropy of Y given X is defined as H(Y|X) \equiv H(X,Y) - H(X). Intuitively, the combined system contains H(X,Y) bits of information. If we learn the value of X, we have gained H(X) bits of information, and the system has H(Y | X) bits remaining.

H(Y | X) = 0 if and only if the value of Y is completely determined by the value of X. Conversely, H(Y | X) = H(Y) if and only if Y and X are independent random variables.

In quantum information theory, the conditional entropy is generalized to the conditional quantum entropy.

01-04-2007 01:18:14
The contents of this article are licensed from Wikipedia.org
under the GNU Free Documentation License. How to see transparent copy