Conditional entropy

probability
Conditional entropy

Suppose we have two random variables $X$ and $Y$ over different domains $A$ and $B$. The conditional entropy is defined by

$$H(Y \vert X) = - \sum_{a \in A} \sum_{b \in B} \mathbb{P}[X = a, Y = b] \log_2(\mathbb{P}[Y = b \vert X = a]).$$