Necessary Conditional Entropy

The necessary conditional entropy [CPC10] quantifies the amount of information that a random variable \(X\) necessarily must carry above and beyond the mutual information \(\I{X : Y}\) to actually contain that mutual information:

\[\H{X \dagger Y} = \H{ X \mss Y | Y}\]

API

necessary_conditional_entropy(dist, rvs=None, crvs=None)[source]

Calculates the necessary conditional entropy \(\H[X \dagger Y]\). This is the entropy of the minimal sufficient statistic of X about Y, given Y.

Parameters:

  • dist (Distribution) – The distribution from which the necessary conditional entropy is calculated.

  • rvs (list, None) – The indexes of the random variable used to calculate the necessary conditional entropy. If None, then the entropy is calculated over all random variables.

  • crvs (list, None) – The indexes of the random variables to condition on. If None, then no variables are conditioned on.

Returns:

H – The necessary conditional entropy.

Return type:

float

Raises:

ditException – Raised if rvs or crvs contain non-existant random variables.

Example