Functional Common Information

The functional common information captures the minimum amount of information neccessary to capture all of a distribution’s share information using a function of that information. In other words:

\[\begin{split}\F{X_{0:n} \mid Y_{0:m}} = \min_{\substack{\ind X_{0:n} \mid Y_{0:m}, W \\ W = f(X_{0:n}, Y_{0:m})}} \H{W}\end{split}\]

Relationship To Other Measures of Common Information

Since this is an additional constraint on the Exact common information, it is generally larger than it, and since its constraint is weaker than that of the MSS Common Information, it is generally less than it:

\[\G{X_{0:n}} \leq \F{X_{0:n}} \leq \M{X_{0:n}}\]

API

functional_common_information(dist, rvs=None, crvs=None)[source]

Compute the functional common information, F, of dist. It is the entropy of the smallest random variable W such that all the variables in rvs are rendered independent conditioned on W, and W is a function of rvs.

Parameters:

  • dist (Distribution) – The distribution from which the functional common information is computed.

  • rvs (list, None) – A list of lists. Each inner list specifies the indexes of the random variables used to calculate the total correlation. If None, then the total correlation is calculated over all random variables, which is equivalent to passing rvs=dist.rvs.

  • crvs (list, None) – A single list of indexes specifying the random variables to condition on. If None, then no variables are conditioned on.

Returns:

F – The functional common information.

Return type:

float