O-Information

The O-information [RMGJ19] is the difference between the Total Correlation and the Dual Total Correlation:

\[\O{X_{0:n}} = \T{X_{0:n}} - \B{X_{0:n}}\]

Positive values indicate that redundant (shared) structure dominates; negative values indicate that synergistic structure dominates. On a three-variable giant bit it is \(+1\) bit; on three-bit parity (xor) it is \(-1\) bit:

In [1]: from dit.multivariate import o_information

In [2]: from dit.example_dists import giant_bit, n_mod_m, Xor

In [3]: o_information(giant_bit(3, 2))
Out[3]: 1.0

In [4]: o_information(Xor())
Out[4]: -1.0

The Cohesion interpolates between \(\T{}\) and \(\B{}\) at finite order \(k\); the O-information is the single-number summary \(T - B\).

API

o_information(dist, rvs=None, crvs=None)[source]

Computes the O-information, defined as the total correlation minus the dual total correlation.

Parameters:

dist (Distribution) – The distribution from which the o-information is calculated.
rvs (list, None) – A list of lists. Each inner list specifies the indexes of the random variables used to calculate the o-information. If None, then the o-information is calculated over all random variables, which is equivalent to passing rvs=dist.rvs.
crvs (list, None) – A single list of indexes specifying the random variables to condition on. If None, then no variables are conditioned on.

Returns:

O – The o-information.

Return type:

float

Examples

>>> d = dit.example_dists.n_mod_m(5, 2)
>>> dit.multivariate.o_information(d)
3.0
>>> dit.multivariate.o_information(d, rvs=[[0], [1], [3], [4]], [2])
-2.0

Raises:: ditException – Raised if dist is not a joint distribution or if rvs or crvs contain non-existant random variables.