S-Information

The S-information (also known as the exogenous information) [RMGJ19] quantifies the total amount of dependency between each individual variable and the rest of a system. It is defined as the sum of the Total Correlation \(\T{}\) and the Dual Total Correlation \(\B{}\):

\[\S{X_{0:n}} = \T{X_{0:n}} + \B{X_{0:n}}\]

Equivalently, it is the sum, over each variable, of the mutual information between that variable and all the others:

\[\S{X_{0:n}} = \sum_{i=0}^{n-1} \I{X_i : X_{\{0:n\} \setminus i}}\]

The S-information is a special case of both the \(\Delta^k\) and \(\Gamma^k\) measures at \(k = 0\); see Delta^k and Gamma^k.

In [1]: from dit.multivariate import s_information

In [2]: from dit.example_dists import n_mod_m

In [3]: d = n_mod_m(5, 2)

In [4]: s_information(d)
Out[4]: 5.0

API

s_information(dist, rvs=None, crvs=None)[source]

Computes the S-information, defined as the sum of the total correlation and the dual total correlation.

Parameters:

  • dist (Distribution) – The distribution from which the s-information is calculated.

  • rvs (list, None) – A list of lists. Each inner list specifies the indexes of the random variables used to calculate the s-information. If None, then the s-information is calculated over all random variables, which is equivalent to passing rvs=dist.rvs.

  • crvs (list, None) – A single list of indexes specifying the random variables to condition on. If None, then no variables are conditioned on.

Returns:

S – The s-information.

Return type:

float

Examples

>>> d = dit.example_dists.n_mod_m(5, 2)
>>> dit.multivariate.s_information(d)
5.0
Raises:

ditException – Raised if dist is not a joint distribution or if rvs or crvs contain non-existant random variables.