DeWeeselike Measures¶
Mike DeWeese has introduced a family of multivariate information measures based on a multivariate extension of the data processing inequality. The general idea is the following: local modification of a single variable can not increase the amount of correlation or dependence it has with the other variables. Consider, however, the triadic distribution:
In [1]: from dit.example_dists import dyadic, triadic
In [2]: print(triadic)
Class: Distribution
Alphabet: ('0', '1', '2', '3') for all rvs
Base: linear
Outcome Class: str
Outcome Length: 3
RV Names: None
x p(x)
000 1/8
022 1/8
111 1/8
133 1/8
202 1/8
220 1/8
313 1/8
331 1/8
This particular distribution has zero coinformation:
In [3]: from dit.multivariate import coinformation
In [4]: coinformation(triadic)
Out[4]: 0.0
Yet the distribution is a product of a giant bit (coinformation \(1.0\)) and the xor (coinformation \(1.0\)), and so there exists within it the capability of having a coinformation of \(1.0\) if the xor component were dropped. This is exactly what the DeWeese construction captures:
In [5]: from dit.multivariate import deweese_coinformation
In [6]: deweese_coinformation(triadic)
Out[6]: 1.0
DeWeese version of the total_correlation, dual_total_correlation, and caekl_mutual_information are also available, and operate on an arbitrary number of variables with optional conditional variables.
API¶

deweese_coinformation
(*args, **kwargs)¶ Compute the DeWeese coinformation.
Parameters:  dist (Distribution) – The distribution of interest.
 rvs (iter of iters, None) – The random variables of interest. If None, use all.
 crvs (iter, None) – The variables to condition on. If None, none.
 niter (int, None) – If specified, the number of optimization steps to perform.
 deterministic (bool) – Whether the functions to optimize over should be deterministic or not. Defaults to False.
 rv_mode (str, None) – Specifies how to interpret rvs and crvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of crvs and rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted, which defaults to ‘indices’.
Returns: val – The value of the DeWeese coinformation.
Return type:

deweese_total_correlation
(*args, **kwargs)¶ Compute the DeWeese total correlation.
Parameters:  dist (Distribution) – The distribution of interest.
 rvs (iter of iters, None) – The random variables of interest. If None, use all.
 crvs (iter, None) – The variables to condition on. If None, none.
 niter (int, None) – If specified, the number of optimization steps to perform.
 deterministic (bool) – Whether the functions to optimize over should be deterministic or not. Defaults to False.
 rv_mode (str, None) – Specifies how to interpret rvs and crvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of crvs and rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted, which defaults to ‘indices’.
Returns: val – The value of the DeWeese total correlation.
Return type:

deweese_dual_total_correlation
(*args, **kwargs)¶ Compute the DeWeese dual total correlation.
Parameters:  dist (Distribution) – The distribution of interest.
 rvs (iter of iters, None) – The random variables of interest. If None, use all.
 crvs (iter, None) – The variables to condition on. If None, none.
 niter (int, None) – If specified, the number of optimization steps to perform.
 deterministic (bool) – Whether the functions to optimize over should be deterministic or not. Defaults to False.
 rv_mode (str, None) – Specifies how to interpret rvs and crvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of crvs and rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted, which defaults to ‘indices’.
Returns: val – The value of the DeWeese dual total correlation.
Return type:

deweese_caekl_mutual_information
(*args, **kwargs)¶ Compute the DeWeese caekl mutual information.
Parameters:  dist (Distribution) – The distribution of interest.
 rvs (iter of iters, None) – The random variables of interest. If None, use all.
 crvs (iter, None) – The variables to condition on. If None, none.
 niter (int, None) – If specified, the number of optimization steps to perform.
 deterministic (bool) – Whether the functions to optimize over should be deterministic or not. Defaults to False.
 rv_mode (str, None) – Specifies how to interpret rvs and crvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of crvs and rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted, which defaults to ‘indices’.
Returns: val – The value of the DeWeese caekl mutual information.
Return type: