Quax Synergy
Quax, Har-Shemesh & Sloot [QHSS17] quantify synergistic information via a synergistic random variable (SRV) \(S\) of the sources \(X\): \(I(S:X) > 0\) while \(I(S:X_i) = 0\) for every source. The synergistic information that a target \(Y\) stores about \(X\) is then \(I(Y:S)\) for an SRV that maximises \(I(S:X)\).
This is not a PID synergy atom: synergistic and unique information can coexist in \(Y\).
In [1]: from dit.multivariate import quax_synergy
In [2]: from dit.example_dists import Xor
In [3]: quax_synergy(Xor(), [[0], [1]], [2], niter=8)
Out[3]: 1.0
API
- quax_synergy(dist, sources, target, crvs=None, niter=None, maxiter=1000, polish=1e-06, bound=None)[source]
Compute the synergistic information I_syn(sources -> target) as defined by Quax, Har-Shemesh & Sloot (2017).
Finds a Synergistic Random Variable (SRV) S that maximises I(S : X) subject to I(S : X_i) = 0 for each source X_i, then returns I(Y : S).
\[I_{\mathrm{syn}}(X \to Y) = \max_{S:\; I(S:X)>0,\; \forall i\, I(S:X_i)=0} I(Y : S)\]- Parameters:
dist (Distribution) – The joint distribution over sources and target.
sources (list of lists) – Each inner list gives the indices (or names) of one source variable group X_i.
target (list) – The indices (or names) of the target variable Y.
crvs (list, None) – Variables to condition on.
niter (int, None) – Number of basin-hopping restarts.
maxiter (int) – Maximum iterations per local optimisation.
polish (float, False) – If a float, perform a polishing pass zeroing probabilities below this threshold. If False, skip polishing.
bound (int, None) – Cardinality bound on S. If None, a theoretical bound is used.
- Returns:
isyn – The synergistic information, in bits (before unit conversion).
- Return type:
- max_synergistic_entropy(dist, rvs=None, crvs=None)[source]
Compute the analytical upper bound on the mutual information that any SRV can have about a set of variables.
\[H(X_1, \ldots, X_n) - \max_i H(X_i)\]This is the maximum possible synergistic entropy of the sources, per Equation 17 of Quax et al. (2017).