Beta Common Information

Yu, Li & Chen [YLC17] interpolate between the Wyner Common Information and the Gács-Körner Common Information using a bound on the conditional maximal correlation:

\[\begin{split}C_{\beta}(X_1 : \ldots : X_n \mid Z) = \inf_{\substack{P_{U \mid X_{1:n} Z} \\ \max_{i \neq j} \rho_m(X_i; X_j \mid U, Z) \le \beta}} I(X_{1:n} ; U \mid Z)\end{split}\]

Special cases: \(\beta = 0\) recovers Wyner common information; \(\beta \to 1\) recovers Gács–Körner common information.

In [1]: from dit.multivariate import beta_common_information

In [2]: from dit.example_dists import Xor

In [3]: beta_common_information(Xor(), beta=0.0, niter=4)
Out[3]: 0.0

API

beta_common_information(dist, beta, rvs=None, crvs=None, niter=None, maxiter=1000, polish=False, bound=None)[source]

Compute the beta-approximate common information (information-correlation function) of dist.

\[C_{\beta}(X_1 : \ldots : X_n | Z) = \inf_{P_{U|X_1 \ldots X_n Z} :\; \max_{i \neq j} \rho_m(X_i; X_j | U, Z) \le \beta} I(X_1 \ldots X_n ; U | Z)\]

This generalises Wyner common information (beta=0) and, in the limit beta -> 1, Gacs-Korner common information. When beta >= rho_m(X;Y) the value is 0 because no auxiliary variable is needed.

Parameters:

  • dist (Distribution) – The distribution from which the common information is computed.

  • beta (float) – The maximal-correlation threshold, 0 <= beta <= 1.

  • rvs (list of lists, None) – A list of lists. Each inner list specifies the indexes of the random variables for one group. If None, each outcome coordinate forms its own group (equivalent to rvs=dist.rvs).

  • crvs (list, None) – A single list of indexes specifying the random variables to condition on. If None, no conditioning is applied.

  • niter (int, None) – Number of basin-hopping restarts.

  • maxiter (int) – Maximum iterations per local optimisation.

  • polish (float or False, optional) – If a float, perform a second optimisation pass with probabilities below this threshold zeroed out. Default is False (no polishing), matching Wyner/ECI common-information optimizers.

  • bound (int, None) – Artificial bound on the cardinality of U. If None, the theoretical bound from Lemma 15a is used.

Returns:

c_beta – The beta-approximate common information.

Return type:

float