Common Informations
These measures all somehow measure shared information, but do not equal the
mutual information in the bivariate case. All of them are importable from
dit.multivariate.
- Gács-Körner Common Information
- Wyner Common Information
- Channel Synthesis
- Exact Common Information
- Exact Common Information Rate
- Functional Common Information
- Kamath-Anantharam Common Information
- Maximum Entropy Function
- MSS Common Information
- Beta Common Information
- Stochastic Gács–Körner Common Information
- Tension Common Information
The directed Kamath–Anantharam quantity
directed_kamath_common_information() is documented on
the Kamath-Anantharam Common Information page.
Several of these quantities are corner points of the Gray–Wyner region; see The Gray-Wyner Network for the operational picture.
Ordering
The common information measures (together with the Dual Total Correlation and CAEKL Mutual Information) form an ordering:
\[\K{X_{0:n}} \leq \J{X_{0:n}}
\leq \B{X_{0:n}}
\leq \C{X_{0:n}}
\leq \G{X_{0:n}}
\leq \F{X_{0:n}}
\leq U(X_{0:n})
\leq \M{X_{0:n}}\]
For a pair of variables the Exact Common Information Rate \(\bar{G}\) fits between the Wyner and exact common informations: \(\C{X : Y} \leq \bar{G}[X : Y] \leq \G{X : Y}\).