Information Measures
dit supports many information measures, ranging from as standard as the Shannon entropy to as exotic as Gács-Körner common information. We organize these quantities into the following groups.
We first have the Shannon-like measures. These quantities are based on sums and differences of entropies, conditional entropies, or mutual informations of random variables:
Common informations quantify shared information through constructions (common random variables, auxiliary channels, sufficient statistics) rather than sums of entropies; in the bivariate case they do not reduce to the mutual information:
- Common Informations
- Gács-Körner Common Information
- Wyner Common Information
- Channel Synthesis
- Exact Common Information
- Exact Common Information Rate
- Functional Common Information
- Kamath-Anantharam Common Information
- Maximum Entropy Function
- MSS Common Information
- Beta Common Information
- Stochastic Gács–Körner Common Information
- Tension Common Information
- Ordering
This next group of measures can not be represented on information diagrams, and can not really be directly compared to the measures above:
There are also measures of “distance” or divergence between two (and in some cases, more) distributions:
Several measures of shared information are related to the ability of two (or more) agents to agree upon a secret key in the face of an eavesdropper: