Perplexity¶
The perplexity is a trivial measure to make the Entropy more intuitive:
\[\P{X} = 2^{\H{X}}\]
The perplexity of a random variable is the size of a uniform distribution that would have the same entropy. For example, a distribution with 2 bits of entropy has a perplexity of 4, and so could be said to be “as random” as a four-sided die.
The conditional perplexity is defined in the natural way:
\[\P{X | Y} = 2^{\H{X | Y}}\]
We can see that the xor distribution is “4-way” perplexed:
In [1]: from dit.other import perplexity
In [2]: from dit.example_dists import Xor
In [3]: perplexity(Xor())
Out[3]: 4.0
API¶
-
perplexity(dist, rvs=None, crvs=None, rv_mode=None)[source]¶ Parameters: - dist (Distribution) – The distribution from which the perplexity is calculated.
- rvs (list, None) – The indexes of the random variable used to calculate the perplexity. If None, then the perpelxity is calculated over all random variables.
- crvs (list, None) – The indexes of the random variables to condition on. If None, then no variables are condition on.
- rv_mode (str, None) – Specifies how to interpret the elements of rvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted.
Returns: P – The perplexity.
Return type: