Tsallis Entropy

The Tsallis entropy is a generalization of the Shannon (or Boltzmann-Gibbs) entropy to the case where entropy is nonextensive. It is given by:

\[\TE{q}{X} = \frac{1}{q - 1} \left( 1 - \sum_{x \in \mathcal{X}} p(x)^q \right)\]
In [1]: from dit.other import tsallis_entropy

In [2]: from dit.example_dists import n_mod_m

In [3]: d = n_mod_m(4, 3)

In [4]: tsallis_entropy(d, 4)
Out[4]: 0.3333163982455249

Non-additivity

One interesting property of the Tsallis entropy is the relationship between the joint Tsallis entropy of two indpendent systems, and the Tsallis entropy of those subsystems:

\[\TE{q}{X, Y} = \TE{q}{X} + \TE{q}{Y} + (1-q)\TE{q}{X}\TE{q}{Y}\]

API

tsallis_entropy(dist, order, rvs=None)[source]

Compute the Tsallis entropy of order order.

Parameters:

  • dist (Distribution) – The distribution to take the Tsallis entropy of.

  • order (float >= 0) – The order of the Tsallis entropy.

  • rvs (list, None) – The indexes of the random variable used to calculate the Tsallis entropy of. If None, then the Tsallis entropy is calculated over all random variables.

Returns:

S_q – The Tsallis entropy.

Return type:

float

Raises:

  • ditException – Raised if rvs or crvs contain non-existant random variables.

  • ValueError – Raised if order is not a non-negative float.