General Information¶
- Documentation:
- Downloads:
- Dependencies:
Python 2.7, 3.3, 3.4, 3.5, or 3.6
Optional Dependencies¶
colorama: colored column heads in PID indicating failure modes
cython: faster sampling from distributions
hypothesis: random sampling of distributions
matplotlib, python-ternary: plotting of various information-theoretic expansions
numdifftools: numerical evaluation of gradients and hessians during optimization
pint: add units to informational values
scikit-learn: faster nearest-neighbor lookups during entropy/mutual information estimation from samples
- Mailing list:
None
- Code and bug tracker:
- License:
BSD 3-Clause, see LICENSE.txt for details.
Quickstart¶
The basic usage of dit
corresponds to creating distributions, modifying
them if need be, and then computing properties of those distributions.
First, we import:
In [1]: In [1]: import dit
Suppose we have a really thick coin, one so thick that there is a reasonable
chance of it landing on its edge. Here is how we might represent the coin in
dit
.
In [2]: In [2]: d = dit.Distribution(['H', 'T', 'E'], [.4, .4, .2])
In [3]: In [3]: print(d)
Class: Distribution
Alphabet: ('E', 'H', 'T') for all rvs
Base: linear
Outcome Class: str
Outcome Length: 1
RV Names: None
x p(x)
E 1/5
H 2/5
T 2/5
In [4]: Class: Distribution
In [5]: Alphabet: ('E', H', 'T') for all rvs
...: Base: linear
...: Outcome Class: str
...: Outcome Length: 1
...: RV Names: None
...:
File "<ipython-input-5-765b249d398d>", line 1
Alphabet: ('E', H', 'T') for all rvs
^
SyntaxError: invalid syntax
Calculate the probability of \(H\) and also of the combination: \(H~\mathbf{or}~T\).
In [6]: In [4]: d['H']
Out[6]: 0.4
In [7]: Out[4]: 0.4
In [8]: In [50]: d.event_probability(['H','T'])
Out[8]: 0.8
In [9]: Out[50]: 0.8
Calculate the Shannon entropy and extropy of the joint distribution.
In [10]: In [10]: dit.shannon.entropy(d)
Out[10]: 1.5219280948873621
In [11]: Out[10]: 1.5219280948873621
In [12]: In [11]: dit.other.extropy(d)
Out[12]: 1.1419011889093373
In [13]: Out[11]: 1.1419011889093373
Create a distribution representing the \(\mathbf{xor}\) logic function. Here, we have two inputs, \(X\) and \(Y\), and then an output \(Z = \mathbf{xor}(X,Y)\).
In [14]: In [6]: import dit.example_dists
Calculate the Shannon mutual informations \(\I[X:Z]\), \(\I[Y:Z]\), and \(\I[X,Y:Z]\).
In [15]: In [12]: dit.shannon.mutual_information(d, ['X'], ['Z'])
---------------------------------------------------------------------------
ditException Traceback (most recent call last)
<ipython-input-15-1c8c59aabbb1> in <module>
----> 1 dit.shannon.mutual_information(d, ['X'], ['Z'])
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/shannon/shannon.py in mutual_information(dist, rvs_X, rvs_Y, rv_mode)
157
158 """
--> 159 H_X = entropy(dist, rvs_X, rv_mode=rv_mode)
160 H_Y = entropy(dist, rvs_Y, rv_mode=rv_mode)
161 # Make sure to union the indexes. This handles the case when X and Y
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/shannon/shannon.py in entropy(dist, rvs, rv_mode)
72 rv_mode = RV_MODES.INDICES
73
---> 74 d = dist.marginal(rvs, rv_mode=rv_mode) # pylint: disable=no-member
75 else:
76 d = dist
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/npdist.py in marginal(self, rvs, rv_mode)
1288 # We parse the rv_mode now, so that we can reassign their names
1289 # after coalesce has finished.
-> 1290 rvs, indexes = parse_rvs(self, rvs, rv_mode, unique=True, sort=True)
1291
1292 ## Eventually, add in a method specialized for dense distributions.
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/helpers.py in parse_rvs(dist, rvs, rv_mode, unique, sort)
334 msg = '`rvs` contains invalid random variables, {0}, {1} {2}.'
335 msg = msg.format(indexes, good_indexes, rv_mode)
--> 336 raise ditException(msg)
337
338 # Sort the random variable names (or indexes) by their index.
ditException: `rvs` contains invalid random variables, ['X'], set() 0.
In [16]: Out[12]: 0.0
In [17]: In [13]: dit.shannon.mutual_information(d, ['Y'], ['Z'])
---------------------------------------------------------------------------
ditException Traceback (most recent call last)
<ipython-input-17-90efbc2156b7> in <module>
----> 1 dit.shannon.mutual_information(d, ['Y'], ['Z'])
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/shannon/shannon.py in mutual_information(dist, rvs_X, rvs_Y, rv_mode)
157
158 """
--> 159 H_X = entropy(dist, rvs_X, rv_mode=rv_mode)
160 H_Y = entropy(dist, rvs_Y, rv_mode=rv_mode)
161 # Make sure to union the indexes. This handles the case when X and Y
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/shannon/shannon.py in entropy(dist, rvs, rv_mode)
72 rv_mode = RV_MODES.INDICES
73
---> 74 d = dist.marginal(rvs, rv_mode=rv_mode) # pylint: disable=no-member
75 else:
76 d = dist
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/npdist.py in marginal(self, rvs, rv_mode)
1288 # We parse the rv_mode now, so that we can reassign their names
1289 # after coalesce has finished.
-> 1290 rvs, indexes = parse_rvs(self, rvs, rv_mode, unique=True, sort=True)
1291
1292 ## Eventually, add in a method specialized for dense distributions.
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/helpers.py in parse_rvs(dist, rvs, rv_mode, unique, sort)
334 msg = '`rvs` contains invalid random variables, {0}, {1} {2}.'
335 msg = msg.format(indexes, good_indexes, rv_mode)
--> 336 raise ditException(msg)
337
338 # Sort the random variable names (or indexes) by their index.
ditException: `rvs` contains invalid random variables, ['Y'], set() 0.
In [18]: Out[13]: 0.0
In [19]: In [14]: dit.shannon.mutual_information(d, ['X', 'Y'], ['Z'])
---------------------------------------------------------------------------
ditException Traceback (most recent call last)
<ipython-input-19-1af669dd1aec> in <module>
----> 1 dit.shannon.mutual_information(d, ['X', 'Y'], ['Z'])
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/shannon/shannon.py in mutual_information(dist, rvs_X, rvs_Y, rv_mode)
157
158 """
--> 159 H_X = entropy(dist, rvs_X, rv_mode=rv_mode)
160 H_Y = entropy(dist, rvs_Y, rv_mode=rv_mode)
161 # Make sure to union the indexes. This handles the case when X and Y
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/shannon/shannon.py in entropy(dist, rvs, rv_mode)
72 rv_mode = RV_MODES.INDICES
73
---> 74 d = dist.marginal(rvs, rv_mode=rv_mode) # pylint: disable=no-member
75 else:
76 d = dist
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/npdist.py in marginal(self, rvs, rv_mode)
1288 # We parse the rv_mode now, so that we can reassign their names
1289 # after coalesce has finished.
-> 1290 rvs, indexes = parse_rvs(self, rvs, rv_mode, unique=True, sort=True)
1291
1292 ## Eventually, add in a method specialized for dense distributions.
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/helpers.py in parse_rvs(dist, rvs, rv_mode, unique, sort)
334 msg = '`rvs` contains invalid random variables, {0}, {1} {2}.'
335 msg = msg.format(indexes, good_indexes, rv_mode)
--> 336 raise ditException(msg)
337
338 # Sort the random variable names (or indexes) by their index.
ditException: `rvs` contains invalid random variables, ['X', 'Y'], set() 0.
In [20]: Out[14]: 1.0
Calculate the marginal distribution \(P(X,Z)\). Then print its probabilities as fractions, showing the mask.
In [21]: In [15]: d2 = d.marginal(['X', 'Z'])
---------------------------------------------------------------------------
ditException Traceback (most recent call last)
<ipython-input-21-b067ba4a93be> in <module>
----> 1 d2 = d.marginal(['X', 'Z'])
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/npdist.py in marginal(self, rvs, rv_mode)
1288 # We parse the rv_mode now, so that we can reassign their names
1289 # after coalesce has finished.
-> 1290 rvs, indexes = parse_rvs(self, rvs, rv_mode, unique=True, sort=True)
1291
1292 ## Eventually, add in a method specialized for dense distributions.
~/checkouts/readthedocs.org/user_builds/dit/conda/latest/lib/python3.7/site-packages/dit/helpers.py in parse_rvs(dist, rvs, rv_mode, unique, sort)
334 msg = '`rvs` contains invalid random variables, {0}, {1} {2}.'
335 msg = msg.format(indexes, good_indexes, rv_mode)
--> 336 raise ditException(msg)
337
338 # Sort the random variable names (or indexes) by their index.
ditException: `rvs` contains invalid random variables, ['X', 'Z'], set() 0.
In [22]: In [16]: print(d2.to_string(show_mask=True, exact=True))