General Information

Documentation:

http://docs.dit.io

Downloads:

https://pypi.org/project/dit/

https://anaconda.org/conda-forge/dit

Dependencies:

• Python 3.9+

• boltons

• debtcollector

• lattices

• loguru

• networkx

• numpy

• PLTable

• scipy

Optional Dependencies

• colorama: colored column heads in PID indicating failure modes

• cython: faster sampling from distributions

• hypothesis: random sampling of distributions

• jax, jaxlib: JAX-based optimization backend with autodiff support

• matplotlib, python-ternary: plotting of various information-theoretic expansions

• numdifftools: numerical evaluation of gradients and hessians during optimization

• pint: add units to informational values

• scikit-learn: faster nearest-neighbor lookups during entropy/mutual information estimation from samples

• torch: PyTorch-based optimization backend with autodiff and GPU support

• xarray: Distribution class for labeled, algebra-friendly distributions

Code and bug tracker:

https://github.com/dit/dit

License:

BSD 3-Clause, see LICENSE.txt for details.

Quickstart

The basic usage of dit corresponds to creating distributions, modifying them if need be, and then computing properties of those distributions. First, we import:

In [1]: import dit

Suppose we have a really thick coin, one so thick that there is a reasonable chance of it landing on its edge. Here is how we might represent the coin in dit.

In [2]: d = dit.Distribution(['H', 'T', 'E'], [.4, .4, .2])

In [3]: print(d)
Class:    Distribution
Alphabet: (('E', 'H', 'T'),)
Base:     linear

x        p(X0)
('E',)   1/5
('H',)   2/5
('T',)   2/5

Calculate the probability of \(H\) and also of the combination: \(H~\mathbf{or}~T\).

In [4]: d['H']
Out[4]: 0.4

In [5]: d.event_probability(['H','T'])
Out[5]: 0.8

Calculate the Shannon entropy and extropy of the joint distribution.

In [6]: dit.shannon.entropy(d)
Out[6]: 1.5219280948873621

In [7]: dit.other.extropy(d)
Out[7]: 1.1419011889093373

Create a distribution representing the \(\mathbf{xor}\) logic function. Here, we have two inputs, \(X\) and \(Y\), and then an output \(Z = \mathbf{xor}(X,Y)\).

In [8]: import dit.example_dists

In [9]: d = dit.example_dists.Xor()

In [10]: d.set_rv_names(['X', 'Y', 'Z'])

In [11]: print(d)
Class:    Distribution
Alphabet: (('0', '1'), ('0', '1'), ('0', '1'))
Base:     linear

x                 p(X,Y,Z)
('0', '0', '0')   1/4
('0', '1', '1')   1/4
('1', '0', '1')   1/4
('1', '1', '0')   1/4

Calculate the Shannon mutual informations \(\I[X:Z]\), \(\I[Y:Z]\), and \(\I[X,Y:Z]\).

In [12]: dit.shannon.mutual_information(d, ['X'], ['Z'])
Out[12]: 0.0

In [13]: dit.shannon.mutual_information(d, ['Y'], ['Z'])
Out[13]: 0.0

In [14]: dit.shannon.mutual_information(d, ['X', 'Y'], ['Z'])
Out[14]: 1.0

Calculate the marginal distribution \(P(X,Z)\). Then print its probabilities as fractions, showing the mask.

In [15]: d2 = d.marginal(['X', 'Z'])

In [16]: print(d2.to_string(show_mask=True, exact=True))
Class:    Distribution
Alphabet: (('0', '1'), ('0', '1'))
Base:     linear

x            p(X,Z)
('0', '0')   1/4
('0', '1')   1/4
('1', '0')   1/4
('1', '1')   1/4

Convert the distribution probabilities to log (base 3.5) probabilities, and access its probability mass function.

In [17]: d2.set_base(3.5)

In [18]: d2.pmf
Out[18]: array([-1.10658951, -1.10658951, -1.10658951, -1.10658951])

Draw 5 random samples from this distribution.

In [19]: d2.rand(5)
Out[19]: [('0', '1'), ('1', '0'), ('0', '0'), ('0', '1'), ('0', '0')]

Enjoy!

Install

The easiest way to install is:

pip install dit

If you want to install dit within a conda environment, you can simply do:

conda install -c conda-forge dit

For development, we recommend uv:

git clone https://github.com/dit/dit.git
cd dit
uv sync --extra dev

This installs dit in editable mode with all development dependencies (tests, docs, linting, type checking, and optional backends).

To install specific optional extras:

# JAX optimization backend
pip install "dit[jax]"

# PyTorch optimization backend
pip install "dit[torch]"