Divergences

Divergences are measures of comparison between distributions:

• Cross Entropy
• Kullback-Leibler Divergence
• Jensen-Shannon Divergence
• Earth Mover’s Distance
• Copy Mutual Information

While the cross entropy and the Kullback-Leibler divergence are not true metrics (they are not symmetric), the square root of the Jensen-Shannon divergence is.