Shannon Entropy Average information content:

$$ H(X) = -\sum_i p(x_i) \log_2 p(x_i) $$

Units: bits (if log base 2), nats (if natural log) 1import numpy as np 2 3def entropy(probabilities): 4 """Calculate Shannon entropy""" 5 p = np.array(probabilities) 6 p = p[p > 0] # Remove zeros 7 return -np.sum(p * np.log2(p)) 8 9# …

Read More