Shannon Entropy Average information content: $$ H(X) = -\sum_i p(x_i) \log_2 p(x_i) $$ Units: bits (if log base 2), nats (if natural log) 1import numpy as np 2 3def entropy(probabilities): 4 """Calculate Shannon entropy""" 5 p = np.array(probabilities) 6 p = p[p > 0] # Remove zeros 7 return …