Understanding Shannon entropy 2 variability and bits

>> YOUR LINK HERE: ___ http://youtube.com/watch?v=beGUI4YzGx4

In this series of videos we'll try to bring some clarity to the concept of entropy. We'll specifically take the Shannon entropy and: • show that it represents the variability of the elements within a distribution, how different are they from each other (general characterization that works in all disciplines) • show that this variability is measured in terms of the minimum number of questions needed to identify an element in the distribution (link to information theory) • show that this is related to the logarithm of the number of permutations over large sequences (link to combinatorics) • show that it is not in general coordinate independent (and that the KL divergence does not fix this) • show that it is coordinate independent on physical state spaces - classical phase space and quantum Hilbert space (that is why those spaces are important in physics) • show the link between the Shannon entropy to the Boltzmann, Gibbs and Von Neumann entropies (link to physics) • • Most of these ideas are from our paper: • https://arxiv.org/abs/1912.02012 • which is part of our bigger project Assumptions of Physics: • https://assumptionsofphysics.org/

#############################

New on site