Derivation of the Gibbs entropy formula from the Boltzmann entropy

There are many ways to arrive at the Gibbs entropy formula

$S= -k_B \displaystyle \sum_{i} p_i \log p_i~,$

which is actually identical, except for units, to the Shannon entropy formula. Here I document a relatively straightforward derivation of the formula which might be the easiest route to develop intuition for undergraduate students who already know the formula for the Boltzmann entropy,

$S(E)= k_B \log \Omega(E)~,$

where $\Omega(E)$ is the effective number of configurations of an isolated system with total energy $E$ . This derivation is not new of course, but I decided to write it up anyway because it is a particularly elegant argument and is not always given emphasis in the usual textbooks.

Let us start by considering some general statistical ensemble such that a state $i$ has energy $E_i$ and probability $p_i=p(E_i)$ which is the same for all states with the same energy $E_i$ . For fixed energy $E_i$ we can assume, from the principle of equal a priori probabilities, which is valid in the microcanonical ensemble, that

$p_i = \displaystyle \frac 1 {\Omega(E_i)}~.$

Hence we obtain for the Boltzmann entropy

$S(E_i)=k_B \log \Omega(E_i) = -k_B \log p_i~.$

For our general ensemble, the mean entropy is given by the weighted average, over all possible states, of the Boltzmann entropy, according to

$S= \langle S(E_i) \rangle _i = \sum_i p_i S(E_i) = \sum_i p_i~ (-k_B \log p_i)$

from which we obtain the famous Gibbs formula for the Boltzmann-Gibbs entropy:

$S = -k_B \displaystyle \sum_i p_i \log p_i~.$

4 thoughts on “Derivation of the Gibbs entropy formula from the Boltzmann entropy”

Eu concordo com o que você diz, Gandhi, é elegante e nem sempre é dado ênfase nos livros didáticos usuais. E sem falar da tamanha importância!

Abraço

Reply ↓

Gandhi Viswanathan on 10/03/2021 at 12:08 said:

Muito obrigado Marcelo! 🙂

Reply ↓

i don’t understand why Pi is the same when you change the ensemble?? (Microcanonical -> general)

Reply ↓

Gandhi Viswanathan on 05/01/2022 at 13:56 said:

In general it is not the same. But for fixed energy, assuming thermal equilibrium, it is reasonable to assume equiprobability.

Reply ↓

Gandhi Viswanathan's Blog

Physics, Math and Life as a Scientist

Derivation of the Gibbs entropy formula from the Boltzmann entropy

4 thoughts on “Derivation of the Gibbs entropy formula from the Boltzmann entropy”

Leave a Reply Cancel reply

Share this:

Like this:

Related

4 thoughts on “Derivation of the Gibbs entropy formula from the Boltzmann entropy”

Leave a Reply Cancel reply