Derivation of the Gibbs entropy formula from the Boltzmann entropy

Posted on by

There are many ways to arrive at the Gibbs entropy formula

S= -k_B \displaystyle \sum_{i} p_i \log p_i~,

which is actually identical, except for units, to the Shannon entropy formula. Here I document a relatively straightforward derivation of the formula which might be the easiest route to develop intuition for undergraduate students who already know the formula for the Boltzmann entropy,

S(E)= k_B \log \Omega(E)~,

where \Omega(E) is the effective number of configurations of an isolated system with total energy E. This derivation is not new of course, but I decided to write it up anyway because it is a particularly elegant argument and is not always given emphasis in the usual textbooks.

Let us start by considering some general statistical ensemble such that a state i has energy E_i and probability p_i=p(E_i) which is the same for all states with the same energy E_i. For fixed energy E_i we can assume, from the principle of equal a priori probabilities, which is valid in the microcanonical ensemble, that

p_i = \displaystyle \frac 1 {\Omega(E_i)}~.

Hence we obtain for the Boltzmann entropy

S(E_i)=k_B \log \Omega(E_i) = -k_B \log p_i~.

For our general ensemble, the mean entropy is given by the weighted average, over all possible states, of the Boltzmann entropy, according to

S= \langle S(E_i) \rangle _i = \sum_i p_i S(E_i) = \sum_i p_i~ (-k_B \log p_i)

from which we obtain the famous Gibbs formula for the Boltzmann-Gibbs entropy:

S = -k_B \displaystyle \sum_i p_i \log p_i~.

 

 

 

4 thoughts on “Derivation of the Gibbs entropy formula from the Boltzmann entropy”

  1. Eu concordo com o que você diz, Gandhi, é elegante e nem sempre é dado ênfase nos livros didáticos usuais. E sem falar da tamanha importância!

    Abraço

    Reply ↓

  2. i don’t understand why Pi is the same when you change the ensemble?? (Microcanonical -> general)

    Reply ↓

Leave a Reply

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s