Entropy and Temperature

Eq. 2.1 introduced the quantity $\Omega(N,V,E)$ as the number of states available to a system under the constraints of constant number of particles, $N$, volume, $V$, and energy $E$. The fundamental postulate of statistical mechanics, also called the “rational basis”, is the following:

In statistical equilibrium, all states consistent with the constraints of $N$, $V$, and $E$ are equally probable.

or

$$P_\nu = 1/\Omega(N,V,E).$$ (13)

This relation is often referred to as a statement of the “equal a priori probabilities in state space.” Another way of saying the same thing: The probability distribution for states in the microcanonical ensemble is uniform.

This postulate reflects the fact that we are maximally uncertain with regards to the probabilities of any particular arrangements of degrees of freedom inside a closed system. A closed system is one which cannot exchange energy, volume, or particles with the environment. As such, there is quite literally no way for us to learn anything at all about how particles are arranged, so we must assume all arrangments that satisfy the given energy, volume and number of particles are equiprobable.

One link between statistical mechanics and classical thermodynamics is given by a definition of entropy:

$$S \equiv k_B\ln\Omega$$ (14)

Note two important properties of $S$. First, it is extensive: if we consider a compound system made of subsystems $A$ and $B$ with $\Omega_A$ and $\Omega_B$ as the respective number of states, the total number of states is $\Omega_A\Omega_B$, and therefore $S = S_A + S_B$. Second, it is consistent with the second law of thermodynamics: putting any constraint on the system lowers its entropy because the constraint lowers the number of accessible states.

Temperature is defined using entropy: $1/T \equiv \left(\partial S/\partial E\right)_{N,V}$, or

$$\beta = \left(k_BT\right)^{-1} = \left(\partial\ln\Omega/\partial E\right)$$ (15)

Now we will consider constraining our system not with constant $E$, but with constant $T$. The set of all possible states satisfying constraints of $N$, $V$, and $T$ is called the canonical ensemble. We now ask, what is the probability of any microstate in this ensemble? Consider a closed system divided into a small subsystem $A$ surrounded by a large “bath” $B$. We imagine that these two subsystems exchange only thermal energy, but no particles, and their volumes remain fixed. We seek to compute the probability of finding the total system in a state such that subsystem $A$ has energy $E_A$. The entire system is microcanonical, so the total energy, $E$ is constant, as is the total number of states available to the system, $\Omega(E)$ (we omit the $N$ and $V$ for simplicity).

When $A$ has energy $E_A$, the total system energy is $E = E_A + E_B$, where $E_B$ is the energy of the bath. By constraining system $A$'s energy, we have reduced the number of states available to the whole system to $\Omega(E-E_A)$. So, using the fundamental postulate, the probability of observing the closed system in a state in which subsystem $A$ has energy $E_A$ is

$$P_A = \frac{\left[\begin{array}{l} \mbox{Number of states for whi... ...} \end{array}\right]} = \frac{\Omega\left(E - E_A\right)}{\Omega\left(E\right)}$$ (16)

We can expand $\Omega\left(E - E_A\right)$ in a Taylor series around $E_A = 0$:

$$P_A \propto \Omega\left(E - E_A\right) = \exp\left[\ln\Omega\left(E-E_A\right)\right]$$ (17)

$$= \exp\left[\ln\Omega\left(E\right) - E_A\frac{\partial\ln\Omega}{\partial E} + \cdots\right],$$ (18)

where the partial derivative implies we are holding $N$ and $V$ fixed. We can truncate the Taylor expansion at the first-order term, because higher order terms become less and less important as the size of subsystem $B$ becomes larger and larger. What results is the Boltzmann distribution law for energies of a system at constant temperature:

$$P_A \propto \exp\left(-\beta E_A\right)$$ (19)

The normalization condition requires that for all energies of subsystem $A$, $E_A$,

$$\sum_A P_A = 1 = \sum_A \exp\left(-\beta E_A\right) \equiv Q\left(N,V,T\right),$$ (20)

which defines the canonical partition function, $Q$. Therefore,

$$P_A = Q^{-1}\exp\left(-\beta E_A\right)$$ (21)

Because some energies can correspond to more than one microstate, we should distinguish between “states” and “energy levels.” We can express the canonical partition function as

$$Q = \sum_{\begin{array}{cc}\nu \mbox{states}\end{array}} e^{-\b... ...n{array}{cc}l \mbox{levels}\end{array}} \Omega\left(E_l\right) e^{-\beta E_l}$$ (22)

where, as we have seen, $\Omega\left(E\right)$ is the number of microstates with energy $E$. Moving to the continuum limit, and assuming a reference energy of $E_{ref} = 0$,

$$Q \rightarrow \int_0^\infty dE \overline\Omega\left(E\right)e^{-\beta E}$$ (23)

where $\overline\Omega\left(E\right)$ is the density of states. What is this equation telling us? It is telling us that $Q$ is the Laplace transform of $\overline\Omega$. We know that transform pairs are unique, and hence, both $Q$ and $\overline\Omega$ contain the same information.

We recognize that for a system described by a canonical ensemble, the energy is a fluctuating quantity. And we now have the probability of observing a state with a given energy, so we can use Eq. 1 to compute the average energy, $\left<E\right>$. Consider

$$\left<E\right> = \left<E_\nu\right> = \sum_\nu P_\nu E_\nu$$ (24)

$$= \left[\sum_\nu E_\nu \exp\left(-\beta E_\nu\right)\right] \left/ \left[\sum_\nu \exp\left(-\beta E_\nu\right)\right] \right.$$ (25)

Notice that

$$\sum_\nu E_\nu \exp\left(-\beta E_\nu\right) = -\left(\partial Q \left/ \partial\beta\right.\right)$$ (26)

Recalling that $d\ln f(x) / dx = 1/f df/dx$, we see that

$$\left<E\right> = -\left(\partial Q \left/ \partial\beta\right.\right)\left/Q\right.$$ (27)

$$= -\left(\partial \ln Q \left/ \partial\beta\right.\right)_{N,V}$$ (28)

Now, let us consider the average magnitude of the fluctuations in energy in the canonical ensemble.

$$\left<\left(\delta E\right)^2\right> = \left<\left(E - \left<E\right>\right)^2\right>$$ (29)

$$= \left<E^2\right> - \left<E\right>^2$$ (30)

$$= \sum_\nu P_\nu E_\nu^2 - \left(\sum_\nu P_\nu E_\nu\right)^2$$ (31)

$$= Q^{-1}\left(\frac{\partial^2Q}{\partial\beta^2}\right)_{N,V} - Q^{-2}\left(\frac{\partial Q}{\partial\beta}\right)^2_{N,V}$$ (32)

$$= \left(\frac{\partial\ln Q}{\partial\beta^2}\right)_{N,V} = -\left(\frac{\partial\left<E\right>}{\partial\beta}\right)_{N,V}$$ (33)

Now, noting that the definition of heat capacity at constant volume, $C_v$, is

$$C_v = \left(\frac{\partial E}{\partial T}\right)$$ (34)

we see that

$$\left<\left(\delta E\right)^2\right> = k_B T^2 C_v$$ (35)

This is an interesting statement. It relates the magnitude of spontaneous fluctuations in the total energy of a system to that system's capacity to store or release energy due to changing its temperature.

The fact (Eq. 28) that the average energy in the canonical ensemble is related to a derivative of the log of the partition function implies that $\ln Q$ is an important thermodynamic quantity. So, let's go back to our undergraduate thermodynamics course(s) and recall the following statement of the 1st and 2nd Law:

$$dA = -SdT - pdV + \mu dN$$ (36)

where $A$ is the Helmholtz free energy, defined in terms of internal energy and entropy as

$$A = \left<E\right> - TS$$ (37)

Now, consider the following derivative of $A$:

$$\left(\frac{\partial\left(A/T\right)}{\partial\left(1/T\right)}\right)_{N,V} = A + \frac{1}{T}\left(\frac{\partial A}{\partial\left(1/T\right)}\right)_{N,V}$$ (38)

$$= A - T\left(\frac{\partial A}{\partial T}\right)_{N,V} = A + TS = \left<E\right>.$$ (39)

Therefore,

$$\left(\frac{\partial\left(\beta A\right)}{\partial\beta}\right)_{N,V} = \left<E\right>.$$ (40)

Considering Eq. 28, we see that

$$\ln Q + C = -\beta A$$ (41)

which does indeed suggest an important link between $\ln Q$ and the important thermodynamic quantity, the Helmholtz free energy. But what is the constant $C$? To evaluate it, consider the “boundary condition” as $T \rightarrow 0$:

$$Q = \sum_\nu e^{-\beta E_\nu} \stackrel{T\rightarrow 0}{\longrightarrow} e^{-\beta E_{\rm ground}}.$$ (42)

Here, we have assumed that the degeneracy of the ground state, $\Omega\left(E_{\rm ground}\right)$ is 1. This tells us that

$$\lim_{T\rightarrow 0} \ln Q = -\beta E_{\rm ground}$$ (43)

Using this fact, and combining Eqs. 37 and 41, as $T \rightarrow 0$, we see that

$$-\beta E_{\rm ground} + C = -\beta \underbrace{\left<E\right>}_{E_{\rm ground}} - \underbrace{\frac{S}{k_B}}_{\rightarrow 0 (\Omega = 1)},$$ (44)

Hence, $C$ = 0. So,

$$\ln Q = -\beta A.$$ (45)

The quantity $-\beta^{-1}\ln Q$ is the Helmholtz free energy, $A$.