1 The Fundamentals of Statistical Mechanics

The energy of the combined system is still $E_{\mathrm{total}}=E_1+E_2$. But the first system can have any energy $E\le E_{\mathrm{total}}$ while the second system must have the remainder $E_{\mathrm{total}}-E$. In fact, there is a slight caveat to this statement: in a quantum system we can’t have any energy at all: only those discrete energies $E_i$ that are eigenvalues of the Hamiltonian. So the number of available states of the combined system is

	$\mathrm{\Omega }(E_{\mathrm{total}})$	$=$	${\displaystyle \sum _{\{E_i\}}}{\mathrm{\Omega }}_1(E_i){\mathrm{\Omega }}_2(E_{\mathrm{total}}-E_i)$		(1.4)
		$=$	${\displaystyle \sum _{\{E_i\}}}\exp \left({\displaystyle \frac{S_1(E_i)}{k_B}}+{\displaystyle \frac{S_2(E_{\mathrm{total}}-E_i)}{k_B}}\right)$

There is a slight subtlety in the above equation. Both system 1 and system 2 have discrete energy levels. How do we know that if $E_i$ is an energy of system 1 then $E_{\mathrm{total}}-E_i$ is an energy of system 2. In part this goes back to the comment made above about the need for an interaction Hamiltonian that shifts the energy levels. In practice, we will just ignore this subtlety. In fact, for most of the systems that we will discuss in this course, the discreteness of energy levels will barely be important since they are so finely spaced that we can treat the energy $E$ of the first system as a continuous variable and replace the sum by an integral. We will see many explicit examples of this in the following sections.

At this point, we turn again to our fundamental assumption — all states are equally likely — but now applied to the combined system. This has fixed energy $E_{\mathrm{total}}$ so can be thought of as sitting in the microcanonical ensemble with the distribution (1.1) which means that the system has probability $p=1/\mathrm{\Omega }(E_{\mathrm{total}})$ to be in each state. Clearly, the entropy of the combined system is greater or equal to that of the original system,

$S(E_{\mathrm{total}})\equiv k_B\log \mathrm{\Omega }(E_{\mathrm{total}})\ge S_1(E_1)+S_2(E_2)$

(1.5)

which is true simply because the states of the two original systems are a subset of the total number of possible states.

While (1.5) is true for any two systems, there is a useful approximation we can make to determine $S(E_{\mathrm{total}})$ which holds when the number of particles, $N$, in the game is very large. We have already seen that the entropy scales as $S\sim N$. This means that the expression (1.4) is a sum of exponentials of $N$, which is itself an exponentially large number. Such sums are totally dominated by their maximum value. For example, suppose that for some energy, $E_{\star }$, the exponent has a value that’s twice as large as any other $E$. Then this term in the sum is larger than all the others by a factor of $e^N$. And that’s a very large number. All terms but the maximum are completely negligible. (The equivalent statement for integrals is that they can be evaluated using the saddle point method). In our case, the maximum value, $E=E_{\star }$, occurs when

${\displaystyle \frac{\partial S_1(E_{\star })}{\partial E}}-{\displaystyle \frac{\partial S_2(E_{\mathrm{total}}-E_{\star })}{\partial E}}=0$

(1.6)

where this slightly cumbersome notation means (for the first term) $\partial S_1/\partial E$ evaluated at $E=E_{\star }$. The total entropy of the combined system can then be very well approximated by

$S(E_{\mathrm{total}})\approx S_1(E_{\star })+S_2(E_{\mathrm{total}}-E_{\star })\ge S_1(E_1)+S_2(E_2)$

It is this simple statement that is responsible for all the irreversibility that we see in the world around us. This is the second law of thermodynamics. As a slogan, “entropy increases”. When two systems are brought together — or, equivalently, when constraints on a system are removed — the total number of available states available is vastly enlarged.

It is sometimes stated that second law is the most sacred in all of physics. Arthur Eddington’s rant, depicted in the cartoon, is one of the more famous acclamations of the law.

And yet, as we have seen above, the second law hinges on probabilistic arguments. We said, for example, that it is “highly unlikely” that the system will return to its initial configuration. One might think that this may allow us a little leeway. Perhaps, if probabilities are underlying the second law, we can sometimes get lucky and find counterexamples. While, it is most likely to find system 1 to have energy $E_{\star }$, surely occasionally one sees it in a state with a different energy? In fact, this never happens. The phrase “highly unlikely” is used only because the English language does not contain enough superlatives to stress how ridiculously improbable a violation of the second law would be. The silly number of possible states in a macroscopic systems means that violations happen only on silly time scales: exponentials of exponentials. This is a good operational definition of the word “never”.

First, note that $T$ has the right units, courtesy of Boltzmann’s constant (1.3). But that was merely a choice of convention: it doesn’t explain why $T$ has the properties that we expect of temperature. To make progress, we need to think more carefully about the kind of properties that we do expect. We will describe this in some detail in Section 4. For now it will suffice to describe the key property of temperature, which is the following: suppose we take two systems, each in equilibrium and each at the same temperature $T$, and place them in contact so that they can exchange energy. Then …nothing happens.

It is simple to see that this follows from our definition (1.7). We have already done the hard work above where we saw that two systems, brought into contact in this way, will maximize their entropy. This is achieved when the first system has energy $E_{\star }$ and the second energy $E_{\mathrm{total}}-E_{\star }$, with $E_{\star }$ determined by equation (1.6). If we want nothing noticeable to happen when the systems are brought together, then it must have been the case that the energy of the first system was already at $E_1=E_{\star }$. Or, in other words, that equation (1.6) was obeyed before the systems were brought together,

${\displaystyle \frac{\partial S_1(E_1)}{\partial E}}={\displaystyle \frac{\partial S_2(E_2)}{\partial E}}$

(1.8)

From our definition (1.7), this is the same as requiring that the initial temperatures of the two systems are equal: $T_1=T_2$.

Suppose now that we bring together two systems at slightly different temperatures. They will exchange energy, but conservation ensures that what the first system gives up, the second system receives and vice versa. So $\delta E_1=-\delta E_2$. If the change of entropy is small, it is well approximated by

	$\delta S$	$=$	${\displaystyle \frac{\partial S_1(E_1)}{\partial E}}\delta E_1+{\displaystyle \frac{\partial S_2(E_2)}{\partial E}}\delta E_2$
		$=$	$\left({\displaystyle \frac{\partial S_1(E_1)}{\partial E}}-{\displaystyle \frac{\partial S_2(E_2)}{\partial E}}\right)\delta E_1$
		$=$	$\left({\displaystyle \frac{1}{T_1}}-{\displaystyle \frac{1}{T_2}}\right)\delta E_1$

The second law tells us that entropy must increase: $\delta S>0$. This means that if $T_1>T_2$, we must have . In other words, the energy flows in the way we would expect: from the hotter system to colder.

To summarise: the equilibrium argument tell us that $\partial S/\partial E$ should have the interpretation as some function of temperature; the heat flowing argument tell us that it should be a monotonically decreasing function. But why $1/T$ and not, say, $1/T^2$? To see this, we really need to compute $T$ for a system that we’re all familiar with and see that it gives the right answer. Once we’ve got the right answer for one system, the equilibrium argument will ensure that it is right for all systems. Our first business in Section 2 will be to compute the temperature $T$ for an ideal gas and confirm that (1.7) is indeed the correct definition.

There is another expression for the heat capacity that is useful. The entropy is a function of energy, $S=S(E)$. But we could invert the formula (1.7) to think of energy as a function of temperature, $E=E(T)$. We then have the expression

${\displaystyle \frac{\partial S}{\partial T}}={\displaystyle \frac{\partial S}{\partial E}}\cdot {\displaystyle \frac{\partial E}{\partial T}}={\displaystyle \frac{C}{T}}$

This is a handy formula. If we can measure the heat capactiy of the system for various temperatures, we can get a handle on the function $C(T)$. From this we can then determine the entropy of the system. Or, more precisely, the entropy difference

$\mathrm{\Delta }S={\displaystyle {\int }_{T_1}^{T_2}}{\displaystyle \frac{C(T)}{T}}𝑑T$

(1.10)

Thus the heat capacity is our closest link between experiment and theory.

The heat capacity is always proportional to $N$, the number of particles in the system. It is common to define the specific heat capacity, which is simply the heat capacity divided by the mass of the system and is independent of $N$.

There is one last point to make about heat capacity. Differentiating (1.7) once more, we have

${\displaystyle \frac{{\partial }^{2}S}{\partial E^2}}=-{\displaystyle \frac{1}{T^{2}C}}$

(1.11)

Nearly all systems you will meet have $C>0$. (There is one important exception: a black hole has negative heat capacity!). Whenever $C>0$, the system is said to be thermodynamically stable. The reason for this language goes back to the previous discussion concerning two systems which can exchange energy. There we wanted to maximize the entropy and checked that we had a stationary point (1.6), but we forgot to check whether this was a maximum or minimum. It is guaranteed to be a maximum if the heat capacity of both systems is positive so that .

If the system has energy $E$, its temperature is

${\displaystyle \frac{1}{T}}={\displaystyle \frac{\partial S}{\partial E}}={\displaystyle \frac{k_B}{ϵ}}\log \left({\displaystyle \frac{Nϵ}{E}}-1\right)$

We can also invert this expression. If the system has temperature $T$, the fraction of particles with spin up is given by

${\displaystyle \frac{N_{\uparrow }}{N}}={\displaystyle \frac{E}{Nϵ}}={\displaystyle \frac{1}{e^{ϵ/k_{B}T}+1}}$

(1.13)

What happens for energies $E>Nϵ/2$, where $N_{\uparrow }/N>1/2$? From the definition of temperature as $1/T=\partial S/\partial E$, it is clear that we have entered the realm of negative temperatures. This should be thought of as hotter than infinity! (This is simple to see in the variables $1/T$ which tends towards zero and then just keeps going to negative values). Systems with negative temperatures have the property that the number of microstates decreases as we add energy. They can be realised in laboratories, at least temporarily, by instantaneously flipping all the spins in a system.

As $T\to 0$, the specific heat drops to zero exponentially quickly. (Recall that $e^{-1/x}$ is a function which tends towards zero faster than any power $x^n$). The reason for this fast fall-off can be traced back to the existence of an energy gap, meaning that the first excited state is a finite energy above the ground state. The heat capacity also drops off as $T\to \mathrm{\infty }$, but now at a much slower power-law pace. This fall-off is due to the fact that all the states are now occupied.

The contribution to the heat capacity from spins is not the dominant contribution in most materials. It is usually dwarfed by the contribution from phonons and, in metals, from conduction electrons, both of which we will calculate later in the course. Nonetheless, in certain classes of material — for example, paramagnetic salts — a spin contribution of the form (1.14) can be seen at low temperatures where it appears as a small bump in the graph and is referred to as the Schottky anomaly. (It is ‘‘anomalous” because most materials have a heat capacity which decreases monotonically as temperature is reduced). In Figure 5, the Schottky contribution has been isolated from the phonon contribution¹¹ 1 The data is taken from Chirico and Westrum Jr., J. Chem. Thermodynamics 12 (1980), 311, and shows the spin contribution to the heat capacity of Tb(OH)${}_3$. The open circles and dots are both data (interpreted in mildly different ways); the solid line is theoretical prediction, albeit complicated slightly by the presence of a number of spin states. The deviations are most likely due to interactions between the spins.

The two state system can also be used as a model for defects in a lattice. In this case, the “spin down” state corresponds to an atom sitting in the lattice where it should be with no energy cost. The “spin up” state now corresponds to a missing atom, ejected from its position at energy cost $ϵ$.

We’ll still use the same notation for the number of states and entropy of the system, but now these quantities will be functions of both the energy and the volume,

$S(E,V)=k_B\log \mathrm{\Omega }(E,V)$

The temperature is again given by $1/T=\partial S/\partial E$, where the partial derivative implicitly means that we keep $V$ fixed when we differentiate. But now there is a new, natural quantity that we can consider — the differentiation with respect to $V$. This also gives a quantity that you’re all familiar with — pressure, $p$. Well, almost. The definition is

$p=T{\displaystyle \frac{\partial S}{\partial V}}$

(1.15)

Despite appearances, the definition of pressure actually has little to do with entropy. Roughly speaking, the $S$ in the derivative cancels the factor of $S$ sitting in $T$. To make this mathematically precise, consider a system with entropy $S(E,V)$ that undergoes a small change in energy and volume. The change in entropy is

$dS={\displaystyle \frac{\partial S}{\partial E}}dE+{\displaystyle \frac{\partial S}{\partial V}}dV$

Rearranging, and using our definitions (1.7) and (1.15), we can write

$dE=TdS-pdV$

(1.16)

Alternatively, you may prefer to run this argument in reverse: if you’re happy to equate squeezing the system by $dV$ with doing work, then the discussion above is sufficient to tell you that pressure as defined in (1.15) has the interpretation of force per area.

What is the interpretation of the first term on the right-hand side of (1.16)? It must be some form of energy transferred to the system. It turns out that the correct interpretation of $TdS$ is the amount of heat the system absorbs from the surroundings. Much of Section 4 will be concerned with understanding why this is right way to think about $TdS$ and we postpone a full discussion until then.

Equation (1.16) expresses the conservation of energy for a system at finite temperature. It is known as the First Law of Thermodynamics. (You may have noticed that we’re not doing these laws in order! This too will be rectified in Section 4).

As a final comment, we can now give a slightly more refined definition of the heat capacity (1.9). In fact, there are several different heat capacities which depend on which other variables are kept fixed. Throughout most of these lectures, we will be interested in the heat capacity at fixed volume, denoted $C_V$,

$C_V={{\displaystyle \frac{\partial E}{\partial T}}|}_V$

(1.17)

Using the first law of thermodynamics (1.16), we see that something special happens when we keep volume constant: the work done term drops out and we have

$C_V={T{\displaystyle \frac{\partial S}{\partial T}}|}_V$

(1.18)

This form emphasises that, as its name suggests, the heat capacity measures the ability of the system to absorb heat $TdS$ as opposed to any other form of energy. (Although, admittedly, we still haven’t really defined what heat is. As mentioned above, this will have to wait until Section 4).

The equivalence of (1.17) and (1.18) only followed because we kept volume fixed. What is the heat capacity if we keep some other quantity, say pressure, fixed? In this case, the correct definition of heat capacity is the expression analogous to (1.18). So, for example, the heat capacity at constant pressure $C_p$ is defined by

$C_p={T{\displaystyle \frac{\partial S}{\partial T}}|}_p$

For the next few Sections, we’ll only deal with $C_V$. But we’ll return briefly to the relationship between $C_V$ and $C_p$ in Section 4.4.

Ludwig Boltzmann was born into a world that doubted the existence of atoms²² 2 If you want to learn more about his life, I recommend the very enjoyable biography, Boltzmann’s Atom by David Lindley. The quote above is taken from a travel essay that Boltzmann wrote recounting a visit to California. The essay is reprinted in a drier, more technical, biography by Carlo Cercignani.. The cumulative effect of his lifetime’s work was to change this. No one in the 1800s ever thought we could see atoms directly and Boltzmann’s strategy was to find indirect, yet overwhelming, evidence for their existence. He developed much of the statistical machinery that we have described above and, building on the work of Maxwell, showed that many of the seemingly fundamental laws of Nature — those involving heat and gases in particular — were simply consequences of Newton’s laws of motion when applied to very large systems.

It is often said that Boltzmann’s great insight was the equation which is now engraved on his tombstone, $S=k_B\log \mathrm{\Omega }$, which lead to the understanding of the second law of thermodynamics in terms of microscopic disorder. Yet these days it is difficult to appreciate the boldness of this proposal simply because we rarely think of any other definition of entropy. We will, in fact, meet the older thermodynamic notion of entropy and the second law in Section 4. In the meantime, perhaps Boltzmann’s genius is better reflected in the surprising equation for temperature: $1/T=\partial S/\partial E$.

Boltzmann gained prominence during his lifetime, holding professorships at Graz, Vienna, Munich and Leipzig (not to mention a position in Berlin that he somehow failed to turn up for). Nonetheless, his work faced much criticism from those who would deny the existence of atoms, most notably Mach. It is not known whether these battles contributed to the depression Boltzmann suffered later in life, but the true significance of his work was only appreciated after his body was found hanging from the rafters of a guest house near Trieste.

We’ll see lots of properties of $Z$ soon. But we’ll start with a fairly basic, yet important, point: for independent systems, $Z$’s multiply. This is easy to prove. Suppose that we have two systems which don’t interact with each other. The energy of the combined system is then just the sum of the individual energies. The partition function for the combined system is (in, hopefully, obvious notation)

	$Z$	$=$	${\displaystyle \sum _{n,m}}{e}^{-\beta (E_n^{(1)}+E_m^{(2)})}$		(1.23)
		$=$	${\displaystyle \sum _{n,m}}{e}^{-\beta E_n^{(1)}}{e}^{-\beta E_m^{(2)}}$
		$=$	${\displaystyle \sum _n}{e}^{-\beta E_n^{(1)}}{\displaystyle \sum _m}{e}^{-\beta E_m^{(2)}}=Z_1{Z}_2$

We can also look at the spread of energies about the mean — in other words, about fluctuations in the probability distribution. As usual, this spread is captured by the variance,

$\mathrm{\Delta }{E}^2=⟨{(E-⟨E⟩)}^2⟩=⟨E^2⟩-{⟨E⟩}^2$

This too can be written neatly in terms of the partition function,

$\mathrm{\Delta }{E}^2={\displaystyle \frac{{\partial }^2}{\partial {\beta }^2}}\log Z=-{\displaystyle \frac{\partial ⟨E⟩}{\partial \beta }}$

(1.26)

There is another expression for the fluctuations that provides some insight. Recall our definition of the heat capacity (1.9) in the microcanonical ensemble. In the canonical ensemble, where the energy is not fixed, the corresponding definition is

$C_V={{\displaystyle \frac{\partial ⟨E⟩}{\partial T}}|}_V$

Then, since $\beta =1/k_{B}T$, the spread of energies in (1.26) can be expressed in terms of the heat capacity as

$\mathrm{\Delta }{E}^2=k_B{T}^2{C}_V$

(1.27)

There are two important points hiding inside this small equation. The first is that the equation relates two rather different quantities. On the left-hand side, $\mathrm{\Delta }E$ describes the probabilistic fluctuations in the energy of the system. On the right-hand side, the heat capacity $C_V$ describes the ability of the system to absorb energy. If $C_V$ is large, the system can take in a lot of energy without raising its temperature too much. The equation (1.27) tells us that the fluctuations of the systems are related to the ability of the system to dissipate, or absorb, energy. This is the first example of a more general result known as the fluctuation-dissipation theorem.

The other point to take away from (1.27) is the size of the fluctuations as the number of particles $N$ in the system increases. Typically $E\sim N$ and $C_V\sim N$. Which means that the relative size of the fluctuations scales as

${\displaystyle \frac{\mathrm{\Delta }E}{E}}\sim {\displaystyle \frac{1}{\sqrt{N}}}$

(1.28)

The limit $N\to \mathrm{\infty }$ in known as the thermodynamic limit. The energy becomes peaked closer and closer to the mean value $⟨E⟩$ and can be treated as essentially fixed. But this was our starting point for the microcanonical ensemble. In the thermodynamic limit, the microcanonical and canonical ensembles coincide.

All the examples that we will discuss in the course will have a very large number of particles, $N$, and we can consider ourselves safely in the thermodynamic limit. For that reason, even in the canonical ensemble, we will often write $E$ for the average energy rather than $⟨E⟩$.

Notice that, unlike in the microcanonical ensemble, we didn’t have to solve any combinatoric problem to count states. The partition function has done all that work for us. Of course, for this simple two state system, the counting of states wasn’t difficult but in later examples, where the counting gets somewhat tricker, the partition function will be an invaluable tool to save us the work.

In fact, we’re going to use a little trick. Suppose that we don’t have just one copy of our system $S$, but instead a large number, $W$, of identical copies. Each system lives in a particular state $|n⟩$. If $W$ is large enough, the number of systems that sit in state $|n⟩$ must be simply $p(n)W$. We see that the trick of taking $W$ copies has translated the probabilities into eventualities. To determine the entropy we can treat the whole collection of $W$ systems as sitting in the microcanonical ensemble to which we can apply the familiar Boltzmann definition of entropy (1.2). We must only figure out how many ways there are of putting $p(n)W$ systems into state $|n⟩$ for each $|n⟩$. That’s a simple combinatoric problem: the answer is

$\mathrm{\Omega }={\displaystyle \frac{W!}{{\prod }_n(p(n)W)!}}$

And the entropy is therefore

$S=k_B\log \mathrm{\Omega }=-k_{B}W{\displaystyle \sum _n}p(n)\log p(n)$

(1.29)

where we have used Stirling’s formula to simplify the logarithms of factorials. This is the entropy for all $W$ copies of the system. But we also know that entropy is additive. So the entropy for a single copy of the system, with probability distribution $p(n)$ over the states is

$S=-k_B{\displaystyle \sum _n}p(n)\log p(n)$

(1.30)

This beautiful formula is due to Gibbs. It was rediscovered some decades later in the context of information theory where it goes by the name of Shannon entropy for classical systems or von Neumann entropy for quantum systems. In the quantum context, it is sometimes written in terms of the density matrix (1.24) as

$S=-k_B\mathrm{Tr}\widehat{\rho }\log \widehat{\rho }$

When we first introduced entropy in the microcanonical ensemble, we viewed it as a function of the energy $E$. But (1.30) gives a very different viewpoint on the entropy: it says that we should view $S$ as a function of a probability distribution. There is no contradiction with the microcanonical ensemble because in that simple case, the probability distribution is itself determined by the choice of energy $E$. Indeed, it is simple to check (and you should!) that the Gibbs entropy (1.30) reverts to the Boltzmann entropy in the special case of $p(n)=1/\mathrm{\Omega }(E)$ for all states $|n⟩$ of energy $E$.

Meanwhile, back in the canonical ensemble, the probability distribution is entirely determined by the choice of temperature $T$. This means that the entropy is naturally a function of $T$. Indeed, substituting the Boltzmann distribution $p(n)=e^{-\beta E_n}/Z$ into the expression (1.30), we find that the entropy in the canonical ensemble is given by

	$S$	$=$	$-{\displaystyle \frac{k_B}{Z}}{\displaystyle \sum _n}{e}^{-\beta E_n}\log \left({\displaystyle \frac{e^{-\beta E_n}}{Z}}\right)$
		$=$	${\displaystyle \frac{k_B\beta }{Z}}{\displaystyle \sum _n}{E}_n{e}^{-\beta E_n}+k_B\log Z$

As with all other important quantities, this can be elegantly expressed in terms of the partition function by

$S=k_B{\displaystyle \frac{\partial }{\partial T}}(T\log Z)$

(1.31)

The partition function in (1.21) is a sum over all states. We can rewrite it as a sum over energy levels by including a degeneracy factor

$Z={\displaystyle \sum _{\{E_i\}}}\mathrm{\Omega }(E_i)e^{-\beta E_i}$

The degeneracy factor $\mathrm{\Omega }(E)$ factor is typically a rapidly rising function of $E$, while the Boltzmann suppression $e^{-\beta E}$ is rapidly falling. But, for both the exponent is proportional to $N$ which is itself exponentially large. This ensures that the sum over energy levels is entirely dominated by the maximum value, $E_{\star }$, defined by the requirement

${\displaystyle \frac{\partial }{\partial E}}{\left(\mathrm{\Omega }(E)e^{-\beta E}\right)}_{E=E_{\star }}=0$

and the partition function can be well approximated by

$Z\approx \mathrm{\Omega }(E_{\star })e^{-\beta E_{\star }}$

(This is the same kind of argument we used in (1.2.1) in our discussion of the Second Law). With this approximation, we can use (1.25) to show that the most likely energy $E_{\star }$ and the average energy $⟨E⟩$ coincide:

$⟨E⟩=E_{\star }$

(We need to use the result (1.7) in the form $\partial \log \mathrm{\Omega }(E_{\star })/\partial E_{\star }=\beta$ to derive this). Similarly, using (1.31), we can show that the entropy in the canonical ensemble is given by

$S=k_B\log \mathrm{\Omega }(E_{\star })$

Let’s start with the microcanonical ensemble, in which we fix the energy of the system so that we only allow non-zero probabilities for those states which have energy $E$. We could then compute the entropy using the Gibbs formula (1.30) for any probability distribution, including systems away from equilibrium. We need only insist that all the probabilities add up to one: ${\sum }_{n}p(n)=1$. We can maximise $S$ subject to this condition by introducing a Lagrange multiplier $\alpha$ and maximising $S+\alpha k_B({\sum }_{n}p(n)-1)$,

${\displaystyle \frac{\partial }{\partial p(n)}}\left(-{\displaystyle \sum _n}p(n)\log p(n)+\alpha {\displaystyle \sum _n}p(n)-\alpha \right)=0\mathit{\hspace{1em}\hspace{1em}}\Rightarrow \mathit{\hspace{1em}\hspace{1em}}p(n)=e^{\alpha -1}$

We learn that all states with energy $E$ are equally likely. This is the microcanonical ensemble.

In the examples sheet, you will be asked to show that the canonical ensemble can be viewed in the same way: it is the probability distribution that maximises the entropy subject to the constraint that the average energy is fixed.

Heuristically, the free energy captures the competition between energy and entropy that occurs in a system at constant temperature. Immersed in a heat bath, energy is not necessarily at a premium. Indeed, we saw in the two-state example that the ground state plays little role in the physics at non-zero temperature. Instead, the role of entropy becomes more important: the existence of many high energy states can beat a few low-energy ones.

The fact that the free energy is the appropriate quantity to look at for systems at fixed temperature is also captured by its mathematical properties. Recall, that we started in the microcanonical ensemble by defining entropy $S=S(E,V)$. If we invert this expression, then we can equally well think of energy as a function of entropy and volume: $E=E(S,V)$. This is reflected in the first law of thermodynamics (1.16) which reads $dE=TdS-pdV$. However, if we look at small variations in $F$, we get

$dF=d⟨E⟩-d(TS)=-SdT-pdV$

(1.33)

This form of the variation is telling us that we should think of the free energy as a function of temperature and volume: $F=F(T,V)$. Mathematically, $F$ is a Legendre transform of $E$.

Given the free energy, the variation (1.33) tells us how to get back the entropy,

$S=-{{\displaystyle \frac{\partial F}{\partial T}}|}_V$

(1.34)

Similarly, the pressure is given by

$p=-{{\displaystyle \frac{\partial F}{\partial V}}|}_T$

(1.35)

The free energy is the most important quantity at fixed temperature. It is also the quantity that is most directly related to the partition function $Z$:

$F=-k_{B}T\log Z$

(1.36)

This relationship follows from (1.25) and (1.31). Using the identity $\partial /\partial \beta =-k_B{T}^2\partial /\partial T$, these expressions allow us to write the free energy as

	$F=E-TS$	$=$	$k_B{T}^2{\displaystyle \frac{\partial }{\partial T}}\log Z-k_{B}T{\displaystyle \frac{\partial }{\partial T}}(T\log Z)$
		$=$	$-k_{B}T\log Z$

as promised.

The probability distribution that we need to use in this case is called the grand canonical ensemble. The probability of finding the system in a state $|n⟩$ depends on both the energy $E_n$ and the particle number $N_n$. (Notice that because $N$ is conserved, the quantum mechanical operator necessarily commutes with the Hamiltonian so there is no difficulty in assigning both energy and particle number to each state). We introduce the grand canonical partition function

$𝒵(T,\mu ,V)={\displaystyle \sum _n}{e}^{-\beta (E_n-\mu N_n)}$

(1.39)

Re-running the argument that we used for the canonical ensemble, we find the probability that the system is in state $|n⟩$ to be

$p(n)={\displaystyle \frac{e^{-\beta (E_n-\mu N_n)}}{𝒵}}$

In the canonical ensemble, all the information that we need is contained within the partition function $Z$. In the grand canonical ensemble it is contained within $𝒵$. The entropy (1.30) is once again given by

$S=k_B{\displaystyle \frac{\partial }{\partial T}}(T\log 𝒵)$

(1.40)

while differentiating with respect to $\beta$ gives us

$⟨E⟩-\mu ⟨N⟩=-{\displaystyle \frac{\partial }{\partial \beta }}\log 𝒵$

(1.41)

The average particle number $⟨N⟩$ in the system can then be separately extracted by

$⟨N⟩={\displaystyle \frac{1}{\beta }}{\displaystyle \frac{\partial }{\partial \mu }}\log 𝒵$

(1.42)

and its fluctuations,

$\mathrm{\Delta }{N}^2={\displaystyle \frac{1}{{\beta }^2}}{\displaystyle \frac{{\partial }^2}{\partial {\mu }^2}}\log 𝒵={\displaystyle \frac{1}{\beta }}{\displaystyle \frac{\partial ⟨N⟩}{\partial \mu }}$

(1.43)

Just as the average energy is determined by the temperature in the canonical ensemble, here the average particle number is determined by the chemical potential. The grand canonical ensemble will simplify several calculations later, especially when we come to discuss Bose and Fermi gases in Section 3.

The relative size of these fluctuations scales in the same way as the energy fluctuations, $\mathrm{\Delta }N/⟨N⟩\sim 1/\sqrt{⟨N⟩}$, and in the thermodynamic limit $N\to \mathrm{\infty }$ results from all three ensembles coincide. For this reason, we will drop the averaging brackets $⟨\cdot ⟩$ from our notation and simply refer to the average particle number as $N$.

We can perform the same algebraic manipulations that gave us $F$ in terms of the canonical partition function $Z$, this time using the definitions (1.40) and (1.41)) to write $\mathrm{\Phi }$ as

$\mathrm{\Phi }=-k_{B}T\log 𝒵$

(1.45)

Suppose we have a system and we double it. That means that we double the volume $V$, double the number of particles $N$ and double the energy $E$. What happens to all our other variables? We have already seen back in Section 1.2.1 that entropy is additive, so $S$ also doubles. More generally, if we scale $V$, $N$ and $E$ by some amount $\lambda$, the entropy must scale as

$S(\lambda E,\lambda V,\lambda N)=\lambda S(E,V,N)$

Quantities such as $E$, $V$, $N$ and $S$ which scale in this manner are called extensive. In contrast, the variables which arise from differentiating the entropy, such as temperature $1/T=\partial S/\partial E$ and pressure $p=T\partial S/\partial V$ and chemical potential $\mu =T\partial S/\partial N$ involve the ratio of two extensive quantities and so do not change as we scale the system: they are called intensive quantities.

What now happens as we make successive Legendre transforms? The free energy $F=E-TS$ is also extensive (since $E$ and $S$ are extensive while $T$ is intensive). So it must scale as

$F(T,\lambda V,\lambda N)=\lambda F(T,V,N)$

(1.46)

Similarly, the grand potential $\mathrm{\Phi }=F-\mu N$ is extensive and scales as

$\mathrm{\Phi }(T,\lambda V,\mu )=\lambda \mathrm{\Phi }(T,V,\mu )$

(1.47)

But there’s something special about this last equation, because $\mathrm{\Phi }$ only depends on a single extensive variable, namely $V$. While there are many ways to construct a free energy $F$ which obeys (1.46) (for example, any function of the form $F\sim V^{n+1}/N^n$ will do the job), there is only one way to satisfy (1.47): $\mathrm{\Phi }$ must be proportional to $V$. But we’ve already got a name for this proportionality constant: it is pressure. (Actually, it is minus the pressure as you can see from (1.44)). So we have the equation

$\mathrm{\Phi }(T,V,\mu )=-p(T,\mu )V$

(1.48)

It looks as if we got something for free! If $F$ is a complicated function of $V$, where do these complications go after the Legendre transform to $\mathrm{\Phi }$? The answer is that the complications go into the pressure $p(T,\mu )$ when expressed as a function of $T$ and $\mu$. Nonetheless, equation (1.48) will prove to be an extremely economic way to calculate the pressure of various systems.

Gibbs provided the first modern rendering of the subject in a treatise published shortly before his death. Very few understood it. Lord Rayleigh wrote to Gibbs suggesting that the book was “too condensed and too difficult for most, I might say all, readers”. Gibbs disagreed. He wrote back saying the book was only “too long”.

There do not seem to be many exciting stories about Gibbs. He was an undergraduate at Yale. He did a PhD at Yale. He became a professor at Yale. Apparently he rarely left New Haven. Strangely, he did not receive a salary for the first ten years of his professorship. Only when he received an offer from John Hopkins of $3000 dollars a year did Yale think to pay America’s greatest physicist. They made a counter-offer of $2000 dollars and Gibbs stayed.