aProbabilistic Systems: An Alternative View of Statistical Mechanics1. Abstract (or why I am writing this and why you should read it)Thermodynamics is cool (pun intended). Its main characters are heat, the equipartition theorem, and the Boltzmann distribution. If you aren't familiar with these figures, you will be soon. But its star is that most elusive of concepts, a mainstay in physics, probability, and computer science. The purveyor of inevitable order and universal doom; entropy. Wanting to understand entropy on a deeper level while avoiding the messiness of the second law and Maxwell's demon, and understanding its application to other fields, is the driving force of this work. Other thermodynamic theorems were in close reach in investigating entropy. This work is the casual presentation of that overarching investigation, trying to be more Feynman lecture than a physics paper.Within this piece, you will find alternative derivations for many significant theorems of thermodynamics. The necessary mathematics isn't exactly a walk in the park. Still, it shouldn't be out of reach if you know a little about calculus, probability and linear algebra. If it is, let me know. And especially let me know where I made one of many inevitable mistakes. The work isn't even that physical. The theorems are all derived with the minimum possible ingredients, making them applicable to a broad class of probabilistic systems rather than the specific class where the laws of physics apply.2. Probabilistic Systems (or, how to answer silly questions)Two people are throwing a ball at each other. Where is the ball? This may seem like a silly question, and it sort of is. We have been given painfully little information! If we knew nothing else, we could only say the ball is at *some* point between the two, or x∈[0,L]. As we have no further information, we'll supply a uniform distribution, 𝜌(x)=1/L for x∈[0,L]. This is saying is that some specific claim about where the ball is, such as it being at point x=0.2, is no more reasonable than saying x=0.7. Our probability distribution would be unfair to assign a greater probability to either one. Therefore, it is uniform from 0 through to L.If we were given more information, we could change this distribution. For example, if it is given that the ball is closer to person A than person B, we would have 𝜌(x)=2/L for x∈[0,L/2]. Therefore, we keep the situations where the ball is closer to A, and discard those where it is closer to B. This is called conditioning, and is how we incorporate new information into a model.If the ball were moving with velocity v to the right, a state where it is in position x would, dt seconds later, be a state where it is in position x+v.dt. We could then update our model to 𝜌(x)=2/L for x∈[v.dt,v.dt+L/2]. Our best model of the ball's position is now a uniform distribution between those two new bounds.Good job, we have just built a model to handle states in a probabilistic system! In physics, we are often dealing with equally silly questions and, indeed, far sillier ones. Where are the gas molecules? All trillions of them. Trillions of trillions of them. Or what we innocently call a "mole of molecules" (Avagadro's number, the number of molecules in a mole, is NA=6.6022×1023≈(1012)2). Thankfully, silly questions that don't give much information can be satisfied with silly answers that give equally little information. I don't have to say exactly how molecule X is going to bounce around, if a general pattern is all which is reasonable. Complex systems with unfathomably many variables are boiled down into simple models which fit our simple observations. Thus we get emergence, a most wonderful process where a compound system adopts properties and patterns not found in its constituents.3. States and ModelsLet's try to mathematically work with these sorts of systems. We'll need to set up some core mechanics for our models. So let's represent states, physical or not, as continuous variables in Rn that evolve deterministically. These individuals states have predictable dynamics, updating according to some evolution map𝜏:Rn→Rn. These can be thought of as offering a naturally evolving path for states,d
dts=𝜏(s)In practice, however, we will work with uncertain systems where one of many states may be the case. A modelΦ gives the likelihood of states by some density function 𝜌(s). This is the proportion of states in some small volume around s, divided by that volume. As the volume gets really really small, halving it will half the proportion of states within it. As its really small, the probability is pretty much the same through out it. So ever smaller volumes give a converged 𝜌(s) density value, assuming necessary smoothness and all that. Models also have bounds dΦ such that states are always contained within the bounds, probability 1. A calculus trick involving surface integrals will make bounds very important, and allow various theorems to be derived. Bounds, after all, ensure models are not completely erratic.3.1. Change in ProbabilityIf we only know how states s(t)evolve, how do we find how models 𝜌(s,t) evolve? Consider a model before and after an evolution and how the density 𝜌 changes. We will have evolutions occur infinitesimally and continuously. Basically, without jumps. Things don't usually teleport, so we should be ok there. For the maths to make sense, we need to be able to think of a sort of map of different states, called a phase space. The probability of states then becomes akin to asking where is everyone? We've already established that we know where everyone is going, by 𝜏, so if we know where people are, we can find out where they will be!Let's draw out an area Z of our map, and see how the number of people there changes over time. This is the same as seeing how many people enter, and how many leave. We go along the boundary, see how many people are nearby with 𝜌(s), and see whether their direction of movement 𝜏(s) means they are entering, exiting, or just travelling along the border dn. Mathematically,
d
dtP(Z)
=-∫dZ𝜌(s)⏠⏣⏣⏡⏣⏣⏢Moveable 𝜌.𝜏(s)⋅dn⏠⏣⏣⏡⏣⏣⏢Motion against the surface of Z
Now we'll be using an incredibly useful mathematical trick. Suppose I wanted to find the difference between this years earnings for some company and its earnings 10 years ago. I could either just, well, take the difference, or add the incremental differences in earnings from year to year.
Δ
=f(10)-f(0)
=(f(10)-f(9))+(f(9)-f(8))+...+(f(1)-f(0))
This same principal can be applied to our volume integral across the surface! Instead of taking the value at the surface of the volume, we add up all the differences inside the volume. With ∇ (called nabla) adding up the changes along each dimension. ∇=∂i, this will be explained in my upcoming linear algebra post. But basically, it acts very similar to a bunch of derivitives in each dimension added together because, well, it is. We get,
d
dtP(Z)
=-∫Z∇(𝜌(s).𝜏(s))·ds
Now we make Z very very small, and compensate for the change in volume. This leaves us with,
d
dt𝜌(s)
=-∇(𝜌(s).𝜏(s))⋅ds/ds
=-∇(𝜌(s).𝜏(s))
And we can expand this, using the derivative product law.
3.2. ConvergenceA converged model ceases to change over time. This occurs when the underlying probability distribution which defines the model is static, and therefore, we have;d
This also implies that when ∇𝜏(s)=0, that convergence is achieved by ∇𝜌(s)=0. In other words, a uniform distribution. We will often deal with systems which we know little about other than that they have converged. When dealing with systems in such an equilibrium, these expressions will be the starting block for figuring out 𝜌(s) values, and will prove useful.4. EntropyA model's entropy SΦ measures its self-uncertainty. In statistics, there is the concept of likelihood, which is the probability of a model producing a set of observations. The closer the set of observations approaches a typical sample from the model, the higher the likelihood. The likelihood of a model over samples drawn from itself is a measure of its inherent randomness. A low self-likelihood means samples drawn appear erratic even when the distribution is known. A high self-likelihood means we have a reasonable idea of what a typical sample looks like.Entropy is two operations away from likelihood. It takes a logarithm followed by negation, so high entropy means a chaotic system, while low entropy means a predictable one. For a continuous model Φ, the expression is,
SΦ
=-∫𝜌Φ(s).ln𝜌Φ(s).ds
This is closely related to ideas from information theory, the mathematics of messiness and lossiness. Ideas from information theory will be used to better understand some of the strange behaviours that entropy exhibits.4.1. Entropy of Models4.1.1. Quick Note on Poisson BracketsPoisson brackets, indicated by <⬜>, gives a short hand for averages according to the current model. In general, we have,
∫f(s).𝜌(s).ds
=<f(s)>
Meaning we can express entropy as,SΦ=<l(s)>, where l(s)=-ln𝜌(s)4.1.2. Change in EntropyLet's try to figure out how entropy changes over time. The relationship between entropy and time is very, very important. It forms the basis of the second law of thermodynamics, which states entropy can never decrease. Fortunately, we can use a few mathematical tricks to get a relatively clean expression for the change of entropy over time. Starting with,
dtSΦ=∫Φ∇(𝜌(s).𝜏(s)).(ln𝜌(s)+1).dsFrom the product rule, we have d(uv)=u.dv+v.du, and therefore du.v=d(uv)-u.dv. This same idea is applied to the multidimensional derivative ∇. You can think of it like a sum of derivatives over multiple dimensions, because that's exactly what it is.
Here we do the surface-volume integral trick from (1) in reverse. The same way a sum of small changes is equal to the total change, a sum of small changes across a region is the same as the difference in values around the edges. This gives us a surface integral around the bounds,
d
dtSΦ
=∫dΦ(ln𝜌(s)+1).𝜌(s).𝜏(s)·dn(s)⏠⏣⏣⏣⏣⏡⏣⏣⏣⏣⏢=0 at the bounds-∫Φ𝜌(s).𝜏(s)·∇ln𝜌(s).ds
The bounds are defined as the volume containing all states, so movement through the bounds has to be zero. If it weren't, they wouldn't be the bounds! This leaves us with,
d
dtSΦ
=-∫Φ𝜌(s).𝜏(s)·∇ln𝜌(s).ds
=<𝜏(s)·∇l(s)>
This is one way of viewing the change in entropy, how aligned are evolutions 𝜏(s) with gradients in l(s). This variable, l(s)=-ln𝜌(s), related to the information-theoretic symbol length, has a very important physical analogue. Spoiler: it's a scalar multiple of energy and/or other conserved variables. The interpretation that energy represents limited resources capable of expression is somewhat justifiable. However, now is the time for maths, not philosophy, so starting at (5) we look at an alternative expression,
d
dtSΦ
=-∫Φ𝜌(s).𝜏(s)·∇𝜌(s)
𝜌(s).ds
=-∫Φ𝜏(s)·∇𝜌(s).ds
Again we use the product rule trick, realising that u.dv=d(uv)-v.du,
d
dtSΦ
=-∫Φ∇(𝜏(s).𝜌(s))-𝜌(s).∇𝜏(s).ds
=-∫dΦ𝜌(s).𝜏(s)·dn(s)⏠⏣⏣⏣⏣⏡⏣⏣⏣⏣⏢=0 at the bounds+∫Φ𝜌(s).∇𝜏(s).ds
=<∇𝜏(s)>
So, the change in entropy is equivalent to the average of ∇𝜏(s). We will call ∇𝜏(s) the feedback, a property of states, and it happens to be zero in Hamiltonian physics. This implies zero natural change in entropy for a perfectly evolving model. We can refine how various axioms are linked to each other to show an equivalence between the second law - entropy of physical models is non-decreasing - and feedback being zero everywhere.4.2. Equivalence of the Second Law and <∇𝜏(s)>=0This section of the work will quickly reveal how much of a logic nerd I am. I think its better to sometimes get the raw symbology out of the way, as paper is more efficient at storing working memory than a carbon-based neural network. It also has the added benefit of achieving one of the central goals of this work: showing that core thermodynamic theorems are equivalent to other constructs, revealing what makes them tick.First lets assume the second law, d
dtSΦ⩾0, holds for all models. This implies that <∇𝜏(s)>⩾0 for all models, which naturally includes models tightly localised around any single point, so we get ∇𝜏(s)⩾0 for all possible states. If we also assume that a converged model which includes all states inside dΦ is possible. In that case, we require that <∇𝜏(s)>=0 holds for that model, as convergence implies no property of the model changes. As no state can have negative feedback, all states within the converged model need to have ∇𝜏(s)=0. Therefore, we get,
∀Φ.d
dtSΦ⩾0 and a converged model containing all statesis possible
⟹
∀s.∇𝜏(s)=0
On the other hand, we can show that,∀s.∇𝜏(s)=0⟹<∇𝜏(s)>=0⟹d
dtSΦ=0⟹d
dtSΦ⩾0We showed in (4) that ∇𝜏(s)=0 and a uniform distribution of states within a bound imply that convergence has occurred. A uniform distribution of states is possible if some sorts of bounds exists, so we get,
∀s.∇𝜏(s)=0 and a bounds exists⟹
∀Φ.d
dtSΦ⩾0 and a converged model containing all states is possible
The pattern A⟹B and B⟹A is known by another name, equivalence! A⟺B is the same as saying two statements are the same.
∀s.∇𝜏(s)=0 and a bounds exists⟺
∀Φ.d
dtSΦ⩾0 and a converged model containing all states is possible
A powerful implication of this is if ∇𝜏(s)=0 and a bounds exists, then d
dtSΦ⩾0, in addition to a few other key properties. A situation where any of the statements on the bottom row are false would imply that at least one of the statements on the top row is as well. Remember, equivalence! We can say that,∃Φ.d
dtSΦ<0⟹∃s.∇𝜏(s)<0 or a bounds does not existAnd in another way,∃Φ.d
dtSΦ<0 and a bounds exists⟹∃s.∇𝜏(s)<0)This solves for Maxwell's demon, and similar paradoxes that try to grapple with the second law. If someone makes the claim that entropy has decreased for some model, they are implying that ∇𝜏(s)<0 holds for some specific state. If the model is bounded, which can be achieved by effectively placing it in a very large closed box, the situation would have to be breaking a physical law somewhere somehow by requiring negative feedback.Now consider the case of Maxwell's demon. In this case, a little devil opens and closes a tiny door so that Oxygen molecules are arranged on one side of a barrier and Nitrogens on another, decreasing the entropy of air. If the starting assumptions of the system are uniformly distributed and mixed molecules, the final system would have to be, on average, equally or more entropic. Otherwise, negative feedback would have had to occured for some state.5. Lossiness and Information TheoryBack in (7) we jumped past ␒SΦ=0 straight to ␒SΦ⩾0 in order to imitate the typical formulation of the second law. If we do not make this jump, we can show that in a perfectly evolving physical system,
∀s.∇𝜏(s)=0 and a bounds exists.
⟺
∀Φ.d
dtSΦ=0, a converged model is possible, and a bounds exists.
␒SΦ⩾0 therefore comes from somewhere, but the requirement from physics that ∀s.∇𝜏(s)=0 isn't it. In fact, we haven't actually used much physics so far. Yes we've been describing physical systems, but with enough distance that a broad range of systems can have the above logic applied. The theorems so far haven't required knowledge of, say, Hamiltonian mechanics other than ∇𝜏(s)=0, or conservation laws.A model Φ gives us a distribution of probabilities over states, 𝜌(0,s). Over time, the states and the model evolves, giving some future 𝜌(t,s). Different states in a model evolve in parallel, so if𝜌(0,s)=(𝜌0(0,s)+𝜌1(0,s))/2 for distributions 𝜌0,𝜌1, then we will have 𝜌(t,s)=(𝜌0(t,s)+𝜌1(t,s))/2. The evolution of a system is the same as the sum of the evolution of its constituent parts. This linear nature makes it a Markov chain, giving 𝜌(t,s)=∫T(t,u,s)⋅𝜌(0,u)⋅du or 𝜌(t)=T(t)·𝜌(0).Ideally, our matrices T(t) are perfectly invertible just like Platonic ideal evolutions. Given some final state 𝜌(t)=T(t)·𝜌(0), we could deduce 𝜌(0)=T-1(t)·𝜌(t). In practice, this is not the case. Each applied time evolution, other than for analytically reducible systems, has to incorporate some lossiness. This lossiness means an initial probability distribution does not evolve perfectly but rather picks up uncertainty over time. This can arise from computational errors, such as rounding and imperfect increments. Or, it can arise from imperfect analytical projections, such as assuming a gas stays uniformly distributed after being expanded.5.1. Rounding ErrorLet's consider a specific error, rounding. Suppose we let rounding collapse multiple continuous states into a single rounded state. In that case, we will find a reduction in entropy, as the space of outputs is smaller than the space of inputs. We effectively imported an assumption, that the system collapses into discrete values, for which we do not exactly have a justification. Instead, let's think of rounding as adding a random aspect to the model. It is arbitrary to run a model on x∈[0,L] instead of y=x+𝛿,y∈[𝛿,L+𝛿]. The 𝛿 gives a random seed for how rounding will impact future outcomes. Now, instead of the model evolving simply by 𝜌(dt)=T(dt)·𝜌(0), it evolves by 𝜌(dt)=R·T(dt)·𝜌(0), where R gives a rounding error. As physical models should evolve independently of the shift 𝛿, our computer simulation incorporates an unacknowledged assumption into the model.Another way to think about it is that running a simulation only once with one choice of a shifting 𝛿 is not physically robust. We would get a more accurate idea by running 10 models with 𝛿={0,0.1,0.2...0.9}. These 10 simulations, though divergent, acknowledge that rounding randomness has been included, and the chaotic fingerprints of the second law arise. This rounding error extends to any invariant, from a shift in positions as shown to a rotation in any dimension. In the case of gases, lifting a barrier or the like will technically result in a state which if played in reverse will eventually return all the molecules behind the now hypothetical barrier. However, this information is typically discarded in analysing such systems. As the information is lost, the gases are assumed to follow the converged model under their new limitations, and hence have a higher entropy.6. Conserved Properties, Stochastic Transfer and the Boltzmann DistributionSuppose all states in a model have some conserved quantity Q(s) and states with effectively random evolutions. This is the case for the most common application of thermodynamics, gases with incalculable collisions. The key to modelling these sorts of systems is to impose conservation on micro-transfers between variables, such that any overall sum of micro-transfers also maintains conservation. The randomness becomes irrelevant, and even in some way useful. Knowing that effectively anything can happen limits us to considering the sorts of rules which have to hold for everything.For some state s∈Rn, we will say variables indexed by k and l are connected if micro-transfers between them are possible, which is to say one can randomly alter the other one by some amount. Let K be a set of tuples of all connected pairs, and wkl(s) be a weighting for 𝛿kl(s). Letting Ql,=∂Q/∂sl, we have 𝛿kl(s) be a micro-transfer equal to,
𝛿kl(s)
=ˆxkQl,(s)-ˆxlQk,(s)
We can rewrite any evolution map 𝜏(s) as,
𝜏(s)
=∑k,l∈Kwkl(s).𝛿kl(s)
The significance of 𝛿kl(s) is that it conserves Q on a local level, and thus any sum of 𝛿kl(s) also maintain Q as constant, even if the weightings are effectively random. Mathematically, we can show this through,
Q(s+dt.∑k,l∈Kwkl(s)(ˆxkQl-ˆxlQk))
=Q(s)+dt.∑k,l∈Kwkl(s)(QkQl-QlQk)
=Q(s)+dt.∑k,l∈Kwkl(s)(0)
=Q(s)
This means our micro-transfers expression captures the possible continuous evolutions.6.1. Change in ExpectationAt convergence, models stop changing. This naturally implies expectations don't change. This gives us,
d
dt<f>
=∫Φf(s).d𝜌(s)
dt.ds
0
=∫Φf(s).∇(𝜌(s).𝜏(s)).ds
=∫Φ∇(f(s).𝜌(s).𝜏(s))-∇f(s)·(𝜌(s).𝜏(s)).ds
0
=∫dΦf(s).𝜌(s).𝜏(s)·dn(s)-∫Φ∇f(s)·𝜏(s).𝜌(s)ds
Now, we can analyse this with our micro-transfers method. To ensure the above holds for the effectively random 𝜏(s) we see in practice, we will analyse it for arbitrary 𝛿(s) values. So we get,
0
=∫dΦf(s).𝜌(s).𝛿kl(s)·dn(s)-∫Φ∇f(s)·𝛿kl(s).𝜌(s)ds
6.2. Unbounded k,l and the Generating FunctionFirst, lets assume k,l are unbounded, or close enough that the probability that either is at their bounds is essentially zero, 𝜌(s)≈0 if s∈dΦ. This could describe the energy of a single particle out of millions or a single consumer in an economy. The small chance that a single particle has all the energy or that one consumer spends all the money is negligible. The below can be thought of as holding for some sufficiently large number of particles N. Looking at (8), we get,
0
=∫Φ∇f(s)·𝛿kl(s).𝜌(s)ds
=<∇f·𝛿kl>
=<fk,.Ql,-fl,.Qk,>
<fk,.Ql,>
=<fl,.Qk,>
This then becomes the generating function. Replacing f with some specific value automatically generates different theorems regarding converged models.6.3. The Equipartition TheoremFirst, we substitute f(s)=sk·sl to get;
<∂
∂sk(sk·sl).Ql,>
=<∂
∂sl(sk·sl).Qk,>
<sl.Ql,>
=<sk.Qk,>
TQl
=TQk
This equilibrium TQl is shared across k and l if they are connected for the conserved variable Q. Next, substituting f(s)=1
2(s2k+s2l) we get,
<sk.Ql,>
=<sl.Qk,>
We now also assume that sk and sl are independently distributed across the model. In addition, we assert that Ql, is not dependent on sk, and Qk, is dependent not on sl. This is an assumption of overall independence within the system, and in a sufficiently chaotic one two random particles should behave in such a way.
<sk><Ql,>
=<sl><Qk,>
If any of the above averages are zero, for any of the connected components, we get,
<sk><Ql,>
=<sl><Qk,>=0
This cascades across connected components. If (9) holds for any l and k, it also has to hold for any variable connected to either one. Finally, we see that we can express the general case, and overall we can state,<si.Qj,>=a
TQi
,i=j
0
,i≠j
For connected components i, j. We get the equipartition theorem in its typical form.6.4. Gravity in a BoxFor those familiar with thermodynamics, why we are going beyond the typical formulation of the equipartition theorem (10) can be explained by an example, a gas in a box. The distribution of gases in the atmosphere is given by the baric equation. Or;
𝜌(h)
∝exp(-E(h)
kBT)=exp(-mgh
kBT)
Using Wolfram Mathematica, we can calculate the expected value of <h.Eh,>=<h.mg> in an infinite atmosphere as,
<h.mg>
=∞∫0hmgexp(-mgh
kBT)dh
∞∫0exp(-mgh
kBT)dh
=(k2BT2
gm)(kBT
gm)-1=kBT
Which gives the equipartition theorem, the movement of particles between 0 and ∞ adds a degree of freedom. Heating an atmosphere is about one degree of freedom more difficult than heating a closed box. However, in a closed box, we do not quite get the equipartition theorem.
<h.mg>
=𝛽∫0hmgexp(-mgh
kBT)dh
𝛽∫0exp(-mgh
kBT)dh
=(kBT(kBT-e-gm𝛽
kBT(kBT+gm𝛽))
gm)((1-e-gm𝛽
kBT)kBT
gm)-1
=kBT-gm𝛽
egm𝛽
kBT-1≠kBT
So the presence of bounds seems to change the nature of the equipartition theorem. To understand the theorem mathematically, we need to be able to understand this case. It will take us on a bit of a journey, but ultimately, will show the Boltzmann distribution can be derived from the stochastic transfer assumption.6.5. Bounded l and the Boltzmann DistributionFrom (8) we got,
0
=∫Φf(s).d
dt𝜌(s)·ds
=∫dΦf(s).𝜌(s).𝛿kl(s)·dn(s)-∫Φ∇f(s)·𝛿kl(s).𝜌(s)ds
We will assume sk is bounded between 𝛽↓k and 𝛽↑k and sl is unbounded. We will also be assuming that their distributions are independent. This reduces (12) down to,
We want to separate out <sl.Ql,>, as we know it to be equal to TQk by the equipartition theorem. We do this by substituting f(s)=sl.g(sk), where g(sk) is some continuous function.6.5.1. Breaking down the boundsAt the boundaries of sk, the first part of (13) breaks down into,
𝛽↑k∫𝛽↓kf(s).𝜌(s).𝛿kl(s)·dn(s)
=𝛽↑k∫𝛽↓ksl.g(sk).𝜌(s).Ql(s).ds\k
𝛿kl(s)·dn(s) at the bounds is the movement of 𝛿kl(s) against the bounds, which gives Ql(s), the scale of movement in the bounded k direction, by the "area" of a volume unit with k removed. We will be generously using the independence of sk and sl, which allows us to expand E[𝜙(sk)𝜓(sl)]=E[𝜙(sk)].E[𝜓(sl)] for any functions 𝜙 and 𝜓.
=<sl.Ql(s)|sk=𝛽↑kor sk=𝛽↓k⏠⏣⏣⏣⏣⏡⏣⏣⏣⏣⏢Independent to sl>.𝛽↑k∫𝛽↓kg(sk).𝜌(s).ds\k⏠⏣⏣⏣⏣⏡⏣⏣⏣⏣⏢Surface integral at both extremes
=<sl.Ql(s)>.[g(sk).𝜌sk(sk)]𝛽↑k𝛽↓k
=TQk.[g(sk).𝜌sk(sk)]𝛽↑k𝛽↓k
6.5.2. Breaking down the interiorLooking at second part of (13), and treating sk and sl as independent variables in the model, we get;
∫Φ∇(sl.g(sk))·𝛿kl(s).𝜌(s).ds
=∫Φa
sl.g'(sk)
g(sk)
T·a
Ql,(s)
-Qk,(s)
.𝜌(s).ds
=∫Φg'(sk).sl.Ql,(s).𝜌(s)-g(sk).Qk,(s).𝜌(s).ds
=<g'(sk)><sl.Ql,>-<g(sk).Qk,(s)>
=<g'(sk)>.TQk-<g(sk).Qk,(s)>
6.5.3. Bringing it togetherCombining (14) and (15) we get,
TQk.[g(sk).𝜌sk(sk)]𝛽↑k𝛽↓k
=<g'(sk)>.TQk-<g(sk).Qk,(s)>
<g(sk).Qk,(s)>
=TQk.(<g'(sk)>-[g(sk).𝜌sk(sk)]𝛽↑k𝛽↓k)
Letting g(x)=x, we get,
<sk.Qk,(s)>
=TQk.(1-[sk.𝜌sk(sk)]𝛽↑k𝛽↓k)
Which explains (11)! However, we can do a slight expansion to get Boltzmann's distribution. One way is to let g(x)=𝛿(x-𝛽), the Kronecker delta. However, we can use a perhaps less gimmicky method with the fundamental lemma of the calculus of variations (what a long and hence terrible name). I'll show you how it works in a second. Looking at (16) we have,
Now let's apply math's worst named theorem. As the right hand side is zero, independent of the choice of g(sk), we need to have,a
0
=(TQk.𝜌'sk(sk)+Qk,(s).𝜌sk(sk))
This is the only way to ensure the right hand side is zero no matter what we choose for g(sk). Rearranging this expression we can derive,
0
=(TQk.𝜌'sk(sk)+Qk,(s).𝜌sk(sk))
𝜌'sk(sk)
=-Qk,(s)
TQk.𝜌sk(sk)
This suggests to us that 𝜌sk(sk) has some sort of exponential form with regards to Q(s). We will define qk(s) as the contribution of k to Q(s), such that Qk,(s)=qk(s). Typically, this qk(s) value should only be a function of sk.
∴𝜌sk(sk)
=A0.exp(-qk(s)
TQk), where qk(s) is the contribution of k to Q(s)
l(sk)
=C0+qk(s)
TQk
Giving the Boltzmann distribution.7. Changing Bounds and EntropySo far, we have only considered internal or endogenous evolution maps which conserve properties. Often we want to look at what happens when bounds change, or when an external force is applied to a system. Let some evolution map on states be dictated by 𝜈, which is distinct from 𝜏, and has no issue not leaving conserved properties Q intact.Letting the path of state under an exogenous change be dictated by the map d
dvs(v)=𝜈(s), we have SΦ(v) evolve similary to (5) and (6), giving,
d
dvSΦ(v)
=<𝜈·∇l>=<∇𝜈>
Rearranging (18), we can get ∇l=∇Q(s)/TQ. If all states are connected so that there is only one TQ value, we have,
d
dvSΦ(v)
=<𝜈·∇Q(s)
TQ>=<∇𝜈>
The change in Q(s) for some specific state from applying 𝜈 is given by 𝜈·∇Q(s), the direction of change times how much Q(s) changes in that direction. We will assume this is constant for all states in the model, so that we only have to deal with one Q(s) value.
d
dvQ(s)
=𝜈·∇Q(s)
1
TQd
dvQ(s)
=<𝜈·∇Q(s)
TQ>
d
dvSΦ(v)
=1
TQd
dvQ(s)=<∇𝜈>
If multiple conserved variables exist, each one contributes to the change in entropy. We can express this as,
dSΦ
=∑i1
TQidQi
7.1. A Note on Reversibility(21) refers to a natural evolution, where the system evolves according to deterministic laws applied on scale. However, entropy can also change because of the factors explained in Section 5. Changes due to lost information, such as rounding error, are irreversible and can cause entropy to change without carefully exchanging quantities between conserved variables. (21) refers to entropy changes under a managed situation, and this caveat applies to all expressions derived from it. More accurately, we could write (21) as,
dSΦ
=dSlost+∑i1
TQidQi
7.2. Volume as a Conserved QuantityVolume and heat are the two conserved variables of particular interest for this work. To perform an expansion of volume, we will map every particle along some dimension from xi to (1+dV/V)xi. This gives an evolution 𝜈=dV
Vxi and feedback,
<∇𝜈(s)>
=N∑i=1⏠⏣⏣⏡⏣⏣⏢Particles∂
∂xi(dV
Vxi)
=N∑i=1(dV
V)
dSΦ
=N.dV
V
Alternatively, we can derive volume using density. To do this, we will assign to each particle an occupancy wi, the volume of space to which it is the closest particle, such that the sum of occupancies gives V. The relevant sk variables are occupancies of different particles, and Vk,(w)=1. The occupancies are effectively unbounded as V and N become large.
Combining the energy and volume expressions we get,
dSΦ
=dU(s)
TU+N.dV
V
7.3. The Ideal Gas Law and the First LawRearranging (26), we get a different expression for dU(s). This gives us the first law of thermodynamics, stated as;
dU(s)
=TU.dSΦ⏠⏣⏣⏡⏣⏣⏢"Heat"-TU.N
V.dV⏠⏣⏣⏣⏡⏣⏣⏣⏢"Work"
"=dQ-p.dV"
We will let TU.N
V=p, for "pressure". We can show that,
p.V
=TU.N
Which is the ideal gas law. Setting dSΦ=0 to analyse a zero entropy natural expansion we see that pressure gives the marginal energy of a naturally changing volume of gas.
dU(s)
=TU.0-TU.N
V.dV
dU(s)
dV
=-TU.N
V=-p
The significance of these derivations is not in what we have done but rather what we haven't. There has been no mention of Hamiltonian mechanics, other than <∇𝜏>=0, or of F=ma, or arguments involving particles hitting the sides of chambers. An implication of this is that the ideal gas law is independent of F=ma, such as in the case of relativistic physics. Both the ideal gas law and the first law of thermodynamics can be derived with little physics. The only necessary assumptions so far are that large chaotic arrangements can be modelled with stochastic transfers with conserved U and V, that endogenous feedback ∇𝜏 is zero, and that external changes lead to a change in U and V which is constant for all states. This imposes a certain relationship between energy and volume.7.4. Generalising PressureThe above derivation is only for a particular case. It is common in thermodynamics for Van der Waals forces and other forms of pressure to arise. This requires an explanation and a more general way of viewing pressure. Pressure is perhaps best thought of as the volumetric density of energy, the sort of latent expansive power from altering the volume of a substance by some amount. So dU/dV will be our starting point. This allows us to think of pressure of conserved variables other than volume. Volume is ultimately just a conserved variable of a system, but others might arise in future investigations, and we will define pressure for them as dU/dQi. A natural evolution with zero entropy change can be taken from (21),dSΦ=∑i1
TQidQiIf we set this to zero and consider only two conserved variables, U and Q𝛼, we get,dQ𝛼
T𝛼=-dU
TUAnd from this we get the pressure ratio,dU
dQ𝛼=-TU
TQ𝛼=-pQ𝛼For volumetric pressure we write,-dU
dV=pV=TU
TVThis is pretty neat! Esepcially if we consider that TQi values have not been obviously intuitive for things other than energy. We can rewrite some previous values, including TQi, in terms of their natural pressures pQi.
TQi
=TU
pQi
=TUdQi
dU
When dealing with a large number of conserved variables, we'd often start dealing with partial derivitives and their associated... difficulties. From (21), we write,
∂S
∂Qi
=1
TQi=pQi
TU
This also allows us to rewrite (22) into what many would consider a more recognisable and standard form,
dSΦ
=dSloss+dU
TU+∑i\{U}pQi
TUdQi
7.5. Van der Waals ForceThe Van der Waals force is an attempt to have a more precise form of pressure than the ideal gas law for certain applications. In many cases, it has nicer experimental validity than the standard ideal gas law. Fortunately, it can be readily integrated into the system we have built up so far. To do this we will look at a form of a Van der Waals equation of state and then see how we can use our previous equations to mathematically examine it. The ideal gas law states that,
N.T
=p.V
However, the attractive force between particles means a small factor reduces pressure as more are added. A convenient way to represent this is to reduce pressure by a quadratic factor a(N/V)2.p→p'-a(N
V)2So we have to compensate our expression for pressure in the ideal gas law by writing,N.T=(p+a(N
V)2).VFurthermore, V is limited by a sort of minimal amount that each particle occupies. Crowding out naturally leaves less free volume for particles to take up, so we instead use the free volume V-nb in the ideal gas law,N.T=(p+a(N
V)2).(V-Nb)This form of the ideal gas law is rather complex, but we're going to try find a way to incorporate it into our current model! We sort of know what pressure means, so if we can isolate it, we can link the Van der Waals equation of state to the rest of our equations.
N.T
=(p+a(N
V)2).(V-Nb)
N.T
V-nb
=p+a(N
V)2
p
=N.T
V-Nb-a(N
V)2
The equation for entropy from (28) links changes in entropy to pressure,
dSΦ
=dU
T+p.dV
T
Substituting in the Van der Waals pressure equation (30),