Entropy and Free Energy in Structural Biology

Hagai Meirovitch is professor Emeritus in the Department of Computational and Systems Biology at the University of Pittsburgh School of Medicine. He earned an MSc degree in nuclear physics from the Hebrew University, a PhD degree in chemical physics from the Weizmann Institute, and conducted postdoctoral training in the laboratory of Professor Harold A. Scheraga at Cornell University. His research focused on developing computer simulation methodologies within the scope of statistical mechanics, as highlighted below. He devised novel methods for extracting the absolute entropy from Monte Carlo samples and techniques for generating polymer chains, which were used to study phase transitions in polymers, magnetic, and lattice gas systems. These methods, together with conformational search techniques for proteins, led to a free energy-based approach for treating molecular flexibility. This approach was used to analyze NMR relaxation data from cyclic peptides and to study structural preferences of surface loops in bound and free enzymes. He developed a new methodology for calculating the free energy of ligand/protein binding, which unlike standard techniques, provides the decrease in the ligand’s entropy upon binding. Dr Meirovitch conducted part of the research depicted above, and other studies, at the Supercomputer Computations Research Institute of the Florida State University, Tallahassee.

• ContentsPreface ..................................................................................................................................................... xvAcknowledgments ...................................................................................................................................xixAuthor .....................................................................................................................................................xxiSection I Probability Theory1. Probability and Its Applications ..................................................................................................... 31.1 Introduction ............................................................................................................................. 31.2 Experimental Probability ........................................................................................................ 31.3 The Sample Space Is Related to the Experiment .................................................................... 41.4 Elementary Probability Space ................................................................................................ 51.5 Basic Combinatorics ............................................................................................................... 61.5.1 Permutations ............................................................................................................. 61.5.2 Combinations ............................................................................................................ 71.6 Product Probability Spaces ..................................................................................................... 91.6.1 The Binomial Distribution .......................................................................................111.6.2 Poisson Theorem ......................................................................................................111.7 Dependent and Independent Events ...................................................................................... 121.7.1 Bayes Formula......................................................................................................... 121.8 Discrete Probability—Summary .......................................................................................... 131.9 One-Dimensional Discrete Random Variables ..................................................................... 131.9.1 The Cumulative Distribution Function ....................................................................141.9.2 The Random Variable of the Poisson Distribution ..................................................141.10 Continuous Random Variables ..............................................................................................141.10.1 The Normal Random Variable ................................................................................ 151.10.2 The Uniform Random Variable .............................................................................. 151.11 The Expectation Value ...........................................................................................................161.11.1 Examples ..................................................................................................................161.12 The Variance ..........................................................................................................................171.12.1 The Variance of the Poisson Distribution ................................................................181.12.2 The Variance of the Normal Distribution ................................................................181.13 Independent and Uncorrelated Random Variables ............................................................... 191.13.1 Correlation .............................................................................................................. 191.14 The Arithmetic Average ....................................................................................................... 201.15 The Central Limit Theorem .................................................................................................. 211.16 Sampling ............................................................................................................................... 231.17 Stochastic Processes—Markov Chains ................................................................................ 231.17.1 The Stationary Probabilities ................................................................................... 251.18 The Ergodic Theorem ........................................................................................................... 261.19 Autocorrelation Functions .................................................................................................... 271.19.1 Stationary Stochastic Processes .............................................................................. 28Homework for Students .................................................................................................................... 28A Comment about Notations ............................................................................................................ 28References ........................................................................................................................................ 29Section II Equilibrium Thermodynamics and Statistical Mechanics2. Classical Thermodynamics ........................................................................................................... 332.1 Introduction ........................................................................................................................... 332.2 Macroscopic Mechanical Systems versus Thermodynamic Systems .................................. 332.3 Equilibrium and Reversible Transformations ....................................................................... 342.4 Ideal Gas Mechanical Work and Reversibility ..................................................................... 342.5 The First Law of Thermodynamics ...................................................................................... 362.6 Joule’s Experiment ................................................................................................................ 372.7 Entropy .................................................................................................................................. 392.8 The Second Law of Thermodynamics .................................................................................. 402.8.1 Maximal Entropy in an Isolated System..................................................................412.8.2 Spontaneous Expansion of an Ideal Gas and Probability ....................................... 422.8.3 Reversible and Irreversible Processes Including Work ........................................... 422.9 The Third Law of Thermodynamics .................................................................................... 432.10 Thermodynamic Potentials ................................................................................................... 432.10.1 The Gibbs Relation ................................................................................................. 432.10.2 The Entropy as the Main Potential ......................................................................... 442.10.3 The Enthalpy ........................................................................................................... 452.10.4 The Helmholtz Free Energy .................................................................................... 452.10.5 The Gibbs Free Energy ........................................................................................... 452.10.6 The Free Energy, H(T,μ) ........................................................................................ 462.11 Maximal Work in Isothermal and Isobaric Transformations ............................................... 472.12 Euler’s Theorem and Additional Relations for the Free Energies ........................................ 482.12.1 Gibbs-Duhem Equation .......................................................................................... 492.13 Summary ............................................................................................................................... 49Homework for Students .................................................................................................................... 49References ........................................................................................................................................ 49Further Reading ................................................................................................................................ 493. From Thermodynamics to Statistical Mechanics ........................................................................513.1 Phase Space as a Probability Space .......................................................................................513.2 Derivation of the Boltzmann Probability ............................................................................. 523.3 Statistical Mechanics Averages ............................................................................................ 543.3.1 The Average Energy ................................................................................................ 543.3.2 The Average Entropy .............................................................................................. 543.3.3 The Helmholtz Free Energy .................................................................................... 553.4 Various Approaches for Calculating Thermodynamic Parameters ...................................... 553.4.1 Thermodynamic Approach ..................................................................................... 553.4.2 Probabilistic Approach ........................................................................................... 563.5 The Helmholtz Free Energy of a Simple Fluid ..................................................................... 56Reference .......................................................................................................................................... 58Further Reading ................................................................................................................................ 584. Ideal Gas and the Harmonic Oscillator ....................................................................................... 594.1 From a Free Particle in a Box to an Ideal Gas ...................................................................... 594.2 Properties of an Ideal Gas by the Thermodynamic Approach ............................................. 604.3 The chemical potential of an Ideal Gas ................................................................................ 624.4 Treating an Ideal Gas by the Probability Approach ............................................................. 634.5 The Macroscopic Harmonic Oscillator ................................................................................ 644.6 The Microscopic Oscillator .................................................................................................. 654.6.1 Partition Function and Thermodynamic Properties ............................................... 664.7 The Quantum Mechanical Oscillator ................................................................................... 684.8 Entropy and Information in Statistical Mechanics ............................................................... 714.9 The Configurational Partition Function ................................................................................ 71Homework for Students .................................................................................................................... 72References ........................................................................................................................................ 72Further Reading ................................................................................................................................ 725. Fluctuations and the Most Probable Energy ............................................................................... 735.1 The Variances of the Energy and the Free Energy ............................................................... 735.2 The Most Contributing Energy E* ....................................................................................... 745.3 Solving Problems in Statistical Mechanics .......................................................................... 765.3.1 The Thermodynamic Approach .............................................................................. 775.3.2 The Probabilistic Approach .................................................................................... 785.3.3 Calculating the Most Probable Energy Term .......................................................... 795.3.4 The Change of Energy and Entropy with Temperature .......................................... 80References ........................................................................................................................................ 816. Various Ensembles ......................................................................................................................... 836.1 The Microcanonical (petit) Ensemble .................................................................................. 836.2 The Canonical (NVT) Ensemble ........................................................................................... 846.3 The Gibbs (NpT) Ensemble .................................................................................................. 856.4 The Grand Canonical (μVT) Ensemble ................................................................................ 886.5 Averages and Variances in Different Ensembles .................................................................. 906.5.1 A Canonical Ensemble Solution (Maximal Term Method) .................................... 906.5.2 A Grand-Canonical Ensemble Solution .................................................................. 916.5.3 Fluctuations in Different Ensembles....................................................................... 91References ........................................................................................................................................ 92Further Reading ................................................................................................................................ 927. Phase Transitions ........................................................................................................................... 937.1 Finite Systems versus the Thermodynamic Limit ................................................................ 937.2 First-Order Phase Transitions ............................................................................................... 947.3 Second-Order Phase Transitions ........................................................................................... 95References ........................................................................................................................................ 988. Ideal Polymer Chains ..................................................................................................................... 998.1 Models of Macromolecules ................................................................................................... 998.2 Statistical Mechanics of an Ideal Chain ............................................................................... 998.2.1 Partition Function and Thermodynamic Averages ............................................... 1008.3 Entropic Forces in an One-Dimensional Ideal Chain..........................................................1018.4 The Radius of Gyration ...................................................................................................... 1048.5 The Critical Exponent ν ...................................................................................................... 1058.6 Distribution of the End-to-End Distance ............................................................................ 1068.6.1 Entropic Forces Derived from the Gaussian Distribution .................................... 1078.7 The Distribution of the End-to-End Distance Obtained from the Central Limit Theorem .... 1088.8 Ideal Chains and the Random Walk ................................................................................... 1098.9 Ideal Chain as a Model of Reality .......................................................................................110References .......................................................................................................................................1109. Chains with Excluded Volume .....................................................................................................1119.1 The Shape Exponent ν for Self-avoiding Walks ..................................................................1119.2 The Partition Function .........................................................................................................1129.3 Polymer Chain as a Critical System ....................................................................................1139.4 Distribution of the End-to-End Distance .............................................................................1149.5 The Effect of Solvent and Temperature on the Chain Size .................................................1159.5.1 θ Chains in d = 3 ...................................................................................................1169.5.2 θ Chains in d = 2 ...................................................................................................1169.5.3 The Crossover Behavior Around θ.........................................................................1179.5.4 The Blob Picture ....................................................................................................1189.6 Summary ..............................................................................................................................119References .......................................................................................................................................119Section III Topics in Non-Equilibrium Thermodynamicsand Statistical Mechanics10. Basic Simulation Techniques: Metropolis Monte Carlo and Molecular Dynamics .............. 12310.1 Introduction ......................................................................................................................... 12310.2 Sampling the Energy and Entropy and New Notations ...................................................... 12410.3 More About Importance Sampling ..................................................................................... 12510.4 The Metropolis Monte Carlo Method ................................................................................. 12610.4.1 Symmetric and Asymmetric MC Procedures ....................................................... 12710.4.2 A Grand-Canonical MC Procedure ...................................................................... 12810.5 Efficiency of Metropolis MC .............................................................................................. 12910.6 Molecular Dynamics in the Microcanonical Ensemble ......................................................13110.7 MD Simulations in the Canonical Ensemble ...................................................................... 13410.8 Dynamic MD Calculations ..................................................................................................13510.9 Efficiency of MD .................................................................................................................13510.9.1 Periodic Boundary Conditions and Ewald Sums .................................................. 13610.9.2 A Comment About MD Simulations and Entropy................................................ 136References ...................................................................................................................................... 13711. Non-Equilibrium Thermodynamics—Onsager Theory .......................................................... 13911.1 Introduction ......................................................................................................................... 13911.2 The Local-Equilibrium Hypothesis .................................................................................... 13911.3 Entropy Production Due to Heat Flow in a Closed System ................................................ 14011.4 Entropy Production in an Isolated System...........................................................................14111.5 Extra Hypothesis: A Linear Relation Between Rates and Affinities ..................................14211.5.1 Entropy of an Ideal Linear Chain Close to Equilibrium .......................................14311.6 Fourier’s Law—A Continuum Example of Linearity ......................................................... 14411.7 Statistical Mechanics Picture of Irreversibility ...................................................................14511.8 Time Reversal, Microscopic Reversibility, and the Principle of Detailed Balance ............14711.9 Onsager’s Reciprocal Relations ...........................................................................................14911.10 Applications ........................................................................................................................ 15011.11 Steady States and the Principle of Minimum Entropy Production .....................................15111.12 Summary ..............................................................................................................................152References .......................................................................................................................................15212. Non-equilibrium Statistical Mechanics ......................................................................................15312.1 Fick’s Laws for Diffusion ....................................................................................................15312.1.1 First Fick’s Law ......................................................................................................15312.1.2 Calculation of the Flux from Thermodynamic Considerations ............................ 15412.1.3 The Continuity Equation ........................................................................................15512.1.4 Second Fick’s Law—The Diffusion Equation ...................................................... 15612.1.5 Diffusion of Particles Through a Membrane ........................................................ 15612.1.6 Self-Diffusion ........................................................................................................ 15612.2 Brownian Motion: Einstein’s Derivation of the Diffusion Equation .................................. 15812.3 Langevin Equation .............................................................................................................. 16012.3.1 The Average Velocity and the Fluctuation-Dissipation Theorem .........................16212.3.2 Correlation Functions.............................................................................................16312.3.3 The Displacement of a Langevin Particle ............................................................. 16412.3.4 The Probability Distributions of the Velocity and the Displacement ................... 16612.3.5 Langevin Equation with a Charge in an Electric Field ..........................................16812.3.6 Langevin Equation with an External Force—The Strong Damping Velocity .......16812.4 Stochastic Dynamics Simulations .......................................................................................16912.4.1 Generating Numbers from a Gaussian Distribution by CLT .................................17012.4.2 Stochastic Dynamics versus Molecular Dynamics................................................17112.5 The Fokker-Planck Equation ...............................................................................................17112.6 Smoluchowski Equation.......................................................................................................17412.7 The Fokker-Planck Equation for a Full Langevin Equation with a Force...........................17512.8 Summary of Pairs of Equations ...........................................................................................175References .......................................................................................................................................17613. The Master Equation ....................................................................................................................17713.1 Master Equation in a Microcanonical System .....................................................................17713.2 Master Equation in the Canonical Ensemble.......................................................................17813.3 An Example from Magnetic Resonance ............................................................................. 18013.3.1 Relaxation Processes Under Various Conditions ...................................................18113.3.2 Steady State and the Rate of Entropy Production ................................................. 18413.4 The Principle of Minimum Entropy Production—Statistical Mechanics Example............185References .......................................................................................................................................186Section IV Advanced Simulation Methods: Polymersand Biological Macromolecules14. Growth Simulation Methods for Polymers .................................................................................18914.1 Simple Sampling of Ideal Chains ........................................................................................18914.2 Simple Sampling of SAWs .................................................................................................. 19014.3 The Enrichment Method ..................................................................................................... 19214.4 The Rosenbluth and Rosenbluth Method ............................................................................ 19314.5 The Scanning Method ......................................................................................................... 19514.5.1 The Complete Scanning Method .......................................................................... 19514.5.2 The Partial Scanning Method ............................................................................... 19614.5.3 Treating SAWs with Finite Interactions ................................................................ 19714.5.4 A Lower Bound for the Entropy ........................................................................... 19714.5.5 A Mean-Field Parameter ....................................................................................... 19814.5.6 Eliminating the Bias by Schmidt’s Procedure ...................................................... 19914.5.7 Correlations in the Accepted Sample ................................................................... 20014.5.8 Criteria for Efficiency ........................................................................................... 20114.5.9 Locating Transition Temperatures ........................................................................ 20214.5.10 The Scanning Method versus Other Techniques .................................................. 20314.5.11 The Stochastic Double Scanning Method ............................................................ 20414.5.12 Future Scanning by Monte Carlo .......................................................................... 20414.5.13 The Scanning Method for the Ising Model and Bulk Systems ............................. 20514.6 The Dimerization Method .................................................................................................. 206References ...................................................................................................................................... 20815. The Pivot Algorithm and Hybrid Techniques ............................................................................21115.1 The Pivot Algorithm—Historical Notes ..............................................................................21115.2 Ergodicity and Efficiency ....................................................................................................21115.3 Applicability ........................................................................................................................21215.4 Hybrid and Grand-Canonical Simulation Methods .............................................................21315.5 Concluding Remarks ............................................................................................................214References .......................................................................................................................................21416. Models of Proteins .........................................................................................................................21716.1 Biological Macromolecules versus Polymers ......................................................................21716.2 Definition of a Protein Chain ...............................................................................................21716.3 The Force Field of a Protein ................................................................................................21816.4 Implicit Solvation Models ....................................................................................................21916.5 A Protein in an Explicit Solvent ......................................................................................... 22016.6 Potential Energy Surface of a Protein ................................................................................ 22116.7 The Problem of Protein Folding ......................................................................................... 22216.8 Methods for a Conformational Search ................................................................................ 22216.8.1 Local Minimization—The Steepest Descents Method ........................................ 22316.8.2 Monte Carlo Minimization ................................................................................... 22416.8.3 Simulated Annealing ............................................................................................ 22516.9 Monte Carlo and Molecular Dynamics Applied to Proteins .............................................. 22516.10 Microstates and Intermediate Flexibility ........................................................................... 22616.10.1 On the Practical Definition of a Microstate .......................................................... 227References ...................................................................................................................................... 22717. Calculation of the Entropy and the Free Energy by Thermodynamic Integration ................23117.1 “Calorimetric” Thermodynamic Integration ...................................................................... 23217.2 The Free Energy Perturbation Formula .............................................................................. 23217.3 The Thermodynamic Integration Formula of Kirkwood ................................................... 23417.4 Applications ........................................................................................................................ 23517.4.1 Absolute Entropy of a SAW Integrated from an Ideal Chain Reference State ..... 23517.4.2 Harmonic Reference State of a Peptide ................................................................ 23717.5 Thermodynamic Cycles ...................................................................................................... 23717.5.1 Other Cycles .......................................................................................................... 24017.5.2 Problems of TI and FEP Applied to Proteins ....................................................... 240References ...................................................................................................................................... 24118. Direct Calculation of the Absolute Entropy and Free Energy ................................................ 24318.1 Absolute Free Energy from E/kBT]> ...................................................................... 24318.2 The Harmonic Approximation ........................................................................................... 24418.3 The M2 Method .................................................................................................................. 24518.4 The Quasi-Harmonic Approximation ................................................................................. 24618.5 The Mutual Information Expansion ................................................................................... 24718.6 The Nearest Neighbor Technique ....................................................................................... 24818.7 The MIE-NN Method ......................................................................................................... 24918.8 Hybrid Approaches ............................................................................................................. 249References ...................................................................................................................................... 24919. Calculation of the Absolute Entropy from a Single Monte Carlo Sample...............................25119.1 The Hypothetical Scanning (HS) Method for SAWs ...........................................................25119.1.1 An Exact HS Method .............................................................................................25119.1.2 Approximate HS Method ...................................................................................... 25219.2 The HS Monte Carlo (HSMC) Method .............................................................................. 25319.3 Upper Bounds and Exact Functionals for the Free Energy ................................................ 25519.3.1 The Upper Bound FB ............................................................................................ 25519.3.2 FB Calculated by the Reversed Schmidt Procedure ............................................. 25619.3.3 A Gaussian Estimation of FB ................................................................................ 25719.3.4 Exact Expression for the Free Energy .................................................................. 25819.3.5 The Correlation Between σA and FA ..................................................................... 25819.3.6 Entropy Results for SAWs on a Square Lattice .................................................... 25919.4 HS and HSMC Applied to the Ising Model ........................................................................ 26019.5 The HS and HSMC Methods for a Continuum Fluid ..........................................................26119.5.1 The HS Method ......................................................................................................26119.5.2 The HSMC Method ............................................................................................... 26219.5.3 Results for Argon and Water ................................................................................. 26419.5.3.1 Results for Argon .................................................................................. 26419.5.3.2 Results for Water .................................................................................. 26619.6 HSMD Applied to a Peptide ............................................................................................... 26619.6.1 Applications .......................................................................................................... 26919.7 The HSMD-TI Method ....................................................................................................... 26919.8 The LS Method ................................................................................................................... 27019.8.1 The LS Method Applied to the Ising Model ......................................................... 27019.8.2 The LS Method Applied to a Peptide ................................................................... 272References .......................................................................................................................................27420. The Potential of Mean Force, Umbrella Sampling, and Related Techniques ........................ 27720.1 Umbrella Sampling ............................................................................................................. 27720.2 Bennett’s Acceptance Ratio ................................................................................................ 27820.3 The Potential of Mean Force .............................................................................................. 28120.3.1 Applications .......................................................................................................... 28420.4 The Self-Consistent Histogram Method ............................................................................. 28520.4.1 Free Energy from a Single Simulation.................................................................. 28620.4.2 Multiple Simulations and The Self-Consistent Procedure.................................... 28620.5 The Weighted Histogram Analysis Method ....................................................................... 28920.5.1 The Single Histogram Equations .......................................................................... 29020.5.2 The WHAM Equations ..........................................................................................29120.5.3 Enhancements of WHAM .................................................................................... 29320.5.4 The Basic MBAR Equation .................................................................................. 29520.5.5 ST-WHAM and UIM ............................................................................................ 29620.5.6 Summary ............................................................................................................... 296References ...................................................................................................................................... 29721. Advanced Simulation Methods and Free Energy Techniques ................................................. 30121.1 Replica-Exchange ............................................................................................................... 30121.1.1 Temperature-Based REM ..................................................................................... 30121.1.2 Hamiltonian-Dependent Replica Exchange .......................................................... 30521.2 The Multicanonical Method ............................................................................................... 30821.2.1 Applications ...........................................................................................................31121.2.2 MUCA-Summary ..................................................................................................31221.3 The Method of Wang and Landau .......................................................................................31221.3.1 The Wang and Landau Method-Applications ........................................................31421.4 The Method of Expanded Ensembles ..................................................................................31521.4.1 The Method of Expanded Ensembles-Applications ..............................................31721.5 The Adaptive Integration Method .......................................................................................31721.6 Methods Based on Jarzynski’s Identity ...............................................................................31921.6.1 Jarzynski’s Identity versus Other Methods for Calculating ΔF ........................... 32321.7 Summary ............................................................................................................................. 324References ...................................................................................................................................... 32422. Simulation of the Chemical Potential ..........................................................................................33122.1 The Widom Insertion Method .............................................................................................33122.2 The Deletion Procedure .......................................................................................................33222.3 Personage’s Method for Treating Deletion ......................................................................... 33422.4 Introduction of a Hard Sphere ............................................................................................ 33622.5 The Ideal Gas Gauge Method ............................................................................................. 33722.6 Calculation of the Chemical Potential of a Polymer by the Scanning Method .................. 33822.7 The Incremental Chemical Potential Method for Polymers ............................................... 34022.8 Calculation of μ by Thermodynamic Integration ................................................................341References .......................................................................................................................................34123. The Absolute Free Energy of Binding ........................................................................................ 34323.1 The Law of Mass Action ..................................................................................................... 34323.2 Chemical Potential, Fugacity, and Activity of an Ideal Gas............................................... 34423.2.1 Thermodynamics .................................................................................................. 34423.2.2 Canonical Ensemble.............................................................................................. 34423.2.3 NpT Ensemble ....................................................................................................... 34523.3 Chemical Potential in Ideal Solutions: Raoult’s and Henry’s Laws ................................... 34523.3.1 Raoult’s Law ......................................................................................................... 34623.3.2 Henry’s Law .......................................................................................................... 34623.4 Chemical Potential in Non-ideal Solutions ......................................................................... 34623.4.1 Solvent ................................................................................................................... 34623.4.2 Solute ..................................................................................................................... 34723.5 Thermodynamic Treatment of Chemical Equilibrium ....................................................... 34723.6 Chemical Equilibrium in Ideal Gas Mixtures: Statistical Mechanics ................................ 34823.7 Pressure-Dependent Equilibrium Constant of Ideal Gas Mixtures .................................... 34923.8 Protein-Ligand Binding ...................................................................................................... 35023.8.1 Standard Methods for Calculating ΔA0 .................................................................35223.8.2 Calculating ΔA0 by HSMD-TI .............................................................................. 35423.8.3 HSMD-TI Applied to the FKBP12-FK506 Complex: Equilibration ................... 35623.8.4 The Internal and External Entropies..................................................................... 35723.8.5 TI Results for FKBP12-FK506 ............................................................................. 36023.8.6 ΔA0 Results for FKBP12-FK506 .......................................................................... 36023.9 Summary ............................................................................................................................. 362References ...................................................................................................................................... 362Appendix ............................................................................................................................................... 367Index ...................................................................................................................................................... 369