This principle fails in the most startling way in second order phase transitions. This is a particularly clean example, because Landau predicted the critical exponents of second-order phase transitions using only the principle that the thermodynamic functions are analytic.
His argument is as follows: given a magnet going through the Curie point, where it loses its magnetization smoothly, the equilibrium magnetization should be the solution of some thermodynamic equation with the derivative some thermodynamic potential is set to zero.
$F(T,m)=0$
At temperatures lower than T_c, the magnetization is nonzero, and at temperatures higher than Tc, the magnetization is 0, and it goes to 0 in a continuous way. How does it go to zero?
Note that the magnetization m and -m are related by rotational symmetry. Shifting T_c to 0 by translating $f(t,m)= F(T_c - t,m)$, you get a new thermodynamic function, which has the property that f has only the trivial solution m=0 for negative t, and has two small nontrivial solutions in m for positive t.
Because m=0 is a solution at t=0, the function $f$ has no constant term in a Taylor expansion. By the symmetry of $m\rightarrow -m$, only even powers of m contribute to its Taylor series.
$f(t,m) = At + Bm^2 + Ct^2 + Dt^3 + E t m^2 ...$
Assuming that $f(t,m)$ is generic, A and B are not exactly zero. So for small enough t, for temperatures close enough to the critical point, you get that
$m \propto \sqrt{t}$
Further, this scaling only fails if one of the coefficients is zero. If A=0,
$m \propto |t|$
But m is then nonzero on both sides of the transition. If B=0, you get
$m \propto t^{1\over 4}$
and m is zero
or if A,B,C are zero, in which case you get
$m \propto |t|^{3/4}$
And each of these cases requires fine tuning of parameters. So Landau predicted that the critical behavior of the magnetization will be as the square root of the temperature at the critical point, and that this behavior will be universal, it won't depend on the system, just on the existence of the phase transition. The Ising model should have the same critical exponent as the physical magnet, a square root dependence of the magnetization on the temperature, and the liquid gas transition will also have a bend in the curve of the density vs. temperature at the critical pressure which goes as the square root.
The exponent turned out to be universal, it was equal for the gas and liquid, and for the Ising model. But it wasn't 1/2, but more like .308 in three dimensions, and .125 in two dimensions, It only turned into Landau's 1/2 in 4 dimensions or higher. This means that Landau's argument fails, and that the thermodynamic function is conspiring to be non-analytic at exactly the place where Landau was expanding. Understanding why it is non-analytic exactly at the phase transition led to the modern renormalization theory.
In mathematics, Rene Thom proposed that a version of Landau's argument is a complete theory of the types of allowed phase transitions in nature. He called the phase transitions "catastrophes", because they showed a sudden change in behavior, and he predicted, based on catastrophe theory all sorts of scaling laws for natural transitions. This was the most ambitious attempt to exploit the observation that naturally occuring functions are nice. This fails for the same reason as Landau's argument: functions describing changes in the critical behavior of interesting systems at a transition point are rarely analytic at this point.