Based on @Mike's answer here, add ScriptLevel -> 1 to Style.

Mathematica uses the ScriptLevel value to determine when it should decrease the font size in equations. The script level increases with each nested fraction/subscript/superscript. I think that Mathematica switches to smaller fonts at a ScriptLevel of 1. In inline formulas (the ones you type with Ctrl-9 in text cells) the starting ScriptLevel is 1, thus fractions become small immediately. Otherwise it is usually 0, so the outermost fractions are large (but things written within them become small).
However, the ScriptLevel is already 1 in Column by default, as opposed to Row. Don't use Row[{..., "\n", ...}] if you mean a Column. You may want to specify alignment using the second argument of Column.
PolarPlot[{LegendreP[5, Cos[theta]],
11/2 If[theta <= Pi, 1, -1] LegendreP[5, Cos[theta]]^2 Sin[
theta]}, {theta, 0, 2 Pi},
PlotStyle -> {Directive[Thickness[0.0025], Blue],
Directive[Thickness[0.0025], Red]},
PlotLabel ->
Column[{Style[
"\!\(\*FormBox[\(\*SubscriptBox[\(P\), \(5\)](cos\\\ \
\[Theta])\),
TraditionalForm]\)", Blue, 16],
Style["\!\(\*FormBox[\(\*FractionBox[\(11\), \(2\)] \
\*SuperscriptBox[\(\*SubscriptBox[\(P\), \(5\)](cos\\\ \[Theta])\), \
\(2\)] sin\\\ \[Theta]\),
TraditionalForm]\)", Red, 16]}]]

All this ugly SuperscriptBox stuff that you see above will format nicely in the front end if you paste it back:

I typed it like this:
- type
"x"
- select the
x and press Control-Shift-T to convert to TraditionalForm
- position the cursor next to the x, then type the formula as you normally would
Personally I prefer to type formulae directly into a single string rather than try to mix strings with HoldForm in a complicated way.
11/2in aStyle. e.g.Style[Style[11/2, 10] HoldForm[ LegendreP[5, " cos \[Theta] "]^2 "sin \[Theta]"], Red, 16]– Edmund Mar 15 '16 at 14:44