You can use \DeclareMathSizes; here's what LaTeX has by default:
\DeclareMathSizes{5}{5}{5}{5}
\DeclareMathSizes{6}{6}{5}{5}
\DeclareMathSizes{7}{7}{5}{5}
\DeclareMathSizes{8}{8}{6}{5}
\DeclareMathSizes{9}{9}{6}{5}
\DeclareMathSizes{\@xpt}{\@xpt}{7}{5}
\DeclareMathSizes{\@xipt}{\@xipt}{8}{6}
\DeclareMathSizes{\@xiipt}{\@xiipt}{8}{6}
\DeclareMathSizes{\@xivpt}{\@xivpt}{\@xpt}{7}
\DeclareMathSizes{\@xviipt}{\@xviipt}{\@xiipt}{\@xpt}
\DeclareMathSizes{\@xxpt}{\@xxpt}{\@xivpt}{\@xiipt}
\DeclareMathSizes{\@xxvpt}{\@xxvpt}{\@xxpt}{\@xviipt}
What does this mean? Let's look at the line
\DeclareMathSizes{\@xpt}{\@xpt}{7}{5}
When the current size is 10pt (\@xpt is a synonym), LaTeX uses 10pt size for normal math, 7pt size for first level sub/superscripts, 5pt for the second level.
If you want to get smaller subscripts, you could say
\makeatletter
\DeclareMathSizes{\@xpt}{\@xpt}{6}{4.5}
\makeatother
Example
\RequirePackage{fix-cm}
\documentclass{article}
\makeatletter
\DeclareMathSizes{\@xpt}{\@xpt}{6}{4.5}
\makeatother
\begin{document}
$a_{b_c}$
\end{document}
Usage of fix-cm is necessary to get font size less than 5pt; alternatively you can load lmodern.
Here is the output; top the “reduced size”, bottom the normal output.


You'll need to redeclare math sizes for all font sizes you need.