The combination of \addlegendimage and \addlegendentry allows to add custom entries. The purpose of the first is to add graphic options and the purpose of the second one is to add the description text.
In your case, the small legend image would probably be nothing but the text "A" or "B" and the description text would... well, describe these groups.
One could define a style which doesn't use a filled area a small legend image, but simply a node with text:
\documentclass{standalone}
\usepackage{pgfplots}
\pgfplotsset{compat=1.10}
\pgfplotsset{
legend image with text/.style={
legend image code/.code={%
\node[anchor=center] at (0.3cm,0cm) {#1};
}
},
}
\begin{document}
\begin{tikzpicture}
\begin{semilogyaxis}[
domain=0:4,
]
\addplot {x}; \addlegendentry{$x$}
\addplot {x^2}; \addlegendentry{$x^2$}
\addplot {x^3}; \addlegendentry{$x^3$}
\addlegendimage{legend image with text=A}
\addlegendentry{$= 42$}
\addlegendimage{legend image with text=B}
\addlegendentry{$\approx 43$}
\addplot {x^(-1)}; \addlegendentry{$x^{-1}$}
\addplot {x^(-2)}; \addlegendentry{$x^{-2}$}
\addplot {x^(-3)}; \addlegendentry{$x^{-3}$}
\end{semilogyaxis}
\end{tikzpicture}
\end{document}
