This is actually described in the manual for pgf/TikZ. See section 92.1 Changing display styles (for v3.0.1a, dated August 29, 2015).
You're almost there in fact, just change the number formatting option for the yticklabels to /pgf/number format/.cd,sci,sci e. If you want a capital e, use sci E instead.
\documentclass[border=4mm]{standalone}
\usepackage{pgfplots}
\begin{document}
\begin{tikzpicture}
\begin{axis}[
scaled ticks=false,
y tick label style={/pgf/number format/.cd,sci,sci e},
ymin=1, ymax=5000,
]
\addplot {pow(x, x)};
\end{axis}
\end{tikzpicture}
\end{document}

Old answer
Is this what you were after?
\documentclass[border=4mm]{standalone}
\usepackage{pgfplots}
\begin{document}
\begin{tikzpicture}
\begin{semilogyaxis}[
log number format basis/.code 2 args={e${\pgfmathprintnumber[showpos=true]{#2}}$}
]
\addplot {exp(x)};
\end{semilogyaxis}
\end{tikzpicture}
\end{document}

xin the notation (unless you mean it as a placeholder for an actual number), but it is related, nonetheless: http://tex.stackexchange.com/questions/110388/siunitx-and-engineering-e-notation – Steven B. Segletes Apr 27 '16 at 20:08siunitx. – Torbjørn T. Apr 27 '16 at 20:25