New solution based on my own answer to Answers at the end of the exam document, this correctly handles when the questions have parts, subparts and subsubparts unlike my earlier answer (below).
Both labels and answer contents are stored in separate seq variables and then reprinted together iterating through both sequences at the same time. The counters are just printed and only repressed if the counter is set to zero, to format the solution number more elegantly would require conditionals based on whether they are zero.
\documentclass[a4paper]{exam}
\usepackage{xparse}
\printanswers
\ExplSyntaxOn
% Counters are not reset on \end{parts} so I add code to reset them
\tl_put_right:Nn \endparts { \setcounter{partno}{0} }
\tl_put_right:Nn \endsubparts { \setcounter{subpart}{0} }
\tl_put_right:Nn \endsubsubparts { \setcounter{subsubpart}{0} }
\seq_new:N \l_exam_endprint_seq
\seq_new:N \l_exam_endprint_labels_seq
\tl_new:N \l__exam_endprint_temp_tl
\NewDocumentCommand \WriteAnswer { +m } {
\seq_gput_right:Nx \l_exam_endprint_labels_seq {\arabic{question}\alph{partno}\roman{subpart}\greeknum{subsubpart}} \seq_gput_right:Nn \l_exam_endprint_seq { #1 }
}
\NewDocumentCommand \EndPrintAnswers { } {
\seq_map_inline:Nn \l_exam_endprint_seq {
\seq_pop_left:NN \l_exam_endprint_labels_seq \l__exam_endprint_temp_tl
\renewcommand{\solutiontitle}{\noindent\textbf{Solution~\l__exam_endprint_temp_tl}:\enspace}
\begin{solution} ##1 \end{solution}
}
}
\ExplSyntaxOff
\begin{document}
\begin{questions}
\addpoints \question
\begin{parts}
\part This is the first part of question one.
\WriteAnswer{This is the solution to part one of question one.}
\part This is the second part of question one.
\begin{subparts}
\subpart This is the first subpart of the second part of question one.
\WriteAnswer{This is the solution to the first subpart of part two of question one.}
\subpart
\begin{subsubparts}
\subsubpart This is the first subsubpart of the second subpart of the second part of question one.
\WriteAnswer{This is the solution to the first subsubpart of the second subpart of the second part of question one.}
\end{subsubparts}
\end{subparts}
\part This is the third part of question one
\WriteAnswer{This is the solution to part three of question one.}
\end{parts}
\addpoints \question This is the second question.
\WriteAnswer{This is the solution to question two.}
\end{questions}
\EndPrintAnswers
\end{document}

This was my original answer which simply numbers each solution through a counter which is incremented each time a solution is printed. If parts are used then this will break numbering (i.e. Solution 1 could be for question 1a, solution 2 for question 1b, and solution 3 for question 2).
Using the MWE from the solution to Answers at the end of the exam document
Adding a new counter for each solution and stepping it with the \solutiontitle macro from the exam class as follows
\newcounter{solution}
\stepcounter{solution}
\renewcommand{\solutiontitle}{\noindent\textbf{Solution \arabic{solution}:}\enspace\stepcounter{solution}}
yields

From the full code
\documentclass[a4paper]{exam}
\usepackage[utf8]{inputenc}
\printanswers
\newcounter{solution}
\stepcounter{solution}
\renewcommand{\solutiontitle}{\noindent\textbf{Solution \arabic{solution}:}\enspace\stepcounter{solution}}
\usepackage{url}
\usepackage{endnotes}
\def\enotesize{\normalsize}
\def\makeenmark{\relax}
\def\notesname{Answers}
\def\answer#1{\endnotetext{\vspace*{-3.5ex}\begin{solution}#1\end{solution}\unskip}}
\def\theanswers{\theendnotes \medskip}
\begin{document}
\begin{questions}
\addpoints \question This is the first question
\answer{This is the solution to question one.}
\addpoints \question This the second question
\answer{This is the solution to question two.}
\end{questions}
\theanswers
\end{document}
\endnotetextI'm not sure how easily this current answer can be hacked to deal with parts. I'll try and have a think about it. – Dai Bowen Oct 20 '16 at 12:56