When I read the question, I first thought of an external file, which is somehow a standard solution. But, with the macro design, you proposed, this is not necessary. A possible (and possibly the most simple but not the fastest -- see the answer of @egreg) implementation with a small test could like like:
\documentclass{book}
\usepackage{amsmath,amsthm}
\newtheorem{problem}{Problem}[chapter]
\makeatletter
\def\solutionsname{Solutions}
\let\@sol=\@empty
\newcommand{\printsolutions}[1][chapter]{\@nameuse{#1}{\solutionsname}\@sol}
\newcommand{\addtosolutions}[1]{%
\protected@xdef\@sol{%
\@sol {\noindent\bfseries\theproblem}\space#1\par\vskip 1em plus 4pt minus 4pt\par
}%
}
\makeatother
\begin{document}
\chapter{Groups}
\begin{problem}
Let $(G,\circ)$ be a finite group and $g\in G$. Prove that the order of $g$ divides the cardinality of $G$.
\addtosolutions{%
Because $G$ is finite there is $\operatorname{ord}_G(g)=:m<\infty$. Hence, there exists a cyclic subgroup $U_g=\lbrace e,g,g^2,\ldots,g^{m-1}\rbrace$. Obviously $\lvert U_g\rvert=m=\operatorname{ord}_G(g)$ holds. Therefore the claim follows from Lagrange theorem.
}
\end{problem}
\begin{problem}
Let $(G,\circ)$ be a group. Show that $m\circ G=G\quad\forall m\in G$.
\addtosolutions{%
The inclusion $m\circ G\subseteq G$ is evident due to the closure of $\circ$ on $G$. Let $m,n\in G$. By calculating
\[
n=e\circ n=(m\circ m^{-1})\circ n=m\circ(m^{-1}\circ n)\in m\in G,
\]
we obtain $G\subseteq m\circ G$.
}
\end{problem}
\appendix
\printsolutions
\end{document}
The synopsis is exactly as described in the OP: (1) \addtosolutions gathers the solution inside the problem environment and (2) the solutions will be printed with the \printsolutions macro at the point you want them to be. (\printsolutions can be called with an optional argument that sets the sectioning level in terms of the sectioning command; default: 'chapter'.)
Note that the \appendix call in my example can be dropped. I just find it well suited for this situation.
answerspackage. Solution text goes with the exercise and is written to an external file. Close that file and then\includeit just before\end{document}. Renumbering takes care of itself. – Ethan Bolker Oct 27 '13 at 13:40exsheetspackage really useful for producing your document. Check it out. – jub0bs Oct 27 '13 at 13:42recordingmacros fromtcolorboxcan be found in How to defer content to a later part of the document? – Ignasi Apr 12 '16 at 15:27