the document class amsbook will use the short version only for running heads; the long version goes into the toc. what must be done is to suppress the "normal" entry from the toc, and explicitly replace it with the desired version.
this is covered in an entry in the ams author faq. briefly, the instruction there is this: in the preamble,
\DeclareRobustCommand{\SkipTocEntry}[4]{}
replacing the 4 by 5 if hyperref is used. then, enter your \chapter information as follows:
\addtocontents{toc}{\SkipTocEntry}
\chapter{Heading that should be replaced}
\addtocontents{toc}{\protect\contentsline{chapter}%
{\protect\tocchapter{Chapter}{\thechapter}%
{Substitute toc text}}%
{\thepage}}
note that the \protects are necessary. this also works for other sectioning commands besides \chapter.
the \SkipTocEntry command cannot be entered before an \include command in a driver file; it must be in the same file as the heading being omitted.
update:
while the method shown here does still work, an improved (and recommended)
method has been provided; see this answer:
\section*[toc]{normal section title} not working for me.
\chapter[short version]{longer version that describes things in a verbose way}will precisely show "short version" in the ToC. – Gonzalo Medina Aug 03 '11 at 00:48