Commands defined in that way are fragile, so you need \protect\testMe in the moving arguments, for instance the caption text.
Let's see why. The argument to \caption needs to be massaged a few times, in particular it gets written to the .aux file for possible inclusion in the list of figures (or tables).
When TeX does a \write operation, it expands commands “all the way”, unless instructed not to do so (which is the purpose of \protect). During this expansion in \write operations, \@ifstar or \@ifnextchar (which \@ifstar is defined upon) break because they need to assign a value to a scratch macro to work and this is not possible during expansion.
Thus you have three possibilities:
use \protect\testMe when in a moving argument
use \DeclareRobustCommand for defining \testMe (same syntax as \newcommand)
use xparse.
The last possibility needs a comment. If you do
\usepackage{xparse}
\NewDocumentCommand{\testMe}{s}
{test \IfBooleanTF{#1}{1}{2}}
you get basically the same as your \testMe, but automatically protected when found in moving arguments.
\protect\testMein the moving arguments, for instance the caption text. – egreg Jun 12 '13 at 08:12\protectbefore every such use? – gablin Jun 12 '13 at 08:17