If you wish to stay with stringstrings, the \isnextbyte function is the one you seek. Because \theresult is an \edef'ed string, you don't want to use \ifx for the comparison, but rather \if.
\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{fp,stringstrings}
\newcommand*{\mytest}[1]{%
\isnextbyte[q]{M}{#1}%
\if T\theresult\textit{M has been recognized}\else%
\textbf{M has NOT been recognized. firstChar=\substring{#1}{1}{1}}\fi
}
\begin{document}\thispagestyle{empty}
Test with Matching: \mytest{Matching}\par
Test with M: \mytest{M}\par
Test with No: \mytest{No}\par
Test with N: \mytest{N}\par
\isnextbyte{M}{Matching}
\isnextbyte{M}{M}
\isnextbyte{M}{No}
\isnextbyte{N}{M}
\end{document}

However, if you prefer your logic to extract the first character and do the comparison, as given in your MWE, this is how it would be done. Rather than trying to assign the substring to a \def, just run the \substring in quiet [q] mode. This will not print out the substring, which has nonetheless been placed via \edefinto \thestring. At that point, the proper test becomes \if M\thestring...\else...\fi.
\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{fp,stringstrings}
\newcommand*{\mytest}[1]{%
\substring[q]{#1}{1}{1}%
\if M\thestring\textit{M has been recognized}\else\textbf{M has NOT been recognized, firstChar='\thestring'}\fi
}
\begin{document}\thispagestyle{empty}
Test with Matching: \mytest{Matching}\par
Test with M: \mytest{M}\par
Test with No: \mytest{No}\par
Test with N: \mytest{N}\par
\end{document}
The OP properly points out in the comments that both of these tests can be fooled if the first character of the string is, for example, a \$. For that possibility, one could invoke some syntax from the ifthen package to take care of the test:
\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{fp,stringstrings,ifthen}
\newcommand*{\mytest}[1]{%
\substring[q]{#1}{1}{1}%
\ifthenelse{\equal{M}{\thestring}}%
{\textit{M has been recognized}}%
{\textbf{M has NOT been recognized, firstChar='\thestring'}}%
}
\begin{document}\thispagestyle{empty}
Test with Matching: \mytest{Matching}\par
Test with M: \mytest{M}\par
Test with No: \mytest{No}\par
Test with N: \mytest{N}\par
Test with \$N: \mytest{\$N}\par
\end{document}
The idea of the package, in placing manipulated strings into \thestring via \edef is so that successive manipulations can be set up on a given text. In this MWE, for example, the phrase "Four score and seven years ago" has the following manipulations sequentially performed: 1) "seven" is changed to "7", 2) all letter-cases are reversed; 3) the first word is rotated to the end of the string; 4) the first character is gobbled.
\documentclass{article}
\usepackage{stringstrings}
\begin{document}
\edef\phrase{Four score and seven years ago}
\convertword[q]{\phrase}{seven}{7}
\changecase[q]{\thestring}
\rotateword[q]{\thestring\ }
\gobblechar[q]{\thestring}
\thestring
\end{document}

\@emptyis required please? – lalebarde May 22 '14 at 13:11\Lettrine{D'}Artagnancapability without the complexity of the second solution that provides it, but I fail. I will post there my failing solution. – lalebarde May 22 '14 at 14:32