You're using the wrong font.
\documentclass{article}
\DeclareFontEncoding{LS1}{}{}
\DeclareFontSubstitution{LS1}{stix}{m}{n}
\DeclareSymbolFont{symbolsstix}{LS1}{stixscr}{m}{n}
\SetSymbolFont{symbolsstix}{bold}{LS1}{stixscr}{b}{n}
\DeclareMathSymbol{\mathvisiblespace}{0}{symbolsstix}{"B6}
\begin{document}
\(\{0,1,\mathvisiblespace\}\)
\end{document}

If you don't need bold math, a simpler strategy will do as well.
\documentclass{article}
\usepackage{amsmath}
\DeclareFontEncoding{LS1}{}{}
\DeclareFontSubstitution{LS1}{stix}{m}{n}
\DeclareRobustCommand{\mathvisiblespace}{%
\mathord{\text{\usefont{LS1}{stixscr}{m}{n}\symbol{"B6}}}%
}
\begin{document}
\(\{0,1,\mathvisiblespace\}\)
\end{document}
For a command that obeys \boldmath, see https://tex.stackexchange.com/a/236903/4427
\documentclass{article}
\usepackage{amsmath,pdftexcmds}
\DeclareFontEncoding{LS1}{}{}
\DeclareFontSubstitution{LS1}{stix}{m}{n}
\DeclareRobustCommand{\mathvisiblespace}{%
\mathord{\text{\usefont{LS1}{stixscr}{\normalorbold}{n}\symbol{"B6}}}%
}
\makeatletter
\newcommand{\normalorbold}{%
\ifnum\pdf@strcmp{\math@version}{bold}=\z@\bfdefault\else\mddefault\fi
}
\makeatother
\begin{document}
\(\{0,1,\mathvisiblespace\}\)
{\boldmath\(\{0,1,\mathvisiblespace\}\)}
\end{document}

\boldmathis in force. – egreg Dec 03 '16 at 17:48