How can I make LaTeX to recognize spaces in my macro (catcode 10)?

Question

I've been messing with TeX primitives, particularly catcodes, to handle diverse LaTeX situations (and to extend my wings).

I created a token gobbler with the help of https://tex.stackexchange.com/a/33536/13552

Why can't I see any 10s in my list of category codes?

Would it be safe to test for \@empty and if true, use \g@addto@macro{\tempa}{\fbox{ }}%

\documentclass{article}
\begin{document}
\makeatletter
\def\scanfunction#1{#1}% Don't know why this is here
\let\tempa\@empty
\def\scan@letters#1#2{% https://tex.stackexchange.com/a/33536/13552
   \bgroup\catcode`\ \active \let\ \fbox{ }\egroup%Try changing space to visible box
   \def\getcategory##1{\the\catcode`##1}% ` changes char into ASCI number required for \catcode
   \def\currcatcode{\getcategory{#1}}% put everything into unified token for \if
   \currcatcode, % spurious space intentional % first character missing
   \if 10\currcatcode%
     \fbox{#2}% if catcode 10 fed, print it as a box
   \fi
   \g@addto@macro{\tempa}{#1\hskip 0pt plus 1sp minus 1sp}%
   \ifx#2\@empty%\@empty is what the end of the line will look like ^^M?
   \else
     \expandafter\scan@letters % recall yourself using #2 until this conditional becomes true
   \fi
#2}
\def\scan#1{%
  %\catcode"0020 11 % Tried changing space to catcode 11
  %\the\catcode"0020 % Test to ensure change happened
  \scan@letters #1@empty
}
\makeatother
\scan{  mac::exception  ==  }
% \scan{     } % crashes with "Improper alphabetic constant."
\begin{itemize}
  \item \tempa
\end{itemize}
\end{document}

Of course, I expect the following from \scan{ mac::exception == }

10, 10, 11, 11, 11, 12, 12, 11, 11, 11, 11, 11, 11, 11, 11, 11, 10, 10, 12, 12, 10, 10,

My Favorites for Reference

Obviously, it is possible to see 10.

\the\catcode`\

result: 13 (mistakenly expected 10)

\the\catcode`\ %

result: 10

\the\catcode"0020

result: 10

• Test whether a token is a control sequence

---

24.06.2016 Update with Practical Solution

This is perhaps useful to know, so I'm putting it here. Using the context and David Carlisle's answer to understand \afterassignment so that I could test for category 10 (space), I modified my code to add glue after every non-space character such that the LaTeX line-breaking mechanism would trigger when

• the total width of all character boxes = \textwidth

Sample

\documentclass{article}
\usepackage{fontspec}
\newfontfamily\macmonofont{Inconsolatazi4-Regular.otf}
%\makeatletter % or \catcode"0040=11
\def\macmono#1{\xscan#1\relax}% calls xscan which looks ahead one token, #1
\def\xscan{\afterassignment\xxscan\let\token= }% assign single token to \token and call \xxscan
\def\xxscan{%
\macmonofont% apply mono font
\ifx\token\relax\normalfont\else%test for end-of-line or end of group and switch back to normal font
  \ifcat\token\space%
    \token% token is catcode 10
    \spaceskip=.5em% remove glue from space for fixed-width space
    \xspaceskip=.5em% remove glue from space for fixed-width space
  \else%
    \token\hskip 0pt plus 1sp minus 1sp % add glue to any non-catcode 10 (space)
   \fi
\expandafter\xscan
\fi}
%\makeatother % or \catcode"0040=12
\parindent=0pt % remove firstpar autoindent
\obeylines% insert \par after each end-of-line (^^M)
\begin{document}
\begin{itemize}
\item Some random error created by C++: \macmono{mac::IOException [File: sharecpp/trunk/PROJECTS/SpecialOps/B/Build/Implementation/Library/GenerateDocumentation.cpp, Line: 999]
    Found documentation overview begin marker more than once in DocumentationManual!
mac::IOException [File: sharecpp/trunk/PROJECTS/SpecialOps/B/Build/Implementation/Library/DocumentationThread.cpp, Line: 934]
    Processing of DocumentationManual failed while generating DocumentationOverview!} and some trailing test to make sure font changing works.  \end{itemize}
\end{document}

Output

Answer 1

An unbraced catcode 10 character is never taken as an undelimited argument so that you can go

\def\foo#1#2#3{...}
\foo a b c

and have a, b and c (rather than the spaces between them) be the three arguments.

The usual solution is to use \futurelet (or as below \afterassignment and \let) but it depends what you want to do.

Your catcode changes such as

\catcode`\ \active

have no effect as the spaces have already been tokenized in the argument to your \scan macro.

10, 10,

Also you should not expect consecutive 10 even if you fixed the scanner as any number of consecutive spaces in the input file are tokenized to a single catcode 10 token

\documentclass{article}

\def\scan#1{\xscan#1\relax}
\def\xscan{\afterassignment\xxscan\let\tmp= }
\def\xxscan{%
\ifx\tmp\relax\else
\ifcat\tmp\space10 \else
\ifcat\tmp a11 \else
\ifcat\tmp 112 \else%...
\fi\fi\fi
\expandafter\xscan
\fi}



\begin{document}

\scan{  mac::exception  ==  }

\end{document}