It can be immediately see from this syntax scheme borrowed from the TeXbook, page 269, that essentially all syntactic constructs requiring a number will break if digits have category code 11 and not 12.
⟨number⟩ → ⟨optional signs⟩⟨unsigned number⟩
⟨optional signs⟩ → ⟨optional spaces⟩
| ⟨optional signs⟩⟨plus or minus⟩⟨optional spaces⟩
⟨unsigned number⟩ → ⟨normal integer⟩ | ⟨coerced integer⟩
⟨normal integer⟩ → ⟨internal integer⟩
| ⟨integer constant⟩⟨one optional space⟩
| '12⟨octal constant⟩⟨one optional space⟩
| "12⟨hexadecimal constant⟩⟨one optional space⟩
| `12⟨character token⟩⟨one optional space⟩
⟨integer constant⟩ → ⟨digit⟩ | ⟨digit⟩⟨integer constant⟩
⟨octal constant⟩ → ⟨octal digit⟩ | ⟨octal digit⟩⟨octal constant⟩
⟨hexadecimal constant⟩ → ⟨hex digit⟩ | ⟨hex digit⟩⟨hexadecimal constant⟩
⟨octal digit⟩ → 012 | 112 | 212 | 312 |
412 | 512 | 612 | 712
⟨digit⟩ → ⟨octal digit⟩ | 812 | 912
⟨hex digit⟩ → ⟨digit⟩ | A11 | B11 | C11 |
D11 | E11 | F11
| A12 | B12 | C12 |
D12 | E12 | F12
⟨one optional space⟩ → ⟨space token⟩ | ⟨empty⟩
⟨coerced integer⟩ → ⟨internal dimen⟩ | ⟨internal glue⟩
Also definitions like \def\cs#1{...} would break, because the syntax prescribes that the digit following # has category code 12, if we want a parameter.
You'd have no problem when printing numbers, but simple things such as \\[1ex] would break horribly.
You might always \detokenize your syntactic numbers, because TeX will perform expansion when in need of a ⟨number⟩:
\hspace{\detokenize{1}em}
would work in the document. Is this worth the pain?
\def\foo{} \def\fooA{} \def\fooB{} \def\fooC{} \def\fooL{} \def\fooX{}, etc. – Manuel Jun 30 '16 at 09:34