Strange behaviour with \detokenize and underscore

Question

I just learned about \detokenize, so this question is likely a naive one.

Nonetheless, based on what I thought that I understood about \detokenize, I would have expected \detokenize{_} to render as _.

However, when I compile the following MWE, \detokenize{_} renders as something that seems to be ~~U+05C4~~ U+02D9 (see @Qrrbrbirlbel's comment).

\documentclass{article}

\begin{document}

A\detokenize{_}B

\end{document}

enter image description here

However, when I add \usepackage[T1]{fontenc} to the preamble, everything works as expected.

\documentclass{article}

\usepackage[T1]{fontenc}

\begin{document}

A\detokenize{_}B

\end{document}

enter image description here

What's going on here? My .tex file is UTF-8 encoded, and I'm compiling on a Mac (OS X 10.9.1) with TeXShop (v. 3.26) if any of that is relevant here.

There’s no _ in the OT1 encoding. It is at position 0x5F (ASCII) in T1. At this position is ˙ in OT1. — Qrrbrbirlbel, Dec 20 '13 at 05:37
Normally one doesn't load a fontencoding manually with XeTeX. Using package fontspec is the standard, which loads en encoding called eu1, giving you a wider support. Frequently loaded packages: Differences between pdfLaTeX and XeLaTeX — Johannes_B, Dec 20 '13 at 08:27
\string_ is easier than \detokenize{_}, isn't it? But \_ is even easier (and works also with OT1). — egreg, Dec 20 '13 at 10:39
@egreg For me, \string_ still produces ˙ unless \usepackage[T1]{fontenc} is loaded. And \_ isn't an option in this case. (This question arose as a result of one of the answers to my question about DOIs and BibTeX.) — Adam Liter, Dec 20 '13 at 16:54
@Adam Of course: \string_ is perfectly equivalent to \detokenize{_} (but more efficient). — egreg, Dec 20 '13 at 16:55

score 5 · Accepted Answer · edited Jun 29 '19 at 16:02

5

As Qrrbrbirlbel points out in the comments, there is no _ in OT1 encoding, which is the default font encoding for LaTeX.

Yet, in T1 encoding, _ is at 5F:

enter image description here

Which is where ˙ is at in OT1 encoding:

enter image description here

Thus, without specifying the output font encoding to be T1 by declaring \usepackage[T1]{fontenc} in the preamble, the output will be ˙ instead of _.

edited Jun 29 '19 at 16:02

apriori

953

answered Jan 04 '14 at 02:23

Adam Liter

12,567

The code point is “5F”, not “5xF”. The ‘x’ is a placeholder in the column head. – Nicola Gigante Dec 06 '17 at 00:44

Strange behaviour with \detokenize and underscore

1 Answers1

Linked