Spacefactor & Unicode Curly Quotes

Question

Unicode curly quotes seem not to have the correct space factor. Consider the following minimal example (not using french spacing):

\documentclass[12pt]{minimal}
\usepackage[utf8]{inputenc} % Or fontspec with XeLaTeX/LuaLaTeX
\begin{document}

Compare:

``Is the spacing correct?\@'' he asked. % Yes, as it should be

``Is the spacing correct?'' he asked.   % No, as it should be

“Is the spacing correct?\@” he asked.   % Yes…

“Is the spacing correct?” he asked.     % Yes -- but it shouldn’t be
\end{document}

Is there a way—either in PDFLaTeX or XeLaTeX/LuaLaTeX—to give the Unicode curly quotes the same space factor (sfcode 0) as the straight quotes have?

Answer 1

The closing double quotes are at position "22 in the OT1 encoding and position "11 in the T1 encoding.

So

\sfcode"22=0

or

\sfcode"11=0

will do, according to the chosen encoding. If you want a magic for setting the space factor code independently of the encoding (OT1 or T1), then

\AtBeginDocument{
  \sfcode\csname\encodingdefault\string\textquotedblright\endcsname=0
}

is what you're looking for. Here's the test document:

\documentclass[12pt]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc} % Or fontspec with XeLaTeX/LuaLaTeX

\AtBeginDocument{
  \sfcode\csname\encodingdefault\string\textquotedblright\endcsname=0
}

\begin{document}

Compare:

``Is the spacing correct?\@'' he asked.

``Is the spacing correct?'' he asked.

“Is the spacing correct?\@” he asked.

“Is the spacing correct?” he asked.
\end{document}

With or without the fontenc line the output is the same.

For XeLaTeX/LuaLaTeX, setting \sfcode`”=0 is correct.

---

Let's examine the case of pdflatex. The definitions found in ts1enc.dfu and t1enc.dfu, which are loaded when the OT1 or T1 encoding are loaded is

\DeclareUnicodeCharacter{201D}{\textquotedblright}

but it's \textquotedblright that changes meaning depending on the encoding. For OT1 it's "typeset the character in position "22", which is equivalent to \"; but for T1 it's the character in position "11. This may be declared by

\sfcode`\^^Q=0

but it's inconvenient having to give different definitions. Fortunately, the cryptic instruction

\sfcode\csname\encodingdefault\string\textquotedblright\endcsname=0

accesses precisely \OT1\textquotedblright or \T1\textquotedblright, which is the symbolic name of the right slot, depending on the default encoding.

Answer 2

Yes there is, and my edit that found the command \sfcode was halfway to the answer. Under XeLaTeX/LuaLaTeX, the curly quote character ” can have its space-factor code set just like any other character. Under PDFLaTex, U+201D ” redirects to the straight double-quote ", which can have its sfcode set.

\documentclass[12pt]{minimal}

\usepackage{ifluatex, ifxetex}
\newif\ifmoderntex
\ifluatex   \moderntextrue  \fi
\ifxetex    \moderntextrue  \fi

\ifmoderntex
    \usepackage{fontspec}
    \setmainfont[Ligatures=TeX]{Constantia}
    \sfcode`\”0
\else
    \usepackage[utf8]{inputenc}
    \sfcode`\"0
\fi

\begin{document}

Compare:

``Is the spacing correct?\@'' he asked. % Yes, correctly

``Is the spacing correct?'' he asked.   % No, correctly

“Is the spacing correct?\@” he asked.   % Yes, correctly

“Is the spacing correct?” he asked.     % No, correctly

\end{document}