28

I would like to hear what from the LuaTeX documentation would help me predict the differences in hyphenation behaviour of LuaTex.

I answered a question about hyphenating already-hyphenated words (like “already-hyphenated”, or “often-hungry”), and this came up. Consider the following input file (uncomment the first three lines if you prefer to run with LaTeX instead of plain TeX):

% \documentclass{article}
% \newcommand{\bye}{\end{document}}
% \begin{document}

\hsize = 0pt
\overfullrule = 0pt
\parindent = 0pt
\parskip=3\baselineskip


a often\penalty10000-\penalty10000\hskip0pt hungry


b often\penalty10000-\hskip0pt hungry


c often-\penalty10000\hskip0pt hungry


d often-\penalty0\hskip0pt hungry


\bye

(I know it's good practice to end \hskip 0pt with \relax in case the next word starts with plus or something, but I've left it out here to avoid clutter.) The output is as below — the left side is what results with TeX / pdfTeX / XeTeX, and the right side with LuaTeX. There is a difference in the “a” and “b” cases:

output

Are these two differences (in the “a” and “b” cases) documented somewhere for LuaTeX? What explains the difference?

  • In the “a” case, with penalty 10000 both before and after the hyphen, and an empty glue after them, at first glance the behaviour of LuaTeX seems to make sense to me: we specified \penalty10000 (aka \nobreak) after the hyphen, so LuaTeX does not break there. But TeX does break there. Is this a bug in LuaTeX, or something that LuaTeX considers an improvement?

  • In the “b” case, all engines other than LuaTeX somehow introduce a blank line… is that correct?

  • The “c” and “d” cases behave similarly: there's a blank line in the “d” case. Note that the “d” case is actually what is recommended in the (usually excellent) TeX FAQ, but it simply doesn't work: it doesn't hyphenate the first word, and it leaves a blank line.

  • The desired output is that of the “a” case on the left, or the “b” case on the right. Is there some sequence of boxes, glue, and penalties that will work for both non-LuaTeX and for LuaTeX?

minorChaos
  • 260
  • 6
ShreevatsaR
  • 45,428
  • 10
  • 117
  • 149
  • 2
    Just noticed something that may be part of an answer: if I change \hsize = 0pt to even \hsize = 0.00001pt (or directly \hsize = 1sp), then there are no blank lines anymore, and the discrepancy in the “b” case goes away. (The one in the “a” case is still there.) So the answer to the final question may be that we can use the “b” case, except with literally zero hsize. – ShreevatsaR Mar 01 '18 at 20:15
  • The case \hsize=0pt is very artificial. For details see “4.4 Applying hyphenation” of the LuaTeX manual. – Henri Menke Mar 01 '18 at 20:20
  • 1
    asking why there is a discrepancy is rather asking the wrong question, hyphenation (and in particular determination of word fragments to consider for hyphenation is almost completely different in luatex, so better to ask why some of the cases are the same. – David Carlisle Mar 01 '18 at 20:30
  • 4
    @DavidCarlisle Good point; I'm not looking for an answer of “because the implementation is completely different” :-), but meant something like “what from the LuaTeX documentation would help me predict this behaviour…” — maybe instead of “what explains…” I should write “where is it explained why…” but really I don't care so much about “where”, just an understandable explanation. (Also even though it's a good point that word fragments may be different, I think the treatment of the basic box-glue-penalty model itself seems different here, so it's more surprising to me.) – ShreevatsaR Mar 01 '18 at 20:37
  • The luatex output is different now (in case c and d). – Ulrike Fischer Jan 31 '19 at 22:32

0 Answers0