Issues with TeX sub-formula formatting

Question

As generally acknowledged TeX offers superb formatting capabilities for math in an automated fashion. Nevertheless there are some areas with clear deficiencies that one either has to accept or manually improve on a case by case basis.

One of the biggies in that respect is TeX's handling of sub-formulas (i.e., material inside a brace group, e.g., and index, but also any group on top-level). TeX sets such sub-formulas always in natural width even if the whole formula is subject to severe stretching or shrinking otherwise. As an example, consider the following (not very sensible) example:

\documentclass{article}

\newlength\x

\begin{document}

\newcommand\formula{$ a+\mathbf{b+c}+d+e+f = \sum_{i=1}^{n-x-y-z} x_{i+j}$}

\settowidth\x{\formula}

\hbox to \x{\formula}    % natural width

\addtolength\x{-20pt}

\hbox to \x{\formula}   % now shrink it a lot

\addtolength\x{100pt}

\hbox to \x{\formula}   % now stretch a lot in the opposite direction

\end{document}

The formula here has a number of obvious sub-formulas in the subscripts and superscript, but for illustration I also added a sub-formula via \mathbf. Of course the usage of \mathbf in this way is wrong!!! (it should not be applied to several symbols), but I'm sure you would find this in documents. In any case just {b+c} would have had the same effect here. Now what do you think happens if we run this?

This:

As one can see the naturally "boxed" sub-formulas are very wrong the moment the rest of the formula is subject to stretching or shrinking.

Until recently no TeX engine successor addressed this issue. With LuaTeX opening up a lot of the internals of TeX I had some hope that this would be different. However, upon studying the manual my conclusion is that this area (sor far it least) has not been addressed (or considered).

As far as I can see the only way something could be done in LuaTeX about this issue would be to use the mlist_to_hlist callback. However, this would really mean replacing the full math typesetting algorithm, which of course could be a way to solve the problem but ... but what is needed is not that (as 99% of this algorithm is next to perfect) but to add support to not simply box sub-formulas at their natural width.

So long text ... here is the short question:

• Is this analysis correct, or did I overlook something?
• And in case anybody knows: are there plans to look into the sub-formula issue and provide support for it eventually? (it is not listed on the "to-dos" for math)

In case somebody wonders that I talk about "this sub-formula issue" as if it is something like a known thing ... it is, it was already raised way back in E-TeX: guidelines to future TeX extensions which at the time a some thorough (if not say heated) discussions.

Answer 1

Old question, but I stumbled upon it, and I think there might now be a way to get this to work, using the luametatex engine*. This is done by unpacking boxes. (This is not applied for square roots, fractions, or in sub/superscripts.) We first take look at the picture.

Here,

1. is just set with \hbox{word word $a + {\bf b + c} + \left(d + e\right) + f + \sqrt{g + h} + \frac{a + b}{c + d} = \sum_{i=1}^{n-x-y-z} x_{i+j}$ word word}.
2. is set with \hbox spread 4cm {word word $a + {\bf b + c} + \left(d + e\right) + f + \sqrt{g + h} + \frac{a + b}{c + d} = \sum_{i=1}^{n-x-y-z} x_{i+j}$ word word}. Note that the spreading is not working in the bold part. But it is working inside the fenced part. As mentioned above, it is not done for the square root, the fraction or the index.
3. is set with \hbox spread 4cm {word word $a + \mathatom unpack \mathordinarycode {\bf b + c} + \left(d + e\right) + f + \sqrt{g + h} + \frac{a + b}{c + d} = \sum_{i=1}^{n-x-y-z} x_{i+j}$ word word}. So, the bold part is also spread. It is unpacked, and then packed as an ord.
4. is set with \hbox spread 4cm{word word $a + \mathbf{b + c} + \left(d + e\right) + f + \sqrt{g + h} + \frac{a + b}{c + d} = \sum_{i=1}^{n-x-y-z} x_{i+j}$ word word}. This shows that the unpacking and repacking is done in \mathbf.
5. is set with \hbox spread -2cm{word word $a + \mathbf{b + c} + \left(d + e\right) + f + \sqrt{g + h} + \frac{a + b}{c + d} = \sum_{i=1}^{n-x-y-z} x_{i+j}$ word word}. We just see that stuff can also be packed together.
6. is set with \hbox spread 4cm{word word $a + \mathatom unpack leftclass \mathordinarycode rightclass \mathbinarycode {\bf c + d} + \left(d + e\right) + f + \sqrt{g + h} + \frac{a + b}{c + d} = \sum_{i=1}^{n-x-y-z} x_{i+j}$ word word}. This is not what we want here, but it shows that we can, when we re-pack, say that the box should behave differently to the left and to the right. Might be useful in other cases. Here, behaving like a binary to the right, makes the plus sign into an ord, and the spacing there gets wrong.

One nice feature of the unpacking and repacking, is that one can have multi line formulas, which breaks inside fences. Example (adapted from this old question):

This formula is set with

\defineformula[XYZ][
  split=text,
  textalign=slanted,
  spaceinbetween=.5\lineheight,
  strut=yes,
  distance=1em,
]
\startXYZformula[margin=3em]
s_{i}^{G}
=
\min 
\left{
    \sqrt{
        \min\left[(x_{i}^{G} - x)^2, (w_{T} - x_{i}^{G} - x)^2\right] + y
        }
    ,\breakhere
    \sqrt{
        \min\left[(x_{i}^{G} - x)^2, (w_{T} - x_{i}^{G} - x)^2\right] 
        + 
        \min\left[(y_{i}^{G} - y)^2, (h_{T} - y_{i}^{G} - y)^2\right]
        }
\right}
\stopXYZformula

*In fact, the \mathbf{b + c} also works in context with luatex. But the unpacking mechanism is new in luametatex. I do not have mkii installed, so I don't know if it also works in pdftex with context.

Answer 2

Looks like the answer is as follows:

• This issue (which is present already in the original program of TeX) is not being solved or addressed by any TeX successor including LuaTeX.

It is true that LuaTeX offers to replace all of the math processing by proviate code but this is more along the lines "demolish the house and build a new one" and not really warranted. After all TeX's algorithm are really great in most respects. A pity, but then perhaps understandable as it would require to take the processing logic of the math formatting appart and reorganize it to improve only a small fraction of it.

Answer 3

There is a way to do this with unicode-math since bold letters in Unicode math are different letters of the same font rather than letters of a different font:

\documentclass{article}

\usepackage{unicode-math}

\setmathfont{XITS Math}

\newlength\x

\begin{document}

\newcommand\formula{$ a+++d+e+f = \sum_{i=1}^{n-x-y-z} x_{i+j}$}

\settowidth\x{\formula}

\hbox to \x{\formula}    % natural width

\addtolength\x{-20pt}

\hbox to \x{\formula}   % now shrink it a lot

\addtolength\x{100pt}

\hbox to \x{\formula}   % now stretch a lot in the opposite direction

\end{document}

I couldn't find a way to make this work with \mathbf, but it should be doable: Essentially \mathbf could be made into something like \begingroup \Umathcode ```a=``` … \endgroup.