Nonstandard (non?)nesting of over- and under-braces

Question

\overbrace{ABCDEFG = HIJKL} = PQRSTUV
ABCDEFG = \underbrace{HIJKL = PQRSTUV}

LaTeX can handle either of the lines above. How can I get both the overbrace and the underbrace, when they overlap like that?

Answer 1

The package oubraces should solve your problem!

Check here: https://www.ctan.org/tex-archive/macros/latex/contrib/oubraces?lang=en

Once you load the package, the command works like this:

\overunderbraces{upper braces}{main formula}{lower braces}

which needs you to split your equation into multiple columns using &, and the braces command is

\br{n}{label} 

where n is the number of columns to span and the label braces can be left blank.

So for your example, you'd have

\[
\overunderbraces{&\br{2}{}&}{&ABCDEFG= & HIJKL &=PQRSTUV}{&&\br{2}{}}
\]

Answer 2

You have to use some trickery. The following uses How to put a big bracket under different parts of a equation so that I could write e.g. a text specific to that part in the equation? as a guide:

\documentclass{article}

\usepackage{mathtools}

\begin{document}

\[
  \renewcommand{\arraystretch}{2.5}% Just for this example
  \begin{array}{c}
    \overbrace{ABCDEFG = HIJKL} = PQRSTUV \\
    ABCDEFG = \underbrace{HIJKL = PQRSTUV} \\
    \mathrlap{\overbrace{\phantom{ABCDEFG = HIJKL}}}
    ABCDEFG =
    \underbrace{HIJKL = PQRSTUV}
  \end{array}
\]

\end{document}