arrows are broken with babel and lualatex

Question

I am starting to lose my mind because of this weird bug that I don't understand how to solve. I do my homework with latex in Hebrew and I use babel to set it up. I have a custom package and class to define my style and commands. After fighting for hours to get it to compile without errors I found out that the \Longrightarrow is broken:

I tried to create a minimal example to reproduce the issue:

% !TeX spellcheck = he_HE
\documentclass[a4paper, 12pt]{extarticle}
\usepackage{amsmath}
\usepackage{enumitem}
\usepackage{witharrows}
\usepackage[bidi=basic, provide=*]{babel}
\babelprovide[alph=letters, Alph=letters]{hebrew}
\babelprovide{english}
\babelfont[hebrew]{rm}[Renderer=Harfbuzz, ItalicFont=*-Regular, ItalicFeatures={FakeSlant=0.3}]{Assistant}
\babelfont[hebrew]{sf}[Renderer=Harfbuzz, ItalicFont=*-Regular, ItalicFeatures={FakeSlant=0.3}]{Assistant}
\babelfont[hebrew]{tt}[Renderer=Harfbuzz, ItalicFont=*-Regular, ItalicFeatures={FakeSlant=0.3}]{Assistant}
\AddToHook{begindocument/end}{\selectlanguage{hebrew}}
\begin{document}
דוגמה \textit{לחישוב}:
\begin{DispWithArrows}
A &= (a + b)^2 \Arrow{נפתח את הסוגריים בביטוי} \
&= a^2 + 2ab + b^2
\end{DispWithArrows}
\underline{דוגמה} \textbf{נוספת}:
\begin{align}
A &= (a + b)^2\
&= a^2 + 2ab + b^2
\end{align}
\begin{otherlanguage}{english}
Some English example.
\end{otherlanguage}
\begin{enumerate}[leftmargin=, label=\alph.]
    \item
בדיקה ראשונה.
    \item
בדיקה שנייה.
\end{enumerate}
[ AB = I \Longrightarrow B = A^{-1} ]
\end{document}

but when compiling it, it worked as expected:

I went over the code trying to disable single features but couldn't find the source of this problem. The code that handles babel is identical to the code in the example above. Here are stripped versions of my files:

my-style.sty:

\NeedsTeXFormat{LaTeX2e}
\ProvidesPackage{my-style}[2023/03/06 My personal style and custom commands]
\RequirePackage[table]{xcolor}
\RequirePackage{tikz}
\RequirePackage{tkz-base, tkz-euclide}
\RequirePackage{pgfplots}
\RequirePackage{graphicx}
\RequirePackage[most]{tcolorbox}
\RequirePackage{wrapfig}
\RequirePackage{titlesec}
\RequirePackage{xifthen}
\RequirePackage{xparse}
\RequirePackage{geometry}
\RequirePackage{amsmath}
\RequirePackage{amsfonts}
\RequirePackage{amssymb}
\RequirePackage{amsthm}
\RequirePackage{mathrsfs}
\RequirePackage{enumitem}
\RequirePackage{moreenum}
\RequirePackage{bm}
\RequirePackage{upgreek}
\RequirePackage[euler]{textgreek}
\RequirePackage{import}
\RequirePackage{esint}
\RequirePackage{mathtools}
\RequirePackage[thicklines]{cancel}
\RequirePackage[per-mode=fraction, exponent-product=\cdot]{siunitx}
\RequirePackage[italicdiff]{physics}
\RequirePackage{nicefrac}
\RequirePackage{parskip}
\RequirePackage{setspace}
\RequirePackage{dsfont}
\RequirePackage{etoolbox}
\RequirePackage[bottom]{footmisc}
\RequirePackage{multicol}
\RequirePackage{ulem}
\RequirePackage{pgfkeys}
\RequirePackage{nicematrix}
\RequirePackage{witharrows}
% Hebrew option
\newboolean{hebrew}
\setboolean{hebrew}{false}
\DeclareOption{hebrew}{\setboolean{hebrew}{true}}
\DeclareOption*{\PackageWarning{my-style}{Unknown '\CurrentOption'}}
\ProcessOptions\relax
% Font and lanuguage settings
\RequirePackage[bidi=basic, provide=*]{babel}
\babelprovide{english}
\babelprovide[alph=letters, Alph=letters]{hebrew}
\babelfont[hebrew]{rm}[Renderer=Harfbuzz, ItalicFont=*-Regular, ItalicFeatures={FakeSlant=0.3}]{Assistant}
\babelfont[hebrew]{sf}[Renderer=Harfbuzz, ItalicFont=*-Regular, ItalicFeatures={FakeSlant=0.3}]{Assistant}
\babelfont[hebrew]{tt}[Renderer=Harfbuzz, ItalicFont=*-Regular, ItalicFeatures={FakeSlant=0.3}]{Assistant}
\ifthenelse{\boolean{hebrew}}{
    \AddToHook{begindocument/end}{\selectlanguage{hebrew}}
}{
    \AddToHook{begindocument/end}{\selectlanguage{english}}
}
% witharrows options
\WithArrowsOptions{displaystyle}
\tikzset{
    WithArrows/arrow/.append style = {
        font= \small
    }
}
\RequirePackage{hyperref}
\RequirePackage{fancyhdr}
\hypersetup{
    colorlinks,
    linkcolor=blue
}
\NewDocumentCommand{\OverSet}{ O{0cm} m m }{
    \overset{\raise #1 \hbox{(#2)}}{#3}
}
\NewDocumentCommand{\UnderSet}{ O{0cm} m m }{
    \underset{\lower #1 \hbox{(#2)}}{#3}
}
\NewDocumentCommand{\claptext}{ m }{\mathclap{\text{#1}}}
\NewDocumentCommand{\Transpose}{ s m }{
    {\IfBooleanTF{#1}{\pqty{#2}}{#2}}^{\mathrm{t}}
}
\NewDocumentCommand{\InverseTranspose}{ s m }{
    {\IfBooleanTF{#1}{\pqty{#2}}{#2}}^{-\mathrm{t}}
}
\NewDocumentCommand{\HermitianTranspose}{ s m }{
    {\IfBooleanTF{#1}{\pqty{#2}}{#2}}^{\dagger}
}
\NewDocumentCommand{\InverseHermitianTranspose}{ s m }{
    {\IfBooleanTF{#1}{\pqty{#2}}{#2}}^{-\dagger}
}
\NewDocumentCommand{\Conjugate}{ s m }{
    \IfBooleanTF{#1}{\pqty{#2}}{#2}^{\ast}
}
% Physics commands
\NewDocumentCommand{\IdentityOperator}{}{\hat{I}}
\NewDocumentCommand{\Change}{ m }{\mathop{\Delta#1}}

homework.cls:

\NeedsTeXFormat{LaTeX2e}
\ProvidesClass{homework}[2022/11/01 My custom class for homework]
\LoadClass[a4paper, 12pt]{extarticle}
\RequirePackage{my-style}
\DeclareOption{hebrew}{\PassOptionsToPackage{hebrew}{my-style}}
\DeclareOption*{\PassOptionsToClass{\CurrentOption}{extarticle}}
\ProcessOptions\relax
\geometry{left=2cm, right=2cm, top=2cm, bottom=2cm}
% HomeWork options
\NewExpandableDocumentCommand{\Course}{}{}
\NewExpandableDocumentCommand{\StudentId}{}{#########}
\NewExpandableDocumentCommand{\StudentEmail}{}{email@example.com}
\NewExpandableDocumentCommand{\HWDate}{}{\today}
\ifthenelse{\boolean{hebrew}}{
    \NewExpandableDocumentCommand{\Homework}{}{תרגיל בית}
    \NewExpandableDocumentCommand{\StudentName}{}{שם הסטודנט}
}
{
    \NewExpandableDocumentCommand{\Homework}{}{Homework assignment}
    \NewExpandableDocumentCommand{\StudentName}{}{}
}
\NewDocumentCommand{\SetCourse}{ m }{\RenewDocumentCommand{\Course}{}{#1}}
\NewDocumentCommand{\SetHomework}{ m }{\RenewDocumentCommand{\Homework}{}{#1}}
\NewDocumentCommand{\SetStudentName}{ m }{\RenewDocumentCommand{\StudentName}{}{#1}}
\NewDocumentCommand{\SetStudentId}{ m }{\RenewDocumentCommand{\StudentId}{}{#1}}
\NewDocumentCommand{\SetStudentEmail}{ m }{\RenewDocumentCommand{\StudentEmail}{}{#1}}
\NewDocumentCommand{\SetHWDate}{ m }{\RenewDocumentCommand{\HWDate}{}{#1}}
% Styling customizations
\ifthenelse{\boolean{hebrew}}{
    % Customize title
    \titlelabel{חלק \thetitle}
}{}
\NewDocumentCommand{\Appendix}{ s }{
    \IfBooleanF{#1}{\newpage}
    \appendix
    \ifthenelse{\boolean{hebrew}}{
        \renewcommand{\thesection}{\arabic{section}}
    % Customize title
    \titlelabel{נספח \thetitle\quad}
}{}

}
% Homework functions
\newcounter{ProblemNum}
\newcounter{SubProblemNum}[ProblemNum]
\NewExpandableDocumentCommand{\TheProblemNum}{}{\arabic{ProblemNum}}
\ifthenelse{\boolean{hebrew}}{
    \NewExpandableDocumentCommand{\TheSubProblemNum}{}{\alph{SubProblemNum}}
}{
    \NewExpandableDocumentCommand{\TheSubProblemNum}{}{\arabic{SubProblemNum}}
}
\NewDocumentCommand{\Section}{ m }{
    \phantomsection
    \addcontentsline{toc}{section}{#1}
    \section*{#1}
}
\NewDocumentCommand{\Problem}{ s o }{
    \IfBooleanF{#1}{\newpage}
    \stepcounter{ProblemNum}
\phantomsection
\ifthenelse{\boolean{hebrew}}{
    \addcontentsline{toc}{subsection}{שאלה \TheProblemNum \IfNoValueF{#2}{\; #2}}
    \subsection*{שאלה \TheProblemNum \IfNoValueF{#2}{\; #2}}
}{
    \addcontentsline{toc}{subsection}{Question \TheProblemNum \IfNoValueF{#2}{\; #2}}
    \subsection*{Question \TheProblemNum \IfNoValueF{#2}{\; #2}}
}

}
\NewDocumentCommand{\Solution}{}{
    \phantomsection
    \ifthenelse{\boolean{hebrew}}{
        \addcontentsline{toc}{subsubsection}{פתרון}
        \subsubsection{פתרון}
    }{
        \addcontentsline{toc}{subsubsection}{Solution}
        \subsubsection{Solution}
    }
}
\NewDocumentCommand{\Part}{ o }{
    \IfNoValueTF{#1}{
        \stepcounter{SubProblemNum}
        \phantomsection
        \ifthenelse{\boolean{hebrew}}{
            \addcontentsline{toc}{subsubsection}{סעיף \texorpdfstring{\TheSubProblemNum}{\arabic{SubProblemNum}}}
            \subsubsection{סעיף \TheSubProblemNum}
        }{
            \addcontentsline{toc}{subsubsection}{Part \texorpdfstring{\TheSubProblemNum}{\arabic{SubProblemNum}}}
            \subsubsection{Part \TheSubProblemNum}
        }
    }{
        \phantomsection
        \ifthenelse{\boolean{hebrew}}{
            \addcontentsline{toc}{subsubsection}{סעיף #1}
            \subsubsection{סעיף #1}
        }{
            \addcontentsline{toc}{subsubsection}{Part #1}
            \subsubsection{Part #1}
        }
    }
}
% Theorems and proofs
\ifthenelse{\boolean{hebrew}}{
    \newtheorem{lemma}{למה}[ProblemNum]
}{
    \newtheorem{lemma}{Lemma}[ProblemNum]
}
% Lists
\newlist{QuestionParts}{enumerate}{2}
\setlist[QuestionParts, 1]{align=left, leftmargin=, label=\alph.}
\setlist[QuestionParts, 2]{align=right, label=\arabic*.}
\newlist{SubParts}{enumerate}{1}
\setlist[SubParts, 1]{align=left, labelsep=0pt, leftmargin=0pt, labelwidth=, label=\uline{חלק \arabic}}
\NewDocumentCommand{\nitem}{ o }{
    \IfNoValueTF{#1}{
        \item\mbox{}\
    }{
        \item[#1]\mbox{}\
    }
}
% Hyperref and fancyhdr setup
\hypersetup{
    pdfauthor={\StudentName},
    pdftitle={\Homework},
}
\pdfstringdefDisableCommands{\let;\empty}
\setlength{\headheight}{15pt}
\ifthenelse{\boolean{hebrew}}{
    \lhead{שאלה \arabic{ProblemNum}}
}{
    \lhead{Question \arabic{ProblemNum}}
}
\chead{\Course}
\rhead{\Homework}
\lfoot{}
\cfoot{\thepage}
\rfoot{}
\pagestyle{fancy}

main.tex:

% !TeX spellcheck = he_HE
\documentclass[hebrew]{homework}
\SetCourse{פיזיקה קוונטית 1}
\SetHomework{תרגיל בית 3}
\begin{document}
\tableofcontents
\import{./}{q1.tex}
\end{document}

q1.tex:

% !TeX spellcheck = he_HE
\Problem[צמוד הרמיטי]
\[
\HermitianTranspose*{\hat{O}^{-1}}\HermitianTranspose{\hat{O}} = \HermitianTranspose*{\hat{O}\hat{O}^{-1}} = \HermitianTranspose{\IdentityOperator} = \IdentityOperator
\Longrightarrow \tcbhighmath{\HermitianTranspose*{\hat{O}^{-1}} = \pqty{\HermitianTranspose{\hat{O}}}^{-1}}
\]

Compiling main.tex with lualatex produces the first picture. As I said before, I compared these files with the example above for hours, but couldn't find the source of this annoying bug. If someone could figure out what is happening here and how how to solve it, I'd really appreciate it.

P.S: I am using the font Assistant.

Answer 1

First of all, the glyph produced by \Longrigtharrow is not actually one glyph, but a composition of ⟹ (\Rightarrow) and = to lengthen its legs. The problem is that the two glyphs comes from different fonts, and their composition is not deigned to be used without the default fonts.

Even with your "unsuccessful" minimal example, if you will look close enough you will see that actually the the legs of and the head does not overlap perfectly:

The reason for that is that your minimal example use 12pt font size, and while the font lmr12 exists (the default font used when fontspec is loaded, which babel loads when the command \babelfont is used), the font cmsy12 (where the head of the arrow is located) isn't, so LaTeX scales cmsy10 but the result is not perfect.

Regarding your main document, in some cases fontspec will define the mathrm font as the documents main font, this is what causing the legs of your arrow to look disturbed, thay are actually the = from the Assistant font.

To suppress that, either pass the no-math option to fontspec, you can do that by adding the line \PassOptionsToPackage{no-math}{fontspec} before the first \babelfont, or you can rename your document class option, or you can laod the unicode-math package before babel. Probably the latter is preferable, as you are using 12pt as the default font size.

The reason you don't not get a similar result with your minimal example is that by passing the hebrew option to the documentclass deceleration, you actually passing it to all the packages as well.

Since you essentially have \RequirePackage[hebrew,bidi=basic, provide=*]{babel} then babel is treating hebrew as the main language, and sets the main font to be the hebrew font.

You can produce similar result with much shorter document:

\documentclass[hebrew]{article}
\usepackage{amsmath}
\usepackage[bidi=basic, provide=*]{babel}
\babelfont[hebrew]{rm}{Assistant}
\begin{document}
[\Longrightarrow]
\end{document}

Answer 2

Updated answer for adding workaround.

Aside from passing no-math option to fontspec (via \PassOptionsToPackage), or to make sure all packages loading amsmath are loaded after fontspec (which is a pain, for example tcolorbox with option "most" does it), the simplest to my mind is to

• add \makeatletter\Umathcharnumdef\std@equal\Umathcodenum`\=\relax\makeatother immediately after \begin{document}.

• or add

    \makeatletter
    \AtBeginDocument{%
      \Umathcharnumdef\std@equal\Umathcodenum`\=\relax
    }%
    \makeatother
  

  anywhere in the preamble, if you are using LaTeX at least as recent as October 2020, or right before \begin{document} for older LaTeX.

You don't need to worry about \std@minus (which if you go reading the rest of this answer you will see mentioned) because fontspec (as far as I checked) does nothing special with the -.

---

This is only a complement to very good answer by @UdiFogiel, as my comments were too long to fit as comments.

Compare the following situations:

1. A pdflatex document:

\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage{dejavu}
\begin{document}
a $\Longrightarrow$
\thispagestyle{empty}
\showoutput
\end{document}
% Local variables:
% TeX-engine: default
% End:

Then one has to considerably zoomed to confirm visually the slight mismatch

and the log contains (don't worry about ) it stands for a slot where is actually the arrow head)

....\OT1/DejaVuSerif-TLF/m/n/12 a
....\glue 3.81479 plus 1.9068 minus 1.27199
....\mathon
....\OT1/cmr/m/n/12 =
....\hbox(0.0+0.0)x-1.99997
.....\kern -1.99997
....\OMS/cmsy/m/n/12 )
....\mathoff

2. Now we try using luatex and fontspec something similar:

\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage{fontspec}
\setmainfont{DejaVu Serif}
\begin{document}
a $\Longrightarrow$
\thispagestyle{empty}
\showoutput
\end{document}
% Local variables:
% TeX-engine: luatex
% End:

and it produces this where mismatch is obvious without zoom

The log contains:

....\TU/DejaVuSerif(0)/m/n/12 a
....\glue(\spaceskip) 3.81445 plus 1.90723 minus 1.27148
....\mathon
....\TU/DejaVuSerif(0)/m/n/12 =
....\hbox(0.0+0.0)x-1.99997, direction TLT
.....\kern-1.99997 (italic)
....\OMS/cmsy/m/n/12 )
....\mathoff

3. Same as 2. but without amsmath:

\documentclass[12pt]{article}
%\usepackage{amsmath}
\usepackage{fontspec}
\setmainfont{DejaVu Serif}
\begin{document}
a $\Longrightarrow$
\thispagestyle{empty}
\showoutput
\end{document}
% Local variables:
% TeX-engine: luatex
% End:

produces, and here I have to zoom a lot:

and the log contains

....\TU/DejaVuSerif(0)/m/n/12 a
....\glue(\spaceskip) 3.81445 plus 1.90723 minus 1.27148
....\mathon
....\hbox(4.33081+0.0)x9.1388, direction TLT
.....\OT1/cmr/m/n/12 =
....\hbox(0.0+0.0)x-1.99997, direction TLT
.....\kern-1.99997 (italic)
....\OMS/cmsy/m/n/12 )
....\mathoff

Note that in these log extracts the shown "font names" are actually the LaTeX "new font selection scheme (or system?) NFSS" assigned macro names.

So we see that the problem is not alone one of fontspec, but is a combination of fontspec and amsmath and a Unicode engine.

Now the amsmath definition of \Longrightarrow is

\renewcommand{\Longrightarrow}{%
  \DOTSB\protect\Relbar\protect\joinrel\Rightarrow}

and the definition of \Relbar is

\ams@DeclareRobustCommand\Relbar{\mathrel\std@equal}

and the definition of \std@equal is the slightly complex

\@ifundefined{Umathcode}
  {%
    \mathchardef\std@minus\mathcode`\-\relax
    \mathchardef\std@equal\mathcode`\=\relax
  }
  {%
   \Umathcharnumdef\std@minus\Umathcodenum`\-\relax
   \Umathcharnumdef\std@equal\Umathcodenum`\=\relax
  }
\@ifundefined{Umathcode}
  {%
    \AtBeginDocument{%
      \mathchardef\std@minus\mathcode`\-\relax
      \mathchardef\std@equal\mathcode`\=\relax
      }%
  }
  {%
    \AtBeginDocument{%
      \Umathcharnumdef\std@minus\Umathcodenum`\-\relax
      \Umathcharnumdef\std@equal\Umathcodenum`\=\relax
    }%
  }

which says that \std@equal is an alias to the = in math mode and amsmath insists upon it as it redoes the definition at begin document.

Without amsmath we have similar things leading to \Relbar except that the latter definition is to be a robust macro which expands to \mathrel{=}. So basically it looks the same as the above stuff.

So why the difference? In the example with amsmath let's do

\makeatletter
a $\std@equal = \Longrightarrow$

then the output is . Hence amsmath went out of its way to make \std@equal and = the same but in the end they are not. Without amsmath the arrow will use \Relbar which will use \mathrel{=} and there is no mismatch.

But here what happens is that \std@equal got frozen to the mathcode of the equal sign, which maps to using a given slot in the "operators" TeX font "family" for math mode. With fontspec this "operators" is modified to match the roman font of the document. At the same time fontspec does modify the = in math mode to not use the = from the text font. But it does not modify the amsmath \std@equal which will thus end up using the document "roman" font.

The modification by fontspec of "operators" in file fontspec-luatex.sty:

  \DeclareSymbolFont{operators}\g_fontspec_encoding_tl\g__fontspec_mathrm_tl\mddefault\shapedefault

This modification by fontspec of =:

  \DeclareSymbolFont{legacymaths}{OT1}{cmr}{m}{n}
...
     \DeclareMathSymbol{=}{\mathrel}{legacymaths}{61}

Then we can check the mathcode of = as well as the math versions

> \symlegacymaths=\char"4.
l.8 \show\symlegacymaths
? 
> 73400381.
l.9 \showthe\mathcode`=
? 
> \temp=\Umathchar"3"04"00003D.
l.11 \show\temp
? 
> \std@equal=\Umathchar"3"00"00003D.
l.12 \show\std@equal
? 

I used for this

\makeatletter
\show\symlegacymaths
\showthe\mathcode`=
\Umathcharnumdef\temp\Umathcodenum`=\relax
\show\temp
\show\std@equal

to disentangle the "Umathcode" of = via an alias \temp which allows to confirm that = is now taken from "legacymaths" (number 4) whereas \std@equal still refers to the glyph from "operators" (number 0). But "operators" now maps to document "roman" font (which in the case of OP using \babelfont will actually be the font for Hebrew).

Conclusion: one of fontspec or amsmath has to take into account the other's existence. Probably this is an already known bug reported on the fontspec issue tracker but I found only one issue mentioning amsmath and it is closed and unrelated.