Using a primed sum operator in display and text mode

Question

I would like to define a primed sum command \sump such that \sump_{x \in X} produces:

• a prime at the top right corner and the x \in X below the sigma in display mode, and
• a prime in the top right corner and the x \in X at the bottom right of the sigma in text mode.

I've tried combinations of \mathchoice and \sideset to get this right, but have not succeeded so far.

If it makes any difference, I'd like to do this not only with sums, but also to define primed \prod, \min and \max commands.

Of course, one could define \sump to take the x \in X as an argument, and then place it correctly using \mathchoice and \sideset, but for writing code that is more readable and includes normal (non-primed) sums I'd prefer not to do this. Help?

If one follows the most accepted answer in the suggested How to create my own math operator with limits?, then when using \sump_{x \in X} in text mode there is an unnecessary space before the x \in X lower limit; the prime pushes the right boundary of the operator out, so that the lower limit doesn't start right next to the Sigma.

Answer 1

You can test for _ after \sump with the help of xparse:

\documentclass{article}
\usepackage{amsmath,mathtools}
\usepackage{xparse}

\makeatletter
\NewDocumentCommand{\sump}{e{_}}
 {%
  \DOTSB
  \mathop{\IfNoValueTF{#1}{\sump@{}}{\sump@{#1}}}%
  \nolimits
 }
\newcommand{\sump@}[1]{\mathpalette\sump@@{#1}}
\newcommand{\sump@@}[2]{%
  \ifx#1\displaystyle
    {\sump@display{#2}}%
  \else
    \sum@\nolimits'_{#2}%
  \fi
}
\newcommand{\sump@display}[1]{%
  \sbox\z@{$\m@th\displaystyle\sum@\nolimits'$}%
  \sbox\tw@{$\m@th\displaystyle\sum@\limits_{#1}$}%
  \sbox\@tempboxa{$\m@th\displaystyle'$}
  \mathop{\sum@\nolimits' \kern-\wd\@tempboxa}\limits_{#1}%
  \ifdim\wd\z@>\wd\tw@
    \kern\dimexpr\wd\z@-\wd\tw@\relax
  \fi
}
\makeatother

\begin{document}

\begin{align}
&\sum_{n} a_n &&\textstyle \sum_{n\ge0} a_n \\
&\sump_{n} a_n &&\textstyle \sump_{n\ge0} a_n \\
&\sum_{n\ge0} a_n &&\textstyle \sum_{n\ge0} a_n \\
&\sump_{n\ge0} a_n &&\textstyle \sump_{n\ge0} a_n \\
&\sum_{1\le n\le 32} a_n &&\textstyle \sum_{1\le n\le 32} a_n \\
&\sump_{1\le n\le 32} a_n &&\textstyle \sump_{1\le n\le 32} a_n
\end{align}

\end{document}

In the “nondisplay” case just typeset \sum'_{#1}, in the display case I measure the prime, so to kern and produce the subscript centered on the summation symbol. Then I measure the summation with the prime and the summation with the subscript, in order to decide whether some kerning is necessary, which it is if the size of “primed summation” exceeds the size of “summation with subscript”.

Answer 2

As already said in the comments, this problem is the main subject of Exercise 18.44 of The TeXbook. Barbara Beeton has already suggested a variation of one of the two solutions that Knuth presents in Appendix A, although her code should be modified to read, more or less,

\DeclareRobustCommand*{\sump}{%
    \mathop{{\sum}^{\mathrlap{\prime}}}%
}

Here’s a complete, compilable example, that also draws attention to a possible flaw of this solution:

% My standard header for TeX.SX answers:
\documentclass[a4paper]{article} % To avoid confusion, let us explicitly 
                                 % declare the paper format.

\usepackage[T1]{fontenc}         % Not always necessary, but recommended.
% End of standard header.  What follows pertains to the problem at hand.

\usepackage{mathtools}

\newcommand*{\sump}{} % check that the name can be used
\DeclareRobustCommand*{\sump}{%
    \mathop{{\sum}^{\mathrlap{\prime}}}%
}



\begin{document}

First in-line:
\( \sump_{i\in I}a_{i} \ne \sum_{i\in I}a_{i} \).
Then in display:
\[
    \sump_{i\in I}a_{i} \ne \sum_{i\in I}a_{i}
    \mbox{.}
\]

You can also say, for instance,
\( \sump\limits_{i\in I}a_{i} \ne \sum\limits_{i\in I}a_{i} \)
and
\[
    \sump\nolimits_{i\in I}a_{i} \ne \sum\nolimits_{i\in I}a_{i}
    \mbox{,}
\]
respectively.

There's a caveat, though: because the prime is not taken into account
when computing the width of the \verb|\sump| symbol, it may bump into
adjacent characters:
\[
    \sump \biggl[\frac{a+b}{c+d}-\frac{x+y}{x-y}\biggr]
        \ne \sum \biggl[\frac{a+b}{c+d}-\frac{x+y}{x-y}\biggr]
\]
The clash is more problematic in in-line math:
\( \sump[a+b] \ne \sum[a+b] \).
However, in practice this is not going to be a problem if you stick to
using the \verb|\sump| symbol only with a (non-empty) subscript.

\end{document}

However, if we have to load the mathtools package, which, in turn, requires amsmath, I think we should also support the [no]sumlimits option of the latter, as well as its \dots feature:

% My standard header for TeX.SX answers:
\documentclass[a4paper]{article} % To avoid confusion, let us explicitly 
                                 % declare the paper format.

\usepackage[T1]{fontenc}         % Not always necessary, but recommended.
% End of standard header.  What follows pertains to the problem at hand.

% Try decommenting the following line:
% \usepackage[nosumlimits]{amsmath}
\usepackage{mathtools}

\makeatletter

% Cannot directly use "\DeclareRobustCommand" with "\DOTSB":
\@ifdefinable\sump@{\DeclareRobustCommand*\sump@{%
    \mathop{{\sum}^{\mathrlap{\prime}}}%
}}
\newcommand*\sump{%
    \DOTSB\sump@\slimits@ % ...or move "\slimits@" into "\sump@"
}

\makeatother



\begin{document}

First in-line:
\( \sump_{i\in I}a_{i} \ne \sum_{i\in I}a_{i} \).
Then in display:
\begin{equation}
    \sump_{i\in I}a_{i} \ne \sum_{i\in I}a_{i}
    \mbox{.}
    \label{eq:displaylimits}
\end{equation}

You can also say, for instance,
\( \sump\limits_{i\in I}a_{i} \ne \sum\limits_{i\in I}a_{i} \)
and
\[
    \sump\nolimits_{i\in I}a_{i} \ne \sum\nolimits_{i\in I}a_{i}
    \mbox{,}
\]
respectively.  And see what happens in the ``default'' case (that is,
in~\eqref{eq:displaylimits}) if you load the \textsf{amsmath} package
with the \texttt{nosumlimits} option.

Finally, note that
\( \sump_{i_{1}}\dots\sump_{i_{p}} x_{i_{1}}\otimes\dots\otimes x_{i_{p}} \)
works exactly in the same way as
\( \sum_{i_{1}}\dots\sum_{i_{p}} x_{i_{1}}\otimes\dots\otimes x_{i_{p}} \)
(as one would expect).  Let's repeat it in display:
\[
    \sump_{i_{1}}\dots\sump_{i_{p}} x_{i_{1}}\otimes\dots\otimes x_{i_{p}}
    \ne
    \sum_{i_{1}}\dots\sum_{i_{p}} x_{i_{1}}\otimes\dots\otimes x_{i_{p}}
\]
(this way the formula is more readable!).

\typeout{A test for robustness: \sump}
\typeout{Compare the above with \sum}

\end{document}

Note that the above code (the second MWE) writes a couple of messages during the compilation, which exemplify how the \sump command is written out to auxiliary files, and shows that its behavior in this respect parallels that of \sum.