I've written a custom \set macro to typeset sets allowing line breaks at every comma:
\documentclass{article}
\usepackage[T1]{fontenc}
%\usepackage{amsmath} %<-- (un)comment to toggle the problem
% this is the colon in expressions like {n : n < 10}
\newcommand*{\setgiven}[1][]{% #1 is discarded
\nonscript\medspace\mathord:\allowbreak% medspace? + : + break?
\nonscript\medspace\mathopen{}% medspace?
}
% \parseset splits a comma-separated list like
% 1,2,3,4,5
% into a sequence stored in \g_setlist_seq
\ExplSyntaxOn
\seq_new:N \g_setlist_seq
\cs_new_nopar:Npn \parseset #1 {
\seq_set_split:Nnn \g_setlist_seq { , } { #1 }
}
% \printset prints list \g_setlist_seq with
% each element separated by ,\allowbreak
\cs_new_nopar:Npn \printset {
\seq_use:Nnnn \g_setlist_seq {,\allowbreak}{,\allowbreak}{,\allowbreak}
}
\ExplSyntaxOff
\NewDocumentCommand{\set}{s o m}{%
% \set{1,2,3} => simple set that allows line breaks at commas
% \set[n]{n<10} => {n : n < 10}
% the starred versions use \left{ and \right} to handle "tall" characters
\parseset{#3}%
\IfValueTF{#2} % WITH optional argument
{\IfBooleanTF{#1} % starred: \left and \right
{\left\lbrace #2\setgiven[\delimsize] \printset \right\rbrace}%
{\lbrace #2\setgiven[\delimsize] \printset \rbrace}%
}
% WITHOUT optional argument
{\IfBooleanTF{#1} % starred: \left and \right
{\left\lbrace \printset \right\rbrace}
{\lbrace \printset \rbrace}%
}%
}
\begin{document}
\begin{tabular}{r l}
With \string\mathellipsis & $\set{1, 2, \mathellipsis}$\
With \string\dots & $\set{1, 2, \dots}$\
%With \string\dotsc & $\set{1, 2, \dotsc}$\ %<-- uncomment when using amsmath
With \string\dots\string\relax & $\set{1, 2, \dots\relax}$\
With \string\dots{} & $\set{1, 2, \dots{}}$
\end{tabular}
\end{document}
This macro should behave well with amsmath, because it's a very common package.
It's also very common to write sets such as \set{1, 2, \dots}, so the macro should also behave well with \dots at the end.
I've found out that while the macro is ok with standard LaTeX,
ending a \set with \dots adds some extra space if you're using amsmath
(\dots in the middle of other things, as in \set{1, \dots, n}, is fine).
Regular LaTeX:
With amsmath:
After examining amsmath.sty and
reading a few questions here,
I know the amsmath version of \dots examines the next token to
decide which kind of dots to use,
but I don't know why in this case it adds extra space.
There's a way to prevent this,
using something else at the end of a \set:
\dots\relax and \mathellipsis work, but
\dots{} and \dotsc don't
(which is not very intuitive, I think people would expect all four of these to work).
So, is there a way to allow a user using amsmath to simply type \dots and
don't get the extra space?
