I want to write a new command for the trace of a matrix that I've defined like:
\newcommand{\trace}[1]{\mathsf{trace}\!\left[#1\right]}
but when I call
\trace{A^\top}
the automatic scaling makes the [ and ] characters \bigl[ and \bigr]. How can I pass an optional additional argument to my new command so that I can control the scaling? Ideally it would be like
\trace[\big]{A^\top}
to get the scaled version and
\trace{A^\top}
to get the standard version.


\operatorname{\mathsf{trace}}– egreg Feb 26 '22 at 08:32