27

TeX offers plenty of ways of doing some things, and it is sometimes difficult to predict which one will be quickest. For instance, how do

  • a macro assignment
  • a toks assignment
  • a macro expansion
  • a toks expansion

compare to each other (depending on the lengths of arguments, and the number of them, of course).

I have been told that \romannumeral-`\0\somelongexpansion is slower than \somelongexpansion. Is that true/how do I test it/has it been tested?

Also, the expl3 source states that to gobble three brace groups it would be faster to gobble two, then one (with the equivalent of \@gobble and \@gobbletwo, rather than \long\def\@gobblethree#1#2#3{}). But preliminary tests with \pdfelapsedtime seem to point to the opposite.

So I guess my question is on what exists in terms of benchmarking the various pieces of TeX (I'm particularly interested in the stomach, but any body part of TeX is appreciated).

yannisl
  • 117,160
Bruno Le Floch
  • 44,937
  • 3
    I really don't think \expandafter\@gobble\@gobbletwo is faster then \@gobblethree. I would really like to know why this should be the case. Update: I made some benchmark which executes \expandafter\@gobble\@gobbletwo{}{}{} and \@gobblethree{}{}{} each 1.000.000 times and the results are: 88706 to 46309 scaled seconds (1/65536s). As you can see \@gobblethree is almost twice as fast and both are so fast that they really do not need to be considered for speed optimizations. – Martin Scharrer Mar 11 '11 at 01:05
  • With some actual content to gobble (119 characters in each argument) the times are 1078126 vs. 1037622. So the reading and tokenisation of the to-be-gobbled arguments takes more time than the expanding of the gobbling macros. However \@gobblethree is still faster. (That was the first time I compiled a 739 MB big .tex file :-) ) – Martin Scharrer Mar 11 '11 at 01:34
  • @Martin: since you already have the relevant files, can you test \long\def\@gobblethree#1{\@gobbletwo}? I expect that this should be faster, since there is no \expandafter. – Bruno Le Floch Mar 11 '11 at 01:37
  • @Bruno Check this out please http://tex.stackexchange.com/questions/tagged/profiling – yannisl Mar 11 '11 at 05:22
  • @Bruno I hope you don't mind, I edited your post to add the profiling tag. – yannisl Mar 11 '11 at 07:34
  • 1
    @Bruno: recently I made a benchmark for tikz-timing which accumulates code in a macro so far. I compared it with a token register and writing the material into an external file. I will post the results later today. In short: Macro and token register are exponential-time because they have to copy the old material, with the token register faster (almost twice) and the file is of course linear-time. – Martin Scharrer Mar 11 '11 at 08:55
  • @Martin: I think you mean quadratic rather than exponential. I'm definitely interested by that. – Bruno Le Floch Mar 11 '11 at 09:31
  • @Bruno: Yes, you are right, it's quadratic. I confused it with some other benchmark I did for my upcoming filemod package. One of the sorting algorithm was exponential because of recursive expansion, until I linearized it with some tricky use of \csname. – Martin Scharrer Mar 11 '11 at 09:49
  • @Martin: I'm a bit suspicious of linear time sorting :). --- Side comment: I discovered this week-end that it is possible to remove the last item of a list (sequence in LaTeX3) in a linear time. Now I only need to construct a linear-time conversion between comma-list and sequence. – Bruno Le Floch Mar 16 '11 at 17:20

2 Answers2

16

For general benchmarking I use a macro \replicate attributed to David Kastrup, to iterate and two macros \startTimer, \stopTimer, based on \pdfsettimer and pdfelapsedtime. I use this for order of magnitude benchmarking (using pdfLaTeX). For example, here is a test routine, that calculates and typesets fibonacci numbers to compare the effect of using \num from siunitx.

\documentclass[a4paper]{article}
    \usepackage{siunitx,fp}
    \def\startTimer{\pdfresettimer}
    \def\stopTimer{%
     \the\pdfelapsedtime\,scaled seconds
     \FPdiv\result{\the\pdfelapsedtime}{65536}
    % \FPmul\result{\result}{1000000} %microseconds
    \FPround\result{\result}{6}
     \result\thinspace s}

    \newcount\numbertimes
    \newcount\numone
    \newcount\numtwo 
    \newcount\savenumone 

    \def\fibonacci#1{0,
    \numbertimes=2\numone=0
    \numtwo=1
    \loop
       \advance\numone  by  \numtwo  
       %\num{\the\numone}, 
       \the\numone,
       \savenumone=\the\numone  \numone=\numtwo  \numtwo=\savenumone 
       \advance\numbertimes  by 1 \ifnum \numbertimes<#1
    \repeat  
    %\ifnum\numbertimes=#1 \advance\numone  by   \numtwo\fi and  \num{\the\numone}.\par
    \ifnum\numbertimes=#1 \advance\numone  by   \numtwo\fi and  \the\numone.\par
    }

    \begin{document}
    \startTimer
    \def\replicate#1#2{\ifnum#1>0 #2%
    \expandafter\replicate\expandafter{\number\numexpr#1-1}{#2}\fi}
    \replicate{4900}{\fibonacci{30}\fibonacci{30}\fibonacci{30}}
     \vskip12pt
    \stopTimer
    \end{document}

If you format it with num you will get a result of at least two orders of magnitude greater than those with plain numbers. This can be useful to identify bottlenecks in the code.

A lot about the pitfalls of this approach can be found in https://tex.stackexchange.com/questions/tagged/profiling

Please see also Bruno's comment below and amend the code to your preference.

Community
  • 1
yannisl
  • 117,160
  • @Yiannis: I know that siunitx is much slower than just using TeX to typeset the numbers! There are two parts to this. First, to get guaranteed control of font in the output, you have do quite a bit of LaTeX mode switching. That part I can't do much about. The second part is parsing and re-assembling the numbers. I've been through about 4 different algorithms for that, and the current one is much more efficient than the previous versions. However, as you observe it is still not that quick. I suspect that the efficiency could be improved, but only at the cost of maintainability. ctd ... – Joseph Wright Mar 11 '11 at 08:11
  • ctd ... If you turn the parser off (\sisetup{parse-numbers = false}), then things are much faster but still nothing like as fast as without siunitx. I'm always willing to try to improve performance, but have to balance that against the features that people want. – Joseph Wright Mar 11 '11 at 08:14
  • Yiannis's solution is essentially the same as mine (\prg_replicate:nn is the same as \replicate): I've therefore deleted my answer. – Joseph Wright Mar 11 '11 at 08:15
  • @Joseph Wright ... maybe I used the wrong example, I was looking for something that would load expl3 on top of LaTeX etc. 'siunitx is a great package, I use it for day to day jobs and the differences in compilation time in normal documents are meaningless. I always opt for readability of code and maintainability. – yannisl Mar 11 '11 at 08:18
  • @Yiannis: I was not taking this as a criticism, I just wanted to explain to the 'passing reader' why there is such a big difference here. – Joseph Wright Mar 11 '11 at 08:34
  • @Yiannis: thank you for the working example. Now I just need to update to TeXLive 2010 (2009's \pdfelapsedtime is broken and only gives multiples of 1 second). – Bruno Le Floch Mar 16 '11 at 17:24
  • One should note that the fp package can be avoided: \pdfelapsedtime sp is a dimension which, in pt is the number of seconds elapsed. So \newdimen\foo\foo\pdfelapsedtime sp\showthe\foo shows that (with the wrong suffix, pt instead of seconds). – Bruno Le Floch Jul 28 '11 at 12:38
10

Here the results of the gobble comparison. This tests runs the same command 1.000.000 times. It gobbles only empty arguments in this test. See also my comment below the question.

The results are:

Total times:

real    0m5.422s
user    0m5.360s
sys 0m0.050s

Results (in scaled seconds, 1/65536s)

\expandafter \@gobbletwo \@gobble : 95638
\expandafter \@gobble \@gobbletwo : 93726
\@gobbletwothenone : 56081
\@gobbleonethentwo : 57910
\@gobblethree : 45746

The \expandafter seems to require quite a bit of time. If the three arguments are gobbled with two macros the order seems not significant. The \@gobblethree macro is the fastest.

When the to-be-gobbled arguments contain some actual text the time of reading this tokens takes more times than the gobbling of it, so all times are very close together.

Benchmark files:

\documentclass{minimal}

\makeatletter
\long\def\@gobblethree#1#2#3{}
\long\def\@gobbleonethentwo#1{\@gobbletwo}
\long\def\@gobbletwothenone#1#2{\@gobbleone}
\begin{document}

\input{a1}
\input{a2}
\input{a3}
\input{a4}
\input{a5}

\end{document}

The input files are:

% a1.tex
\def\name{\expandafter\@gobbletwo\@gobble}%
\@onelevel@sanitize\name
\pdfresettimer
\expandafter\@gobbletwo\@gobble{}{}{}%
% ...
\typeout{\name: \the\pdfelapsedtime}

% a2.tex
\def\name{\expandafter\@gobble\@gobbletwo}%
\@onelevel@sanitize\name
\pdfresettimer
\expandafter\@gobble\@gobbletwo{}{}{}%
% ...
\typeout{\name: \the\pdfelapsedtime}

% a3.tex
\def\name{\@gobbletwothenone}%
\@onelevel@sanitize\name
\pdfresettimer
\@gobbletwothenone{}{}{}%
% ...
\typeout{\name: \the\pdfelapsedtime}

% a4.tex
\def\name{\@gobbleonethentwo}%
\@onelevel@sanitize\name
\pdfresettimer
\@gobbleonethentwo{}{}{}%
% ...
\typeout{\name: \the\pdfelapsedtime}

% a5.tex
\def\name{\@gobblethree}%
\@onelevel@sanitize\name
\pdfresettimer
\@gobblethree{}{}{}%
% ...
\typeout{\name: \the\pdfelapsedtime}
Martin Scharrer
  • 262,582