Why is this loop so much faster under Windows?

Question

In a recent question, Yiannis Lazarides was asking about benchmarking loops. Strangely his results were about 800 times faster than mine. I couldn't quite accept that my computer is that outdated (as I only bought it a couple of months ago). The main difference seemed to be that I was compiling the file on Linux while he was compiling it on Windows. So I installed MikTeX on my Windows partition to try it myself and the result has quite surprised me.

This is a slightly simplified version of Yiannis' code (to be run with pdflatex)

\documentclass{article}
\usepackage{ifthen}

\def\startTimer{\pdfresettimer}
\def\stopTimer{\the\pdfelapsedtime\,scaled seconds}

\begin{document}
\startTimer
\newcounter{acount}
\whiledo{\value{acount}<888}{%
\stepcounter{acount}% 
 \theacount, }
\stopTimer
\end{document}

On Linux (Ubuntu 10.10 64bit, TeX Live 2010 (both 32bit and 64bit)) the timer reports about 1700 scaled seconds. On Windows (7, 64bit, MikTeX 2.9 (32bit?)) the result is 2 scaled seconds.

I am pretty sure that a factor of 850 is not solely attributable to the operating system (both tests are run on the same machine). Also, compiling a “real life” document seems to take about the same time on both operating systems. Where does the difference come from?

Answer 1

At this (very small) amount of running time, the results are likely to be dominated by the implementation differences between the windows and linux system function calls that are used by pdftex. Pdftex does not run the same code on both platforms, and the function implementation that is used on Windows (ftime) is infamous for being unreliable at small intervals.

Besides that, also bear in mind that \pdfelapsedtime measures wall clock time differences, not the actual computer usage of pdftex. Lots of other stuff could (and probably will) be going on on the computer at the same time. Intervening task switches of the operating system itself can have a pretty large effect on the results.

As I wrote in that other thread already: you should not trust any benchmarking values that are less than a whole second (65536 scaled seconds). Also make sure you run the code multiple time on a machine that does as little else as possible.

Answer 2

Responding to the comments: Here is a test that is a bit more realistic.

TeX file:

\documentclass{article}
\usepackage{ifthen}

\newwrite\BenchmarkStream
\def\startTimer{%
  \pdfresettimer
  \immediate\openout\BenchmarkStream=\jobname.dat
}
\def\stopTimer{%
  \immediate\write\BenchmarkStream{\number\pdfelapsedtime}%
  \immediate\closeout\BenchmarkStream
}

\begin{document}
\startTimer
\newcounter{acount}
\whiledo{\value{acount}<28888}{%
\stepcounter{acount}% 
 \theacount, }
\stopTimer
\end{document}

Python script:

#!/usr/bin/env python2.6

from __future__ import unicode_literals, division, print_function

import subprocess
import numpy

def run():
    args = ["pdflatex", "--interaction=batchmode", "test.tex"]
    subprocess.check_call(args)
    with open("test.dat", "rt") as stream:
        return int(stream.read()) / 0x10000

def main():
    # warm up
    for i in xrange(10):
        run()
    count = 50
    result = numpy.empty(count)
    for i in xrange(count):
        print("Iteration", i)
        result[i] = run()
    imin = result.argmin()
    imax = result.argmax()
    print("Count:", count)
    print("Minimum:", result[imin], "at", imin)
    print("Maximum:", result[imax], "at", imax)
    print("Mean:", result.mean())
    print("Median:", numpy.median(result))
    print("Standard deviation:", result.std())

if __name__ == "__main__":
    main()

Results on my Linux system are:

Minimum: 1.05038452148 at 0
Maximum: 1.20942687988 at 30
Mean: 1.07401489258
Median: 1.05834197998
Standard deviation: 0.039375040446