Timing response of externally called commands via write18: blocking or not?

Question

I was wondering: when we use \immediate\write18{...} to run commands in an external shell from Latex, is the \write18 a blocking call or not? In other words, will it wait for the external code to complete and return (with success, or not) - or will Latex just initiate execution in a separate thread in a non-blocking manner, and continue immediately afterwards?

Can an example be supplied that illustrates this?

Answer 1

The documentation for Web2c explains the shell escape feature for TeX engines based on Web2c, e.g. TeX Live. Section "4.5 Shell escapes":

The shell-command string is passed to the command shell (via the C library function system). The output of shell-command is not diverted anywhere, so it will not appear in the log file, or anywhere but the terminal output. The exit status of the system call is also not available to TeX.

The system function is explained in its manual page:

system() returns after the command has been completed.

Thus system is a "blocking call". That allows that the command creates a file that can be read after the \write in LaTeX, for example. Otherwise it would not be possible, because there is no interface for checking the status of the child (no wait, ...); it could happen that LaTeX finishes \end{document}, before the command execution is finished.

In short:

• system is a "blocking call"
• Be aware of the difference \immediate\write18 and \write18 without immediate. The latter execution is deferred to the next page shipout.
• Keep in mind security. There are options to control, whether commands can be executed. The new restricted mode (enabled by default) has further restrictions.

Answer 2

Here is a self-answer, as I didn't find this info anywhere explicitly - \immediate\write18 seems to have a blocking behavior.

I used perl on Linux, with a for loop with a sleep inside as the test program; it turns out, it's a bit itchy to escape the linefeed \n as raw ASCII for the external shell command - so, one needs to be careful about that. Otherwise, the MWE below produces the following output, when Latex is called with -shell-escape:

$ pdflatex -shell-escape test.tex 
This is pdfTeX, Version 3.1415926-2.3-1.40.12 (TeX Live 2011)
 \write18 enabled.
entering extended mode
(./test.tex
LaTeX2e <2011/06/27>
Babel <v3.8m> and hyphenation patterns for english, dumylang, nohyphenation, lo
aded.
(/path/to/texlive/2011/texmf-dist/tex/latex/base/article.cls
Document Class: article 2007/10/19 v1.4h Standard LaTeX document class
(/path/to/texlive/2011/texmf-dist/tex/latex/base/size10.clo))
(./test.aux)
Check: outputting command: 
 echo 'for($ix=0;$ix<5;$ix++){ print("perl $ix \n");sleep 1;};' ; perl -e 'for(
$ix=0;$ix<5;$ix++){ print("perl $ix \n");sleep 1;};' 

 \LaTeX typeout: Before write18 

for($ix=0;$ix<5;$ix++){ print("perl $ix 
");sleep 1;};
perl 0 
perl 1 
perl 2 
perl 3 
perl 4 

 \LaTeX typeout: After write18 

(./test.aux) )
No pages of output.
Transcript written on test.log.

Clearly, the \write18 waited for the perl code to execute first - before allowing Latex to continue onwards.

Here is the MWE, test.tex:

\documentclass{article}

\begin{document}

% http://tex.stackexchange.com/a/69294/2595
% must escape backslash like this, to have escaped linefeed  
% character \n in \write18 output for the shell (raw ASCII)!
\begingroup
  \catcode `~=11
  \gdef\mytilde{~}
  \catcode `\|=0
  \catcode `\\=11
  |gdef|LF{\n} % \LF becomes (ASCII) "\n" (verbatim char!)
  |gdef|n{\n} % \n becomes (ASCII) "\n" (verbatim char!)
  |gdef|ELF{\\n} % \ELF becomes (ASCII) "\\n" - escaped backslash for shell!
|endgroup

\edef\cmd{%
  %echo 'for($ix=0;$ix<5;$ix++){ print("perl $ix \LF");sleep 1;};'
  % below works, with a previous definition of linefeed (with
  % non-escaped backslash) \n as macro; then linefeed char is verbatim
  %echo 'for($ix=0;$ix<5;$ix++){ print("perl $ix \n");sleep 1;};'
  % like this for escaped-backslash-linefeed to propagate through shell;
  % the programs see escaped backslash then:
  %echo 'for($ix=0;$ix<5;$ix++){ print("perl $ix \ELF");sleep 1;};' ;
  %
  % The proper one is with \n/\LF; don't forget `;` separators for bash
  % (as latex will compact these in a single line anyway)
  %
  echo 'for($ix=0;$ix<5;$ix++){ print("perl $ix \n");sleep 1;};' ;
  perl -e 'for($ix=0;$ix<5;$ix++){ print("perl $ix \n");sleep 1;};'
}

\typeout{%
  Check: outputting command: ^^J
  \cmd% OK
}

\typeout{^^J \LaTeX typeout: Before write18 ^^J}

\immediate\write18{%
  \cmd% OK
}%

\typeout{^^J \LaTeX typeout: After write18 ^^J}

\end{document}