Write18: capturing shell (script) output as command/variable?

Question

Let's say I compile the following file with pdflatex -shell-escape test.tex:

\documentclass{minimal}

\begin{document}

File listing is:

\immediate\write18{ls /usr}

\end{document}

This will send the output of the command ls /usr to report/log of pdflatex (primarily to stdout).

There are then two cases I'd like to utilize:

• The output of ls /usr being included directly in the document (LaTeX stream).
• The output of ls /usr becoming the contents of a \newcommand (by this I mean, that I'd like the script to be executed when the \newcommand is executed first - and upon subsequent calls to the new command, the shell code should not be executed anew).

I have read through How to execute shell script from LaTeX?, but I'm not sure if this "pipe input" can be applied to \newcommand.

I have also read through tex - How can I save shell output to a variable in LaTeX? - and it seems that Tex' file I/O should be used; but I'm quite disliking the fact that I'd still have to redirect the script output (actually, in this case, the ls /usr output) to a file, and then read it in, to have it as contents of a command.

So, is there an easier way to achieve what I'd want (hopefully, illustrated through an example based on the above code)?

EDIT: Ehm, I should have asked one more question earlier :) I'll try with an edit here, although it will probably get missed .. :)

I originally asked for a \newcommand that will execute shell code only upon its definition (i.e., in a sense it is "cached"); and the answer from @egreg does exactly that. But then - would it be possible to have a different \newcommand definition, such that each time this newcommand is called, the shell command is executed anew? I.e. executing something like \@@input|"cat tempfile" (which cannot be executed as such), where tempfile changes between calls?

Answer 1

\documentclass{article}

\begingroup\makeatletter\endlinechar=\m@ne\everyeof{\noexpand}
\edef\x{\endgroup\def\noexpand\TeXpath{\@@input|"which tex" }}\x

\begin{document}
File listing is

{\catcode`_=12 \ttfamily
\input{|"ls /usr" }

}

\TeX{} is \TeXpath
\end{document}

We must use \@@input (the primitive \input command) because \input in LaTeX does assignments. The setting of \endlinechar is to avoid a spurious space in the expansion of \TeXpath.

When shell escape is active and the primitive \input finds a |, it accepts as input the standard output of the following shell command.

There should be a package by H. Oberdiek that does something of this kind.

Note An assignment is any TeX operation that gives a meaning or a value to a control sequence or register. During the \edef operation, TeX expands all commands it finds between the braces until only unexpandable tokens remain, but doesn't perform any assignment; rather, something like \catch=22 (where \catch is the name of a count register) remains completely inaltered. Since the definition of \input in LaTeX is

\@ifnextchar\bgroup\@iinput\@@input

the implicit assignments performed by \@ifnextchar would not be performed and both \@input and \@@input would be expanded, which results in a complete disaster. Conversely, the \input primitive (that LaTeX saves as \@@input) is expandable and its expansion consists in causing TeX to read the named file. One has, of course, to be careful about what this file contains, as also this will be expanded. So other precautions have to be taken when doing this kind of operations, depending on the nature of the tokens produced by the command we want to perform and this "solution" is only a skeleton for possible "real" applications.

---

Update 2019

After some years, things have changed and better methods are available.

For instance, with xparse and expl3 the code can be improved:

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentCommand{\captureshell}{som}
 {
  \sdaau_captureshell:Ne \l__sdaau_captureshell_out_tl { #3 }
  \IfBooleanT { #1 }
   {% we may need to stringify the result
    \tl_set:Nx \l__sdaau_captureshell_out_tl
     { \tl_to_str:N \l__sdaau_captureshell_out_tl }
   }
  \IfNoValueTF { #2 }
   {
    \tl_use:N \l__sdaau_captureshell_out_tl
   }
   {
    \tl_set_eq:NN #2 \l__sdaau_captureshell_out_tl
   }
 }

\tl_new:N \l__sdaau_captureshell_out_tl

\cs_new_protected:Nn \sdaau_captureshell:Nn
 {
  \sys_get_shell:nnN { #2 } { } #1
  \tl_trim_spaces:N #1 % remove leading and trailing spaces
 }
\cs_generate_variant:Nn \sdaau_captureshell:Nn { Ne }
\ExplSyntaxOff

\begin{document}

\captureshell*[\TeXpath]{which tex} % we need to stringify it because of _

File listing is

{\ttfamily\captureshell{ls \jobname.*}\par}

\TeX{} is \texttt{\TeXpath}

\end{document}

We could add an error message if the user doesn't pass the -shell-escape option for the LaTeX run.

Check also texosquery (requires Java).

Answer 2

Here is a simple way of doing this, using my bashful package

\documentclass{article}
\usepackage[a6paper]{geometry}
\usepackage{bashful}

\begin{document}
\bash[script,stdout]
ls -F /usr
\END
\end{document}

which generates

Answer 3

I suppose this snippet will be helpful in this thread.

https://gist.github.com/w495/7328b76e76aee49657e0bd7a3b46c870

% !TeX encoding = UTF-8
\ProvidesPackage{bashline}[2016/10/24 v. 0.1]

\makeatletter
    \newcommand{\bashline@file@name}[1]{%
        /tmp/${USER}-${HOSTNAME}-\jobname-#1.tex%
    }
    \newread\bashline@file
    \newcommand{\bashline@command@one}[2][tmp]{%
        \immediate\write18{#2 > \bashline@file@name{#1}}
        \openin\bashline@file=\bashline@file@name{#1}
        % The group localizes the change to \endlinechar
        \bgroup
            \endlinechar=-1
            \read\bashline@file to \localline
            % Since everything in the group is local, 
            % we have to explicitly make the assignment global
            \global\let\bashline@result\localline
        \egroup
        \closein\bashline@file
        % Clean up after ourselves
        \immediate\write18{rm \bashline@file@name{#1}}
        \bashline@result
    }
    \newcommand{\bashline@command@many}[2][tmp]{%
        \immediate\write18{#2 > \bashline@file@name{#1}}
        \openin\bashline@file=\bashline@file@name{#1}
        % The group localizes the change to \endlinechar
        \newcount\linecnt
        \bgroup
            \endlinechar=-1
            \loop\unless\ifeof\bashline@file 
                \read\bashline@file to \localline%
                \localline
                \newline
            \repeat
        \egroup
        \closein\bashline@file
        % Clean up after ourselves
        \immediate\write18{rm \bashline@file@name{#1}}
    }
    \newcommand{\bashline}[2][tmp]{%
        \bashline@command@one[#1]{#2}%
    }
    \newcommand{\bashlines}[2][tmp]{%
        \bashline@command@many[#1]{#2}%
    }
\makeatother

\newcommand{\urandomstring}[1]{%
    \bashline{cat /dev/urandom | tr -dc "A-Za-z0-9" | fold -c#1 | head -1}%
}

\newcommand{\bashdate}{%
    \bashline{date --iso-8601}%
}

\newcommand{\bashdatetime}{%
    \bashline{date --iso-8601=seconds}%
}

\newcommand{\commit}{%
    \bashline{git describe --dirty }%
}

\newcommand{\commitlog}{%
    \bashline{git log -1 --oneline}%
}

\newcommand{\branch}{%
    \bashline{git describe --all}%
}

\endinput

It is based on Antal's answer in «How can I save shell output to a variable in LaTeX?». For example check \urandomstring. It generates new random string with every call. Also, see \bashlines macros. It works for me like native bash.

Here an example: https://www.sharelatex.com/project/580e8926fe7b0dfd2ef8ae52

As you can see, \bashdatetime gives a different nanoseconds each time.

Answer 4

You can use my package iexec:

\documentclass{minimal}
\usepackage{iexec}
\begin{document}
File listing is:
\iexec{ls /usr}
\end{document}