How to dump listings to a file?

Question

I thought to share my solution to a problem I had. (This is one of those “answer your own question” things, plus, this way, people can improve upon it – dual-licence under The MirOS Licence please. Other solutions to the problem class are welcome, too.)

When you have a listing (using lstlisting from the listings package, as appears to be the currently recommended one), you sometimes cannot copy and paste it correctly from the PDF. There is a variety of solutions, but why not ship the listing as plaintext file instead?

If your listings are in separate files, the solution is easy – \lstinputlisting – but that precludes substitution, for example by…

\newcommand{\myversion}{1.2}

\begin[escapeinside={〈}{〉}]{lstlisting}
tar -xzf myprogram-〈\myversion{}〉.tgz
\end{lstlisting}

… as well as managing the listings within the same file as the surrounding text (to prevent them from going stale), and some might prefer to have all listings from a single PDF contained in a single listing file.

Stuff posted on the SE network fall under a default CC-BY-SA license. A new MIT license is suggested to commence Jan 16, 2016 on [so] and [se], and on all other sites on Feb 1, 2016. — Werner, Dec 21 '15 at 20:26
@Werner doesn’t change my ability to also licence it under a separate, more free licence, nor my ability to ask editors to put their changes also under this licence, nor their ability to do so. The CC-Wiki licence is just from me to SO/SE, but by no means exclusive. (I’m the head of an OSS operating system project and thus have, unfortunately, enough knowledge about licencing.) — mirabilos, Dec 21 '15 at 20:58

score 2 · Accepted Answer · edited Apr 13 '17 at 12:34

Solution (source)

I’ve dug deep in both the Teχ SE and the listings source code and have, eventually, come up with the following SSCCE:

% © 2015 mirabilos <t.glaser@tarent.de>, published also under The MirOS Licence
\documentclass{scrartcl}

% used packages
\RequirePackage[utf8]{inputenc}
\RequirePackage[LY1,TS1,T1]{fontenc}
% font for listings
\RequirePackage[varqu,varl]{inconsolata}
% the listings package
\RequirePackage[writefile]{listings}

% some useful basic definitions
\lstset{
    basicstyle=\ttfamily,
    upquote=true,
    keepspaces,
literate=
    {á}{{\'a}}1 {é}{{\'e}}1 {í}{{\'i}}1 {ó}{{\'o}}1 {ú}{{\'u}}1
    {Á}{{\'A}}1 {É}{{\'E}}1 {Í}{{\'I}}1 {Ó}{{\'O}}1 {Ú}{{\'U}}1
    {à}{{\`a}}1 {è}{{\`e}}1 {ì}{{\`i}}1 {ò}{{\`o}}1 {ù}{{\`u}}1
    {À}{{\`A}}1 {È}{{\'E}}1 {Ì}{{\`I}}1 {Ò}{{\`O}}1 {Ù}{{\`U}}1
    {ä}{{\"a}}1 {ë}{{\"e}}1 {ï}{{\"i}}1 {ö}{{\"o}}1 {ü}{{\"u}}1
    {Ä}{{\"A}}1 {Ë}{{\"E}}1 {Ï}{{\"I}}1 {Ö}{{\"O}}1 {Ü}{{\"U}}1
    {â}{{\^a}}1 {ê}{{\^e}}1 {î}{{\^i}}1 {ô}{{\^o}}1 {û}{{\^u}}1
    {Â}{{\^A}}1 {Ê}{{\^E}}1 {Î}{{\^I}}1 {Ô}{{\^O}}1 {Û}{{\^U}}1
    {æ}{{\ae}}1 {Æ}{{\AE}}1 {ß}{{\ss}}1 {œ}{{\oe}}1 {Œ}{{\OE}}1
    {ő}{{\H{o}}}1 {Ő}{{\H{O}}}1 {ű}{{\H{u}}}1 {Ű}{{\H{U}}}1
    {å}{{\r a}}1 {Å}{{\r A}}1 {ç}{{\c c}}1 {Ç}{{\c C}}1 {ø}{{\o}}1
    {©}{{\textcopyright}}1 {™}{\texttrademark}1 {–}{-}1 {§}{\S}1
    {£}{{\pounds}}1 {°}{\textdegree{}}1 {„}{{\quotedblbase}}1
    {“}{{\textquotedblleft}}1 {”}{{\textquotedblright}}1
    {‘}{{\textquoteleft}}1 {’}{{\textquoteright}}1
    {«}{\guillemotleft}1 {»}{\guillemotright}1
    {…}{{\fontencoding{LY1}\selectfont\symbol{'205}}}1,
    columns=flexible,
    showstringspaces=false,
    extendedchars=true,
    breaklines=true,
    prebreak=\raisebox{0ex}[0ex][0ex]{\ensuremath{\hookleftarrow}},
    frame=single,
    showtabs=false,
    showspaces=false,
    showstringspaces=false,
    escapeinside={〈}{〉},
}

% lst dumps definitions
\makeatletter
\newcounter{lstdumps}
\setcounter{lstdumps}{0}
\newcommand{\lstdumpname}{}
\lstnewenvironment{lstdump}[3][]
    {\lst@TestEOLChar{#3}%
     \lstset{#1}%
     \csname\@lst @SetFirstNumber\endcsname%
     \stepcounter{lstdumps}%
     \renewcommand{\lstdumpname}{ (#2)}%
     \lst@BeginAlsoWriteFile{\jobname.lst}%
     \immediate\write\lst@WF{-----BEGIN LISTING \thelstdumps\lstdumpname-----}%
    }
    {
     \immediate\write\lst@WF{-----END LISTING \thelstdumps\lstdumpname-----}%
     \immediate\write\lst@WF{}%
     \endgroup%
     \csname\@lst @SaveFirstNumber\endcsname%
    }
\lstnewenvironment{lstdumpx}[2][]
    {\lst@TestEOLChar{#2}%
     \lstset{#1}%
     \csname\@lst @SetFirstNumber\endcsname%
     \stepcounter{lstdumps}%
     \renewcommand{\lstdumpname}{ [unnamed]}%
     \lst@BeginAlsoWriteFile{\jobname.lst}%
     \immediate\write\lst@WF{-----BEGIN LISTING \thelstdumps\lstdumpname-----}%
    }
    {
     \immediate\write\lst@WF{-----END LISTING \thelstdumps\lstdumpname-----}%
     \immediate\write\lst@WF{}%
     \endgroup%
     \csname\@lst @SaveFirstNumber\endcsname%
    }
% usage: \lstdumpesc[file]{pdf} or \lstdumpesc{bothfileandpdf}
\def\lstdumpesc{\@ifnextchar[{\lstdumpesc@two}{\lstdumpesc@one}}
\def\lstdumpesc@one#1{\lst@WFAppend{#1}#1}
\def\lstdumpesc@two[#1]#2{\lst@WFAppend{#1}#2}
\makeatother

% document body

\newcommand{\myversion}{1.2}

\begin{document}

This is the MyProgram \myversion{} installation manual.
You need to have \texttt{myprogram-\myversion{}.tgz}
downloaded already. Start with:

\begin{lstdump}[language=sh]{extract}
tar -xzf myprogram-〈\lstdumpesc{\myversion}〉.tgz
cd 〈\lstdumpesc[myprogram-\myversion]{my*/«tab»}〉
\end{lstdump}

Oh, by the way, check this out:

\begin{lstdumpx}
zcat /usr/share/doc/mksh/examples/uhr.gz | mksh
\end{lstdumpx}

The second listing has no name in the \texttt{.lst} file.

\end{document}

Solution (output)

The PDF output looks like this:

When saved as demo.tex a file called demo.lst will be generated with the following contents:

-----BEGIN LISTING 1 (extract)-----
tar -xzf myprogram-1.2.tgz
cd myprogram-1.2
-----END LISTING 1 (extract)-----

-----BEGIN LISTING 2 [unnamed]-----
zcat /usr/share/doc/mksh/examples/uhr.gz | mksh
-----END LISTING 2 [unnamed]-----

Basically, all lstdump and lstdumpx listings are numbered (I didn’t show these numbers in the PDF, it could maybe useful to print them in lightgrey on the margin – we have extremely small margins as the PDFs are mostly for online use, and we don’t use the listing numbers, but they were cheap to get and some people may like them) and all lstdump ones are named, and they show up in the plaintext file. Additionally, we can escape from listings to Teχ (for substitution), and the \lstdumpesc command ensures that the escaped text is also written to the file (if an optional argument is given, it is written instead).

extra information

The literate part allows use of some Unicode characters in listings, but they just are omitted from the .lst file – anything outside of ASCII will either not show up or (e.g. in listing names for the lstdump environment) result in random 8-bit characters in the output.

More features

listing numbers

If you want the listing numbers to additionally show up in the PDF (which makes cross-referencing easier), add the marginnote and possibly xcolor package and insert the line…

        \marginnote{{\Huge\color{Gray}\thelstdumps}}%

… after the stepcounter line in each definition.

file and chapter intro lines in the listing file

From this answer I found a method to include the current chapter/section/subsection number in the output file each time they differ, but this needs the hyperref package loaded. Note that this may require a third pdflatex run, to fix the LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right. warning and get the margin notes on the right side of the page.

The lst dump definitions including both features, complete again, are:

% lst dumps definitions
\makeatletter
\newcounter{lstdumps}
\setcounter{lstdumps}{0}
\newcommand{\lstdumpname}{}
\xdef\lstdumpsection{\@empty}%
\lstnewenvironment{lstdump}[3][]
    {\lst@TestEOLChar{#3}%
     \lstset{#1}%
     \csname\@lst @SetFirstNumber\endcsname%
     \stepcounter{lstdumps}%
     \marginnote{{\huge\color{Gray}\thelstdumps}}%
     \renewcommand{\lstdumpname}{ (#2)}%
     \lst@BeginAlsoWriteFile{\jobname.lst}%
     \ifx\lstdumpsection\@currentlabel\else%
      \ifx\lstdumpsection\@empty%
       \immediate\write\lst@WF{Listing dump for \jobname.tex}%
       \immediate\write\lst@WF{}%
      \fi%
      \immediate\write\lst@WF{>>> (\@currentlabel) \@currentlabelname}%
      \immediate\write\lst@WF{}%
     \fi%
     \immediate\write\lst@WF{-----BEGIN LISTING \thelstdumps\lstdumpname-----}%
    }
    {
     \immediate\write\lst@WF{-----END LISTING \thelstdumps\lstdumpname-----}%
     \immediate\write\lst@WF{}%
     \endgroup%
     \csname\@lst @SaveFirstNumber\endcsname%
     \xdef\lstdumpsection{\@currentlabel}%
    }
\lstnewenvironment{lstdumpx}[2][]
    {\lst@TestEOLChar{#2}%
     \lstset{#1}%
     \csname\@lst @SetFirstNumber\endcsname%
     \stepcounter{lstdumps}%
     \marginnote{{\huge\color{Gray}\thelstdumps}}%
     \renewcommand{\lstdumpname}{ [unnamed]}%
     \lst@BeginAlsoWriteFile{\jobname.lst}%
     \ifx\lstdumpsection\@currentlabel\else%
      \ifx\lstdumpsection\@empty%
       \immediate\write\lst@WF{Listing dump for \jobname.tex}%
       \immediate\write\lst@WF{}%
      \fi%
      \immediate\write\lst@WF{>>> (\@currentlabel) \@currentlabelname}%
      \immediate\write\lst@WF{}%
     \fi%
     \immediate\write\lst@WF{-----BEGIN LISTING \thelstdumps\lstdumpname-----}%
    }
    {
     \immediate\write\lst@WF{-----END LISTING \thelstdumps\lstdumpname-----}%
     \immediate\write\lst@WF{}%
     \endgroup%
     \csname\@lst @SaveFirstNumber\endcsname%
     \xdef\lstdumpsection{\@currentlabel}%
    }
% usage: \lstdumpesc[file]{pdf} or \lstdumpesc{bothfileandpdf}
\def\lstdumpesc{\@ifnextchar[{\lstdumpesc@two}{\lstdumpesc@one}}
\def\lstdumpesc@one#1{\lst@WFAppend{#1}#1}
\def\lstdumpesc@two[#1]#2{\lst@WFAppend{#1}#2}
\makeatother