`latexmk`can include files created during the `latexmk` run, except with `lstinputlistings`

Question

My latex file looks roughly like this:

...
\usepackage{listings}
...
\lstinputlisting{F4_01fig2dev.ptx}
...

and the file F4_01fig2dev.ptx is to be created. In fact I want to build with latexmk. Normally the technique is, to make latexmk run a latex processor in scrollmode finding out the missing files, create the missing files with additional rules and then rerun the latex processor.

If I embed the file with \input{F4_01fig2dev.ptx} or with \includegraphics (using the graphicx package) or with \import (using the import package), the usual technique with latexmk works.

It does not work with \lstinputlisting. Originally, I told here that the reason is that for some reason \lstinputlisting does not respect scrollmode, but this is not the difference. I don't want to make another wrong suspicion, but i observed that the FLS files don't include the file to be included in any case, so no difference. But the log file is a bit different:

! Package Listings Error: File `F4_01fig2dev(.ptx)' not found.

for the listings package versus

! LaTeX Error: File `F4_01fig2dev.ptx' not found.

for the other kinds of inclusions. Maybe latexmk does not recognize the ending that way. I must say, I find listings package more precise.

Ok, now either latexmk changes or listings package changes or I do some workaround.

I thought of conditional code, bypassing if the file does not exist. Any other thoughts?

As cabohah suggested, I shall add my .latexmkrc file. I will post in full length,

#!/usr/bin/env perl
to create pdf via lualatex
#$pdflatex = 'lualatex -file-line-error %O %S';
PDF-generating modes are:
1: pdflatex, as specified by $pdflatex variable (still largely in use)
2: postscript conversion, as specified by the $ps2pdf variable (useless)
3: dvi conversion, as specified by the $dvipdf variable (useless)
4: lualatex, as specified by the $lualatex variable (best)
5: xelatex, as specified by the $xelatex variable (second best)
$pdf_mode = 4;# specifies creation of pdf via lualatex
If additional parameters must be passed to lualatex
#'svg' package.
It converts raw SVG files to the PDF+PDF_TEX combo using InkScape.
$lualatex = "lualatex --shell-escape";
note that -recorder is implicitly added
$latex2pdfOptions = "-interaction=nonstopmode -synctex=1 -shell-escape";
$lualatex = "lualatex $latex2pdfOptions %O %S";
#$postscript_mode = $dvi_mode = 0;
to configure bibtex
$bibtex_use=2;
this cannot be done according to the according latex maven plugin,
because the according parameter maxNumReRunsLatex may be set to -1
which signifies an infinite number of runs.
$max_repeat=30;
default are tex and eps, but could also be pdf and ptx and mps
Currently, all those files are given with explicit endings,
so no extensions to be added.
#add_input_ext('');
It is what it seems to be: clean inludes what was generated by cus
$cleanup_includes_cusdep_generated = 1;
$cleanup_includes_generated = 1;
#$makeindex = 'makeindex %O -o %D %S'; # the default
TBD: clarify: xdv and dvi seem to be internal.
maybe missing other extensions in conjunction with synctex
$clean_ext .= " %R.synctex.gz";
bbl does not work
#@generated_exts = (@generated_exts, 'lol', 'bbl', 'glo', 'ist') 
#print "Hello!"
#foreach (@generated_exts) {
#print "Generated exts: $_\n";
#}
#print "clean_ext\n";
Here @generated_exts is ('aux', 'fls', 'log', # generated by latex already
'toc', 'lof', 'lot', 'out', # generated by latex conditionally
'idx', 'ind', 'blg', 'ilg', # concerning indices
# strange enough: nothing for bibtex
'xdv', 'bcf'
extensions ext to be deleted by latexmk -c
Note that the file names are %R.ext.
this may cause problems with extensions containing a dot.
Also this is not general enough
if the generated file deviates from %R by more than an extension.
In this case, use $clean_ext$
Here, $clean_ext is empty.
list of listings, whereas lof and lot are already present.
push @generated_exts, "lol";
push @generated_exts, "dvi", "xdv";
for beamer class
push @generated_exts, "nav", "snm", "vrb";
why are .ist and xdy not under generated_exts?
note that currently, either %R or what is present is the extension only!
this does not make sense very much.
$clean_ext .= " stateMachine.log"; does not work, because stateMachine.log is the extension!
should be under indexing
$clean_ext .= " %R.ist %R.xdy %R-.ind %R-.idx %R-.ilg %R-.ind";
add_cus_dep('fig', 'ptx', 0, 'fig2dev');
sub fig2dev {
  $file = $_[0];
  print("$file");
  rdb_add_generated("$file.eps", "$file.pdf", "$file.ptx");
  my $GenOptions = "";
  my $PtxOptions = "";
  my $PdfEpsOptions = "";
  print("create from $file.fig");
  rdb_add_generated("$file.ptx", "$file.pdf", "$file.eps");
  #fig2dev -L pstex    <fig2devGenOptions> <fig2devPdfEpsOptions>        xxx.fig xxx.eps

  #fig2dev -L pdftex   <fig2devGenOptions> <fig2devPdfEpsOptions>        xxx.fig xxx.pdf

  #fig2dev -L pdftex_t <fig2devGenOptions> <fig2devPtxOptions>    -p xxx xxx.fig xxx.ptx
my $ret1 = system("fig2dev -L  pstex   $GenOptions $PdfEpsOptions       $file.fig $file.eps");
  my $ret2 = system("fig2dev -L pdftex   $GenOptions $PdfEpsOptions       $file.fig $file.pdf");
  my $ret3 = system("fig2dev -L pdftex_t $GenOptions $PtxOptions -p $file $file.fig $file.ptx");
return ($ret1 or $ret2 or $ret3);
}
add_cus_dep('gp', 'ptx', 0, 'gnuplot');
sub gnuplot {
  $file = $_[0];
  print("create from $file.gp");
  rdb_add_generated("$file.ptx", "$file.pdf", "$file.eps");
  my $nuplotOptions = "";
  my $ret1 = system("gnuplot -e &quot;set terminal cairolatex pdf $gnuplotOptions;

            set output '$file.ptx';

            load '$file.gp'&quot;");
my $ret2 = system("gnuplot -e &quot;set terminal cairolatex eps $gnuplotOptions;\
set output '$file.ptx';\
load '$file.gp'&quot;");
return $ret;
}
metapost rule from http://tex.stackexchange.com/questions/37134
#add_cus_dep('mp', 'mps', 0, 'mpost');
add_cus_dep('mp', 'mps', 0, 'mpost');
sub mpost {
  my $file = $_[0];
  print("create from $file.mp");
  rdb_add_generated("$file.mps", "$file.fls", "$file.log");
  my ($name, $path) = fileparse($file);
  pushd($path);
  my $return = system("mpost -interaction=nonstopmode -recorder -s prologues=2 -s 'outputtemplate=&quot;%j.mps&quot;' $name");
  popd();
  return $return;
}
add_cus_dep('svg', 'ptx', 0, 'inkscape');
sub inkscape {
  my $file = $_[0];
  print("create from $file.svg");
  rdb_add_generated("$file.ptx", "$file.pdf");
  my $ret1 = system("inkscape -D --export-filename=$file.pdf --export-latex $file.svg ");
  #my $ret2 = system("inkscape -D --export-filename=$file.eps --export-latex $file.svg ");
  #use File::Copy;
This works only for pdf, not for eps.
#unlink($file.pdf_tex) or die "cannot unlink $file.pdf_tex";
  rename("$file.pdf_tex", "$file.ptx");# or die "cannot move $file.pdf_tex";
  return $ret1;# or $ret2;
}
graphics for xfig (not appropriate for mixed tex/pdf)
add_cus_dep('fig', 'pdf', 0, 'fig2pdf');
sub fig2pdf {
system( "fig2dev -Lpdf &quot;$[0].fig&quot; &quot;$[0].pdf&quot;" );
}
use splitindex
$makeindex = 'internal splitindex';
sub splitindex {
Use splitindex instead of makeindex.
The splitindex programe starts from an .idx file, makes a set of
other .idx files for separate indexes, and then runs makeindex to
make corresponding .ind files.
However, it is possible that the document uses the splitindex
package, but in a way compatible with the standard methods
compatible with makeindex, i.e., with a single index and with the
use of the \printindex command.
Then we need to invoke makeindex.
In addition, latexmk assumes that makeindex or its replacement makes
an .ind file from an .idx file, and latexmk gives an error if it
doesn't exist, we need to make an .ind file.
Both problems are solved by running makeindex and then splitindex.
Note: errors are returned by makeindex and splitindex for things
like a missing input file.  No error is returned for lines in an
input file that are in an incorrect format; they are simply
ignored.  So no problem is caused by lines in the .idx file
that are generated by splitindex in a format incompatible with
makeindex.
my $ret1 = system( "makeindex", $$Psource );
   my $ret2 = system( "splitindex", $$Psource );
   return $ret1 || $ret2;
}
add_cus_dep( 'acn', 'acr', 0, 'makeglossaries' );
add_cus_dep( 'glo', 'gls', 0, 'makeglossaries' );
push @generated_exts, 'glo', 'gls', 'glg';
push @generated_exts, 'acn', 'acr', 'alg';
push @generated_exts, "ist"; # index stylefile created by the glossaries package
#$clean_ext .= " acr acn alg glo gls glg";# TBD: clarify: better in @generated_exts?
sub makeglossaries {
     my ($base_name, $path) = fileparse( $_[0] );
     my @args = ( "-q", "-d", $path, $base_name );
     if ($silent) { unshift @args, "-q"; }
     return system "makeglossaries", "-d", $path, $base_name; 
}
sub run_makeglossaries {
    my ($base_name, $path) = fileparse( $_[0] ); #handle -outdir param by splitting path and file, ...
    pushd $path; # ... cd-ing into folder first, then running makeglossaries ...
if ( $silent ) {
    # system &quot;makeglossaries -q '$base_name'&quot;; #unix
    system &quot;makeglossaries&quot;, &quot;-q&quot;, &quot;$base_name&quot;; #windows
}
else {
    # system &quot;makeglossaries '$base_name'&quot;; #unix
    system &quot;makeglossaries&quot;, &quot;$base_name&quot;; #windows
};

popd; # ... and cd-ing back again

}
!!! ONLY WORKS WITH VERSION 4.54 or higher of latexmk
#TBD: take into account: modified:
'$_[0]'->

# makeindex

@ist = glob("*.ist");
if (scalar(@ist) > 0) {
$makeindex = "makeindex -s $ist[0] %O -o %D %S";
}
glossaries new with bib2gls
push @generated_exts, 'glstex', 'glg';
add_cus_dep('aux', 'glstex', 0, 'run_bib2gls');
sub run_bib2gls {
if ( $silent ) {
my $ret = system "bib2gls --silent --group $_[0]";
} else {
my $ret = system "bib2gls --group $_[0]";
};
my ($base, $path) = fileparse( $_[0] );
if ($path && -e "$base.glstex") {
rename "$base.glstex", "$path$base.glstex";
}
# Analyze log file.
local *LOG;
$LOG = "$_[0].glg";
if (!$ret && -e $LOG) {
open LOG, "<$LOG";
while (<LOG>) {
if (/^Reading (.*.bib)\s$/) {
rdb_ensure_file( $rule, $1 );
}
}
close LOG;
}
return $ret;
}
$pythontex = 'pythontexW %R';#'pythontexW %O %R';
push @generated_exts, "pytxcode", "plg";
push @generated_exts, "depytx", "dplg";
$clean_ext .= " pythontex-files-%R/* pythontex-files-%R";
#$extra_rule_spec{'pythontex'}  = [ 'internal', '', 'mypythontex', "%Y%R.pytxcode", "%Ypythontex-files-%R/%R.pytxmcr", "%R", 1 ];
$extra_rule_spec{'pythontex'} = [ 'internal', '', 'mypythontex', "%R.pytxcode", "pythontex-files-%R/%R.pytxmcr", "%R", 1 ];
sub mypythontex {
   my $result_dir = $aux_dir1."pythontex-files-$$Pbase";
   my $ret = Run_subst( $pythontex, 2 );
   rdb_add_generated( glob "$result_dir/*" );
   #my $fh = new FileHandle $$Pdest, "r";
   open( my $fh, "<", $$Pdest );
   if ($fh) {
      print "path: $ENV{PATH}";
      while (<$fh>) {
         if ( /^%PythonTeX dependency:\s+'([^']+)';/ ) {
         print "Found pythontex dependency '$1'\n";
             rdb_ensure_file( $rule, $aux_dir1.$1 );
     }
      }
      undef $fh;
   }
   else {
       warn "mypythontex: I could not read '$$Pdest'\n",
            "  to check dependencies\n";
   }
   return $ret;
}
for htlatex
push @generated_exts, "4tc", "4ct", "tmp", "xref", "css", "idv", "lg";
TBD: for -C remove also html and xhtml
TBD: check that this plugin also removes all these extensions.. think of lg.
biblatex
push @generated_exts, "run.xml";# does run.xml work?
$clean_ext .= " %R-blx.bib";

Answer 1

You can use the command in functional to judge file if exist.

\documentclass{article}
\usepackage{functional}
\usepackage{listings}
\NewDocumentCommand{\LstIfExistInputListing}{m}{\fileIfExistT{#1}{\lstinputlisting{#1}}}
\begin{document}
\fileIfExistT{F4_01fig2dev.ptx}{\lstinputlisting{F4_01fig2dev.ptx}}
or
\LstIfExistInputListing{F4_01fig2dev.ptx}
\end{document}

Answer 2

If you just need to change the F4_01fig2dev(.ptx) in the error message to F4_01fig2dev.ptx,you can do this using:

\documentclass{article}
\usepackage{listings}
\usepackage{xpatch}
\makeatletter
\newcommand*{\NewLine}{^^J}%
\xpatchcmd{\lst@MissingFileError}
  {#1(.#2)' not found.} {#1.#2' not found.\NewLine}
  {%
    \typeout{File ending patch for \string\lst@MissingFileError\space done.}%
  }{%
   \typeout{File ending patch for \string\lst@MissingFileError\space failed.}%
  }
\makeatother
\begin{document}
Test
\lstinputlisting{F4_01fig2dev.ptx}
\end{document}

Here this results in the error message:

! Package Listings Error: File `F4_01fig2dev.ptx' not found.
Type X to quit or <RETURN> to proceed,or enter new name. (Default extension: ptx)

If needed you could also replace "Package Listings" by "LaTeX":

\documentclass{article}
\usepackage{listings}
\usepackage{xpatch}
\makeatletter
\newcommand*{\NewLine}{^^J}%
\xpatchcmd{\lst@MissingFileError}
  {Package Listings Error: File #1(.#2)' not found.} {LaTeX Error: File#1.#2' not found.\NewLine}{%
    \typeout{File ending patch for \string\lst@MissingFileError\space done.}%
  }{%
    \typeout{File ending patch for \string\lst@MissingFileError\space failed.}%
  }
\makeatother
\begin{document}
Test
\lstinputlisting{F4_01fig2dev.ptx}
\end{document}

Which results in:

! LaTeX Error: File `F4_01fig2dev.ptx' not found.
Type X to quit or <RETURN> to proceed,or enter new name. (Default extension: ptx)

In both cases, the default extension is still shown in the error message. But because of the patch, the space token before or is missing. If this is important, you would have to replace the whole message, not only the first line.

Disclaimer: I don't know if an how latexmk triggers to one of these messages, because the corresponding parts of your latexmk configuration are missing in your question, so I cannot test it.

Answer 3

The answer is very close to what Clara suggested. The line of code is

\fileIfExistTF{F4_01fig2dev.ptx}{\lstinputlisting{F4_01fig2dev.ptx}}{\input{F4_01fig2dev.ptx}}

In the first lualatex run F4_01fig2dev.ptx does not exist, but latexmk detects the missing file through \input{F4_01fig2dev.ptx} and creates it by an according rule. In all following runs, F4_01fig2dev.ptx and so \lstinputlisting{F4_01fig2dev.ptx} is used.

It is not really satisfactory, because of course, the change from \input{F4_01fig2dev.ptx} to \lstinputlisting{F4_01fig2dev.ptx} changes the decomposition into pages causing additional lualatex runs.

Also it polutes the code a bit (lstinputlisting comes with a lot of parameters in my case. )

So I would maybe prefer a better solution.