Inclusion of graphics with on-the-fly conversion using XeTeX

Question

I would like to make \inclugraphics{file.ext} perform on-the-fly conversion of graphics files with unsupported formats, e.g. .ext is .svg or .gif.

This use case happens at least with the package moodle.sty, where users might want to include local graphic files adapted to the web (SVG, GIF), and have them passed as-is to the XML output. This XML output contains a question bank to be imported in the Moodle LMS for building quizzes. In order to keep the process as simple as possible for the user, I would like to avoid requiring that two versions of the graphic file are manually kept synchronized locally (one for the XML, one for the standard LaTeX output file).

Here is a MWE that compiles without error and does what I expect with: Linux, ImageMagick for conversion, TeX Live 2020 and both compilation calls pdflatex -shell-escape MWE.tex and lualatex -shell-escape MWE.tex.

\documentclass{minimal}
\usepackage{graphicx}
\makeatletter
\edef\Gin@extensions{\Gin@extensions,.svg}%
\DeclareGraphicsRule{.svg}{png}{.png}{%
    `convert '#1' \noexpand\Gin@base-svg-converted-to.png
}%
\makeatother
\begin{document}
\begin{filecontents*}{pict.svg}

<?xml version="1.0" encoding="UTF-8"?>
<svg width="10" height="10" version="1.1" viewBox="0 0 10 10" 
xmlns="http://www.w3.org/2000/svg">
<circle cx="5" cy="5" r="4"/>
</svg>

\end{filecontents*}
This is a black disk: \includegraphics{pict}
\end{document}

Unfortunately, I am failing to make this work with XeTeX. Here are parts of the log file when using \tracingmacros1

This is XeTeX, Version 3.14159265-2.6-0.999992 (TeX Live 2020/Debian) (preloaded format=xelatex 2020.10.2)  11 DEC 2020 11:20
entering extended mode
 \write18 enabled.
 %&-line parsing enabled.
**MWE.tex
(./MWE.tex
LaTeX2e <2020-02-02> patch level 5
L3 programming layer <2020-07-17>
...
\Gin@ext ->.svg
\Gin@rule@.svg #1->{png}{.png}{`convert '#1' \Gin@base -svg-converted-to.png }
#1<-\Gin@base \Gin@ext
\Gin@setfile #1#2#3->\ifx \#2\\Gread@false \fi \ifGin@bbox \else \ifGread@ \c
sname Gread@\expandafter \ifx \csname Gread@#1\endcsname \relax eps\else #1\fi 
\endcsname {\Gin@base #2}\else \Gin@nosize {#3}\fi \fi \Gin@viewport@code \Gin@
nat@height \Gin@ury bp\advance \Gin@nat@height -\Gin@lly bp\Gin@nat@width \Gin@
urx bp\advance \Gin@nat@width -\Gin@llx bp\Gin@req@sizes \expandafter \ifx \csn
ame Ginclude@#1\endcsname \relax \Gin@drafttrue \expandafter \ifx \csname Gread
@#1\endcsname \relax @latex@error {Can not include graphics of type: #1}@ehc 
\global \expandafter \let \csname Gread@#1\endcsname @empty \fi \fi \leavevmod
e \ifGin@draft \hb@xt@ \Gin@req@width {\vrule \hss \vbox to \Gin@req@height {\h
rule @width \Gin@req@width \vss \edef @tempa {#3}\rlap { \ttfamily \expandaft
er \strip@prefix \meaning @tempa }\vss \hrule }\hss \vrule }\else @addtofilel
ist {#3}\ProvidesFile {#3}[Graphic file (type #1)]\setbox \z@ \hbox {\csname Gi
nclude@#1\endcsname {#3}}\dp \z@ \z@ \ht \z@ \Gin@req@height \wd \z@ \Gin@req@w
idth \box \z@ \fi 
#1<-png
#2<-.png
#3<-`convert '\Gin@base \Gin@ext ' \Gin@base -svg-converted-to.png
\Gread@eps #1->\Gread@generic {#1}\Gread@eps@aux 
#1<-\Gin@base .png
...
->\errmessage  LaTeX Error: File `\Gin@base .png' not found.
See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help@err@
\Gin@base ->pict
@err@                                                                  ->
! LaTeX Error: File `pict.png' not found.

Since the file pict.png is not created, I guess that the call to convert is not made.

Here is what I tried unsuccessfully:

• removing the part -svg-converted-to in the \DeclareGraphicsRule does not help (still the conversion call is not made I guess)
• epstopdf does not seem to help
• solutions involving the QuickTime library seem to be macOS-only, see here and there

My questions are:

• Am I missing something obvious?
• Why is the conversion call not made when compiling with XeTeX?
• Is there something we can do specifically for XeTeX to achieve what the MWE does with pdfTeX and LuaTeX?

Answer 1

Here is an attempt to answer my own question after receiving help from Ulrike Fisher and David Carlisle.

Disclaimer: running TeX engines with the -shell-escape option is to be considered risky for general use. When doing this, you should either trust the files and executables on your system or use a sandboxed environment for compilation.

Am I missing something obvious?

Well... Maybe I will learn not to expect too much from things that are not documented. The grfguide says that the last argument of \DeclareGraphicsRule is usually empty and shows an example where this argument is a system command (zcat...) preceded with a backtick (`). It is not said that it will work with any command nor any TeX engine.

Why is the conversion call not made when compiling with XeTeX?

As David Carlisle and Joseph Wright pointed out, the backtick syntax never worked with XeTeX. This engine performs no graphics conversion by default and outputs a file in the XDV format. A PDF file is obtained after a processing by dvipdfmx. In this separate pass, if necessary, graphics are converted. The graphics bundle defines no interface to pass conversion commands from TeX to dvipdfmx.

Is there something we can do specifically for XeTeX to achieve what the MWE does with pdfTeX and LuaTeX?

Playing with graphics internals, I came up with a solution that seems to work with latex, pdflatex, xelatex and lualatex. The interface of the command \DeclareGraphicsAlien is inspired by \DeclareGraphicsRule and epstopdf's\epstopdfDeclareGraphicsRule. The three arguments are:

1. the alien extension (including leading dot) that is declared,
2. the target native extension (including leading dot) to map it to,
3. system command for conversion using \SourceFile and \OutputFile for referring to the source file (alien format) and the result of the conversion process (native format).

\usepackage{graphicx}% including pictures
\usepackage{shellesc}% run system commands from TeX engine
\usepackage{etoolbox}% for \csdef, \csletcs
\makeatletter
\def\@firstofthree#1#2#3{#1}%
\def\@secondofthree#1#2#3{#2}%
\newcommand{\DeclareGraphicsAlien}[3]{%
  \edef\Gin@extensions{#1,\Gin@extensions}%
  \DeclareGraphicsRule{#1}{\@gobble#1}{#1}{}%
  \csdef{Gread@\@gobble#1}##1{%
    \edef\SourceFile{\Gin@base\Gin@ext}%
    \edef\Gin@base{\Gin@base-\@gobble#1-converted-to}%
    \edef\Gin@ext{#2}%
    \edef\OutputFile{\Gin@base\Gin@ext}%
    \edef\targetfmt{\expandafter\expandafter\expandafter
                    \@firstofthree\csname Gin@rule@\Gin@ext\endcsname\relax}%
    \edef\targetext{\expandafter\expandafter\expandafter
                    \@secondofthree\csname Gin@rule@\Gin@ext\endcsname\relax}%
    \IfFileExists{\SourceFile}{\ShellEscape{#3}}{}%
    \csletcs{Ginclude@\@gobble#1}{Ginclude@\targetfmt}%
    \csname Gread@\targetfmt\endcsname{\Gin@base\targetext}%
  }%
}%
\makeatother

(this is guaranteed 100% Cargo Cult Programming, bugs and room for improvement are to be expected)

Now, we can 1) declare an alien format, 2) pair it with a native format of the TeX engine used, and 3) specify the command line to be used for conversion.

Here is an example using two FOSS Swiss Army Knives: ImageMagick and Inkscape, for raster and vector images, respectively. I used the SVG file from the question (named vector.svg here) and this animated GIF (raster.svg).

\usepackage{iftex}% determine TeX engine
\ifnum 0\ifPDFTeX 1\fi\ifpdf 0\fi=1% EPS is a native format with latex
  \DeclareGraphicsAlien{.svg}{.eps}{%
    inkscape --export-type=eps --export-area-page '\SourceFile' -o 
    '\OutputFile' 2>/dev/null}%
  \DeclareGraphicsAlien{.gif}{.eps}{convert '\SourceFile[0]' '\OutputFile'}%
\else% PDF and PNG are native formats with pdftex, xetex, and luatex.
  \DeclareGraphicsAlien{.svg}{.pdf}{%
    inkscape --export-type=pdf --export-area-page '\SourceFile' -o 
    '\OutputFile' 2>/dev/null}%
  \DeclareGraphicsAlien{.gif}{.png}{convert '\SourceFile[0]' \OutputFile}%
\fi
\begin{document}
\begin{itemize}
\item Vector graphic \includegraphics{vector.svg}
\item Rasterized graphic \includegraphics{raster.gif}
\end{itemize}
\end{document}

Calls to latex must be detected because with this engine, PDF and PNG are not native graphics formats, contrarily to pdflatex, xelatex and lualatex.

Only the first frame of the animated GIF is converted (note the syntax [0] to ask this to ImageMagick's convert) and passed to the resulting document.

Additional note I should probably define what I call a native graphics format. I am not comfortable with this. Let say that in your TeX's engine driver for graphics (dvips.def, pdftex.def, xetex.def, or luatex.def), you find lines like \@namedef{Gin@rule@.<XYZ>}#1{{<FMT>}{.<EXT>}{#1}}. I would say that native formats are shown with their (non-empty) extension .<EXT>. For instance, I could not make BMP work as a native format with latex/dvips.