Externalization to other format, Makefile. Add new rules to the makefile

Question

Through using Christian's method of externalization, shown below, one can convert the pdf's to any chosen format right after the externalization at hand.

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{external}
\tikzexternalize[mode=list and make]
\tikzset{
    % Defines a custom style which generates BOTH, .pdf and .png export
    png export/.style={
        external/system call/.add={}%
        {; convert -density 300 -transparent white "\image.pdf" "\image.png"}
    }
}
\begin{document}
\bgroup
\tikzset{png export} % Will be exported to a png as-well
\begin{tikzpicture}
  \draw[red] (0,0) -- (1,1);
\end{tikzpicture}
\egroup
\begin{tikzpicture}
  \draw (0,0) -- (1,1);
\end{tikzpicture}
\end{document}

The above will externalize two images. One of them (the first) will also be created as a png. To create the images you also need to call: make -f \jobname.makefile.

This question should be equivalent on both Linux and Windows as long as you can use Makefiles.

Using this I will always get my png images out immediately. However there is often times where I have large figures and thus the processing takes way more time because of the png creation (first TeX -> Pdf, then Pdf -> Png, then next picture).

I always use the key external/mode=list and make to make/re-make the images as needed upon data change etc. This makes it more robust for me.

So how do one externalize frames and add the proper information to the Makefile so that one can make the pdf and the png's separately, however the png's need to have a dependency on the pdf?

PS. I found a method, but would like others to join in on possible other solutions. I will supply my answer along the others in a couple of days.

Answer 1

I can see that maybe my question is very restricted, in terms of user base, or there is simply not that many that have been dealing with this issue.

So I will here try to elaborate on the use of it.

Why is it needed/implemented

The need for quick typesetting is in interest for everyone. How often have you not recompiled a document to see if your equation gets typeset correctly, or that you need another final touch to adjust the spacing, etc.

This means that you would like to compile within the second(s). For this to happen, in a fairly large document, you have several choices:

• Make a separate document with your page settings and set in your equation, process, edit, process edit.
• Force your current ship-out in TeX to disregard any images or tikzpicture or any other lengthy macro (I refrained from using long, as it would probably be mistaken with \long :) ). I believe that I have heard of such packages although I have never used them myself.
• fill in your own method, my method is filed below

For repeated use of the above everybody wishes this switch to be fast and easy. In this regard my fix is quick.

How do I use the externalisation

My use of externalisation has only to do with the tikzpicture environment. However, this idea could be used in other contexts.

I love Makefiles. I will not lie! They make my life easier.

This is why I quickly fell for the method of externalization by using Makefiles. All you need to do is:

\usepackage{tikz}
\usetikzlibrary{external}
\tikzexternalize[mode=list and make]

This will do the following:

1. On first compilation every tikzpicture gets a designated name and a corresponding entry of the job-call gets added to a separate makefile called: \jobname.makefile.
   • Instead of actually compiling the tikzpicture in a separate command, as would the external library do it ordinarily, it will insert a text showing that the image has been discarded:

     [[ Image Discarded Due To `/tikz/external/mode=list and make' ]]

2. On the second run the same thing will happen, the image will not be typeset.
3. You have to call the command make -f \jobname.makefile in order to generate the pictures.

So what is the purpose?

First of all using the makefile method you can more easily setup dependencies (which the library sets up for you!). Say you use pgfplots and you use \addplot table {data.dat}; the dependency scheme will automatically derive that that figure needs only be recompiled when the file data.dat changes.
How convenient!

The implementation even allows for custom setup of dependencies if you require files in other ways. See the command \tikzpicturedependsonfile.

Further a key feature of this is that one can now utilize the power of multiple processors as one is allowed to call: make -j 4 -f \jobname.makefile which will run parallel on 4 processors (each compiling their own figure). This is also mentioned in the manual.
Now that is one swift way of compiling your document.

What is then needed

Further down the line of your compilation/production of the document you find that it would be nice to have all the figures created in png's or any other format. The manual also describes how this can be done.

Now what I found was that it took immense times if I had large images and wanted to convert them to png's. This was simply because I either had to use the conversion to pdf's and png's or only pdf's.

What I wanted was to decisively say now I want png's or now I want both.

Hence, I scoured around and had to find my own way of doing it. The solution is to add the commands to the Makefile as the other macros are done.

I thus made a couple of keys added to the external library which easily added the functionality of my need.

\makeatletter
\tikzset{%
    external/mode=list and make,
    % Make the keys for automatic generation of images
    external/new make rule/.code args={make #1; ext .#2}{%
        % Add the rules for the new format
        \tikzexternalwritetomakefile{}%
        \tikzexternalwritetomakefile{ALL_FIGURES_#2=\tikzexternal@DOLLARchar(ALL_FIGURE_NAMES:\tikzexternal@PERCENTchar=\tikzexternal@PERCENTchar .#2)}%
        \tikzexternalwritetomakefile{}%
        \tikzexternalwritetomakefile{#1: \tikzexternal@DOLLARchar(ALL_FIGURES_#2)}%
        \tikzexternalwritetomakefile{\tikzexternal@TABchar @echo All #2 images exist now. Use make -B to re-generate them.}%
        \tikzexternalwritetomakefile{}%
    },
    external/add fig rule/.code args={.#1 depends: .#2,cmd:#3}{%
        % Generate the extra system call for the format
        \tikzset{%
            % The \@firstoftwo Removes the \tikzexternal@TABchar, which is inserted
            external/system call/.add={}{\noexpand\@firstoftwo ^^J},
            external/system call/.add={}{\noexpand\@firstoftwo ^^J\image.#1: \image.#2},
            external/system call/.add={}{^^J#3}
        }
    }
}
\makeatother

This piece of code lets you call:

\tikzset{
    external/new make rule={%
        make allimagespng; ext .png
    },
    external/add fig rule={%
        .png depends: .pdf,cmd:convert -density 300 "\image.pdf" "\image.png"
    },
    external/new make rule={%
        make allimagesjpeg; ext .jpeg
    },
    external/add fig rule={%
        .jpeg depends: .pdf,cmd:convert -density 300 "\image.pdf" "\image.jpeg"
    }
}

which first makes a new rule called allimagespng. The next makes the rule for the individual files, i.e. makes pngs created from the pdf's. Then I also have the same for jpeg. In principle you can do this as much as you like.

Try it out and check out the \jobname.makefile.

A MWE

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{external}
\tikzexternalize[mode=list and make]
\makeatletter
\tikzset{%
    external/mode=list and make,
    external/new make rule/.code args={make #1; ext .#2}{%
        \tikzexternalwritetomakefile{}%
        \tikzexternalwritetomakefile{ALL_FIGURES_#2=\tikzexternal@DOLLARchar(ALL_FIGURE_NAMES:\tikzexternal@PERCENTchar=\tikzexternal@PERCENTchar .#2)}%
        \tikzexternalwritetomakefile{}%
        \tikzexternalwritetomakefile{#1: \tikzexternal@DOLLARchar(ALL_FIGURES_#2)}%
        \tikzexternalwritetomakefile{\tikzexternal@TABchar @echo All #2 images exist now. Use make -B to re-generate them.}%
        \tikzexternalwritetomakefile{}%
    },
    external/add fig rule/.code args={.#1 depends: .#2,cmd:#3}{%
        \tikzset{%
            external/system call/.add={}{\noexpand\@firstoftwo ^^J},
            external/system call/.add={}{\noexpand\@firstoftwo ^^J\image.#1: \image.#2},
            external/system call/.add={}{^^J#3}
        }
    }
}
\makeatother
\tikzset{
    external/new make rule={make allimagespng; ext .png},
    external/add fig rule={.png depends: .pdf,cmd:convert -density 300 "\image.pdf" "\image.png"}
}
\begin{document}
\begin{tikzpicture}
  \begin{axis}
    \addplot coordinates {(1,20)(2,23)(3,24.5)};
  \end{axis}
\end{tikzpicture}
\end{document}

Hope this clarifies what I was searching for. If any body has any comments or better ways to do it please add!!! :)