How do LaTeX macros interact with the TeX engine?

Question

From wiki on LaTeX

the basic requirement is to have a TeX compiler (which is used to generate output files from source), fonts, and the LaTeX macro set.

Assuming I have a TeX engine and I would like to add to it LaTeX support. I would expect it to be similar to any language where I should #include or import or whatever is the equivalent, all the LaTeX macros, which are written already in the TeX language. Then, when I have to build a LaTeX written file, I would first have to pre compile the macro files (to be able to use LaTeX tags) and after the file the user has written.

I did find the svn repository of the latest LaTeX release. But I am confused why there is a build.lua file? What exactly is it building?

Can someone tell me if my logic regarding adding LaTeX support is correct? If not, how exactly should I proceed? I have a problem understanding the global picture and mainly how LaTeX macros interact with the TeX engine.

More or less. Including the latex macros is called "creating a format" in a tex system. The tex system will do it for you at installation time and create appropriate wrapper commands (e.g. pdflatex or lualatex). — Ulrike Fischer, Jun 25 '15 at 13:05
@UlrikeFischer can you expend the part tex system will do it for you? What exactly will it do? i.e. how a format is created? — Alexandru Barbarosie, Jun 25 '15 at 13:09
It creates 'formats' which are sets of pre-compiled macros. This gets done when you install TeX or when you update the installation (if required). So if you install TeX Live, for example, when you update it using the TeX Live Manager, various things are done automatically. One of those is the recreation of formats if that is necessary. — cfr, Jun 25 '15 at 13:12
When you compile a document, you set the appropriate engine and format by using the appropriate wrapper command name. For example, if you say pdflatex <filename>.tex then the pdfTeX engine will be used and the LaTeX format. If you say lualatex <filename>.tex then LuaTeX will be used with the LaTeX format. Roughly. — cfr, Jun 25 '15 at 13:16
A format is created (more or less) with the command pdftex --ini pdflatex.ini. This inputs the macros and outputs the pdflatex.fmt. All texsystems have scripts to create the standard formats (which explains why I'm not sure about the exact syntax -- it's long ago that I had to do it manually.) — Ulrike Fischer, Jun 25 '15 at 13:16
@cfr I am somewhat familiar with what you said, I am trying to understand how it works internally, what is under the hood. How do this wrappers use the LaTeX macros and the TeX engine; how do the two interact. — Alexandru Barbarosie, Jun 25 '15 at 13:30
@UlrikeFischer and how can one use this .fmt files after they are created? — Alexandru Barbarosie, Jun 25 '15 at 13:32
Maybe take a look at texmf-dist/scripts/texlive/fmtutil.pl? Other scripts in that directory may also be of interest. — cfr, Jun 25 '15 at 13:48
http://ctan.org/pkg/texlive-source might be of interest, too. — cfr, Jun 25 '15 at 13:54
Welcome to TeX.SX! You can have a look at our starter guide to familiarize yourself further with our format. — Martin Schröder, Jun 26 '15 at 00:03
See also http://www.seanallred.com/tex/2015/05/25/tex-terminology.html — Sean Allred, Jun 26 '15 at 04:19
I'm not sure about the dupe here: this question seems to be specifically about format files — Joseph Wright, Jun 27 '15 at 09:03
@JosephWright yes, it has nothing to do with the duplicate, I am trying to figure out what are the existing TeX engines doing under the hood with format files, and so far I haven't seen any info regarding it on this forum. — Alexandru Barbarosie, Jun 27 '15 at 12:30

Joseph Wright · Accepted Answer · 2015-06-27T09:01:51.527

A TeX engine in the initial state ('IniTeX') knows only the primitives and a very small number of pre-defined settings. For almost all practical use you want a set of macros on top of this. That can be achieved by doing

tex -ini

and then typesetting them in interactively, but is much more practically achieved by having them saved into a file. You could read that file at the start of every run you did, so start your input with

\input mymaros\relax

or similar.

In most real cases this standard file will end with the primitive \dump: that ends the TeX run and saves the current state of TeX's memory so it can be reread later. Your 'real' document then does not ever need to worry about those pre-defined macros: they are loaded at start up from the dump file. Historically, you would have told TeX to read a dump using

tex &<dump-file-name>

so for example

tex &latex

for the LaTeX format (latex.fmt). However, you don't need to do that as the command latex is available as more-or-less equivalent to the same thing. (There are technical differences, and importantly a modern latex will be more akin to pdftex -output-format=dvi &latex.)

Note that a format file is a memory dump. As such, it is only usable by exactly the same binary that made it: same OS, same version, etc. As such, format files are not distributed, but are created on the system where they are used.

If you look at the LaTeX Team SVN you will find a variety of files concerned with the process of building and testing LaTeX for release. Our build system, l3build, does make format files for local use in testing. However, this is not the same as making a format file for a user. Indeed, the official release of LaTeX, like most other TeX software, is via CTAN, and we do not send the build system material there (users do not need it).

Most end users don't get material directly from CTAN in any case: they get LaTeX as part of a TeX system, likely TeX Live or MiKTeX. These systems do the format building and saving automatically, and users therefore very rarely need to be concerned with the detail.

So roughly speaking any existing tex software comes already with a latex.fmt dump file? I checked CTAN and from what I understand LaTeX is composed of multiple required packages, all of them composed of multiple .ins and .dtx files, so then how does the software put them together in a single .fmt file? i.e. could you please elaborate more on what this dump file actually is? — Alexandru Barbarosie, Jun 27 '15 at 08:32
@AlexandruBarbarosie The latex.fmt format contains only the LaTeX kernel. Packages are add ons to LaTeX and are not built-in to the format, but are instead loaded using \usepackage. The latter reads a .sty file, which is just a plain text file. Often (but not a requirement) .sty files re extracted from .dtx sources using an .ins file. All of these are plain text with different extensions just for convenience. — Joseph Wright, Jun 27 '15 at 09:05

How do LaTeX macros interact with the TeX engine?

1 Answers1