21

Reading through pi.tex, I discovered the trick of using \fontdimen as an array of dimensions (which is equivalent to storing integers between ±(2^31-1)sp). Namely,

\font\x=cmr10 at 1sp % or whatever font that is not used in the document.
\fontdimen 10000 \x = 1pt
\showthe\fontdimen 10000 \x % => 1pt
\bye

This method of storage appears to be very efficient memory-wise (in TeXLive 2012, pdfTeX and XeTeX are happy to store a bit less than 3000000 such parameters, LuaTeX extends memory as needed), and speed-wise (I benchmarked access and modification to be just as fast as for usual dimen registers, once the space has been allocated).

The main drawback I see is that the size of the array must be declared as soon as the new font is introduced (or rather, before another font is defined):

\font\x=cmr10 at 1sp
\font\y=cmr10 at 2sp
\fontdimen1000\x=1pt % => error: '\x only has 7 fontdimen parameters'
\bye

Are there any other drawbacks to using font memory for the storage of large amounts of data? Is it frequent for a document to reach the bounds of […]TeX's font memory (in which case I shouldn't use it for other purposes than fonts)? If not, I am thinking of using this technique for l3regex, and perhaps to store some tables of values for l3fp.

Bruno Le Floch
  • 44,937
  • The TeXbook puts no limits on how many additional font parameters can be created, other than as you've noted requiring that they are created as soon as the font is loaded. The obvious downsides are that these are global assignments (so good for constants but not so great for other things), and that you need to avoid any 'real' font or pick high-numbered parameters so you are 'safe'. But you already know all of this ... – Joseph Wright Jan 28 '13 at 08:39
  • @JosephWright I think it is useful to collect all downsides in a single answer (including the few that I mention, and the fact that assignments are global). Actually, I am wondering what the high-numbered \fontdimen parameters are used for, except for that trick. Should I ask a separate question? Also, there may be a risk that the values of those parameters are included in the resulting pdf? – Bruno Le Floch Jan 28 '13 at 15:50
  • Knuth left the font dimensions flexible: after all, you need more for a math font than a text font, and LuaTeX/XeTeX are using this to backport the additions Microsoft have made to this area. I guess Knuth envisaged more developments/specialism in the same way he imagined people might create new types of whatsit, which did not really happen. – Joseph Wright Jan 28 '13 at 16:18
  • Reading through tex.web I see another potential pitfall: the extra fontdimen parameters are included in the format. For l3regex I would be using the fontdimen parameters as (global) scratch space, which means that l3regex should not be used to help setting up the kernel (not much of a problem, I guess). – Bruno Le Floch Jan 30 '13 at 08:53
  • Not an issue now, but you can imagine that we might put l3regex into the kernel at some point simply to avoid the 'now load X' business you have in LaTeX2e: easier if all of these things are 'out of the box'. That said, I'm not clear on how we might div up 'thinks in a format file' and 'thinks loaded after the format file'. – Joseph Wright Jan 30 '13 at 08:55
  • @JosephWright Including the code of l3regex in the format file won't be a problem in this approach, but running l3regex functions before dumping would be a problem. I don't see us needing those functions to construct kernel-level things. – Bruno Le Floch Jan 30 '13 at 09:03

1 Answers1

5

It's possible to do this, the main disadvantage of course is they are global. Having loaded the font you can use other parts of its structure as well. This old TUGBoat paper of Jonathan Fine's discusses using the ligature tables of such fake fonts to encode state machines.

Joseph Wright
  • 259,911
  • 34
  • 706
  • 1,036
David Carlisle
  • 757,742
  • This article is very interesting, thank you. A couple follow-up questions: is it possible to edit the ligature table from within TeX (I think not, except perhaps in LuaTeX), or \write a tfm file for TeX to read back? – Bruno Le Floch Feb 03 '13 at 17:20
  • no to both: well for the latter you could write the ascii pl version of the tfm and then \write18 a call to pltotf to make a tfm file. – David Carlisle Feb 03 '13 at 17:44
  • The main question if we want to write tfm files is to know if a valid tfm file can end with a trailing new line. Can it? Building the tfm file could reasonably be done from TeX, I guess, since we would only use a very restricted set of features. – Bruno Le Floch Feb 05 '13 at 08:29
  • the tfm itself is binary you couldn't write that (could you?) but the pl file can have trailing white space. – David Carlisle Feb 05 '13 at 12:19
  • I'd rather avoid the need for shell escape, so writing a pl file would not be good enough. You can write binary data with pdftex -8bit and with luatex, not with xetex (since that engine can only write valid utf8). See http://tex.stackexchange.com/questions/8729/ . Would another font file format be "less binary"? The pfb I can find in my system seem to have some non-binary parts. – Bruno Le Floch Feb 05 '13 at 12:44