5

MikTeX provides access to the latest binaries through the miktex-nxt update channel. While being windows-only all these years, in 2018, it looks like Mr Schenk is extending his distribution to certain GNU/Linux distros. The problem is that I am not on one of the supported distros and therefore using TL2018 on my linux distro.

The question of updating TL binaries was asked before here for which no satisfactory answer was provided. The only answer there does not provide an actual answer to the OP's question, but instead simply says that the recommended way is to wait for next release of TexLive. The closest explanation for this comes from a comment by Joseph Wright, who says it is non-trivial and that drop-in binaries for non-Windows versions do not exist.

I find all this to be very confusing. There is certainly a binary available for GNU/Linux from the contextgarden/luatex development team here. What is the installation difficulty with this? Why is miktex able to do it while texlive cannot? I am looking for technical answers.

For example, as of today (31/Aug/2018), the latest luatex version is 1.09, but that included with TL 2018 is v1.07. Looking at the reference manual, luatex 1.09 does seem to indicate a performance boost. Furthermore, a new lightweight pdf library is promised. What is the recommended way to update if one does not want to wait until TL2019 is released (which will jump to luatex v1.10)?

I am asking because I have a large thesis with custom lua commands that takes nearly 7-8 minutes to compile on a fast machine and any performance boost to speed this up is welcome.

Dr Krishnakumar Gopalakrishnan
  • 3,117
  • 4
    it;s not that hard to compile the binary from source and drop it into texlive or (I assume) you could use the one from context distrib and similarly copy it in to place. (I don't suppose it helps to know that when I started 10-15 minutes per page was quite common so 8 minutes for a thesis seems pretty fast:-) – David Carlisle Aug 31 '18 at 13:25
  • 3
    note there are good reasons why texlive doesn't update binaries mid-year or encourage users to do so, notably latex isn't extensively tested with the dev builds of luatex (at least the regression tests running on every checkin do not use that) so if (as is often the case) a major luatex update requires format changes to match, you may be "on your own" if you update luatex ahead of the main texlive update. – David Carlisle Aug 31 '18 at 13:27
  • 3
    Concentrate on writing your thesis (focus on the content) and don't waiste time hunting for minuscule performance increases of the tools you use. – Martin Schröder Aug 31 '18 at 13:36
  • 1
    The binaries for ConTeXt are the ones that go into TL (they are compiled by Mojca): you could just grab those if you are desperate. – Joseph Wright Aug 31 '18 at 14:05
  • @JosephWright thank you. I see a luatex binary in the ConTeXt download area. How do I obtain the corresponding binary that works with the latex format, i.e. how to generate the lualatex executable? Is lualatex merely a symbolic link to luatex. If so, how does it work with the context format? – Dr Krishnakumar Gopalakrishnan Aug 31 '18 at 14:07
  • @DavidCarlisle thank you for pointing out about the lack of rigorous testing. – Dr Krishnakumar Gopalakrishnan Aug 31 '18 at 14:08
  • @MartinSchröder thank you for the well-meaning comment. Yes, I have nearly written the document, and it is time to make some fine-tuning of stuff. – Dr Krishnakumar Gopalakrishnan Aug 31 '18 at 14:09
  • 1
    @Krishna Exactly the same as you always would: fmtutil-sys --byengine luatex. – Joseph Wright Aug 31 '18 at 14:10
  • 1
    lualatex is just a symbolic link to luatex so you just need to update luatex (save the old copy first:-) then rebuild the formats as @JosephWright just showed – David Carlisle Aug 31 '18 at 14:10
  • @JosephWright thank you. I am going to try this out in a while. – Dr Krishnakumar Gopalakrishnan Aug 31 '18 at 14:11
  • @DavidCarlisle If it is mereley a symlink, how does this same executable work with the context format? I guess fmtutil will make luatex understand the required formats. – Dr Krishnakumar Gopalakrishnan Aug 31 '18 at 14:12
  • 2
    programs know the name under which they are called (argv[0] in C speak) – David Carlisle Aug 31 '18 at 14:13
  • Note that by coincidence luatex 1.08 was announced today http://tug.org/pipermail/luatex/2018-August/006912.html so if you install 1.09 you are well into untested development code. Are you sure you want to use such an installation for your thesis? – David Carlisle Aug 31 '18 at 19:31
  • @DavidCarlisle that looks like a typo to me. As per the roadmap 1.09 was supposed to be released at the conference next week in Prague. – Dr Krishnakumar Gopalakrishnan Aug 31 '18 at 19:55
  • @DavidCarlisle yup. That looks like a typo. As per the official luatex website, "version 1.08 was released shortly after TeXLive 2018 and has some low level changes, most noticeably reworked direction nodes. The interfaces are downward compatible. This version also default to Lua 5.3 which gives a small performance boost." How do we alert Hans, Taco or whoever is the maintainer of that changelog to the typo? – Dr Krishnakumar Gopalakrishnan Aug 31 '18 at 19:57
  • just reply to that message on the list I guess, but even if 1.09 is released today I wouldn't use it for a thesis until it's been tested in texlive. It's Ok to run some test documents on the side, but a big risk to use on a document that you actually want. – David Carlisle Aug 31 '18 at 20:18
  • @DavidCarlisle The version numbering on the announcement was correct. Apparently, v1.08 was in context since April 2018 for testing purposes, and they made an official release now. Luigi has also updated the website with this information. – Dr Krishnakumar Gopalakrishnan Sep 01 '18 at 12:19

1 Answers1

7

tl;dr Download LuaTeX 1.08.0 from the releases page of my LuaTeX repository on GitHub.

Setup Travis CI for your own fork of LuaTeX

It is very easy to build LuaTeX yourself using GitHub and Travis CI.

  1. First of all you will need a GitHub account and associate this account with Travis CI by visiting https://travis-ci.org/ and logging in with GitHub.

  2. Then you fork the official LuaTeX mirror: https://github.com/TeX-Live/luatex/fork

  3. Go to https://travis-ci.org/<your-username>/luatex and activate Travis CI for this repository.

  4. Generate a personal access token with visibility public_repo on https://github.com/settings/tokens

  5. Register the access token in the section Environment Variables on https://travis-ci.org/<your-username>/luatex/settings under the name GH_TOKEN.

  6. Tag a new releases on your LuaTeX fork https://github.com/<your-username>/luatex/releases/new. Travis CI will automatically build the release and deploy the binaries to GitHub.

Build LuaTeX locally using Docker

If you don't want to create a GitHub account, you can also build LuaTeX locally.

  1. Clone the GitHub repository.

    git clone --depth 1 --single-branch https://github.com/TeX-Live/luatex

    The repository is huge, so we only clone the latest commit and a single branch.

  2. Go to the cloned directory.

    cd luatex

  3. Execute these commands to avoid timestamping issues.

    find . -name \*.info -exec touch '{}' \;
touch ./source/texk/web2c/web2c/web2c-lexer.c
touch ./source/texk/web2c/web2c/web2c-parser.c
touch ./source/texk/web2c/web2c/web2c-parser.h

    Otherwise the build will crash with makeinfo: command not found

  4. Run the build script. You need bash, gcc, g++, and make for that.

    ./build.sh --parallel --jit

    This will build LuaTeX and LuaJITTeX. The binaries will be generated in the subfolder build/texk/web2c.

    Alternatively, if you can build LuaTeX in Docker.

    docker run -e JOBS_IF_PARALLEL=`nproc --all` \
    -v `pwd`:/luatex -w /luatex -it --rm debian:jessie sh -c \
    "apt-get update; apt-get install -y --no-install-recommends bash gcc g++ make; ./build.sh --parallel --jit"

Using my experimental TeX Live repository

If you want to go bleeding edge and always have the latest version of LuaTeX to enjoy all the nice bugs and random crashes, you can use my personal luatex-dev repository for TeX Live. Instructions on how to install are given on the GitHub page:

https://github.com/hmenke/texlive-luatex-dev

Henri Menke
  • 109,596
  • thank you. I was able to get the latest binaries from the project's download area for linux x86_64 and regenerate the formats successfully using fmtutil. However, v1.08 is unable to compile even the simplest of hello world documents. See here for a very simple example and the problems reported in the log file. – Dr Krishnakumar Gopalakrishnan Sep 01 '18 at 12:39
  • @Krishna yes that is a known issue, you need to install a forked version of luaotfload – David Carlisle Sep 01 '18 at 12:55
  • @DavidCarlisle Yup. Got it. I downloaded Ulrike's version of LuaOTFload and everything works now!. Is this documented anywhere? – Dr Krishnakumar Gopalakrishnan Sep 01 '18 at 12:56
  • No (other than in Ulrike's fork) it will of course work "out of the box" by the time there is a supported release, but if you want to build untested development builds of lualatex you need to be ahead of the documentation sometimes. I really strongly recommend that you only use this for testing and not for any important documents. @Krishna – David Carlisle Sep 01 '18 at 12:59
  • @DavidCarlisle I understand the conservative advice given to general audience. I am in computational modelling and like to work on bleeding edge software. I just tried v1.08 with Ulrike's fork and everything works fine (about 30% faster) on my 250 page PhD thesis. (Sadly, Imperial doesn't care whether I used LuaTeX 1.07 or v1.08.....In fact, they probably do not care if about LuaTeX at all)....Curiously, v1.08 (in testing since April) has been merged into the stable download of context mkIV. A new context user will get the benefit of the new PDF library and lua 5.3. I wonder why TL is behind. – Dr Krishnakumar Gopalakrishnan Sep 01 '18 at 13:10
  • @Krishna TL were asked not to ship that version for tl2018 as it wasn't ready and they don't change binaries mid year. It is of course up to you but there has, to a good approximation, been no testing of luatex post 1.07 with latex at all, so if you are using a lualatex based on 1.08 for your thesis that will be one of the first documents to use that, if you are Ok with that that's fine: it is your document, but it is far from being a supported version of latex at the present time. – David Carlisle Sep 01 '18 at 13:20
  • @DavidCarlisle thank you for your explanation. Much appreciate the way you took the effort to answer a question on out of topic/unsupported software. I cannot thank you enough for this and generally all the contributors who answer questions tirelessly here. – Dr Krishnakumar Gopalakrishnan Sep 01 '18 at 13:24
  • I cloned and added Ulrikes fork (https://github.com/u-fischer/lua-font-pond) as an aux-tree, compiled luatex 1.08, put it in the right place, renewed all formats, everything works, and at least the second compilation of a *.tex file is reasonably fast. So for all the folks pondering the issue: Yes, ok, maybe we get a little bit closer to bid farewell to pdfTeX. – Keks Dose Sep 02 '18 at 10:04
  • 1
    @Krishna If you always want the latest LuaTeX you can also use my personal repository for experimental LuaTeX. I added a section about it to my answer. – Henri Menke Sep 16 '18 at 05:54