138

Every year TeX Live provides a new version. My recollection is that soon after a new version is released, the old version's tlmgr stops fetching updates to packages. They provide upgrading instructions with the disclaimer

This procedure is not bullet-proof, or especially recommended

There is no upgrade path for Windows. Looking at the release history, it is not clear to me what is actually changing between releases such that a full new install is required/recommended. I always thought of TeX Live as providing useful binaries (e.g., pdflatex, biber, and makeindex) and a convenient way of updating CTAN packages. What am I missing? Why does TeX Live need a new install every year?

cmhughes
  • 100,947
StrongBad
  • 20,495
  • Maybe just because of legal issues? – Mario S. E. Apr 05 '13 at 13:07
  • 1
    I don't think of legal issues, @MarioS.E. Due to legal issues, some packages are removed, if they are not licensed correctly, but this happens automatically during update with tlmgr update command. – Stephan Lukasczyk Apr 05 '13 at 14:39
  • @StephanLukasczyk, I was thinking of legal issues more as in the way Windows always requires you have the latest drivers, the latest service packs and the latest everything to avoid compatibility issues. If you are not in compliance with this, they simply cannot give you any support – Mario S. E. Apr 05 '13 at 14:53
  • @MarioS.E. Didn't get this with my last comment, sorry. This could also be a topic. I remember problems with installing on Vista as the installation was to C:\Program File\texlive by default. If the installation failed and you restared it with same settings, the rest of the files went to C:\Program\texlive. The error was caused from the space in the folder name. Next TL-version went to C:\texlive by default to ship around this. So these issues are things to request a whole reinstallation. – Stephan Lukasczyk Apr 05 '13 at 14:58
  • Binaries are only installed during an upgrade. So if you've got an outdated bibtex/luatex/... only a fresh install of texlive will bring you the current releases. – topskip Apr 05 '13 at 17:28
  • @topskip I am pretty sure biber is a binary and I have upgraded it with tlmgr from 0.9.x to 1.5. What am I missing? – StrongBad Apr 05 '13 at 17:48
  • @DanielE.Shub the non-update policy is probably restricted to the 'main' binaries such as luatex, pdftex and such. Perhaps the update of biber has to do with the fact that it is a perl script converted to "exe", but I don't know. – topskip Apr 05 '13 at 19:24
  • 1
    @DanielE.Shub: biber is special case as the binary is built by its developers not TeX Live builders. – خالد حسني Apr 06 '13 at 09:51
  • 3
    biber is built using the excellent Par::Packer module of Perl (and biber is all perl). The build process is not like any other TL binary as a result and it would have been hard to get all the TL servers updated with the necessary things. So I have a build farm of VMs to do this myself. It makes for a more rapid dev cycle as I control the VMs. – PLK Apr 24 '13 at 18:50
  • See also this answer which clarifies a lot (IMO the true answer to this question -- and it's even by the same author as the accepted answer to this question). – ShreevatsaR Dec 07 '18 at 23:39

4 Answers4

99

Ok, here's an answer as the main developer of tlmgr and the whole TeX Live infrastructure. You have to decide two things:

  • the freeze period before release of a new version
  • upgradability from one release to the next

Concerning the former, freeze period: Normally during the year we do not make updates to the actual binaries, but only to scripts and the packaging. That means, that in preparation of a new release all the binaries have to be recompiled, which is a huge tasks and involved lots of time, and iterations. Bugs are fixed, code adapted so that it builds on all platforms (and there are many!). During this time we do not want to make partial upgrades of some binaries, because sometimes that needs to be accompanied with library file updates.

Another point is that during the freeze period critical new features are sometimes included in the texlive infrastructure and the tlmgr, which would be too dangerous to be released to the world in the normal course.

And finally, it is also about getting into a state that can be pressed onto DVD.

Now for the upgradability between releases: The reason in the first years were changes in the internals that did not allow upgrades (like format of the internal coding of options, etc.). This was in the first years (say 2008-2010) the most common reason. A normal upgrade was simply not trivially possible. Of course, one could write an upgrade script and make an NSIS installer for Windows, but we do not have time for that. We are volunteers and have to concentrate on the important things.

In the last years (say since 2010) there always was an upgrade procedure, although we normally didn't give it a lot of testing; that is the reason why we don't recommend it. Disk space is nowadays quite abundant, and having two installations in parallel is thus not such a pain. But still, there always was the way to upgrade.

On Windows this is unfortunately not so trivial, as the uninstaller and the registry etc etc is linked to release years, it is simply a pain on Windows, but that is a special case.

Finally, one more reason: In many cases in the last year, an update or a new installation would not have changed much in the amount of downloaded data, as often all the packages were updated in one way or the other (due to internal changes), which meant an upgrade would have involved downloading all packages, just like the installation.

I hope that all this makes our intention a bit more clear, and if there is anything unclear, or if someone has better ideas how to deal with it, we are open to suggestions!

barbara beeton
  • 88,848
norbert
  • 8,235
  • 3
    I appreciate this answer from the lead developer. Can you answer the question I posed to @vonbrand in my first comment: why does the entire CTAN tree have to be considered "maintained" by the TeX Live team, to the extent of binding its upgrades to the yearly releases? Whereas with Linux the distro maintainers are to an extent the package maintainers as well, does TeX Live do any patching of packages or just provide them as-is? And if the latter, why not allow continuing updates? – Ryan Reich Apr 06 '13 at 15:39
  • Why not continuous updates: please read my answer, it has nothing to do with the packages itself as they are on CTAN, but with our internal repackaging and format of distribution. We don't change the CTAN packages at all! It is all about internal changes, as I wrote. – norbert Apr 07 '13 at 03:14
  • As in Ryan Reich's comment, it is still not clear to me whether or not TeX Live "maintains" (its versions of) the entire CTAN tree. Specifically, if I had installed TeX Live 2015 (say), and someone updates a package on CTAN in 2017 (or adds a new package), will I be able to update it with tlmgr? (If not, why not? This is the part that is still not clear.) – ShreevatsaR Nov 03 '16 at 16:43
  • 3
    Well, that cannot be answered so easily: For fonts and macros we include everything that is not marked as obsolete and license-wise compatible. Concerning support scripts and programs, there are some requirements (mostly relocability and auto* integration), but other than this the same requirements apply. So in principle, yes, in about 95% of the cases an update of new package on CTAN will end up in TeX Live within a few days. – norbert Nov 03 '16 at 22:36
  • Thanks, that clarifies it somewhat. So I gather that tlmgr is a tool to install those packages on CTAN that have been explicitly selected for inclusion into TeX Live, and not a tool for installing arbitrary packages from CTAN the way CPAN.pm is for installing packages from CPAN, cabal from Hackage, gem from RubyGems, apt from Debian repositories, npm from the npm registry, etc. It is not clear to me what is the need or what are the advantages of this (though I can guess a few), and I think this was crux of question, but this is just my ignorance and I'll figure it out someday :-) – ShreevatsaR Nov 04 '16 at 03:32
  • 1
    Look into how packages look like on CTAN and how they look in TeX Live! Converting from CTAN, that is how the authors uploaded, to TDS is not trivial. – norbert Nov 04 '16 at 03:41
  • PS, feel free to contribute your own installation routine! – norbert Nov 04 '16 at 03:42
  • 5
    Thank you!! I hadn't even heard of TDS before (that's the level of my ignorance), and with that hint I was able to look up links like https://tug.org/tds/tds.html and https://tug.org/texlive/pkgcontrib.html and ctan2tds and ctan2tl and https://www.ctan.org/tex-archive/systems/texlive/tlnet/archive etc., which finally answered my question: …(contd) – ShreevatsaR Nov 04 '16 at 05:32
  • 3
    …(contd): TeX Live does indeed maintain a copy of nearly the entire CTAN tree, because CTAN is not a "proper" package repository; only TeX Live's version is. I've always had a lot of respect for TeX Live but this information shows what a staggering amount of work it is! Thank you… I might write this as an answer to the question sometime, to help others like me (and a few others on this thread I suspect, from the question and comments). This was the most important piece of information that I think those who knew the answer already just took for granted :-) – ShreevatsaR Nov 04 '16 at 05:32
  • 1
    You're welcome. And yes, it is a lot of work, but fortunately we managed to automatize large parts of it so that today most work is done by maybe two, Karl mostly and a bit myself. – norbert Nov 04 '16 at 06:13
51

I can't speak for TeX Live in particular, but I have been involved with Linux distributions from their very beginning, and I believe my observations there are relevant for TeX Live too (both are collections of disparate pieces that have to be selected, integrated and convinced to work nice together, each piece evolves at their own pace).

While in a perfect world such a software collection would be maintained forever, this isn't practical. New packages show up, old packages are overhauled, some packages are superseded by new ways of getting the work done. To maintain, say, TL-2011 would mean to keep "the old way" working, even if upstream has abandoned the package (or changed it so much that it isn't a drop-in replacement anymore). That means endless backporting and fixing old bugs. Volunteers for that are only found among dyed-in-the-wool masochists.

TeX Live maintenance requires knowledgeable people in a rather esoteric area, and those are in very short supply. Either they maintain 2011 forever (and no 2012, 2013, ...) or they concentrate on building and shipping the last version. Besides, working on shiny&new is always more fun, so there is more potential to attract new recruits there.

Perhaps they are enough to keep two versions reasonably up to date, but I guess not (as they have decided that they don't have the manpower, so mandatory yearly upgrade).

barbara beeton
  • 88,848
vonbrand
  • 5,473
  • 2
    Makes complete sense to me – Mario S. E. Apr 05 '13 at 13:15
  • 10
    This doesn't answer, though, why the entire CTAN tree needs to be considered "released" on a yearly basis for TeX Live purposes. Though dependencies are resolved by tlmgr, it would be almost trivial to add a switch "--no-deps" that just installs the explicitly requested packages, and make that switch mandatory in outdated releases. Along with forbidding updates of the stuff that's actually maintained by the TeX Live team, this could keep old releases usable indefinitely without extra work. So I interpret the question as asking: why not do this? – Ryan Reich Apr 05 '13 at 13:56
  • 1
    Rarely, do I have to do a complete reinstall of my Linux distribution. For example, when Debian moves from Squeeze to Wheezy, upgrading will (hopefully) be as easy as apt-get dist-upgrade. When you "upgrade" TeXLive it downloads all of the packages again from CTAN. That just seems silly (but I am guessing there is a reason). – StrongBad Apr 05 '13 at 14:03
  • @RyanReich, the --no-deps and such road leads to what I have installed is completely different from yours, each breaks/malfunctions in endearingly different ways due to missing dependencies. That means a even much greater burden in maintenance than having several versions in parallel. Been there, seen that done (participated a bit myself), seen the resulting suffering. – vonbrand Apr 05 '13 at 14:19
  • @vonbrand: But hey, not maintained, use-at-your-own-risk and all that. If you want a streamlined system, use the current one. I think anyone opting to keep an old version should be informed that only minor updates of few packages are considered safe. The burden of maintenance is on the user at this point, and honestly, that's a fine trade-off for, say, having been able to get pgf-2.10 the other year without upgrading 200M of other packages. – Ryan Reich Apr 05 '13 at 14:27
  • @DanielE.Shub, when I update my Fedora it replaces almost all too. Sure, it is just a fedup-cli run away. I'd be surprised if Debian's apt-get dist-upgrade is much different. And keeping that kind of infrastructure running smoothly isn't trivial (witness the birth woes of Fedora 18, several times delayed precisely for bugs in installing/updating). Offering clean install and upgrade from whatever mess the user has is very, very hard. Have the enthusiastic manpower to do so? Go ahead! But I guess it isn't there. – vonbrand Apr 05 '13 at 14:29
  • @RyanReich, if you want not maintained, install the packages by hand. That way it is made doubly clear that if you break it, you keep the pieces. Plus TL people don't get to maintain fragile functionality that just causes headaches later on. Win-win. – vonbrand Apr 05 '13 at 14:31
  • @vonbrand Definitely lose-win. Installing TeX packages by hand is a major hassle. That's exactly why I want a script like tlmgr. – Ryan Reich Apr 05 '13 at 15:01
  • @RyanReich, TL folks win by (1) not having to create the infrastructure for random updates, (2) not having to triage bug reports for random assemblages of packages. Win-win. – vonbrand Apr 05 '13 at 15:46
  • 2
    That's not what win-win usually means... – Ryan Reich Apr 05 '13 at 17:20
  • 4
    Sorry to say, but concerning TeX Live the above fits only partially. We maintain TeX Live without a break, it should be much more considered a continuous development with short breaks in preparation for new binaries and DVD. What is true is that if someone wants, he could check out the svn repo before freeze and continue including package updates using our scripts and distributing them. We don't want to do that, that is all. No time, no space. – norbert Apr 06 '13 at 08:23
  • @norbert, Linux distribution development isn't that different from this. There isn't a break either, just a drive to stabilize/clean up when a release comes up. – vonbrand Apr 06 '13 at 09:01
  • @vonbrand: That is not how Debian is developed :) – خالد حسني Apr 06 '13 at 14:30
  • @KhaledHosny, perhaps you are just too young to remember the last freeze-before-release flamefest there... – vonbrand Apr 06 '13 at 14:32
  • I don't think this answers the question about why TeX Live needs yearly updates, and additionally the claim that "release cycles are required in the Linux world" is plainly wrong: several distributions already use a rolling release model (Arch, Gentoo) and many others are considering switching to it (Ubuntu, OpenSuse, Debian to name a few). Frankly, I don't see the benefits of a fixed release schedule for TeX Live when most of its parts (the packages) are already updated on a rolling release on CTAN... – Xavier Apr 24 '13 at 19:50
  • 1
    @Xavier Arch, at least, is rather different. It expects to be able to post a news announcement saying 'This upgrade involves a fundamental change to the system. To ensure your system still boots, follow this set of instructions, check this list of complications and make sure your system satisfies this set of conditions.' And if they miss something because they didn't realise that users with a certain config would be unable to start a display manager or to login, users need to know enough to workaround and fix it. Can you imagine if tlmgr update --all came with similar requirements? – cfr Jul 02 '17 at 01:55
  • 1
    @Xavier The switch from init to systemd did not require reinstalling, for example. And so on and so forth. TeX Live has a very different audience in mind. It is just not the same at all. You need to compare a distro which is as undemanding of its users as TeX Live. And your last sentence says you just didn't read nobert's answer. Note, too, that there is absolutely no guarantee at all that when I upload my new version of a package to CTAN, other packages will not break. It is not updated as a system. Individual packages are updated is all. – cfr Jul 02 '17 at 01:58
35

in addition to @vonbrand's excellent answer, one important reason to upgrade and "freeze" tex live annually is to provide a stable snapshot on physical media, largely for the benefit of members of the various user groups, but also for tex users who don't have convenient internet access.

this dvd (it no longer fits on a cd) is also physically packaged with some (la)tex-related books, particularly in germany; this (commercial) packaging is the reason why anything with a restrictive (non-free) license must be excluded from the collection.

barbara beeton
  • 88,848
  • 12
    I can understand the need to take and release snapshots. What I don't get is the need to reinstall. It seems tlmgr should be able to update the entire release (it has a mechanism to update itself). – StrongBad Apr 05 '13 at 14:33
  • 4
    @Daniel sometimes the format of the database that provides the information changes, and older tlmgr will not be able to use it. – norbert Apr 06 '13 at 08:39
15

To add some more information to the answers of @vonbrand and @barbara beeton:

One other reason is to change infrastructural things. Some time ago the tlmgr management tool was introduced with a graphical interface for (mainly) windows users. Also the package format was changed (I think it was from TL2008 to TL2009) from LZO to XZ which was not possible if not requesting a new installation as the whole infrastructure was changed.

But nevertheless it often is possible to update e.g. from TL2011 to TL2012 without any new full installation. But be careful: This is not supported by the developers and is not recommended! It works in some cases and in others not.

I have an hope that there will be kind of a rolling release scheme (as e.g. in Arch Linux or Gentoo), as this was a topic several times on TeX Live's mailing list.

Stephan Lukasczyk
  • 2,552