0

I don't know if this is the place to ask that, but I don't know where else…

So I just finished the first version of my robust-externalize library (after doing an important rewrite, thanks a lot everybody here for the help), but before pushing it to CTAN, I'd like to check if it is compatible with windows (I need -shell-escape and I don't really know if the command I used are actually really cross platform, they should but who knows how windows reacts).

Would someone be kind enough to test it for me, as I do not have any windows with me? The procedure to test is simple:

$ git clone https://github.com/leo-colisson/robust-externalize
$ cd robust-externalize/doc

Then, to disable the build of bash code (not available on windows), just modify the file doc/robust-externalize.tex by adding after \begin{document}:

\robExtConfigure{
  % bash code will not be compiled (bash does not exist on windows)
  bash/.append style={
    disable externalization
  },
}

and compile the file using:

$ pdflatex -shell-escape robust-externalize.tex
$ pdflatex -shell-escape robust-externalize.tex

(if you are afraid by the -shell-escape, you can also add a line |enable manual mode,| in \robExtConfigure and run the commands in |JOBNAME-robExt-compile-missing-figures.sh| manually, but part of the goal is to check if -shell-escape works as expected)

If you get no error, and if the code produces a file robust-externalize.pdf that looks like this one https://raw.githubusercontent.com/leo-colisson/robust-externalize/master/doc/robust-externalize.pdf then everything is fine! You can either report it here, or in the github issue created for that! Thanks!

tobiasBora
  • 8,684
  • Are you aware you could say \cs_generate_variant:Nn \str_replace_all:Nnn { Nnx , Nnv , ... } rather than doing each variant one-by-one? (I'm not sure if the code may be clearer doing them one at a time, so this isn't a criticism.) \str_set:Nn {#1} {#2} is incorrect. This is true wherever you have N, you shouldn't have braces. So \str_set:Nn #1 {#2}. – cfr Oct 06 '23 at 02:33
  • @cfr oh good to know, thanks – tobiasBora Oct 06 '23 at 11:02
  • Maybe this question is stupid, I didn't look at your code, but why do you use pgfkeys if you're using expl3? – Skillmon Oct 06 '23 at 20:57
  • @Skillmon first because I'm much more familiar with pgfkeys that I find quite flexible. Second because my users will need to define their own style, and I would expect them to know pgfkeys better that expl3, that require for instance to load expl3 syntax etc… – tobiasBora Oct 07 '23 at 08:53
  • You can define a user-level macro, or even key, to define new styles/keys. No need for any user to ever touch the expl3-layer. But it's fine if you want to use pgfkeys, I was merely curious. – Skillmon Oct 07 '23 at 15:39

2 Answers2

1

While it would be great to have a robust way to externalise images and other things, please reconsider pushing this to CTAN in its current state. The clarity of expl3 code requires people to abide by the naming conventions and use functions which include, rather than exclude, checks for conflicts.

\cs_set:Nn \str_set_hash_robust:Nn % No!

You really, really, really shouldn't do this. Not only is str the name of an existing module, it is the name of a kernel module. You are not adding this as a member of the team responsible for that module, so you should choose your own module name and all the functions and variables you create should be prefixed by that module name.

You should also not do set unless you've previously done new. And you absolutely definitely shouldn't do it with a name which rightfully belongs to the existing module str.

\str_set:Nn {#1} {#2} % No!

is incorrect and the same for similar constructions. Wherever you have N, you shouldn't have braces. So

\str_set:Nn #1 {#2}

Your variables are systematically misnamed as they all omit the suffix which should indicate the kind of variable it is.

\str_new:N \l_robExt_template % No!

is malformed, for example, and should be

\str_new:N \l_robExt_template_str

This is true for every expl3 variable you create.

Note that you are also declaring this variable to be suitable for public use. That means users can legitimately expect updates to take account of the fact that they may be using \l_robExt_currentCompilationCommand, for example. This is fine if you are prepared to support this, but it is somewhat unusual for a package to make use of no internal functions or variables.

You are opening at least 3 new writes. That is a hefty chunk of resources for a package with a relatively limited role. TeX allows at most 16 writes. Your package demands almost 20% of them. (LuaTeX allows 128, so less of a concern.) This can be worked around with morewrites, but I don't recall using any package which helped itself to three (one only for temporary writes?).

If commands, functions or variables shouldn't be called by end users, name them appropriately. For expl3, the guidance is clear. For 2e commands, things are less well-defined, but \robExt@... would flag up the likelihood the command isn't meant for external use. Don't rely on comments or documentation to communicate this.

Although you declare local variables, you don't seem to make much use of grouping. Perhaps this is necessary and by-design but, if not, it is worth seeing whether you can do things more locally.

cfr
  • 198,882
  • Thanks for this review. This project is one of my first project using expl3, so it comes at no surprise that the code is not extremely clean, but I will try to fix this for the next release. To my defense, I find it really hard to understand what is allowed or not since the code "Just works™" even when we do not follow notations (and the manual clearly states that the naming are just conventions, so I was not expecting it to be such a problem not to follow them). I am indeed relying on doc for what is supposed to be usable for end users, but I should make it clearer in the source as well. – tobiasBora Oct 06 '23 at 09:02
  • Thanks for letting me know about the writes, I was not aware of that issue (but still I think it is better to consume 20% of writes but provide a caching functionality than not providing caching at all). The main problem I have is that I don't know how to append stuff to a file in LaTeX without doing really dirty things like reading it all and writing it again… which is horrible from an efficiently perspective). If you know how to avoid this, I might be able to reduce them. That being said, I’m still surprise that in 2023 LaTeX has still such limitations…What is the point of these limitations? – tobiasBora Oct 06 '23 at 09:02
  • One of the aims of expl3 is consistent, transparent naming and avoiding (1) conflicting names and (2) blurring of internal and external names. However, the specified naming scheme is hard or impossible to enforce programmatically. So you have to rely on people abiding by it. But your code is ignoring 2e conventions, too, by using names without @ you regard as internal. A variable name should be \<scope>_<module>_<description>_<type> if public or \<scope>__<mod>_<des>_<type> if internal. A function should be \<mod>_<des>:<arg spec> if public, \__<mod>_<desc>:<arg spec> if internal. – cfr Oct 06 '23 at 16:59
  • The limitation is imposed by TeX. It is not a decision made by LaTeX in 2023, but a function of the resources available when TeX was written. – cfr Oct 06 '23 at 17:00
  • Thanks, I'll change it when I have some time. So you don't have a clean solution to append to a file without openning multiple iow, or reading the whole file before writteng to it? I never understood: why can't we upgrade TeX, or create Tex2? Basing a software on a so old codebase creates tons of idiocrasies (e.g. the number of files that can be opened at once), and the time to fix them quickly overcome the time to fix tex. – tobiasBora Oct 07 '23 at 08:43
  • 1
    @tobiasBora TeX is frozen. That's Knuth's decision and legally binding. Creating TeX2 is something like ConTeXt has done, but you lose backwards compatibility etc. So if you want software not restricted by TeX, you could try ConTeXt. – cfr Oct 07 '23 at 14:28
  • @tobiasBora I'm not sure what you mean about appending to a file without multiple iow etc. Any time you add a line to .toc, for example, you're appending to that file. You don't need multiple writes or to read the whole thing first. But I assume that's obvious, so you mean something different, but I'm not sure what. – cfr Oct 07 '23 at 14:32
  • you told me that I should have a single iow_new instead of 3. But 2 of them are used to write to two fixed files during the compilation (one writes the list of all figures that are used to help cleaning the cache, one writes the list of all compilation commands to run if the user disables shell-escape). If I use a single iow_new, I need to find a way to open/write/close the file every time I want to write a new line in the file. And latex cannot open in append mode, so everytime I open a file it is erased… which is not what I want for the two files described above. – tobiasBora Oct 07 '23 at 17:14
  • Oh it is even legally enforced?! That's so sad. The problem of context is that it really breaks backward compatibility too much. I guess we could extend tex in a backward compatible way. For instance having a real programming engine would be great, and lualatex is a great idea, why not merge it into pdflatex? It would make life of package writters much simpler. – tobiasBora Oct 07 '23 at 17:17
  • Also, I’m curious, why can’t we extend tex using something like etex to allow more file writes? – tobiasBora Oct 07 '23 at 17:36
  • @tobiasBora Really, you should ask new questions. Not least because you'll get answers from people who actually know about this stuff. There are ways of reducing the number of writes you need. For example, there's a package which lets you write multiple indexes to a single .idx and then splits them. As for extending TeX, as I said, you can use morewrites which gives you more writes. But it's not default. – cfr Oct 07 '23 at 18:57
  • Sure sorry, I understand, just since you told me that it was not good to have so much writes I just commented on that. Anyway, thanks for your return. – tobiasBora Oct 07 '23 at 19:15
  • @tobiasBora You're just going beyond my extremely limited knowledge. Anything I told you at this point would be sub-optimal, at best, and simply wrong, at worst. :( You could also ask some of the gurus to look at this in chat. – cfr Oct 07 '23 at 19:29
0

Actually, I managed to run an old windows VM, and it works!

tobiasBora
  • 8,684