Why is `openin_any = p` restricting access to packages in TEXMFDIST?

Question

I'm using openin_any = p (set in a custom TEXMFCNF) as an additional line of defense in an application that runs on (partially) user-provided LaTeX code. This used to work fine, but after updating to the latest texlive 2020, building pretty much any document now fails:

/opt/texlive/bin/x86_64-linux/xelatex: Not reading from /opt/texlive/texmf-dist/tex/latex/l3packages/xparse/xparse-generic.tex (openin_any = p).
! LaTeX3 Error: File 'xparse-generic.tex' not found.

From what I understood, even with openin_any = p files in TEXMFDIST are supposed to be fine. Is this a bug, or is this an intentional change. In any case, I'm looking for a solution that allows access to anything in TEXMFDIST in addition to the directory containing my tex file.

are you referencing the file by an absolute path or \input{xparse-generic} and it being found on the path? — David Carlisle, Sep 23 '20 at 19:42
I'm not accessing it directly at all. The tex file is just using various packages with \usepackage{...} — ThiefMaster, Sep 23 '20 at 19:45
I guess this is something due to the implementation of \file_input:n. — egreg, Sep 23 '20 at 19:47
I tried to trim down the tex file to a minimal example, and \usepackage{fontspec} seems to be the main culprit. — ThiefMaster, Sep 23 '20 at 19:50
@ThiefMaster no it is a binary level feature a minimal example is \documentclass{article}\usepackage{xparse}\begin{document}\end{document} which gives the error you show in pdftex and xetex, in luatex nothing loads at all, article.cls is blocked. — David Carlisle, Sep 23 '20 at 20:07
for luatex see https://tex.stackexchange.com/questions/292363/luatex-doesnt-open-installed-packages-with-openin-any-p-options — David Carlisle, Sep 23 '20 at 20:19
yeah, I found that luatex question as well - but it's quite old and not the distribution i'm using — ThiefMaster, Sep 23 '20 at 20:31
yes but luatex hasn't changed and I see the same in texlive 2020 — David Carlisle, Sep 23 '20 at 20:47

score 2 · Answer 1 · answered Sep 24 '20 at 09:23

This was indeed a bug in texlive; after asking on the mailing list they fixed it.

Details (also from the mailing list) on why exactly this happened:

With openin_any=p you can load most LaTeX packages normally. The issue specific to xparse(-generic.tex) is that it uses a \pdffilesize-based (expandable) test to check whether a file exists, and as I said in my previous mail, \pdffilesize fails with openin_any=p:
export openin_any=p && pdftex '\pdffilesize{sample2e.tex}\bye'
and that's what Akira-san's patch addresses.

This "old behaviour" you mention is that until a few months ago xparse was all loaded in one go in xparse.sty as most packages are (and work as they always did), but now it is split into the "loader" (and legacy---do not use---interfaces) and the bulk of the code in xparse-generic.tex. Since the latter uses \pdffilesize, it fails with openin_any=p.

Why is `openin_any = p` restricting access to packages in TEXMFDIST?

1 Answers1