1

Rationale: The goal I have is to create a system where I pretend -- via Lua -- that a file is being read, while in reality the contents come from elsewhere. A concept elsewhere called VFS (virtual file system), which is the name I borrowed.

Consider the following MWE (I know it has flaws):

% !TeX encoding = UTF-8
% !TEX TS-program = lualatex
\documentclass{article}
\usepackage{luacode}

\begin{luacode*} do -- Table which contains "reader" functions for the virtual "files" given as keys vfs = { ["ZaphodBeeblebrox"] = function() return nil end }

local function luatexbase_log(text, ...)
    texio.write_nl("log", string.format(text, ...))
end

local function get_vfs_realname(name)
    local pfx = "virtual." -- virtual "file" prefix
    if name:find(pfx, 1, true) == 1 then
        local lookup_name = name:sub(#pfx + 1, #name)
        if vfs[lookup_name] ~= nil and type(vfs[lookup_name]) == "function" then
            return lookup_name
        end
    end
    return nil -- not found
end

local function hook_find_read_file(id, name)
    kpse_name = kpse.find_file(name)
    if kpse_name then
        return kpse_name
    end
    local rname = get_vfs_realname(name)
    if rname then
        luatexbase_log([[Found VFS \input lookup: %s (id=%d)]], rname, id)
        return name
    end
end

local function emulate_default_open_read_file(name)
    return {
        ["fhandle"] = assert(io.open(name,"r")),
        ["reader"] = function(self)
                return self.fhandle:read("*l")
            end,
        ["close"] = function(self)
                self.fhandle:close()
            end,
    }
end

local function hook_open_read_file(name)
    kpse_name = kpse.find_file(name)
    if kpse_name then
        return emulate_default_open_read_file(kpse_name)
    end
    local rname = get_vfs_realname(name)
    if rname then
        return { ["reader"] = vfs[rname] }
    end
end

luatexbase.add_to_callback("find_read_file", hook_find_read_file, "vfs")
luatexbase.add_to_callback("open_read_file", hook_open_read_file, "vfs")

end \end{luacode*}

\begin{document} \input{virtual.ZaphodBeeblebrox} bla \end{document}

As far as I can tell, the code works flawlessly. However, there's one thing I dislike: emulate_default_open_read_file().

I totally made up that function, trying to guess what LuaLaTeX does by default.

Question

  1. Instead of coming with my own version of what I think LuaLaTeX may be doing by default upon "open_read_file", is there a way can access the default behavior? The goal here is to be 100% identical in behavior to the default, except for when I deal with my own virtual "files".
  2. Assuming there is such a way above, how do I detect failure by the default function and kick in with my fallback? Or would the only way here to switch the order, detect the virtual "files" before and failing that fall back to the default?

NB: I am aware that the current "ZaphodBeeblebrox" reader function is flawed. It should return data line-wise and end with nil to signal end of file. But I got stuck before fleshing out this function. So please bear with me.

PS: I took the liberty to rewrite luatexbase_log from texdoc ltluatex to use the string.format() facility directly, which saves some typing. PPS: I am trying this in order to avoid having to write temporarily to the disk and in order to dodge the weird behavior I get when using \directlua or \luadirect to tex.print() a bunch of LaTeX+TikZ code. Probably related to expansion. But writing the output of the Lua code to disk (manually), then using \input on the written file, happens to work. Which is what I am trying to emulate here.

  • Can't test it right now, but doing return false in the callback function should make it fall back on the default – Max Chernoff May 05 '23 at 02:01
  • "But writing the output of the Lua code to disk (manually), then using \input on the written file, happens to work. Which is what I am trying to emulate here." that exactly describes \scantokens – David Carlisle May 05 '23 at 07:39
  • 1
    @MaxChernoff thanks. Oh, so you're saying it distinguishes between false and nil. Interesting. Will test that and also see if I can have a look at the source code (never actually did). – 0xC0000022L May 05 '23 at 09:34
  • @DavidCarlisle thanks, saw your answer on the topic and will read up further. Perhaps I can leverage that knowledge. Perhaps I even end up with two methods to achieve the same thing. – 0xC0000022L May 05 '23 at 09:36
  • @MaxChernoff: Returning false in the callback it crashes. Now I know why it crashes with false. In terms of Lua the values nil or a valid (C) char* is permissible (CALLBACK_LSTRING in lcallbacklib.c). But I agree, it would have been great were that the behavior implemented in texfileio.c. Alas, it's not. Having the ability to invoke the default behavior with false would be sensible. For luatex_find_read_file() it's visible there is no fallback no facility to do what I want (other than reimplementing). – 0xC0000022L May 08 '23 at 22:15
  • 1
    Oh, and interestingly I found this: For some minor speed gain, you can assign the boolean false to the non-file related callbacks, doing so will prevent LuaTEX from executing* whatever it would execute by default (when no callback function is registered at all). Be warned: this may cause all sorts of grief unless you know exactly what you are doing!* Perhaps you thought of this? – 0xC0000022L May 08 '23 at 22:27
  • "9.3.2 General file readers" lists a number of readers. Three of those alias to readbinfile() (texfileio.c). There appears to be no way to mimic the default for text files, i.e. lua_a_open_in() (texfileio.c), which functionally corresponds to the open_read_file callback. – 0xC0000022L May 08 '23 at 22:48
  • 1
    I was thinking of the node-based callbacks where luatexbase can take multiple callbacks, and if you return true, then it will just move on to the next. Returning false from the raw file callbacks would be bad, but I was hoping that luatexbase had a wrapper that would fallback on the default if you returned a boolean. But for open_read_file, luatexbase just sets the raw callback directly, so returning a boolean makes bad things happen (as you've figured out). A bad guess on my part. – Max Chernoff May 09 '23 at 07:19

0 Answers0