5

I am struggling to control expansion in code recorded with ltproperties. In the following example, I am trying to record the body of an environment as stated, without expansion (so as if with \exp_not:n). Of course, to work with ltproperties it needs to first be stored in a variable, so I use \exp_not:V on the variable in the fourth argument of \property_new:nnnn.

\documentclass{article}

\ExplSyntaxOn \tl_new:N \l__recordenv_contents_tl

\property_new:nnnn { recordenv/contents } { now } { } { \exp_not:V \l__recordenv_contents_tl }

\NewDocumentEnvironment{recordenv}{ m +b } { \tl_set:Nn \l__recordenv_contents_tl { #2 } \tl_show:N \l__recordenv_contents_tl \property_record:nn { #1-recordenv } { recordenv/contents } } {} \ExplSyntaxOff

\begin{document}

\section{First section}

\begin{recordenv}{foo} some text and \thesection \end{recordenv}

\section{Second section}

\end{document}

If I check the log, the tl variable contains what I want:

> \l__recordenv_contents_tl=some text and \thesection

However the aux file contains

\relax 
\@writefile{toc}{\contentsline {section}{\numberline {1}First section}{1}{}\protected@file@percent }
\new@label@record{foo-recordenv}{{recordenv/contents}{some text and 2}}
\@writefile{toc}{\contentsline {section}{\numberline {2}Second section}{1}{}\protected@file@percent }
\gdef \@abspage@last{1}

So when written to the aux, the body is still being expanded since \thesection becomes a number. Furthermore, this expansion is happening at shipout since the number \thesection expands to is 2, not 1 as you'd expect since the property was set with now.

Now, I thought maybe this was the same issue as in this question I asked about expansion in \addtocontents. There the issue was that two separate expansions were happening so adding an extra \exp_not:n was necessary. However, \exp_not:n { \exp_not:V \l__recordenv_contents_tl } just leads to an empty argument in the aux so I assume that's not what's happening here. I also tried \exp_not:o { \l__recordenv_contents_tl } but the effect is the same as with \exp_not:V.

My questions are

  1. why is there expansion happening on the tl variable after being V-expanded, and
  2. why is this second expansion happening at shipout?

Added

I think I now understand why \exp_not:V alone can't work the way I want, and why \exp_not:n { \exp_not:V <tlvar> } fails. The content is expanded once at definition, stripping the \exp_not:n, then expanded again when writing to the file during shipout at which time <tlvar> is empty. Is this correct? Looking at the ltproperties code, I am still confused why \iow_shipout_x:Nx is used in \property_record:nn for all properties, even those with setpoint now.

mbert
  • 4,171
  • Well, you are preventing the expansion of \thesection at the time the property is stored by using \exp_not:V. If you just remove it, you'll get {{recordenv/contents}{some text and 1}} as expected (as I'd expect, at least). Off-topic, are you sure trying to store arbitrary content as a property in you aux file is the best method to store such content? Some pitfalls there... – gusbrs Jan 31 '24 at 02:51
  • Also, as you seem to expect to find some text and \thesection as the value of your property in your aux file, for that you'd need \property_new:nnnn { recordenv/contents } { now } { } { \tl_to_str:V \l__recordenv_contents_tl }, but be really careful there... I wouldn't recommend it. Again, the question sounds like an XY problem and that this is not quite the right tool for the purpose. – gusbrs Jan 31 '24 at 02:53
  • To understand why your MWE produces what it does (and some more), see texdoc tex-by-topic section "12.6.3 Expansion and \write". – gusbrs Jan 31 '24 at 03:20
  • I've never used ltproperties and just read the first couple of pages of documentation, so take this with a pinch of salt: isn't the point of ltproperties to record the expansion (e.g. 1) and retrieve it elsewhere (e.g. 1 at shipout)? It seems odd to pick this tool to record the non-expansion. What am I misunderstanding? – cfr Jan 31 '24 at 04:01
  • @cfr Agreed. \thesection is a great example of why you actually want to expand it in place. As it is, if the MWE did what the OP claims to want, \thesection would give the value at the place the reference is made which makes no sense. That's why I think the MWE is over simplified and we still don't know what the actual problem is (XY so far). – gusbrs Jan 31 '24 at 04:14
  • @cfr Yes, a much more common usage is to expand fully whatever code is being recorded. I've had success writing environment contents to a file with \exp_not:n{#1} where #1 is the env body, then reading those in at begindocument and storing in macros so they can be referenced anywhere in the document. This question is just to see if I can do the same thing with ltproperties, then I'll benchmark to see which approach is faster. The ltproperties approach has the benefit of not writing to a new file – mbert Jan 31 '24 at 04:18
  • @gusbrs \tl_to_str:V does seem to suit my needs. I am aware of catcode issues with what I'm doing. Are there other downsides? – mbert Jan 31 '24 at 04:22
  • @gusbrs And yes, sorry, my example using \thesection was artificial and not instructive for the first question. I used it only to express confusion in the second question about why it's being expanded at shipout when the property is set with now – mbert Jan 31 '24 at 04:23
  • 1
    @mbert catcodes are enough trouble. I guess you could control expansion so as not to nuke them with \tl_to_str:V but, regardless, you still have to deal with the difference between catcodes when the argument is read and when the .aux file is read... Also, the time things expand is critical to the results, as the \thesection shows. Furthermore, you may find trouble in control expansion at reading time (https://github.com/latex3/latex2e/issues/1194). Etc., etc. So, I'm still saying, are you really sure about this the way to go for whatever you are trying to do?... – gusbrs Jan 31 '24 at 04:27
  • @gusbrs I am fully aware of the expansion timing. I can always add a keyval option expand for recordenv that expands the recorded contents with a \protected@edef or \text_expand:n or something else. But having the option allows flexibility for environments where I may want the recorded content intentionally unexpanded so that it depends on its context or, say, the definition of some macro at the time the retrieving macro is called – mbert Jan 31 '24 at 04:33
  • @mbert In this case, I will stop trying to dissuade you. :-) (Though you should probably infer from that issue linked above, about expansion control at retrieval time, that Ulrike will curse you for doing that... ;-) – gusbrs Jan 31 '24 at 04:40
  • @gusbrs Yes I just finished reading the exchange. Hopefully my question makes it clear which side I lean towards :) – mbert Jan 31 '24 at 04:44
  • @mbert Regarding your added question. I haven't examined the ltproperties code, but I'd presume \iow_shipout_x:Nx is used because it is just one label, and it may potentially contain a shipout property, so you can't use an immediate write. The difference between a now and a shipout property is then whether the contents are expanded immediately by other means. zref does have a \zref@wrapper@immediate, but you need to be sure no shipout properties are in the list to use it... And I'm not sure ltproperties has an equivalent construct (yet?). – gusbrs Jan 31 '24 at 10:50
  • @gusbrs \iow_shipout_x:Nx is simply an error which is already corrected in the development code: https://github.com/latex3/latex2e/issues/1200 – Ulrike Fischer Jan 31 '24 at 12:57
  • @UlrikeFischer Oh, I see. Your fault then. ;-) But I was only trying to explain why not \immediate which \protected@write still isn't. – gusbrs Jan 31 '24 at 13:03
  • 1
    Generally I don't think that is a good idea to use properties for this. At least not if the environment can have arbitrary content. Controlling expansion and use of commands can get very tricky here, for example if the content contains verbatim material. That means that you would have to do quite a lot sanitizing first. – Ulrike Fischer Jan 31 '24 at 13:11
  • @mbert I told ya. :-) A final comment, you said you are trying ltproperties to check if it is faster than your current approach of writing to a separate file. Probably not, since the .aux is read twice per compilation. – gusbrs Jan 31 '24 at 14:33
  • @mbert I said "final", but... You haven't yet told us thus far if you really need the stored contents of the environment before the environment itself. If you don't, you don't need the .aux file at all, or any external file for that matter. See postnotes which does something of the sort for end notes. – gusbrs Jan 31 '24 at 14:41
  • @gusbrs Thanks for the comment about speed. The point of this is to be able to use the contents of the environment like a label-reference, so indeed before the environment – mbert Jan 31 '24 at 14:44
  • @mbert Than you will need to send it to an external file, no alternative. And be it with ltproperties or using a dedicated file, the complications are similar, so... a matter of choice. – gusbrs Jan 31 '24 at 14:46
  • @gusbrs There are still some things I'd like to fix such as hash doubling, which I think can be handled by the answers to this question, but I can live without verbatim or other catcode-changing material – mbert Jan 31 '24 at 14:47
  • @mbert This stuff is out of my league. You'll have to rely on the experts. ;-) – gusbrs Jan 31 '24 at 14:48

1 Answers1

4

LaTeX writes on the .aux file with \protected@write that performs full expansion (but preserves commands declared robust).

You need to protect the contents from expansion at a different level.

\documentclass{article}

\ExplSyntaxOn \tl_new:N \l__recordenv_contents_tl

\property_new:nnnn { recordenv/contents } { now } { } { \exp_not:V \l__recordenv_contents_tl }

\NewDocumentEnvironment{recordenv}{ m +b } { \tl_set:Nn \l__recordenv_contents_tl { \exp_not:n { #2 } } %\tl_show:N \l__recordenv_contents_tl \property_record:nn { #1-recordenv } { recordenv/contents } } {} \ExplSyntaxOff

\begin{document}

\section{First section}

\begin{recordenv}{foo} some text and \thesection \end{recordenv}

\section{Second section}

\end{document}

The .aux file

\relax 
\@writefile{toc}{\contentsline {section}{\numberline {1}First section}{1}{}\protected@file@percent }
\new@label@record{foo-recordenv}{{recordenv/contents}{some text and \thesection }}
\@writefile{toc}{\contentsline {section}{\numberline {2}Second section}{1}{}\protected@file@percent }
\gdef \@abspage@last{1}
egreg
  • 1,121,712