How to extract parts of (La)TeX document?

Question

I want to run queries against LaTeX documents like these:

1. get list of all equations, even if one declares \def\be{\begin{equation}} and uses it;
2. get list of all arguments of some command (e. g. \marginpar), where it has been used in document (again, even if one declares \def\m\marginpar and calls this macro);
3. get the theorem with given counter value.

The task seems to require full compiling of all macros, but I want to get arguments in not expanded form --- that is not as blocks and glue, but as user's input (if possible).

How can I do such things?

Update: ideally, I want to use "semantic markup" to mark parts of documents and generate different views of them. Assume I have smth like this:

\beginBlock{calculus}
\beginBlock{content}
... lectures...
\endBlock
\beginBlock{exam}
... problems...
  \beginBlock{comments}
  ... comments about criteria and how to evaluate typical mistakes...
  \endBlock
\endBlock[exam]
\endBlock[calculus]

and similar for linear algebra in other file; I want to generate a report with sections for calculus and linear algebra, containing only exam problems, e. g.

Answer 1

Let's look at \marginpar. You want to redefine this command to save its arguments in some list to be printed, say, at end document.

One has to look the definition of \marginpar, discovering that it's not a command with arguments and it performs a rather complicated series of tasks. So we can't simply add some action to it without examining in detail the macro that are used.

A good strategy is to gather the arguments and then put them back for the original \marginpar.

\makeatletter
\let\saved@marginpar\marginpar
\renewcommand\marginpar[2][]{%
  \if\relax\detokenize{#1}\relax
    \let\saved@mp@optional\@empty
  \else
    \def\saved@mp@optional{[#1]}%
  \fi
  \def\saved@mp@mandatory{{#2}}%
  \saved@mp@add{\saved@mp@print{#1}{#2}}%
  \expandafter\expandafter\expandafter\saved@marginpar
    \expandafter\saved@mp@optional\saved@mp@mandatory}
\newtoks\saved@mp@toks
\def\saved@mp@add#1{\global\saved@mp@toks=\expandafter{\the\saved@mp@toks#1}}
\def\printmarginpars{\the\saved@mp@toks}
\def\saved@mp@print#1#2{\par\noindent Marginpar\\
  Optional: #1\\
  Mandatory: #2\par\medskip}
\makeatother

If you say, after this code,

\newcommand{\mpx}{\marginpar}

then it will work the same.

What do the macro do?

1. \marginpar is redefined to have an optional and a mandatory argument, after having saved a copy of the original command.

2. If the optional argument is empty, then we define \saved@mp@optional to be \@empty, otherwise we define it to expand to the optional argument in brackets.

3. Similarly, we define \saved@mp@mandatory to expand to the mandatory argument in braces.

4. We add the contents of the arguments to the token list we have allocated.

5. We put back the arguments after \saved@marginpar, so LaTeX will do its usual work as if nothing had happened.

Example

\documentclass{article}
\makeatletter
\let\saved@marginpar\marginpar
\renewcommand\marginpar[2][]{%
  \if\relax\detokenize{#1}\relax
    \let\saved@mp@optional\@empty
  \else
    \def\saved@mp@optional{[#1]}%
  \fi
  \def\saved@mp@mandatory{{#2}}%
  \saved@mp@add{\saved@mp@print{#1}{#2}}%
  \expandafter\expandafter\expandafter\saved@marginpar
    \expandafter\saved@mp@optional\saved@mp@mandatory}
\newtoks\saved@mp@toks
\def\saved@mp@add#1{\global\saved@mp@toks=\expandafter{\the\saved@mp@toks#1}}
\def\printmarginpars{\the\saved@mp@toks}
\def\saved@mp@print#1#2{\par\noindent Marginpar\\
  Optional: #1\\
  Mandatory: #2\par\medskip}
\makeatother

\newcommand{\mpx}{\marginpar}

\begin{document}
a\marginpar{xxx}
b\mpx[yyy]{zzz}

\printmarginpars
\end{document}

Answer 2

I think you will be better off with an external script in your favourite language such as python or perl.

If you want to do it with LaTeX an approach that could work, at least for the command that are not environments is to save the contents of the commands either on file or within a list.

I will demonstrate it for the use of `marginpars. You first define a list:

  \def\alist{}

and then you define macros to add to the list:

  \def\addtolist#1#2{
    \lst@lAddTo\alist{#2}
  }

  \long\gdef\addmp#1#2{\addtolist\alist{#1,}}

  \def\AddMP#1#2{%
     \long\expandafter\gdef\csname#1\endcsname{\textbf{#1}: #2}
     \addmp{#1}{#2}\marginpar{#2}
  }

Notice on the last one, you save the contents of the marginpar by using addmp and then you print it.

Full MWE:

\documentclass[11pt]{book}
\usepackage{lstdoc,lipsum}
\begin{document}
\makeatletter
\def\alist{}

\def\addtolist#1#2{
  \lst@lAddTo\alist{#2}
}

\long\gdef\addmp#1#2{\addtolist\alist{#1,}}

\def\AddMP#1#2{%
\long\expandafter\gdef\csname#1\endcsname{\textbf{#1}: #2}
\addmp{#1}{#2}\marginpar{#2}
}

\def\PrintMP{%
  \@for \i:=\alist\do{%
  \csname\i\endcsname}
}
%example
\AddMP{mp}{\lipsum[2]}
\AddMP{mp}{\lipsum[3]}
\AddMP{mp}{\lipsum[1]}
% print the marginpars
\Printmp
\makeatother
\end{document}

Edit

Based on new information provided by the OP, one could use a .dtx file with the doc/docstrip system. This would allow you to save into different files sections of the document etc.

Answer 3

Ok, now I understand this better. It is a rather common idea to have one "master" document which will produce different output in different "modes".

This would mean you'd make different "configurations" where the environments you mention are defined differently. By activating one "configuration" with a specific definition of the environments you get a different version of the document. One "configuration" could be a .tex file containing environment definitions (one of which is then \input) or different options for a dedicated package which represent different environment definitions.

Packages which might be relevant: comment, optional, versions, version, renditions, tagging.

Another very sophisticated, large package which can be used for this kind of thing: The AcroTeX eDucation Bundle.

Questions which might be relevant:

Showing Solutions of the Questions “separately”

Comparing packages which facilitate typesetting exercises and solutions: exercise vs. answers vs. probsoln

Commenting out large sections

Including only the parts of a document corresponding to a conditional expression

There was another question highly relevant to this, but I fail to find it now. It was concerned with how to store different content parts in a common file, assembling different documents from them. Maybe someone else remembers it.