Extracting the structure of a LaTex document, including comments

Question

I like to write a "special" comment with a title at the beginning of each paragraph in my documents, something like this:

\section{Introduction}

% 1.0 Visually tracking balls is an interesting problem.  
Text related to why tracking balls is important, blah, blah, blah.   
Blah, blah, blah. 

% 1.1 Tracking balls is difficult because of these challenges.  
Text relating the challenges, blah, blah, blah.  
More blah, blah, blah. 

% 2.0 Previous work.  
Text relating the existing work on tracking balls, blah, blah, blah.  
Blah, blah, blah.

This helps me keeping the content of my documents well structured.

What I'd like to do is to extract the structure of a document: sectioning commands (i.e, \chapter, \section and the like) and "special" comments at the beginning of each paragraph.
I'd like to parse a LaTex source file (or a group of them, in case the main file includes other source files) and produce a new file which contains only the sectioning commands and the "special" comments, turned into regular text (uncommented) or, better yet, turned into a bulleted list.

So the output generated running the parser on the previous code would be:

\section{Introduction}

\begin{itemize}  
  \item 1.0 Visually tracking balls is an interesting problem.  
  \item 1.1 Tracking balls is difficult because of these challenges.  
  \item 2.0 Previous work.  
\end{itemize}

The best solution I got so far consists in me labelling the special comments with a character sequence (i.e., starting them with "%$"), then grepping the occurrences of "%$" and of the sectioning commands in the source file.

Thanks in advance.

After a fashion, this is sort of what a \tableofcontents does, using information that has been written out to the .toc file. Maybe something could be adapted along those lines. — Steven B. Segletes, Sep 22 '14 at 16:18
Welcome to TeX.SX! You can have a look at our starter guide to familiarize yourself further with our format. — Symbol 1, Sep 22 '14 at 16:20
You would probably enjoy emacs’ org-mode; see https://tex.stackexchange.com/questions/54205/org-mode-export-to-latex-temptation-or-nuisance — Thérèse, Sep 22 '14 at 16:41
As far as I can see, you're not going to be able to do it easily with simple comment syntax. — Sean Allred, Sep 22 '14 at 16:58

David Carlisle · Accepted Answer · 2014-09-22T19:56:15.567

12

This is easily done with any command line tools here I use grep and sed (in cygwin bash on windows) but other systems have similar or you could use perl

If zz.tex is your original file,

The command line of

$ grep "\(sub\)*section\|^%" zz.tex | sed -e '/^\\s/ a \\\\begin{itemize}' -e 's/^%/\\item /' -e '$ a \\\\end{itemize}'

outputs

\section{Introduction}
\begin{itemize}
\item  1.0 Visually tracking balls is an interesting problem.  
\item  1.1 Tracking balls is difficult because of these challenges.  
\item  2.0 Previous work.  
\end{itemize}

edited Sep 22 '14 at 19:56

answered Sep 22 '14 at 19:17

David Carlisle

757,742

Sorry it took me years to mark this solution as the accepted one, I had lost track of this question. – user623515 Apr 18 '19 at 15:30

Steven B. Segletes · Answer 2 · 2014-09-23T11:04:59.630

Here I use a "table of contents" approach, by which I mean I explicitly write information to the aux file, and process it from there. EDITED to present two approaches: 1) where the "XYZecutive summary" is contained in the original document, and 2) where the summary is an external document.

In both cases, I have my tex file in which I can repeatedly use the macro \addxyzline{type}{content} to add various lines to the aux file. I also have the \writexyz macro, which will read the .aux file and create an .xyz file. It can appear in the source document after \begin{document}, either before any sectioning has occurred, or after all sectioning has occurred.

APPROACH 1: XYZecutive Summary in Same Document

In this approach, not only does the \writexyz macro write things to the .xyz file, it also processes that document on the second pass, formatting the contents appropriately.

The executive summary (containing all the \addxyzline information) appears at the invocation of \writexyz. In this MWE, I place it after the toc but before the document matter. I play with the \addtocontents macro definition so that the executive summary sectioning doesn't end up as duplicative sectioning in the toc.

\documentclass{article}
\newcommand\addxyzline[2]{\addtocontents {xyz}{\protect \xyzline {#1}{#2}}}
\let\svaddtocontents\addtocontents
\makeatletter
\newcommand\writexyz{%
  \renewcommand\addtocontents[2]{\relax}%
  \clearpage\setcounter{section}{0}\noindent%
  {\Huge\hrulefill XYZecutive Summary\hrulefill\par}%
  \newcommand\xyzline[2]{\expandafter\csname##1\endcsname{##2}}\@starttoc{xyz}%
  \par\noindent\hrulefill\clearpage\setcounter{section}{0}%
  \let\addtocontents\svaddtocontents
}
\makeatother
\begin{document}
\tableofcontents% CAN UNCOMMMENT TO SEE THAT toc WORKS FINE
\writexyz

\section{Introduction}
\addxyzline{section}{Introduction}

\addxyzline{begin}{itemize}
\addxyzline{item}{1.0 Visually tracking balls is an interesting problem.}
Text related to why tracking balls is important, blah, blah, blah.   
Blah, blah, blah. 

\addxyzline{item}{1.1 Tracking balls is difficult because of these challenges.}
Text relating the challenges, blah, blah, blah.  
More blah, blah, blah. 

\addxyzline{item}{2.0 Previous work.}
Text relating the existing work on tracking balls, blah, blah, blah.  
Blah, blah, blah.
\addxyzline{end}{itemize}

\end{document}

enter image description here

APPROACH 2: Separate Summary Document

Here, I create my document body as before, but the \writexyz will not write the summary to the same document, but merely to the .xyz file.

\documentclass{article}
\newcommand\addxyzline[2]{\addtocontents {xyz}{\protect \xyzline {#1}{#2}}}
\makeatletter
\newcommand\writexyz{\newcommand\xyzline[2]{}\@starttoc{xyz}}
\makeatother
\begin{document}
%\tableofcontents% CAN UNCOMMMENT TO SEE THAT toc WORKS FINE
\section{Introduction}
\addxyzline{section}{Introduction}

\addxyzline{begin}{itemize}
\addxyzline{item}{1.0 Visually tracking balls is an interesting problem.}
Text related to why tracking balls is important, blah, blah, blah.   
Blah, blah, blah. 

\addxyzline{item}{1.1 Tracking balls is difficult because of these challenges.}
Text relating the challenges, blah, blah, blah.  
More blah, blah, blah. 

\addxyzline{item}{2.0 Previous work.}
Text relating the existing work on tracking balls, blah, blah, blah.  
Blah, blah, blah.
\addxyzline{end}{itemize}

\writexyz
\end{document}

As far as screen output, the compilation produces the output of the original document, without regard to the \addxyzline content:

enter image description here

However, it also creates a .xyz file (in this case xyz.xyz, since I had called my original file xyz.tex), with the following information:

\xyzline {section}{Introduction}
\xyzline {begin}{itemize}
\xyzline {item}{1.0 Visually tracking balls is an interesting problem.}
\xyzline {item}{1.1 Tracking balls is difficult because of these challenges.}
\xyzline {item}{2.0 Previous work.}
\xyzline {end}{itemize}

containing all of my \addxyzline information.

Then, I have a very simple second tex file:

\documentclass{article}
\newcommand\xyzline[2]{\csname#1\endcsname {#2}\par}
\begin{document}
\input{xyz.xyz}% NAME OF .xyz FILE GOES HERE IN THE ARGUMENT
\end{document}

which provides the xyz summary info

enter image description here

POSTSCRIPT

As one can see, the first argument of \addxyzline is the text of any macro name. In my MWE, I have used "section", "begin", "item", and "end". Other useful incantations might include

\addxyzline{relax}{this is just text added on the same line}
\addxyzline{newline}{this is just text on a new line}

Aditya · Answer 3 · 2014-09-23T19:51:10.713

Here is a ConTeXt solution that uses the "table of content" approach. Pick a section head that you are not using, say subsubject and use that to annotate your outline in the main document:

\setuphead[subsubject][placehead=empty, before=,after=, incrementnumber=yes]

\starttext

\section{Introduction}

\subsubject{1.0 Visually tracking balls is an interesting problem.}
Text related to why tracking balls is important, blah, blah, blah.   
Blah, blah, blah. 

\subsubject{1.1 Tracking balls is difficult because of these challenges.}
Text relating the challenges, blah, blah, blah.  
More blah, blah, blah. 

\subsubject{2.0 Previous work. }
Text relating the existing work on tracking balls, blah, blah, blah.  
Blah, blah, blah. 

\stoptext

In the first line, placehead=empty tells ConTeXt not to place the head (subsubject, in this case) in the output; incrementnumber=yes is need to ensure that the ToC entry is written in a way that is accessible through other files.

Next create a separate file to create an outline. I'll assume that the name of the main file was test.tex.

\starttext
\ctxlua{job.load("test.tuc")}
\placelist[section,subsubject]
          [pagenumber=no, headnumber=no]   
\stoptext

which gives

enter image description here

Extracting the structure of a LaTex document, including comments

3 Answers3

Linked