How to automatically include several text documents into a LaTeX document?

Question

I have several .txt or .dat files, whose names are of the form: dn-ddmmyyyy.txt. For instance:

d0-01021989.txt
d1-21021989.txt
d2-15021994.txt
d3-12121996.txt
d4-14032014.txt
d5-22022035.txt
...many more...

What is the best way to include all the files without using manually \include for each one of them?

Is there a way to create a title (like a chapter title) for each document, as "Document N", and extract the number present in the file name, which is a date, to put it in the title?

Is there a way to do this only with TeX?

Answer 1

I don't know about a TeX only solution (I could however imagine that lualatex could by handy) but what I usually do is the following:

1) use a command like dir /b *.tex > allFiles.txt? to get a file of all the TeX files. (the /b suppresses everything but the file name)

2) open up Excel, paste the content of allFiles.txt and use string concatenation to build the include commands.

For once-only this is the most efficient solution. In cases where I regularly have to update my TeX file I'd write a short Python/Bash/Batch file.

EDIT:

I have found some suitable code to include via lualatex Lua-calls in one of the answers here and was able to adjust it a little. It uses \write18 as well, so it must be used with --shell-escape. I am pretty sure that a Lua-only solution is possible, but this is clearly beyond my Lua-knowledge.

%!TEX TS-program = LuaLaTeX

\documentclass[12pt,ngerman]{scrartcl}
\usepackage{luacode}

\begin{document}
Some text before the Lua call.

\begin{luacode}
function scandir(directory)
    local i, t, popen = 0, {}, io.popen
    for filename in popen('dir "'.. directory .. '" /b d*.tex'):lines() do
        i = i + 1
        t[i] = filename
       tex.print('\\input{includes/' .. filename .. '} from ' .. filename ..'\\clearpage')  
    end
    return t
end
scandir("C:/Users/Uwe/LuaLula/includes/")
\end{luacode}

\end{document}

Answer 2

This solution is only for Microsoft Windows users. For Linux, MAC, Unix users, you have to adapt the batch-related parts. Sorry for this inconvenience and thank you for your cooperation.

This code below was actually also prepared several decades ago here (click). Compile with pdflatex -shell-escape filename. Make sure that

• the input files to be included are put in a sub directory Sub-Dir.
• no spaces in path are allowed (to simplify the code).

Generic Minimal Working Example

% filename.tex

\documentclass[preview,border=12pt,12pt,varwidth]{standalone}

\usepackage{filecontents}
\begin{filecontents*}{batch.txt}
rem batch.bat
echo off

rem %1 path (relative to the main input file) to the files to be iterated 
rem %2 output file name
rem remaining args represent the extension of file to be iterated

set curdir=%CD%
cd "%~1"
shift

rem output must be enclosed with "" to allow spaces in the path
set output="%curdir%\%~1.list"

if exist %output% del %output%
copy nul %output%
shift

:loop
if "%~1"=="" goto :eof
dir /b *.%~1 >> %output%
shift
goto :loop
\end{filecontents*}


\begin{filecontents*}{oh-my-ghost.tex}
No body no body but you!
\[
E\not = mc^2
\]
\end{filecontents*}

\begin{filecontents*}{hahaha.tex}
PSTricks is fun!
\[
pV = nRT
\]
\end{filecontents*}


\immediate\write18{%
    del batch.bat && 
    rename batch.txt batch.bat && 
    move oh-my-ghost.tex Sub-Dir/oh-my-ghost.tex && 
    move hahaha.tex Sub-Dir/hahaha.tex
}


\newread\reader
\newcount\TotalFiles    


\makeatletter
\newcommand\IterateInputFiles[2][tex]{%
% #1: directory path with a trailing /
% #2: a list of file extensions: tex 
\immediate\write18{batch "#2" \jobname\space #1}
\openin\reader=\jobname.list\relax
\loop
    \read\reader to \filename
    \unless\ifeof\reader
    \filename@parse{\filename}
    \section{\filename@base}% change to \chapter if you want 
    \include{"#2\filename@base"}
    \advance\TotalFiles1\relax
\repeat
\closein\reader
}
\makeatother



\begin{document}
\IterateInputFiles{Sub-Dir/}

\section{Summary}
There is(are) \the\TotalFiles\ file(s) in total.
\end{document}

Output

The following output is the proof that I am not lying.

---

Multi-platform Version

% filename.tex

\documentclass[preview,border=12pt,12pt,varwidth]{standalone}

\newif\ifDOS
%\DOStrue
\DOSfalse

\usepackage{filecontents}
\ifDOS
\begin{filecontents*}{batch.txt}
rem batch.bat
echo off

rem %1 path (relative to the main input file) to the files to be iterated
rem %2 output file name
rem remaining args represent the extension of file to be iterated

set curdir=%CD%
cd "%~1"
shift

rem output must be enclosed with "" to allow spaces in the path
set output="%curdir%\%~1.list"

if exist %output% del %output%
copy nul %output%
shift

:loop
if "%~1"=="" goto :eof
dir /b *.%~1 >> %output%
shift
goto :loop
\end{filecontents*}
\else
\begin{filecontents*}{batch.txt}
#!/bin/bash
# bash batch.sh Sub-Dir output tex

# $1 path (relative to the main input file) to the files to be iterated
# $2 output file name
# remaining args represent the extension of file to be iterated

output="$2.list"

if [ -e "$output" ]; then
    rm "$output"
fi

for ext in ${@:3}; do
    ls "$1"/*."$ext" >> "$output"
done
# $ (to squelch bad syntax highlighting in my editor)
\end{filecontents*}
\fi

\begin{filecontents*}{oh-my-ghost.tex}
No body no body but you!
\[
E\not = mc^2
\]
\end{filecontents*}

\begin{filecontents*}{hahaha.tex}
PSTricks is fun!
\[
pV = nRT
\]
\end{filecontents*}


\ifDOS
\immediate\write18{%
  del batch.bat &&
  rename batch.txt batch.bat &&
  move oh-my-ghost.tex Sub-Dir/oh-my-ghost.tex &&
  move hahaha.tex Sub-Dir/hahaha.tex
}
\else
\immediate\write18{%
  mv batch.txt batch.sh &&
  mkdir -p Sub-Dir &&
  mv oh-my-ghost.tex hahaha.tex Sub-Dir/
}
\fi


\newread\reader
\newcount\TotalFiles

\makeatletter
\newcommand\IterateInputFiles[2][tex]{%
% #1: directory path with a trailing /
% #2: a list of file extensions: tex
\ifDOS
\immediate\write18{batch "#2" \jobname\space #1}
\else
\immediate\write18{bash batch.sh "#2" \jobname\space #1}
\fi
\openin\reader=\jobname.list\relax
\let\filename\relax
\loop
  \read\reader to \filename
  \typeout{<<< here again}
  \unless\ifeof\reader
  \filename@parse{\filename}
  \section{\filename@base}% change to \chapter if you want
  \include{"#2\filename@base"}
  \advance\TotalFiles1\relax
\repeat
\closein\reader
}
\makeatother

\begin{document}
\IterateInputFiles{Sub-Dir/}

\section{Summary}
There is(are) \the\TotalFiles\ file(s) in total.
\end{document}