pythontex does not consider certain changes unless I delete its files folder?

Question

In the past, I have asked a question about including multiple images through Python in a tex document and I was given a solution which worked great (Link to the old question).

My new problem: Now I have noticed that adding new images after already having compiled the document in LaTeX results into new images not being considered through pythontex. The new images are only added after I delete the pythontex-files-[documentname] folder manually. I find this manual task a little bit annoying and I am looking for either a way to automate this without breaking my code or maybe finding another simpler solution. Maybe I am just doing something wrong?

Example: Let's say I have the following file structure:

├── Images
│   ├── Interesting image.jpg
│   └── Some image.jpg
├── pythontex-files-test
│   └── [Some files made by pythontex]
├── test.pdf
├── test.tex
└── [.aux, .log, .pytxcode, and .gz files] 

Now I run my code which works fine to add those images:

\documentclass{article}
\usepackage{pythontex}
\usepackage{graphicx}
\begin{document}
My images:
\begin{pycode}
import os
import textwrap
directory = 'Images'
extension = '.jpg'
files = [fn for fn in os.listdir(directory) if fn.lower().endswith(extension)]
files.sort()
figs = []
for filename in files:
    caption = filename.replace(r'', r'\')
    caption = caption.split('.')[0]
    directory_file = directory+'/'+filename
    fig = fr'''
    \begin{{figure}}[!ht]
    \centering
    \includegraphics[height=4cm]{{{directory_file}}}
    \caption{{{caption}}}
    \end{{figure}}
    '''
    figs.append(textwrap.dedent(fig))
print(''.join(figs))
\end{pycode}
\end{document}

The result looks like this and I am happy with it:

Afterwards I am adding a new image to the Images-folder called new-image.jpg.

Images
├── Interesting image.jpg
├── new-image.jpg
└── Some image.jpg

I would like to have the new image added automatically when compiling again. My code runs fine without any errors BUT the new image is not added unless I delete the pythontex-files-[documentname] folder. Only after the manual deletion of that folder, the new image is added.

• I would like to know the reason behind this - I could not find anything related to this in the documentation on CTAN.

• Is there maybe an easy method to delete the folder automatically before compiling without doing the compilation in a Python file using os.system('pdflatex '+filename+'.tex')? I have already tried the following but it did not work:

  \begin{pycode}   
   import os
   import textwrap
  pythontex_folder = 'pythontex-files-test'
  pythontex_files = [os.path.join(pth, f) for pth, dirs, files in os.walk(pythontex_folder) for f in files]
  for pth in pythontex_files:
       os.remove(pth)
   [...]
   \end{pycode}
  

• Is there maybe different way to achieve what I want (for example by slightly changing my existing code)?

Thanks in advance for any help!

Answer 1

The caching mechanism of pythontex can create some problems. I think a really simple workaround is to save your code in a separate Python file and call it from LaTeX. In order to do this, we use the --shell-escape option in the compiler. In TeXStudio, this can be done by setting txs:///pdflatex --shell-escape as the standard compiler. Instead of printing out the code for figures, you can save them to a file and \input from LaTeX. For example, this can be your Python file figure.py:

import os
import textwrap
directory = 'Images'
extension = '.jpg'
files = [fn for fn in os.listdir(directory) if fn.lower().endswith(extension)]
files.sort()
figs = []
for filename in files:
    caption = filename.replace(r'', r'\')
    caption = caption.split('.')[0]
    directory_file = directory+'/'+filename
    fig = fr'''
    \begin{{figure}}[!ht]
    \centering
    \includegraphics[height=4cm]{{{directory_file}}}
    \caption{{{caption}}}
    \end{{figure}}
    '''
    figs.append(textwrap.dedent(fig))
save figure content to figure.vrb
with open('figure.vrb', 'w') as outfile:
    outfile.write(''.join(figs))

Here is a minimum working example for your TeX file:

\documentclass{article}
\usepackage{expl3}
\begin{document}
\ExplSyntaxOn
% call python script, here I assume the binary name is python3
\sys_shell_now:n {python3~figure.py}
\ExplSyntaxOff
% input the generated figure file
\input{figure.vrb}
\end{document}