Syncing Python code with LaTeX document using Listings package

Question

I have a Tex document that uses the listings package to print a Python source code. I also want to separately print certain Python function signatures in my document. I do not want to use the line number functionality from the listings package since the Python code might be edited in the future.

Is there any way to automatically sync certain parts of the Python code with the Tex document (without using line numbers)?

Thank you very much!

Answer 1

The main problem is that you need to analyze the Python source code, which TeX can't (easily) do. Luckily Python is capable of analyzing itself, using abstract syntax trees (see also some more thorough documentation). The idea is that we have the Python ast analyze the code and write what it finds to a TeX file.

#!/usr/local/bin/python3

import ast

""" The python file we want to analyze.  Happens to be itself """
pythonfilename = 'pythonlinenumbers.py'

newcommands = []

def makenewcommand(command,
                   output):
    """ Turns the command and line number into the appropriate command.
    The signature is split onto two lines to make it more complicated.
    We have to play tricks with the trailing \, because we can't end a string
    with a single backslash. """
    return r'\newcommand{''\\'+command+'}{'+str(output)+'}\n'

class FuncLister(ast.NodeVisitor):
    def visit_FunctionDef(self, node):
        """ Recursively visit all functions, determining where each function
        starts, where its signature ends, and where the function ends.  Store
        these in the TeX variables \firstline@funcname, \sigend@funcname,
        \docend@funcname, and \lastline@funcname. """
        newname=node.name.replace('_','@') # _ isn't allowed in a TeX command
        newcommands.append(makenewcommand('firstline@'+newname,node.lineno))
        sigend = max(node.lineno,lastline(node.args))
        newcommands.append(makenewcommand('sigend@'+newname,sigend))
        docstring = ast.get_docstring(node)
        docstringlength = len(docstring.split('\n')) if docstring else -1
        newcommands.append(makenewcommand('docend@'+newname,sigend+docstringlength))
        newcommands.append(makenewcommand('lastline@'+newname,lastline(node)))
        self.generic_visit(node)

def lastline(node):
    """ Recursively find the last line of a node """
    return max( [ node.lineno if hasattr(node,'lineno') else -1 , ]
                +[lastline(child) for child in ast.iter_child_nodes(node)] )

with open(pythonfilename) as f:
    code = f.read()
FuncLister().visit(ast.parse(code))
with open('linenumbers.tex','w') as f:
    for newcommand in newcommands:
        f.write(newcommand)

Creates the file linenumbers.tex:

\newcommand{\firstline@makenewcommand}{10}
\newcommand{\sigend@makenewcommand}{11}
\newcommand{\docend@makenewcommand}{15}
\newcommand{\lastline@makenewcommand}{16}
\newcommand{\firstline@visit@FunctionDef}{19}
\newcommand{\sigend@visit@FunctionDef}{19}
\newcommand{\docend@visit@FunctionDef}{23}
\newcommand{\lastline@visit@FunctionDef}{32}
\newcommand{\firstline@lastline}{34}
\newcommand{\sigend@lastline}{34}
\newcommand{\docend@lastline}{35}
\newcommand{\lastline@lastline}{37}

Then, you can have your main TeX file input the derived TeX file, and use what it found:

\documentclass{article}

\usepackage{listings}

\immediate\write18{python3 pythonlinenumbers.py}
\makeatletter
\input{linenumbers}
\newcommand{\showfunc}[1]{%
 #1 signature:
 \lstinputlisting[
    firstline=\csname firstline@#1\endcsname,
    lastline=\csname sigend@#1\endcsname,
    language=Python]
 {pythonlinenumbers.py}

 #1 in its entirety:
 \lstinputlisting[
    firstline=\csname firstline@#1\endcsname,
    lastline=\csname lastline@#1\endcsname,
    language=Python]
 {pythonlinenumbers.py}}
\makeatother

\begin{document}

Here's the lastline function:

\showfunc{lastline}

\bigskip

Here's the makenewcommand function (also has a multi-line signature):

\showfunc{makenewcommand}

\end{document}

This results in

Answer 2

Here's another approach for unix-like systems with sed available (tested on windows+cygwin). You need to run with --enable-write18

\documentclass{article}
\usepackage{listings}

% Here we read the occurence of "def #2" and "end def #2" from 
% file #1 into lstart and lstop respectively  
% method taken from http://tex.stackexchange.com/a/70858/28808 thanks 
% to Antal Spector-Zabusky
\newread\myinput
\newcommand{\readfunc}[2]{% first parameter is the filename, second the function name
    \leavevmode
    \immediate\write18{sed -n '/def #2/=;//q' <#1 >\jobname_start.txt}
    \immediate\write18{sed -n '/end def #2/=;//q' <#1 >\jobname_stop.txt}
    \openin\myinput=\jobname_start.txt
    \bgroup
        \endlinechar=-1
        \read\myinput to \localline
        % Since everything in the group is local, we have to explicitly make the
        % assignment global
        \global\let\lstart\localline
    \egroup
    \openin\myinput=\jobname_stop.txt
    \bgroup
        \endlinechar=-1
        \read\myinput to \localline
        \global\let\lstop\localline
    \egroup
    \lstinputlisting[firstline=\lstart,lastline=\lstop]{#1}
}

This uses GNU sed to find the "def myfunc..." line, and relies on a matching "end def myfunc" comment. I have some pretty long definitions in my python script so tend to do this anyway. Test code:

\begin{filecontents}{test.py}
#a comment
def myfunc():
    print "This is a function"
#end def myfunc

myfunc()
\end{filecontents}

\begin{document}
\subsection{Listing 1}
\readfunc{test.py}{myfunc}
\end{document}

Temporary files are used to hold the output of sed and aren't deleted. The input code could probably be optimised from what I've written.