Copy-pasting leading whitespace and blank lines in listings package (pdf)

Question

I'm trying to use the listings package to typeset some code in a PDF document, in a way that allows me to copy-paste the resulting code. This mostly works well, if basicstyle=\ttfamily,columns=flexible is used as suggested here: Phantom spaces in listings (pdf)

The problem is that the leading whitespace is not copy-pasted correctly. Sure, it visually shows up correctly in the PDF, but selecting it and copy-pasting it omits leading whitespace and blank lines.

\documentclass{article}
\usepackage{listings}
\lstset{basicstyle=\ttfamily, columns=flexible}
\begin{document}
Some text.

\begin{lstlisting}
def f(x):
    return x+1

Previous line intentionally left blank.
\end{lstlisting}
\end{document}

Is there anything I can do to make it copy-paste correctly? I prefer using listings unless it's absolutely impossible with this package. (The document base I'm working on is pretty large so I'd like to make as few changes as possible to prevent unforeseen bugs.)

Answer 1

I found a solution to the leading whitespace!
Edit: Actually, it doesn't work in Adobe reader. It works in SumatraPDF.
Thanks to Daniel and cgnieder for their comments above.

As described by Martin Monperrus you can use the accsupp package to differentiate text that you copy-paste from the type-set text.

Note that the linked page is not usable out of the box. The way it is written there, surplus spaces are introduced during the accsupp replacement. Removing all the whitespace in the \newcommand seems to solve it. Here is a minimal example that does work:

\documentclass{article}
\usepackage{listings}
\lstset{
  basicstyle=\ttfamily,
  columns=fullflexible,
  literate={\ }{{\copyablespace}}1
}

\usepackage[space=true]{accsupp}
% requires the latest version of package accsupp
\newcommand{\copyablespace}{\BeginAccSupp{method=hex,unicode,ActualText=00A0}\ \EndAccSupp{}}

\begin{document} 
Some text.
\begin{lstlisting}
def f(x):
    return x+1

Previous line intentionally left blank.
\end{lstlisting}
\end{document}

Notes:
The basicstyle=\ttfamily and the columns=fullflexible are required to get the font monospaced and to prevent alignment-spaces from being introduced in the middle of your code, as detailed here.

The blank newlines still can't be copy pasted. Does anyone know if this can be done with accsupp as well?

I have for the time being ignored the special unicode characters and such that Martin Monperrus talks about. See that page if you have any problems with that. (Using "upquotes=true" fixes the most common problem)

---

If you don't want to use the accsupp package, this also works somewhat:

If you set showspaces=true, it turns all spaces into a visible space character, including leading spaces. This doesn't look very pretty, but they are indeed copyable.

The listings package allows you to set visible tab characters yourself, but unfortunately not visible space characters. Perhaps someone with more LaTeX knowledge can offer a suggestion on how to replace the visible space character with some form of invisible space character. I assume the copy-paste-ability would be maintained.

This also doesn't solve the blank newlines problem.

Proof of concept:

\documentclass{article}
\usepackage{listings}
\lstset{basicstyle=\ttfamily, columns=flexible, showspaces=true}
\begin{document}
Some text.

\begin{lstlisting}
def f(x):
    return x+1

Previous line intentionally left blank.
\end{lstlisting}
\end{document}

Answer 2

I cannot see any way of getting the blank lines copied either (inserting a line feed using \BeginAccSupp{method=hex,ActualText=0A}\EndAccSupp{} only works if there is a copying non-space character somewhere on the same line).

However, in the following, I present an alternative (more complicated) approach for copying "invisible" spaces at the start of lines. The comments provide some information about what packages can be loaded in combination. I believe it has wider viewer support (see below for compatibility table).

%http://tex.stackexchange.com/questions/142617
\documentclass{article}
%\usepackage[resetfonts]{cmap}% BAD: \textvisiblespace will be copied as ?. wrap \textvisiblespace in appropriate accsupp to fix
\usepackage[T1]{fontenc}
%\usepackage{lmodern}

\usepackage{textcomp}% for upquote
\usepackage{listings}
\lstset{
  basicstyle=\ttfamily,
  columns=fullflexible,
  tabsize=4,
  upquote=true,
  showstringspaces=true,
  %showspaces=true,% show all spaces except those at the beginning of a line
}
% from MinionPro manual, page 7 (http://www.tex.ac.uk/CTAN/fonts/minionpro/MinionPro.pdf)
%\input{glyphtounicode}
%\pdfgentounicode=1% BAD: \textvisiblespace will be copied as ?. wrap \textvisiblespace in appropriate accsupp to fix


\usepackage{xcolor}
\makeatletter
%an invisible visible space
% is a visible space character
% in the same font color as the listings background
% (or white if no background color is set)
% similiar to [How to make listings code indentation remain unchanged when copied from PDF?](http://tex.stackexchange.com/a/19978)
\newcommand{\lst@invisiblevisiblespace}{%
    \textcolor{white}{\lst@bkgcolor{\lst@visiblespace}}}%

% save old definitions
\AtBeginDocument{%delay in case any other package patches these
    \let\lst@newlineold@ProcessSpace\lst@ProcessSpace
    \let\lst@newlineold@ProcessTabulator\lst@ProcessTabulator
    \let\lst@newlineold@Append\lst@Append
}

\newcommand*{\lst@beginline}{%activate special whitespace handling
    \lst@ifdisplaystyle% but only for display listings, not for inline/TextStyle listings
        \let\lst@ProcessSpace\lst@newline@ProcessSpace
        \let\lst@ProcessTabulator\lst@newline@ProcessTabulator
        \let\lst@Append\lst@newline@Append
    \fi
}

% at the end of every line, prepare to deal with whitespace at the beginning of the next line
\lst@AddToHook{EOL}{%
    \lst@beginline
}

% spaces at the start of a line are made into visible spaces
% which are hidden by the choice of font color
\newcommand*{\lst@newline@ProcessSpace}{%
    % pretend keepspaces=true at the moment
    % but always use invisiblevisiblespace instead of outputspace
    % regardless of the setting of showspaces
    \lst@PrintToken
    \lst@whitespacetrue
    \lst@AppendOther\lst@invisiblevisiblespace
    \lst@PrintToken
}

% tabs at the start of a line are treated like consecutive spaces
% note that these will copy-paste as spaces instead of tabs
\newcommand*{\lst@newline@ProcessTabulator}{%
    \@tempcnta=\lst@tabsize\relax
    \loop
    \ifnum\@tempcnta>\z@
        \lst@newline@ProcessSpace
        \advance\@tempcnta\m@ne
    \repeat
}

\newcommand*{\lst@newline@Append}[1]{%
    \ifx#1\lst@invisiblevisiblespace
    \else
        % as soon as we append something to the output line
        % that isn't our special space, go back to the normal processing of spaces and tabulators
        \let\lst@Append\lst@newlineold@Append
        \let\lst@ProcessSpace\lst@newlineold@ProcessSpace
        \let\lst@ProcessTabulator\lst@newlineold@ProcessTabulator
    \fi
    \lst@newlineold@Append{#1}%
}

% the start of every listing is the beginning of a line
\lst@AddToHook{Init}{%
    \lst@beginline
}

\makeatother



%\lstset{backgroundcolor=\color{gray}}
\begin{document}
\pagestyle{empty}
\begin{lstlisting}[language=C++]
  int test(int x)
  {
    std::cout << "Hello world"; // print
    return x  +1;               // return
  }

    // Previous line intentionally left blank.
\end{lstlisting}

Note that spaces are still ignored at the start of inline listings: \lstinline^  test(5);^

\end{document}

This does not work in all viewers. For those that have been tested, see the compatibility table below. Feel free to add your own observations. Adding accsupp appears to further reduce viewer compatibility, not improve it.

Supported viewers:

• Adobe Reader XI
• PDF-XChange Viewer 2.5.213

Unsupported viewers:

• Firefox 25.0 pdf.js built-in viewer
• SumatraPDF v2.4
• Foxit Reader 5.3
• PDF Converter Professional 7.1