114

One frequent aspect of pdflatex output that could be improved is to what extent one can select text output from the generated pdf-file and copy-and-paste it. This is essentially an issue of having the right output encoding. There are at least two important aspects to this:

  • In the ideal case, Unicode would be the output encoding.
  • Ligature glyphs (such as fi) are ideally de-ligated in the output (here: "fi").

It seems that output encodings are specified through the fontenc package. What is the range of possible output encodings, and most importantly: Is there a way to specify a Unicode output encoding and one that deals with ligatures in the intended way?

(Note: It is important to distinguish output from input encodings. How to use different input encodings in LaTeX has been documented widely. My understanding is that this is best handled by the inputenx package, as it supersedes the inputenc package.)

Quick guide to the solutions: The information in the answers and answer threads is a bit distributed, so here the short summary: one approach is to use \input glyphtounicode together with \pdfgentounicode=1; the other approach is to use cmap/mmap.

Addendum: Sometimes one will need to load the package accsupp and enclose one's macro definition in \BeginAccSupp{method=hex,unicode,ActualText=<codepoint>}[...]\EndAccSupp{} to generate a specific code point. See the caveat about non-BMP code points here and the fix (starting from accsupp version v0.4, 2012/11/18) here, which provides the new unichar package option.

Community
  • 1
Lover of Structure
  • 22,025
  • 3
    Please avoid creating new tags when there are very similar ones existing already (output-encodings -> font-encodings; locales -> languages). If you think another name for that tag would be better, post it as a question on [meta]. – doncherry Jul 21 '12 at 12:43
  • @doncherry Thanks for the info; what is the best place to look for a comprehensive list of tag definitions and differentiation? – Lover of Structure Jul 21 '12 at 19:03
  • 1
    When I'm asking a question, I start typing in what I think would be a good tag; if an existing tag pops up, I use it. If not, I try to think of synonyms. Seeing the some 20k questions we have, you can be pretty sure that there are existing tags for the most important concepts. The tag wiki excerpts and the tag wikis (if they exist yet) give you a rough idea if you're using the tag for the right area. If they don't exist, take a look at the questions that are tagged with that tag. A rough rule of thumb really is: Don't create new tags. There are numerous exceptions to this rule, of course. – doncherry Jul 21 '12 at 23:29
  • 1
    @doncherry Another good method that I've discovered to work quite well for me is to look at the tags of the "Questions that may already have your answer". – Lover of Structure Oct 08 '12 at 23:23
  • \usepackage{ae} is a partial solution - the copy'n'paste doesn't produce gibberish, but characters gets copied decomposed (e.g. 'ó' -> '`o'). – Jakub Narębski Feb 25 '15 at 14:58
  • 1
    Even with \usepackage{lmodern}, \usepackage[T1]{fontenc} and \usepackage[utf8]{inputenc} (a combination which permits to copy many characters, including those given in the OP examples, from the PDF), copying \succeq from the resulting PDF does not work. Adding \input glyphtounicode \pdfgentounicode=1 makes it work: \succeq gets copied to unicode ⪰. – Olivier Cailloux Feb 26 '19 at 22:57

6 Answers6

59

Do you Know the package cmap (look at CTAN:cmap)? It does this for you. Load it with \usepackage{cmap} as first package.

Update: I did a little research and found some hints how to use cmap or mmap. Result: cmap and mmap can't handle fonts based on virtual fonts (files *.vf or *.vpl in the font directory). So if your used font needs virtual fonts cmap or mmap can't work. You should better use glyphtounicode. This hint could be useful.

BTW: cmap is based on the character maps of Adobe. See PDF reference Version 1.6 for more information.

Update2: With texdoc encguide you will get a document describing the T1 Cork encoding. There you will find a table showing all the glyphs you can directly write in your pdf file. Because there is a glyph ä you can find ä in your pdf file. If there is no glyph ä LaTeX has to build it with two characters: "a. You will see ä in your pdf file but you can't find it (to search "a could help; copy and paste give you this back).

Update 3: glyphtounicode supports only the T1-encoding, cmap supports many more encodings. One advantage of glyphtounicode is that one can add with one command more glyphs which haven't yet been included in the official list. More information about \glyphtounicode is here: tb91thanh-fonts.pdf. Make sure to get the latest version of glyphtounicode.tex from LCDF Type Software and see this caveat on how to properly use it.

Community
  • 1
Mensch
  • 65,388
  • @user14996 glyphtounicode functionality that cmap does not offer (f.ex. support for virtual fonts). But cmap offers more encodings. If you only need T1-encoding, use glyphtounicode. – Sveinung Jul 30 '12 at 10:17
  • @LoverofStructure What makes you think \pdfgentounicode only supports T1 encoding? – Lev Bishop Feb 18 '13 at 01:48
  • @LevBishop I don't remember how this fact made it into here; I think it was merged from another answer or comment. Please correct if justified. – Lover of Structure Feb 18 '13 at 11:21
  • @Sveinung Perhaps you remember about the T1-part (see previous comment by Lev Bishop)? – Lover of Structure Feb 18 '13 at 11:22
  • 1
    +1 for a working answer; however, the answer would be improved if it said what cmap is/does. As is, I need to click the external link to get a description. – vossad01 Feb 14 '17 at 16:53
  • cmap doesn't seem to work with \usepackage{libertinus} -- is the order important? – Fuhrmanator Dec 24 '19 at 00:27
18

As you don't provide a minimal example, I am not sure if this really addresses your problem. Have you tried loading glyphtounicode.tex?

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}

\input glyphtounicode
\pdfgentounicode=1

\begin{document}

ff ffi ffl ä ü ö ß

auffallen

\end{document}

However, even without using glyphtounicode, pdftotext produces unicode text on my system. With glyphtounicode, the ligatures get de-ligated.

Michael
  • 1,294
  • Thanks for the starter code. Some things: With the inputenc-line it is unclear what the difference is if I compile the code with or without the two glyphtounicode-lines. Therefore I have replaced your first text line with ff ffi ffl \"a \"u \"o \ss, removed the inputenc-line, saved it in ANSI format and compiled (with and without the fontenc-line). Result: (1) With the fontenc-line, it oddly makes no difference in the output whether we use the two glyphtounicode-lines. ... – Lover of Structure Jul 24 '12 at 19:02
  • 1
    ... (2) Without the fontenc-line, the two glyphtounicode-lines are required for sensible output, but even then the ligatures will copy-and-paste as ¨a ¨u ¨o. – Lover of Structure Jul 24 '12 at 19:03
  • Update (a correction to my very first assertion ("With the inputenc-line [...] two glyphtounicode-lines.")): With the original äöüß-chars and the inputenc-line, the results of my experimentation is actually exactly the same: With the fontenc-line, the two glyphtounicode-lines don't make a difference; without the fontenc-line, the two glyphtounicode-lines indeed enable proper copy-and-pasting but the umlaut characters paste as ¨a ¨o ¨u, which is surely not as intended. So I am unclear about the exact role of glyphtounicode and its interaction with fontenc. – Lover of Structure Jul 24 '12 at 19:20
  • Which font are you using? I am using minionpro and never experienced any interaction. With fontenc and without glyphtounicode, I got strange result when copying and pasting from the .PDF-files. – Sveinung Jul 25 '12 at 08:11
  • By the way, without fontenc, accented characters are composed of a letter and a diacrit (ä is ¨a etc.) so they copy/paste ‘correctly’. – Sveinung Jul 25 '12 at 13:13
  • @Sveinung I think you could argue that they do paste correctly, though you could also argue they should paste with a Unicode combining diaeresis. As for my setup, I've just started exactly with Michael's code and then modified it as I described. // Basically my question is why glyphtounicode doesn't make a difference in output or pasting behavior when the fontenc-line is used. – Lover of Structure Jul 26 '12 at 20:27
  • @user14996 I have updated my answer with some of my own findings. – Sveinung Jul 26 '12 at 21:40
  • @user14996 Still not satisfied with the answers? – Sveinung Jul 27 '12 at 21:38
  • @Sveinung Many thanks! I ended up accepting Kurt's answer because of his summary research, but your contribution is valuable too. Too bad attributing partial points is so cumbersome in SE (works only with bounty mechanics). // One more thing about "a: I'd strongly argue that "correct" pasting behavior yields compact characters whenever possible. That means when the output encoding is Unicode, I'd prefer ä over either clumsy "a or "a with a combining diaeresis thereafter". But you can justify "a if the output encoding doesn't have "ä", so you're indeed right about the case w/o fontenc. – Lover of Structure Jul 28 '12 at 20:49
  • @user14996 Therefore I put the word correctly in citation marks, to emphasize that correctly should not be interpret literally. – Sveinung Jul 29 '12 at 09:39
  • When loading T1 font encoding, you should also load a vector font like lmodern as discussed in this question: http://tex.stackexchange.com/a/1293 – matth Oct 27 '12 at 10:15
6

If you are using memoir, you can compile your document with the option ms. The output is like something that are written with a typewriter, but since it has no hyphenation, it is easy to copy and paste the text for reuse in another program, such as Word or LibreOffice Writer.

I often use this option as a quick and dirty way to get the text in a LaTeX document into Word.

Since memoir use 1.5 line spacing in ms-mode, it is important to set the number of lines per pages to a number that is not overfilling the page.

A MWE:

\documentclass{ms,article}{memoir}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{lipsum}

\input{glyphtounicode}
\pdfgentounicode=1

\setulmargins{*}{*}{1,5}
\setlength{\textheight}{32\baselineskip}
\checkandfixthelayout[lines]


\begin{document}

\lipsum[1-21]

\end{document}

EDIT: user14996 asked: Why glyphtounicode doesn't make a difference in output or pasting behavior when the fontenc-line is used?

The answer to that is: It depends on your font. I use MinionPro, which use minuscule figures (oldstylenums) as standard. I also use spaced small capitals. With the glyphtounicode, but without the fontenc, loaded, Norwegian characters (æøå ÆØÅ) is not copied correctly from the PDF-file. On the other hand, all numbers and small caps can be copied and pasted. If I do the opposite, remove glyptounicode but keep fontenc, the numbers comes out as Greek characters, and the small caps disappear. Also, all ligatures (fi ffi fj) disappear.

Using cmap in stead of glyphtounicode does not help.

So the answer is that sometimes fontenc is the only package you need, but if you use special features in a font (small caps, old style figures etc.) which require virtual fonts, you need to load glyphtounicode. cmap does not support virtual fonts. Sometimes not even glyphtounicode solves all problems.

Sveinung
  • 20,355
4

I am not proficient in latex, but in order to achieve copyable Cyrillic PDFs you can use a document of the following kind.

\documentclass{article}
\usepackage{cmap}
\usepackage[utf8x]{inputenx}
\usepackage[T2A]{fontenc}
\usepackage[T2B]{fontenc}
\usepackage[T2C]{fontenc}

\begin{document}

Кирилица

\end{document}

T2A, T2B and T2C are sets of different variants of Cyrillic alphabets.

Sogartar
  • 141
4

I had this problem for quite some time in the past. I solved it in a way which worked well for me, and I found the following notes about it:

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

I agree that fontenc (as encoding of the output) is what you would expect. However, I have been using the german package as well. And that tends to assemle german umlauts as strange custom tokens (like an 'a' with two dots drawn on top of it rather than the correct unicode char). I used inputenc to fix that.

I understand that this answer addresses mainly the question "How can I make pdflatex output copy-pastable pdfs?". It leaves the predicate "good" for others who might be able to answer your specific questions. But perhaps this is still useful as it seemed to work fine, or at least fine enough.

Christian Feuersänger
  • 55,840
  • @Michael There seems to be some interaction between glyphtounicode and fontenc, which I haven't yet figured out. I'll be happy about any insights about the precise nature of this interaction. – Lover of Structure Jul 24 '12 at 19:26
3

It is been a while but I'd like to contribute to offer a solution for LyX.

Type:

\usepackage{cmap}
\usepackage[T2A]{fontenc}

in the preamble (Document-> Preferences).

Then activate under languages (Document->Preferences->Language) for the encoding: "Different -> Unicode(utf8).

Close and save.

Lukas
  • 143