htlatex converting ligatures to null bytes

Question

I am familiar with questions 162626 and with 204822, and have followed the instructions therein, but have been unable to get a good solution. htlatex is converting all of the ligatures in a document into null bytes. MWE:

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage[oldstyle,proportional]{libertine}
\begin{document}
find the fly; it's difficult, but fluffy.
\end{document}

My command line:

htlatex test.tex "xhtml, -css, charset=utf-8" " -utf8 -cunihtf" "" -shell-escape;

The problem seems to be in libertine; when I don't load it, the ligatures come out fine, but when I do, the ligatures are null. This suggests the solution of simply removing the libertine package and then compiling with htlatex, but I'd really like to avoid having to modify my document in order to convert it to HTML.

Does anyone have a suggested solution to this?

Answer 1

There is still no full support for Linux Libertine in tex4ht, as there are literally dozens fonts for various font encodings, styles, sizes and weights. But, it is not that hard to add support for basic T1 fonts, using htfgen.

You can find that some font support is missing from tex4ht command output:

tex4ht test.dvi " -cunihtf -utf8"

this will print really lot of text, because it prints contents of tex4ht.env file, which isn't exactly small. But at the beginning, you can find this:

tex4ht.c (2012-07-25-19:36 kpathsea)
tex4ht sample.dvi 
(/usr/local/texlive/2015/texmf-dist/tex4ht/base/unix/tex4ht.env)
(/usr/local/texlive/2015/texmf-dist/tex4ht/ht-fonts/iso8859/1/html/charset/unicode.4hf)
(/usr/local/texlive/2015/texmf-dist/fonts/tfm/public/libertine/LinLibertineTB-osf-t1.tfm)
(LinLibertineTB-osf-t1.htf)
Searching `LinLibertineT-osf-t1.htf' for `LinLibertineTB-osf-t1.htf'
--- warning --- Couldn't find font `LinLibertineT-osf-t1.htf' (char codes: 0--255)

----------------------------------------------------
environment file
----------------------------------------------------
% tex4ht.env-unix (2015-02-15-10:26), generated from tex4ht-env.tex

so, we need to create file LinLibertineT-osf-t1.htf, which corresponds to LinLibertineT-osf-t1.tfm. We can inspect used font encoding and whether the font is virtual or real with lstexenc command provided by htfgen project:

$ lstexenc LinLibertineT-osf-t1
LinLibertineT-osf-t1    vf      EXTENDED TEX FONT ENCODING - LATIN
  LinLibertineT-osf-t1--base    tfm     EXTENDED TEX FONT ENCODING - LATIN

so we see that it is virtual font and it uses some non-standard font encoding. It uses LinLibertineT-osf-t1--base as a base font, sometimes virtual fonts contains several fonts, so this isn't that bad.

Exact font encoding can be sometimes found in the .map file. Easiest way to find is to search with Google for fontname enc, so LinLibertineT-osf-t1--base enc in our case (yes, the enc file is provided for the base fonts. If the virtual font contain several real fonts, we need to find encodings for each one). The search finds file libertine.map, which provides encodings for all Libertine real fonts. We have to search it for LinLibertineT-osf-t1--base:

LinLibertineT-osf-t1--base LinLibertineT "AutoEnc_ulngvdga3jyvcsz3zod63p4gna ReEncodeFont" <[lbtn_ulngvd.enc <LinLibertineT.pfb

important is <[lbtn_ulngvd.enc part, which contains the encoding name, lbtn_ulngvd. Now we can create helper table for LinLibertineT-osf-t1:

tfmtochars LinLibertineT-osf-t1 lbtn_ulngvd > LinLibertineT-osf-t1.tsv

and finally to create the .htf file:

htfgen LinLibertineT-osf-t1.tsv > LinLibertineT-osf-t1.htf

the result is following:

LinLibertineT-osf-t1 0 255
'`'  ''  grave
'´'  ''  acute
'ˆ'  ''  circumflex
'˜'  ''  tilde
'¨'  ''  dieresis
'˝'  ''  hungarumlaut
'˚'  ''  ring
'ˇ'  ''  caron
'˘'  ''  breve
'¯'  ''  macron
'˙'  ''  dotaccent
'¸'  ''  cedilla
'˛'  ''  ogonek
'‚'  ''  quotesinglbase
'‹'  ''  guilsinglleft
'›'  ''  guilsinglright
'“'  ''  quotedblleft
'”'  ''  quotedblright
'„'  ''  quotedblbase
'«'  ''  guillemotleft
'»'  ''  guillemotright
'–'  ''  endash
'—'  ''  emdash
'&#xnil;'   ''  .notdef
'₀'  ''  perthousandzero
'ı'  ''  dotlessi
'ȷ'  ''  dotlessj
'fi'  ''  f_i
'ffi'  ''  f_f_i
'ff'  ''  f_f
'fl'  ''  f_l
'ffl'  ''  f_f_l
'␣'  ''  uni2423
'!'  ''  exclam
'"'  ''  quotedbl
'#'  ''  numbersign
'$'  ''  dollar
'%'  ''  percent
'&'  ''  ampersand
'’'  ''  quoteright
'('  ''  parenleft
')'  ''  parenright
'*'  ''  asterisk
'+'  ''  plus
','  ''  comma
'-'  ''  hyphen
'.'  ''  period
'/'  ''  slash
''  ''  zero.oldstyle
''  ''  one.oldstyle
''  ''  two.oldstyle
''  ''  three.oldstyle
''  ''  four.oldstyle
''  ''  five.oldstyle
''  ''  six.oldstyle
''  ''  seven.oldstyle
''  ''  eight.taboldstyle
''  ''  nine.oldstyle
':'  ''  colon
'&#x003B;'  ''  semicolon
'<'  ''  less
'='  ''  equal
'>'  ''  greater
'?'  ''  question
'@'  ''  at
'A'  ''  A
'B'  ''  B
'C'  ''  C
'D'  ''  D
'E'  ''  E
'F'  ''  F
'G'  ''  G
'H'  ''  H
'I'  ''  I
'J'  ''  J
'K'  ''  K
'L'  ''  L
'M'  ''  M
'N'  ''  N
'O'  ''  O
'P'  ''  P
'Q'  ''  Q
'R'  ''  R
'S'  ''  S
'T'  ''  T
'U'  ''  U
'V'  ''  V
'W'  ''  W
'X'  ''  X
'Y'  ''  Y
'Z'  ''  Z
'['  ''  bracketleft
'\'  ''  backslash
']'  ''  bracketright
'^'  ''  asciicircum
'_'  ''  underscore
'‘'  ''  quoteleft
'a'  ''  a
'b'  ''  b
'c'  ''  c
'd'  ''  d
'e'  ''  e
'f'  ''  f
'g'  ''  g
'h'  ''  h
'i'  ''  i
'j'  ''  j
'k'  ''  k
'l'  ''  l
'm'  ''  m
'n'  ''  n
'o'  ''  o
'p'  ''  p
'q'  ''  q
'r'  ''  r
's'  ''  s
't'  ''  t
'u'  ''  u
'v'  ''  v
'w'  ''  w
'x'  ''  x
'y'  ''  y
'z'  ''  z
'{'  ''  braceleft
'|'  ''  bar
'}'  ''  braceright
'~'  ''  asciitilde
'-'  ''  hyphen
'Ă'  ''  Abreve
'Ą'  ''  Aogonek
'Ć'  ''  Cacute
'Č'  ''  Ccaron
'Ď'  ''  Dcaron
'Ě'  ''  Ecaron
'Ę'  ''  Eogonek
'Ğ'  ''  Gbreve
'Ĺ'  ''  Lacute
'Ľ'  ''  Lcaron
'Ł'  ''  Lslash
'Ń'  ''  Nacute
'Ň'  ''  Ncaron
'Ŋ'  ''  Eng
'Ő'  ''  Ohungarumlaut
'Ŕ'  ''  Racute
'Ř'  ''  Rcaron
'Ś'  ''  Sacute
'Š'  ''  Scaron
'Ş'  ''  Scedilla
'Ť'  ''  Tcaron
'Ţ'  ''  Tcedilla
'Ű'  ''  Uhungarumlaut
'Ů'  ''  Uring
'Ÿ'  ''  Ydieresis
'Ź'  ''  Zacute
'Ž'  ''  Zcaron
'Ż'  ''  Zdotaccent
'IJ'  ''  IJ
'İ'  ''  Idotaccent
'đ'  ''  dcroat
'§'  ''  section
'ă'  ''  abreve
'ą'  ''  aogonek
'ć'  ''  cacute
'č'  ''  ccaron
'ď'  ''  dcaron
'ě'  ''  ecaron
'ę'  ''  eogonek
'ğ'  ''  gbreve
'ĺ'  ''  lacute
'ľ'  ''  lcaron
'ł'  ''  lslash
'ń'  ''  nacute
'ň'  ''  ncaron
'ŋ'  ''  eng
'ő'  ''  ohungarumlaut
'ŕ'  ''  racute
'ř'  ''  rcaron
'ś'  ''  sacute
'š'  ''  scaron
'ş'  ''  scedilla
'ť'  ''  tcaron
'ţ'  ''  tcedilla
'ű'  ''  uhungarumlaut
'ů'  ''  uring
'ÿ'  ''  ydieresis
'ź'  ''  zacute
'ž'  ''  zcaron
'ż'  ''  zdotaccent
'ij'  ''  ij
'¡'  ''  exclamdown
'¿'  ''  questiondown
'£'  ''  sterling
'À'  ''  Agrave
'Á'  ''  Aacute
'Â'  ''  Acircumflex
'Ã'  ''  Atilde
'Ä'  ''  Adieresis
'Å'  ''  Aring
'Æ'  ''  AE
'Ç'  ''  Ccedilla
'È'  ''  Egrave
'É'  ''  Eacute
'Ê'  ''  Ecircumflex
'Ë'  ''  Edieresis
'Ì'  ''  Igrave
'Í'  ''  Iacute
'Î'  ''  Icircumflex
'Ï'  ''  Idieresis
'Ð'  ''  Eth
'Ñ'  ''  Ntilde
'Ò'  ''  Ograve
'Ó'  ''  Oacute
'Ô'  ''  Ocircumflex
'Õ'  ''  Otilde
'Ö'  ''  Odieresis
'Œ'  ''  OE
'Ø'  ''  Oslash
'Ù'  ''  Ugrave
'Ú'  ''  Uacute
'Û'  ''  Ucircumflex
'Ü'  ''  Udieresis
'Ý'  ''  Yacute
'Þ'  ''  Thorn
'&#xnil;'   ''  .notdef
'à'  ''  agrave
'á'  ''  aacute
'â'  ''  acircumflex
'ã'  ''  atilde
'ä'  ''  adieresis
'å'  ''  aring
'æ'  ''  ae
'ç'  ''  ccedilla
'è'  ''  egrave
'é'  ''  eacute
'ê'  ''  ecircumflex
'ë'  ''  edieresis
'ì'  ''  igrave
'í'  ''  iacute
'î'  ''  icircumflex
'ï'  ''  idieresis
'ð'  ''  eth
'ñ'  ''  ntilde
'ò'  ''  ograve
'ó'  ''  oacute
'ô'  ''  ocircumflex
'õ'  ''  otilde
'ö'  ''  odieresis
'œ'  ''  oe
'ø'  ''  oslash
'ù'  ''  ugrave
'ú'  ''  uacute
'û'  ''  ucircumflex
'ü'  ''  udieresis
'ý'  ''  yacute
'þ'  ''  thorn
'ß'  ''  germandbls
LinLibertineT-osf-t1 0 255

(you need to add blank line at the end of the file, Stack Exchange eats is)

We should also add support for bold and italics. We don't have to repeat the whole process if the fonts use the same encoding. Unfortunately, each font use different encoding, but after diff it doesn't seem that they differ substantially, so we can just create .htf files, which doesn't include all unicode values for all characters, but just link to LinLibertineT-osf-t1. This second type of the htf file looks like this:

LinLibertineTB-osf-t1.htf:

.LinLibertineT-osf-t1
htfcss:  LinLibertineTB-osf-t1  font-weight: bold; font-family: "Linux Libertine O",serif;  

LinLibertineTI-osf-t1:

.LinLibertineT-osf-t1
htfcss:  LinLibertineTI-osf-t1  font-weight: normal; font-style: italic; font-family: "Linux Libertine O",serif;  

If you haven't disabled the CSS, you would see something like:

As you can see, the normal font isn't styled, but bold and italics are. You can fix it with simple config file:

\Preamble{xhtml}
\Css{body{font-family: "Linux Libertine O", serif;}}
\begin{document}
\EndPreamble

Also note that unicode old-style numbers are used instead of normal ones. Final result: