9

I have just changed computers, and running Biber 2.11 now produces this error:

Utils.pm:209> ERROR - Data file ... cannot be read in encoding 'UTF-8': 
Can't decode ill-formed UTF-8 octet sequence <96> at C:\ ... Slurper.pm line 63

My .bib file is huge, having accumulated over years. Thus, I suspect that there's something malformed in it. Is there any rapid way to trouble-shoot it?

The new machine runs Win 10, Biber 2.11, XeLaTeX 2018.10.31, LaTeX2e 2018-04-01 and TeXLive 2018. The old machine ran Win 7 and slightly earlier versions of all of the above.

A MWE of the driver .tex file:

\documentclass{article} % also with beamer

\usepackage[utf8]{inputenc} % problem arises whether or not include this
\usepackage[backend=biber,bibencoding=utf8]{biblatex} %  problem arises whether or not include this
\addbibresource{bib-mwe.bib}

\begin{document}

cite reference \cite{mal-rwws}

\printbibliography

\end{document}

Thanks!

Colin Rowat
  • 477
  • 3
    save a copy of the bib file (just in case:-), then load it into a text editor and re-save it as utf-8 encoding. – David Carlisle Nov 13 '18 at 18:23
  • 1
    Unfortunately, the underlying library that Biber uses does not provide a line number for the error, see also https://github.com/plk/biber/issues/226. Copy-and-paste or normal recoding might be an option, but depending on how permissive your editor is that might not be enough. You may want to have a look at https://unix.stackexchange.com/q/6516 and similar things on the web. Of course it could be possible that your complete .bib file is consistently in an encoding different from UTF-8, then recoding will probably be a good idea. – moewe Nov 13 '18 at 21:11

3 Answers3

5

As explained in https://github.com/plk/biber/issues/226 the underlying library that Biber uses to read .bib files does not provide a line number for the error. So Biber can't tell you the place of the error.

Usually the problem is simply that the entire file does not have the guessed/declared encoding. Note that the entire file content matters, so even the comments could be an issue.

For the MWE the .bib file must be encoded in UTF-8 (or a compatible encoding like US-ASCII). Even without the explicit bibencoding=utf8, biblatex would guess from the \usepackage[utf8]{inputenc} that you are using UTF-8, and if your LaTeX is newer than 2018-04-01, UTF-8 would even be assumed without inputenc since it is the standard now.

If a .bib file acts up, you should go through the following steps.

  1. Determine which encoding it should have (usually UTF-8 is a good idea, but if you have declared a different encoding in your .tex file, biblatex will normally assume that your .bib file uses the same encoding, of course that can be overridden with bibencoding).
  2. Find out which encoding your .bib file actually has.
  3. If the encodings don't match, you need to recode your .bib file or tell biblatex to try a different encoding.

It may work to just copy and paste the contents of your .bib file to a new file that is explicitly set up as UTF-8.

moewe
  • 175,683
  • Is not working. At least when I compile file in repository with GitHub actions. I still get the same error – Mafsi Sep 11 '21 at 13:06
2

I ran into the same problem. Instead of using a text editor, I opened the .bib file in jabref. You can change the library encoding using 'Library Properties' from the 'File' menu. Then save the library. This also changed the encoding specification on the first line of the .bib file, which I would otherwise have had to change manually (when I would have used the text editor).

Biba Blendi
  • 21
  • I had a couple of "non-ISO" characters causing the same error. Fixed with JabRef, Quality-->Check Integrity... finding in one reference non-ASCII encoded character found. Click on the reference. Quick look and fixed deleting it. – Hastur Oct 07 '20 at 22:39
1

Thanks for the helpful comments.

Following David's, the problem is solved by:

  1. open the .bib file in a text editor (e.g., Notepad)
  2. set the encoding in WinEdt (my LaTeX editor) to UTF-8 by clicking on the menu bar at the bottom (originally set to ANSI)
  3. create a blank document in WinEdt
  4. paste the contents of the original .bib file from Notepad into WinEdt
  5. save the new WinEdt document as a .bib file
  6. compile using the new .bib file
khatchad
  • 454
Colin Rowat
  • 477