0

I am trying to write a PhD thesis in TexStudio backended with biber & biblatex. I've been using JabRef to handle the bib file until several days ago. However decided to switch to Mendeley, but parsing a lot of PDFs often with hopeless metadata entries (as Mendeley struggled to cross reference them online) seems to have created some non-UTF8 encoded strings in my new bib file and now upon passing through biber during TexStudio build I am getting the following error:

INFO - This is Biber 2.15 (beta)
INFO - Logfile is 'Thesis.blg'
INFO - Reading 'Thesis.bcf'
INFO - Found 26 citekeys in bib section 0
INFO - Processing section 0
INFO - Looking for bibtex format file 'Bib/Mendeley/libraryjab.bib' for section 0
ERROR - Data file 'Bib/Mendeley/libraryjab.bib' cannot be read in encoding 'utf8': Wide character at C:\Users\user\AppData\Local\Temp\par-776f6c6665\cache-b32225defac3568d1e921af5a7245ca571b236aa\inc\lib/Win32/Unicode/File.pm line 163.

INFO - ERRORS: 1

I have checked the file, it is almost definitely UTF8 encoded itself (exported file has a comment line at the top confirming it), I tried saving it again as suggested here, which hasn't helped. I checked the preamble and the

\usepackage[utf8]{inputenc}

\usepackage
    [backend=biber,
    bibencoding=utf8]{biblatex}

are definitely there.

I ran the JabRef entry clean-up tool to convert all text fields to Unicode. No dice. I even tried to use the older version of the bib file pre-dating switch to Mendeley - which DOES work fine with my .tex file, so the problem is most certainly with one/some of the new entries.

I also tried removing all of the files generated by the Build and Building again. Same problems persist. I have tried to manually prune whatever erroneous entries I could find, but I continue to get the error.

Is there a way I could to find and remove the wide characters causing the problem in my .bib file? There are hundreds of references so I'd rather avoid continuing to investigate them manually.

Wolfe79
  • 303
  • 2
    that is a bug in biber, it has problems with too long bib-files. Try the development version, see https://github.com/plk/biber/issues/332 – Ulrike Fischer Oct 14 '20 at 21:23
  • cat input.bib | iconv -f utf-8 -t latin1 -c | iconv -f latin1 -t utf-8 > output.bib maybe? – Oleg Lobachev Oct 14 '20 at 21:23
  • Going by what @UlrikeFischer says, can anyone advise if switching to LF encoding would potentially cause issues? I'm not sure splitting the bib file into smaller chunks that biber can cope with makes any sense for a lengthy bibliography that I'll end up having. – Wolfe79 Oct 14 '20 at 21:35
  • why don't you simply install biber 2.16dev? you only need to put the binary in your path https://sourceforge.net/projects/biblatex-biber/files/biblatex-biber/development/binaries/Windows/ – Ulrike Fischer Oct 14 '20 at 21:39
  • @DavidCarlisle I don't know what exactly happens, I only know that it broke if the file exceeded a certain length, the content didn't matter (it broke also if it was ascii). – Ulrike Fischer Oct 14 '20 at 21:41
  • what is LF encoding? – David Carlisle Oct 14 '20 at 21:45
  • Forgive me being a noob with these things - how do I install biber from a zipped executible? It seems to do nothing when I launch it straight from a download folder and I usually downloaded these via the MikTeX console. @UlrikeFischer – Wolfe79 Oct 14 '20 at 21:53
  • 2
    you only need to unzip the zip, it contains a biber.exe. You can either replace your existing biber.exe with it, but you can also leave it in the folder and call it as C:\path\to\the\folder\biber, or you can rename it to eg. biber-dev.exe and put somewhere where it is found and then call biber-dev.exe instead. – Ulrike Fischer Oct 14 '20 at 22:00

1 Answers1

2

Ok, @UlrikeFischer's solution for this biber bug has worked! Bibliography has loaded without issues. Looks like it's indeed an issue with .bib file length. For anyone experiencing this in the future, replace the biber 2.15 with 2.16.

Many thanks folks!

Wolfe79
  • 303
  • And see the other topic https://tex.stackexchange.com/q/562607/206959 for the details. This seems to be an issue Win only bug. – Henning Oct 27 '20 at 14:14