Malformed bcf file not recreated by latexmk after error

Question

I'm using latexmk with pdflatex to compile my thesis, with biblatex for references and biber as backend. It compiles fine and creates correct PDF output. If a change to the source files introduces an error the first run of pdflatex fails but a bcf file is created. The run of biber then complains about a malformed bcf file indicating that the last biblatex run failed and the compilation is stopped. However, after fixing the error latexmk thinks the pdflatex run was fine and invokes biber, but the bcf file is still malformed. latexmk somehow doesn't seem to notice that there were file changes.

Removing the bcf file or cleaning it with latexmk -C makes latexmk call pdflatex first and recreate a correct bcf file. Calling pdflatex manually also work but defeats the purpose of latexmk. I tried to use -halt-on-error as option to pdflatex but that doesn't work. I seem to recall that it worked with TexLive 2014, after a failed attempt to run biber latexmk would run pdflatex first on the next attempt.

I'm using TexLive 2015, the version of latexmk is 4.43a, biber has the version 2.3. The output produced is

Latexmk: This is Latexmk, John Collins, 5 February 2015, version: 4.43a.
Rule 'biber thesis': File changes, etc:
   Non-existent destination files:
      'thesis.bbl'
------------
Run number 1 of rule 'biber thesis'
------------
------------
Running 'biber  "thesis"'
------------
Latexmk: applying rule 'biber thesis'...
INFO - This is Biber 2.3
INFO - Logfile is 'thesis.blg'
ERROR - thesis.bcf is malformed, last biblatex run probably failed. Deleted thesis.bbl
INFO - ERRORS: 1
Latexmk: Failed to find one or more biber source files:
  NONE
Collected error summary (may duplicate other messages):
  biber thesis: Could not find all biber source files for 'thesis'
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs of latex/pdflatex.
Biber error: [33] Utils.pm:163> ERROR - thesis.bcf is malformed, last   biblatex run probably failed. Deleted thesis.bbl
Latexmk: Errors, so I did not complete making targets

Obligatory mwe.tex:

\documentclass[paper=a4]{scrartcl}

\usepackage[backend=biber]{biblatex}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\addbibresource{Bibliography.bib}

\begin{document}
\autocite{smith_pixel_1995}
\end{document}

Bibliography.bib

@article{smith_pixel_1995,
    title = {A Pixel Is Not A Little Square, A Pixel Is Not A Little Square, A Pixel Is Not A Little Square!},
    volume = {6},
    url = {http://ftp.alvyray.com/Memos/CG/Microsoft/6_pixel.pdf},
    journaltitle = {Microsoft Computer Graphics, Technical Memo},
    author = {Smith, Alvy Ray},
    urldate = {2016-02-04},
    date = {1995}
}

Steps to reproduce:

1. run latexmk -pdf mwe
2. introduce an error in mwe.tex like an undefined control sequence
3. run latexmk -pdf mwe
4. when prompted abort compilation with by pressing x
5. fix error
6. run latexmk -pdf mwe -> biber error
7. run latexmk -pdf mwe -> same biber error, won't go away

So the question is, how do I get latexmk to re-create the bcf file so that biber can use it without having to manually clean or remove files?

Answer 1

As already mentioned in a comment, the solution is to use the new version of latexmk (4.44 at the time I write this answer), which is now available at http://www.ctan.org/pkg/latexmk/

Answer 2

Update May 2019 March 2020:

This is still an issue with 18.04 LTS of Ubuntu (bionic), because they deliver version 4.41.

As stated by John this is fixed since version 4.44. Newer versions of latexmk are delivered for cosmic, disco, eoan, focal,..

There are at least three ways to upgrade, choose one of them:

a) Download and install the .deb package manually, e.g. from https://packages.ubuntu.com/focal/all/latexmk/download and hopefully it does not require any further dependencies to be upgraded.

b) I fixed that issue on my machine, by temporarily use the cosmic repository:

1. edit /etc/apt/sources.list at line of universe repo from bionic to cosmic, save
2. apt update
3. apt install latexmk
4. revert changes in /etc/apt/sources.list
5. apt update

Both is in general not recommended, but latexmk is a very simple app/script with few dependencies. Pinning is not required because hopefully the maintainer won't choose a version between 4.41 and 4.44, current version in cosmic is 4.59.

c) Another approach would be loading the newest package from CTAN and install it like in the documentation, basically replace and adjust permissions. See How can I upgrade latexmk / why very old version is contained in Ubuntu repository?.