How to use latexdiff to compare changes in .bib files?

Question

I read several of frederik's answers on latexdiff, but I still struggle to get latexdiff to mark differences for references (i.e., printed via \printbibliography).

Suppose I have two .tex and corresponding .bib files, all residing in the same folder.

Old version

• old.tex:

  \documentclass{article}
  \usepackage[american]{babel}
  \usepackage[style = apa, backend = biber]{biblatex}
  \usepackage{csquotes}
  % Library settings.
  \DeclareLanguageMapping{american}{american-apa}
  % Load the library.
  \addbibresource{old.bib}
  % Main content.
  \begin{document}
  This is a simple citation \cite{hastieElementsStatisticalLearning2009}.
  % References.
  \printbibliography
  \end{document}
  

• old.bib:

  @book{hastieElementsStatisticalLearning2009,
      title = {The Elements of Statistical Learning: Data Mining, Inference, and Prediction},
      author = {Hastie, Trevor and Tibshirani, Robert and Friedman, Jerome},
      date = {2009},
      edition = {2},
      publisher = {Springer-Verlag},
      url = {https://www.springer.com/gp/book/9780387848570}
  }
  

New version

• new.tex:

  \documentclass{article}
  \usepackage[american]{babel}
  \usepackage[style = apa, backend = biber]{biblatex}
  \usepackage{csquotes}
  % Library settings.
  \DeclareLanguageMapping{american}{american-apa}
  % Load the library.
  \addbibresource{new.bib}
  % Main content.
  \begin{document}
  This is an updated citation \cite{hastieElementsStatisticalLearning2009}.
  % References.
  \printbibliography
  \end{document}
  

• new.bib:

  @book{hastieElementsStatisticalLearning2009,
      title = {The Elements of Machine Learning: Data Mining, Inference, and Prediction},
      author = {Tibshirani, Trevor and Hastie, Robert and Friedman, Jerome},
      date = {2009},
      edition = {2},
      publisher = {Springer-Verlag},
      url = {https://www.springer.com/gp/book/9780387848570}
  }
  

Using latexdiff, I would like to see not only the differences in the .tex files, but also the differences in the .bib files. For instance, the change in author name (i.e., swapped Hastie with Tibshirani), and the change in the title field (i.e., replaced Learning with Machine in the title). For clarity, these are the changes between the .tex files:

• old.tex:
  • This is a simple citation \cite{hastieElementsStatisticalLearning2009}.
• new.tex:
  • This is an updated citation \cite{hastieElementsStatisticalLearning2009}.

And these are the changes between the .bib files:

• old.bib:
  • title = {The Elements of Statistical Learning: Data Mining, Inference, and Prediction},
  • author = {Hastie, Trevor and Tibshirani, Robert and Friedman, Jerome},
• new.bib:
  • title = {The Elements of Machine Learning: Data Mining, Inference, and Prediction},
  • author = {Tibshirani, Trevor and Hastie, Robert and Friedman, Jerome},

To do this, I ran the following commands in the root folder where the .tex and .bib files are located:

# Create a `build` directory.
mkdir ./build
Run pdflatex on the .tex files to generate .bcf files.
pdflatex -output-directory=build -interaction=nonstopmode old.tex
pdflatex -output-directory=build -interaction=nonstopmode new.tex
Run biber on the .bcf files to generate .bbl files.
biber --output-directory=build --input-directory=build old.bcf
biber --output-directory=build --input-directory=build new.bcf

According to frederik's answer, I should now run latexdiff on the .bbl files and produce a dif.bbl files with the differences between the old.bib and new.bib files.

# Run `latexdiff` on the `.bbl` files.
latexdiff ./build/old.bbl ./build/new.bbl > ./build/diff.bbl

Then, I can go ahead and run latexdiff on the actual .tex files to produce the diff.tex file.

# Run `latexdiff` on the `.tex` files.
latexdiff old.tex new.tex > ./build/diff.tex

Finally, now I have to run pdflatex on the diff.tex to generate the .pdf

# Run `pdflatex` to produce the `.pdf`.
pdflatex -output-directory=build -interaction=nonstopmode diff.tex

This produces a warning regarding undefined references:

LaTeX Warning: There were undefined references.
Package biblatex Warning: Please (re)run Biber on the file:
(biblatex)                diff
(biblatex)                and rerun LaTeX afterwards.

However, the .pdf is generated, but the bibliography differences are not being picked up by latexdiff, as it can be seen below:

If I follow the instructions in the warning and re-run biber on diff.bcf, the new diff.bbl that gets created overrides the previously created diff.bbl (i.e., via the command latexdiff ./build/old.bbl ./build/new.bbl > ./build/diff.bbl).

I found another answer by frederik where it is recommended to use --append-textcmd=field and --flatten flags. So I retried the following, which sadly produces exactly the same thing as above:

# Remove directory.
rm -rf ./build/
Create a build directory.
mkdir ./build
Run pdflatex on the .tex files to generate .bcf files.
pdflatex -output-directory=build -interaction=nonstopmode old.tex
pdflatex -output-directory=build -interaction=nonstopmode new.tex
Run biber on the .bcf files to generate .bbl files.
biber --output-directory=build --input-directory=build old.bcf
biber --output-directory=build --input-directory=build new.bcf
Run latexdiff on the .bbl files.
latexdiff ./build/old.bbl ./build/new.bbl > ./build/diff.bbl
Run latexdiff on the .tex files with the recommended flags.
latexdiff --append-textcmd=field --flatten old.tex new.tex > ./build/diff.tex
Run pdflatex to produce the .pdf.
pdflatex -output-directory=build -interaction=nonstopmode diff.tex

I also followed the instruction from the compiler output and run the following subsequent commands:

# Run `biber` on the `diff.bcf` to update the `diff.bbl`.
biber --output-directory=build --input-directory=build diff.bcf
Re-run pdflatex to re-generate diff.pdf.
pdflatex -output-directory=build -interaction=nonstopmode diff.tex

And, again nothing changes, the output is the same:

Do you have any ideas oh how I can get latexdiff to mark differences in the References section, as well as in the citation itself?

---

Update (1) based on frederik's comments

1. Replacing \printbibliography with \bibliography:

• I replaced \printbibliography with, for example, \bibliography{old.bib} in old.tex and it did not work. The file compiled and produced the output, although there was a ! LaTeX Error in the compiler output. However, neither the citations, nor the references were created.

2. Replacing \printbibliography with the .bbl contents

• this resulted in a fatal error, and no .pdf file was produced
• there were a lot of ! Undefined control sequence. messages (e.g., for \field, or \strng)

3. Adding the --append-textcmd=field option

• added the --append-textcmd=field when generating the diff.bbl using latexdiff, i.e.:

  latexdiff --append-textcmd=field ./build/old.bbl ./build/new.bbl > ./build/diff.bbl
  

• removed the --flatten option from the latexdiff old.tex new.tex > ./build/diff.tex, i.e.:

  latexdiff old.tex new.tex > ./build/diff.tex
  

• so, the sequence of commands ran now is:

  # Remove directory.
  rm -rf ./build/
  Create a build directory.
  mkdir ./build
  Run pdflatex on the .tex files to generate .bcf files.
  pdflatex -output-directory=build -interaction=nonstopmode old.tex
  pdflatex -output-directory=build -interaction=nonstopmode new.tex
  Run biber on the .bcf files to generate .bbl files.
  biber --output-directory=build --input-directory=build old.bcf
  biber --output-directory=build --input-directory=build new.bcf
  Create the diff.bbl file.
  latexdiff --append-textcmd=field ./build/old.bbl ./build/new.bbl > ./build/diff.bbl
  Run latexdiff on the .tex files with the options removed.
  latexdiff old.tex new.tex > ./build/diff.tex
  Run pdflatex to produce the .pdf.
  pdflatex -output-directory=build -interaction=nonstopmode diff.tex
  

And this is already an improvement because now the changes in the bibliography are picked up:

The compiler output still recommends to rerun biber on diff.bcf followed by pdflatex, but doing that removes all the commands added by latexdiff in the diff.bbl file.

The only thing not being picked up is the change in the author name. Perhaps the option --append-textcmd=field should be adjusted to include something else other than \field?

---

Update (2)

It seems that adjusting the option --append-textcmd does the trick. For example, adjusting it to also include the name:

latexdiff --append-textcmd="field,name" ./build/old.bbl ./build/new.bbl > ./build/diff.bbl

Produces the following:

Answer 1

Assuming the setup with the old.tex and new.tex files provided in the question, the following commands will produce the desired output in a build directory.

• create build directory

  mkdir ./build
  

• run pdflatex on each .tex file to produce the .bcf files

  pdflatex -output-directory=build -interaction=nonstopmode old.tex
  pdflatex -output-directory=build -interaction=nonstopmode new.tex
  

• run biber on each .bcf file to produce the .bbl files

  biber --output-directory=build --input-directory=build old.bcf
  biber --output-directory=build --input-directory=build new.bcf
  

• run latexdiff on the .bbl files to mark the changes

  • Note. The option --append-textcmd is crucial here for latexdiff to know what kind of changes we are interested in. We can look inside one of the .bbl files to understand how each piece of information is stored. In the question above, the changes are in the title (i.e., \name{author}) and author (i.e., \field{title}). So, we can add those two to --append-textcmd as --append-textcmd="field,name".

  latexdiff --append-textcmd="field,name" ./build/old.bbl ./build/new.bbl > ./build/diff.bbl
  

• run latexdiff on the .tex files to produce a .tex file with the changes

  • Note. If I understand the documentation of latexdiff correctly, adding the --flatten option to the command below would spare us the need to run latexdiff on the .bbl files as we did above. With this option, latexdiff would replace the \bibliography{...} command with the contents of diff.bbl (i.e., generated on the fly) in the output .tex. However, since we use biber as the citation backend, we have \printbibliography instead of \bibliography{...}. As frederik indicated in the comments above, latexdiff is not configured to look for \printbibliography commands. We can, in principle, still create a "flat" document by adding the contents of a .bbl file manually as shown in this answer.

  latexdiff old.tex new.tex > ./build/diff.tex
  

• finally, run pdflatex in the diff.tex to produce the .pdf

  pdflatex -output-directory=build -interaction=nonstopmode diff.tex
  

The above will result in the marked changes shown below.