11

Maybe the question is also of interest beyond biblatex but here's the thing: I recently published my biblatex style biblatex-fiwi and I'm always having trouble making sure that updates don't lead to unexpected behavior. For example, I just realized that my last update includes a stupid error where a colon is dropped at an important place. The colon was there in previous versions but since I wasn't careful enough it's gone at the moment.

What I would like to have is some kind of automated testing environment which would check a set of bibliographic entries against a defined outcome. At the moment I always have to check the final PDF myself which is error-prone.

What's to easiest way to do this?

Simifilm
  • 3,168
  • 23
  • 29
  • Follow-up: is there a way to do just the same, but with my whole latex distribution ? I often see package or font updates en up breaking stuff in subtle ways (in particular, ways that pdftotext would not notice). but maybe that deserves a question on its own. – Gyom Dec 09 '11 at 07:09
  • @Gyom: compare also with a diff the two log files. –  Dec 09 '11 at 07:16
  • See also http://tex.stackexchange.com/q/36351/5763 – Martin Schröder Dec 09 '11 at 16:47
  • @Herbert, MartinSchröder: thanks a lot for these suggestions. I'll definitely try these as soon as I find a little bit of time to work on that topic ! – Gyom Dec 13 '11 at 18:49

2 Answers2

8

I have the same issue with biblatex-apa. I opted for a perl script using the CAM::PDF module which can compare PDFs and tell you the differences. I save a "reference" PDF of the test document included with the style and then run this script to compare the this PDF with the one output from the same test document after changes. If all ok, this becomes the new reference document. The output of the script isn't really nice - it compares page by page and says "ok" if the page is the same and otherwise outputs a fairly useless error message but at least I know which page to check for differences at that point ... Script is below. You can see where my two PDFs are mentioned. I just run this in the same directory as the two PDFs.

#!/usr/bin/perl

use Test::More;
use Test::Deep;
use CAM::PDF;

cmp_pdf('biblatex-apa-test.pdf', 'reference.pdf');
done_testing;


sub cmp_pdf ($$) {
  my ($got, $expected) = @_;
  $got = CAM::PDF->new($got) or die "Could not create CAM::PDF instance with : $got";
  $expected = CAM::PDF->new($expected) or die "Could not create CAM::PDF instance with : $expected";

  foreach my $page_num (1 .. $expected->numPages()) {
    my $tree1 = $got->getPageContentTree($page_num, "verbose");
    my $tree2 = $expected->getPageContentTree($page_num, "verbose");
    cmp_deeply($tree1->{blocks}, $tree2->{blocks}, "Page $page_num");
  }
}
doncherry
  • 54,637
PLK
  • 22,776
7
  • Save the old output,
  • create the new one,
  • convert both with pdftotext to true text,
  • compare the output with diff.

can all be done with a script