Can't replace non-ascii characters with biblatex

Question

This is a follow-up to my question Translate content in fields in bibliography entries (biblatex), which revealed a problem when trying to replace content in a .bib file when that content contain non-ascii characters like æøå.

Compiling the following MWE works fine. In this case, the string def is replaced by abc:

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "def",
    YEAR = "1968"}
\end{filecontents}
\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {def}, 
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

But when the string contains non-ascii characters like æøå, no output is produced with LaTeX:

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "æøå",
    YEAR = "1968"}
\end{filecontents}
\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {æøå}, 
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

The console produces this output (biblatextest64.tex being the name of the document) :

Failure to make 'biblatextest64.pdf'
Collected error summary (may duplicate other messages):
  biber biblatextest64: Command for 'biber biblatextest64' gave return code 256
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs of latex/pdflatex.
Latexmk: Errors, so I did not complete making targets
C:\texlive2013\bin\win32\runscript.tlu:650: command failed with exit code 12:
perl.exe c:/texlive2013/texmf-dist/scripts/latexmk/latexmk.pl -pdf biblatextest64.tex

A pdf is produced when compiling with XeLaTeX, but no substitution has taken place:

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage{fontspec}
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "æøå",
    YEAR = "1968"}
\end{filecontents}
\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {æøå}, 
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

Answer 1

It seems that the value given to match is passed through TeX's expansion mechanism, which is a big problem, but also a help, since we can (also selectively) use \detokenize.

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "æøå",
    YEAR = "1968"}
\end{filecontents}

\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {\detokenize{æøå}},
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

This is what is written in the .bcf file:

  <!-- SOURCEMAP -->
  <bcf:sourcemap>
    <bcf:maps datatype="bibtex" level="user">
      <bcf:map>
        <bcf:map_step map_field_source="title" map_match="æøå" map_replace="abc"/>
      </bcf:map>
    </bcf:maps>

Without \detokenize, the result is

  <!-- SOURCEMAP -->
  <bcf:sourcemap>
    <bcf:maps datatype="bibtex" level="user">
      <bcf:map>
        <bcf:map_step map_field_source="title" map_match="\T1\ae \T1\o <E5>" map_replace="abc"/>
      </bcf:map>
    </bcf:maps>

So it seems that \detokenize works correctly. However, the replacement doesn't take place, which seems to be a problem with Biber.

To wit, I tried the following Perl script (adapted from https://stackoverflow.com/a/1572156)

use warnings;
use strict;
use utf8;
binmode STDOUT, "utf8";
my $string = "æøå";

$string =~ s/æøå/abc/gi; 

print "$string\n";

and the result on the terminal was

abc