Ignore specific URL fields in biber?

Question

I am using Biber for my literature. Usually the URL should be printed out but some entries have "ugly" URLs like "http://dx.doi.org/..." etc. Is it possible to not print the URL if the URL field contains a specific string? I don't want to clean my bib file because maybe I want to use all URLs later.

I have tried the advice from karlkoeller, but it still doesn't work. Why the following code doesn't work?

\DeclareSourcemap{
  \maps[datatype=bibtex]{
    \map{
      \step[fieldsource=url,
        match=\regexp{dx.doi.org|dl.acm.org(.+)}, null=true
        ]
    }
  }
}

I don't know if this helps or not: If you put your DOIs into 'doi = {}' fields it's easy to exclude them with the BibLaTeX option 'doi=false'. — twsh, Sep 18 '13 at 00:07
Indeed, DOIs are best handled in the field doi. You can map specific URLs to the doi field as explained in Biblatex: Convert doi-url into doi field, that particular solution maps certain URLs to the doi field, it can easily be modified to dispose of those URLs though. — moewe, Sep 18 '13 at 06:37
Thx for your answer but is there a more general solution available? Some URL fields contain e.g. http://dl.acm.org/citation.cfm?id=xxxx which I also want to remove. — user4811, Sep 18 '13 at 13:34
The sourcemap answer below should do it for you, and is the right way, but having had the same problem I've adjusted my .bib so that all the DOIs are actually DOIs and not URLs to the DOI nameserver. They're still clickable with hyperref. — Chris H, Apr 02 '14 at 12:54

score 9 · Accepted Answer · edited Jun 10 '20 at 12:32

The solution for you is to use biblatex and biber's sourcemap features.

Add the following code snippet to your preamble.

\DeclareSourcemap{
  \maps[datatype=bibtex]{
    \map{
      \step[fieldsource=url, match=\regexp{http://(dx.doi.org/|dl.acm.org/)}, final]
      \step[fieldset=url, null]
      \step[fieldset=urldate, null]
    }
  }
}

You can extend the list of URLs to be deleted by specifying the URL in match=\regexp{http://(dx.doi.org/|dl.acm.org/)}. To filter example.com, make the line match=\regexp{http://(dx.doi.org/|dl.acm.org/|example.com)}, though you should probably use match=\regexp{http://(www.)?(dx.doi.org/|dl.acm.org/|example.com)} to also catch www.example.com.

biber uses Perl's Regular Expressions, you can read more about them here.

If biber finds a match, the url and urldate are purged, otherwise nothing happens.

The MWE

\documentclass{article}
\usepackage[style=authoryear,backend=biber]{biblatex}
\DeclareSourcemap{
  \maps[datatype=bibtex]{
    \map{
      \step[fieldsource=url, match=\regexp{http://(dx.doi.org/|dl.acm.org/)}, final]
      \step[fieldset=url, null]
      \step[fieldset=urldate, null]
    }
  }
}
\begin{filecontents}{\jobname.bib}
@article{testarto,
  author        = {Arnold Uthor},
  title         = {L'Article},
  date          = {2013},
  journal       = {Journal of Interesting Articles},
  volume        = {42},
  url           = {http://dx.doi.org/10.234/56789},
  urldate       = {2013-09-18},
}
@article{testartt,
  author        = {William Riter},
  title         = {Research},
  date          = {2012},
  journal       = {Journal of Interesting Articles},
  volume        = {12},
  url           = {http://dx.doi.org/10.234/56789},
  urldate       = {2013-09-18},
}
@article{testartf,
  author        = {Steve C. Ientist},
  title         = {Some Paper},
  date          = {2012},
  journal       = {Journal of the ACM},
  volume        = {12},
  url           = {http://dl.acm.org/citation.cfm?id=2487241&picked=prox&CFID=362895712&CFTOKEN=85933440},
  urldate       = {2013-09-19},
}
@online{blog,
  author        = {Bernie Logger},
  title         = {A Very Opinionated Blogpost},
  date          = {2013-09-12},
  url           = {http://www.example.com/blog/2013/09/12/opinion},
  urldate       = {2013-09-19},
}
\end{filecontents}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

yields enter image description here

Edit

biber's features enable us to implement a more sophisticated handling of those URLs.

The following MWE will map DOIs, stable JSTOR identifiers and arXiv links to the appropriate doi or eprintfields. If you do not want to have those displayed, just add doi=false and eprint=false to biblatex's options.

\documentclass{article}
\usepackage{hyperref}
\usepackage[style=authoryear,backend=biber]{biblatex}
\DeclareSourcemap{
  \maps[datatype=bibtex]{
    \map{% JSTOR stable
      \step[fieldsource=url, match=\regexp{https?://(www.)?jstor.org/stable/}, final]
      \step[fieldset=eprint, origfieldval]
      \step[fieldsource=eprint, match=\regexp{https?://(www.)?jstor.org/stable/}, replace={}]
      \step[fieldset=eprinttype, fieldvalue={jstor}]
      \step[fieldset=url, null]
      \step[fieldset=urldate, null]
    }
    \map{% arXiv
      \step[fieldsource=url, match=\regexp{https?://arxiv.org/(abs|pdf|ps|format)/}, final]
      \step[fieldset=eprint, origfieldval]
      \step[fieldsource=eprint, match=\regexp{https?://arxiv.org/(abs|pdf|ps|format)/}, replace={}]
      \step[fieldset=eprinttype, fieldvalue={arxiv}]
      \step[fieldset=url, null]
      \step[fieldset=urldate, null]
    }
    \map{% DOIs
      \step[fieldsource=url, match=\regexp{https?://dx.doi.org/}, final]
      \step[fieldset=doi, origfieldval]
      \step[fieldsource=doi, match=\regexp{https?://dx.doi.org/}, replace={}]
      \step[fieldset=url, null]
      \step[fieldset=urldate, null]
    }
    \map{% get rid of
      \step[fieldsource=url, match=\regexp{https?://(dl.acm.org/)}, final]
      \step[fieldset=url, null]
      \step[fieldset=urldate, null]
    }
  }
}
\begin{filecontents}{\jobname.bib}
@article{testarto,
  author        = {Arnold Uthor},
  title         = {L'Article},
  date          = {2013},
  journal       = {Journal of Interesting Articles},
  volume        = {42},
  url           = {http://dx.doi.org/10.234/56789},
  urldate       = {2013-09-18},
}
@article{testartt,
  author        = {William Riter},
  title         = {Research},
  date          = {2012},
  journal       = {Journal of Interesting Articles},
  volume        = {12},
  url           = {http://dx.doi.org/10.234/56789},
  urldate       = {2013-09-18},
}
@article{testartf,
  author        = {Steve C. Ientist},
  title         = {Some Paper},
  date          = {2012},
  journal       = {Journal of the ACM},
  volume        = {12},
  url           = {http://dl.acm.org/citation.cfm?id=2487241&picked=prox&CFID=362895712&CFTOKEN=85933440},
  urldate       = {2013-09-19},
}
@online{blog,
  author        = {Bernie Logger},
  title         = {A Very Opinionated Blogpost},
  date          = {2013-09-12},
  url           = {http://www.example.com/blog/2013/09/12/opinion/},
  urldate       = {2013-09-19},
}
@online{HawSpT,
  author        = {S. W. Hawking},
  title         = {The Nature of Space and Time},
  date          = {1994-09-30},
  url           = {http://arxiv.org/abs/hep-th/9409195},
  urldate       = {2013-09-19},
}
@online{HelfGold,
  author        = {H. A. Helfgott},
  title         = {Major arcs for Goldbach's theorem},
  date          = {2013-06-14},
  url           = {http://arxiv.org/abs/1305.2897v2},
  urldate       = {2013-09-19},
}
@article{FregeNumber,
  author        = {G. Frege},
  title         = {The Whole Number},
  journal       = {Mind},
  series        = {newseries},
  volume        = {79},
  number        = {316},
  pages         = {481-486},
  date          = {1970-10},
  url           = {http://www.jstor.org/stable/2252434},
  urldate       = {2013-09-19},
}
\end{filecontents}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

enter image description here

Thanks a lot. I have thought "final" skips further processing and "null" removes the entry. I don't know why your example works, but anyway it works!!! ;) — user4811, Sep 18 '13 at 17:59

score 1 · Answer 2 · answered Apr 02 '14 at 12:47

The above will not work properly with https links. The desired behaviour will occur if replace={} is used instead of replace=\regexp{$1}, like so:

\documentclass{article}
\usepackage{hyperref}
\usepackage[style=authoryear,backend=biber]{biblatex}

\DeclareSourcemap{
  \maps[datatype=bibtex]{
    \map{% JSTOR stable
      \step[fieldsource=url, match=\regexp{http(s)?://(www.)?jstor.org/stable/}, final]
      \step[fieldset=eprint, origfieldval]
      \step[fieldsource=eprint, match=\regexp{http(s)?://(www.)?jstor.org/stable/}, replace={}]
      \step[fieldset=eprinttype, fieldvalue={jstor}]
      \step[fieldset=url, null]
      \step[fieldset=urldate, null]
    }
    \map{% arXiv
      \step[fieldsource=url, match=\regexp{http(s)?://arxiv.org/(abs|pdf|ps|format)/}, final]
      \step[fieldset=eprint, origfieldval]
      \step[fieldsource=eprint, match=\regexp{http(s)?://arxiv.org/(abs|pdf|ps|format)/}, replace={}]
      \step[fieldset=eprinttype, fieldvalue={arxiv}]
      \step[fieldset=url, null]
      \step[fieldset=urldate, null]
    }
    \map{% DOIs
      \step[fieldsource=url, match=\regexp{http(s)?://dx.doi.org/}, final]
      \step[fieldset=doi, origfieldval]
      \step[fieldsource=doi, match=\regexp{http(s)?://dx.doi.org/}, replace={}]
      \step[fieldset=url, null]
      \step[fieldset=urldate, null]
    }
    \map{% get rid of
      \step[fieldsource=url, match=\regexp{http(s)?://(dl.acm.org/)}, final]
      \step[fieldset=url, null]
      \step[fieldset=urldate, null]
    }
  }
}    

\begin{filecontents}{\jobname.bib}
@article{testarto,
  author        = {Arnold Uthor},
  title         = {L'Article},
  date          = {2013},
  journal       = {Journal of Interesting Articles},
  volume        = {42},
  url           = {http://dx.doi.org/10.234/56789},
  urldate       = {2013-09-18},
}
@article{testartt,
  author        = {William Riter},
  title         = {Research},
  date          = {2012},
  journal       = {Journal of Interesting Articles},
  volume        = {12},
  url           = {http://dx.doi.org/10.234/56789},
  urldate       = {2013-09-18},
}
@article{testartf,
  author        = {Steve C. Ientist},
  title         = {Some Paper},
  date          = {2012},
  journal       = {Journal of the ACM},
  volume        = {12},
  url           = {http://dl.acm.org/citation.cfm?id=2487241&picked=prox&CFID=362895712&CFTOKEN=85933440},
  urldate       = {2013-09-19},
}
@online{blog,
  author        = {Bernie Logger},
  title         = {A Very Opinionated Blogpost},
  date          = {2013-09-12},
  url           = {http://www.example.com/blog/2013/09/12/opinion/},
  urldate       = {2013-09-19},
}
@online{HawSpT,
  author        = {S. W. Hawking},
  title         = {The Nature of Space and Time},
  date          = {1994-09-30},
  url           = {http://arxiv.org/abs/hep-th/9409195},
  urldate       = {2013-09-19},
}
@online{HelfGold,
  author        = {H. A. Helfgott},
  title         = {Major arcs for Goldbach's theorem},
  date          = {2013-06-14},
  url           = {http://arxiv.org/abs/1305.2897v2},
  urldate       = {2013-09-19},
}
@article{FregeNumber,
  author        = {G. Frege},
  title         = {The Whole Number},
  journal       = {Mind},
  series        = {newseries},
  volume        = {79},
  number        = {316},
  pages         = {481-486},
  date          = {1970-10},
  url           = {http://www.jstor.org/stable/2252434},
  urldate       = {2013-09-19},
}
\end{filecontents}
\addbibresource{\jobname.bib}

\begin{document}
\nocite{*}

Thanks for catching that problem. I think it would have been more appropriate as a comment on my post though. — moewe, Apr 09 '14 at 07:57

Ignore specific URL fields in biber?

2 Answers2

Edit