Biblatex get cited references from multiple documents

Question

I'm currently working on a huge document that concatenates every publications of my lab during past year. I've split it into volumes for printing convenience. I now want to add the bibliography in each volume, using the cited category, but taking into account what happens in every volumes… Is it possible?

Vol1.tex

% !TeX document-id = {59748d60-1764-4a68-aa7d-506dedf42c2a}
% !TeX TS-encoding = utf8
% !TeX TS-spellcheck = fr_FR
% !BIB TS-program = biber
% !TeX TS-program = lualatex

\documentclass[12pt, a4paper]{memoir}

\usepackage{fontspec}
\defaultfontfeatures{Ligatures=TeX}%,Numbers=OldStyle}% ,Scale=MatchLowercase} bug in current Biolinum
\setmainfont{Linux Libertine O}
\setsansfont{Linux Biolinum O}

\usepackage{polyglossia}
\setmainlanguage{french}
\setotherlanguage{english}
\usepackage[hyphens,spaces,obeyspaces]{url}
\urlstyle{same}
\newcommand{\murl}[2]{\href{#2}{#1}\footnote{\url{#2}}}
\let\oldurl\url
\renewcommand{\url}[1]{\begin{english}\oldurl{#1}\end{english}}

\usepackage{hyperref}
\hypersetup{
    pdfauthor={Bur Nicolas},
    pdftitle={Recueil des publications ESTIA},
    pdfencoding=auto,
    pdffitwindow = false,
    pdfstartview = {FitH},
    bookmarksopen = true,
    bookmarksnumbered = true,
    bookmarksopen = true,
    bookmarksopenlevel = 3,
    bookmarksdepth = 4,
    colorlinks = true,    %colorise les liens
    breaklinks = true,    %permet le retour à la ligne dans les liens trop longs
    urlcolor = black,        %couleur des hyperliens
    linkcolor = black,        %couleur des liens internes
    citecolor = black,
    backref=page,
    pdfpagelabels,
    pdfencoding=auto,
    unicode,
    naturalnames,
}

\newcommand{\Publi}[1]{%
    \clearpage
    \phantomsection
    \addcontentsline{toc}{chapter}{\fullcite{#1}}
    \markboth{\footnotesize\citetitle{#1}}{\footnotesize\citetitle{#1}}
     \null
     \vfill
     {\Large \fullcite{#1}}
     \vfill
     \clearpage
 }

 \usepackage{csquotes}
 \usepackage[%
     language=french,
     backend=biber,
     sorting=nyt,
     backref=true,
     indexing=true,
     maxnames=99,
     style=authoryear-icomp,
     url=false,
 ]{biblatex}
 \DeclareBibliographyCategory{cited}
 \AtEveryCitekey{\addtocategory{cited}{\thefield{entrykey}}}

 \addbibresource{Extraction_Hal_2017.bib}

 \begin{document}
     \Publi{abiakle:hal-01438790}
     \Publi{abiakle:hal-01664371}
     \Publi{abiakle:hal-01664375}

     \nocite{*}
     \printbibliography[heading=bibintoc, category=cited] % from both Vol1 and Vol2
     \cleardoublepage
 \end{document}

Vol2.tex (with exact same preamble)

\begin{document}
    \setcounter{page}{4}
    \Publi{ambrosino:hal-01534697}
    \Publi{badets:hal-01655230}
    \Publi{barquissau:hal-01655914}

    \nocite{*}
    \printbibliography[heading=bibintoc, category=cited] % from both Vol1 and Vol2

    \cleardoublepage
\end{document}

Extraction_Hal_2017.bib

@inproceedings{abiakle:hal-01664375,
    TITLE = {{Helping inhabitants in energy saving and getting inputs from usage for eco-design: Cooking case study}},
    AUTHOR = {Abi Akle, Audrey and Lizarralde, Iban},
    URL = {https://hal.archives-ouvertes.fr/hal-01664375},
    BOOKTITLE = {{The 21st International Conference on Engineering Design (ICED17 )}},
    ADDRESS = {Vancouver, Canada},
    YEAR = {2017},
    MONTH = Aug,
    HAL_ID = {hal-01664375},
    HAL_VERSION = {v1},
    USERA = {COMCL},
}
@article{abiakle:hal-01438790,
    TITLE = {{Information visualization for selection in Design by Shopping}},
    AUTHOR = {Abi Akle, Audrey and Minel, St{\'e}phanie and Yannou, Bernard},
    URL = {https://hal.archives-ouvertes.fr/hal-01438790},
    JOURNAL = {{Research in Engineering Design}},
    PUBLISHER = {{Springer Verlag}},
    VOLUME = {28},
    NUMBER = {1},
    PAGES = {99-117},
    YEAR = {2017},
    MONTH = Jan,
    DOI = {10.1007/s00163-016-0235-2},
    KEYWORDS = { Computer aided design (CAD) ;  Decision making ; Visual Design ; Computer Aided Design (CAD) ; Decision Making},
    PDF = {https://hal.archives-ouvertes.fr/hal-01438790/file/RiED%202016%20-%20AbiAkle%20Minel%20Yannou.pdf},
    HAL_ID = {hal-01438790},
    HAL_VERSION = {v1},
    USERA = {ART},
}
@inproceedings{abiakle:hal-01664371,
    TITLE = {{Design space visualization for efficiency in knowledge discovery leading to an informed decision}},
    AUTHOR = {Abi Akle, Audrey and Minel, St{\'e}phanie and YANNOU, Bernard},
    URL = {https://hal.archives-ouvertes.fr/hal-01664371},
    BOOKTITLE = {{The 21st International Conference on Engineering Design (ICED17 )}},
    ADDRESS = {Vancouver, Canada},
    YEAR = {2017},
    MONTH = Aug,
    KEYWORDS = {Visualization ; Computer aided design (CAD) ; Decision making ; Design by shopping},
    PDF = {https://hal.archives-ouvertes.fr/hal-01664371/file/ICED%202017%20-%20Abi%20Akle%20et%20al.pdf},
    HAL_ID = {hal-01664371},
    HAL_VERSION = {v1},
    USERA = {COMCL},
}
@inproceedings{ambrosino:hal-01534697,
    TITLE = {{IdeaBulb: A Smart and Tangible User Interface for Monitoring Ideation During Creative Sessions}},
    AUTHOR = {Ambrosino, Julien and Daniel, Maxime and Masson, Dimitri and Legardeur, J{\'e}r{\'e}my},
    URL = {https://hal.archives-ouvertes.fr/hal-01534697},
    BOOKTITLE = {{Sixth International Conference on Smart Cities, Systems, Devices and Technologies (SMART 2017)}},
    ADDRESS = {Venice, Italy},
    ORGANIZATION = {{IARIA}},
    EDITOR = {XPS (Xpert Publishing Services)},
    PUBLISHER = {{Berntzen, L. et al.}},
    SERIES = {SMART INTERFACES 2017, The Symposium for Empowering and Smart Interfaces in Engineering},
    PAGES = {pp. 70-74},
    YEAR = {2017},
    MONTH = Jun,
    KEYWORDS = {idea generation ; ideation ; tangible interface ; creativity},
    PDF = {https://hal.archives-ouvertes.fr/hal-01534697/file/Smart%20Interface%20IdeaBulb_v2.1.pdf},
    HAL_ID = {hal-01534697},
    HAL_VERSION = {v2},
    USERA = {ACTCL},
}
@inproceedings{badets:hal-01655230,
    TITLE = {{Human efficiency for reducing Lean limits }},
    AUTHOR = {Badets, Patrick and Merlo, Christophe and Pilniere, V{\'e}ronique},
    URL = {https://hal.archives-ouvertes.fr/hal-01655230},
    BOOKTITLE = {{12e congr{\`e}s international de g{\'e}nie industriel}},
    ADDRESS = {Compi{\`e}gne, France},
    YEAR = {2017},
    MONTH = May,
    KEYWORDS = {Lean approach ; D{\'e}marche Lean ; mod{\`e}le de performance ; mod{\'e}lisation d'entreprise ; -- Lean approach ; performance model ; enterprise modelling ; GRAI},
    PDF = {https://hal.archives-ouvertes.fr/hal-01655230/file/CIGI2017_paper48-6finale.pdf},
    HAL_ID = {hal-01655230},
    HAL_VERSION = {v1},
    USERA = {COMM},
}
@inproceedings{barquissau:hal-01655914,
    TITLE = {{Les forums d'actionnaires individuels : un outil de r{\'e}duction du risque ?}},
    AUTHOR = {Barquissau, {\'E}ric and Darras-Barquissau, Nathalie},
    URL = {https://hal.archives-ouvertes.fr/hal-01655914},
    BOOKTITLE = {{Oriane - 15{\`e}me colloque francophone sur le risque}},
    ADDRESS = {Bayonne, France},
    ORGANIZATION = {{IUT Bayonne Pays Basque and Universit{\'e} de Pau et des Pays de l'Adour}},
    YEAR = {2017},
    MONTH = Sep,
    KEYWORDS = {Actionnaires individuels ; R{\'e}seaux sociaux ; Risque financier ; Risque per{\c c}u ; Netnographie ; Forum boursier},
    PDF = {https://hal.archives-ouvertes.fr/hal-01655914/file/Oriane%202017-Barquissau-Darras.pdf},
    HAL_ID = {hal-01655914},
    HAL_VERSION = {v1},
    USERA = {COMM},
}

Edit to add some clarification.

It seems feasible to print all cited references as @gusbrs pointed out in the first comment. However I'd like to have the backref to the page (and ideally to the volume) in which the reference is cited.

From the python script I can know how many volumes I'll have; using temporary file and \immediate\write at the end of a volume with \pageref{derniere_page} and \read the previous saved value at the very beginning, I'm able to set a sequential pagination through volumes.

This is a first thought. I suppose you could extract the cited references in each volume with https://tex.stackexchange.com/q/41821/105447. Then add both files as bibresources for each volume and use \nocite{*}. But perhaps someone will think of something cleverer. — gusbrs, Mar 20 '18 at 10:54
Sure that will do the job. But it'll lack the backref (see p. x).
I'd like to mention everything is done via a python script:

get bib file and pdf from HAL,

write tex files inputing pdf not to exceed a given page number

compile.

Thus, the “something cleverer” could be added in it. — NBur, Mar 20 '18 at 11:13
Well, in that case, you'll need someone cleverer too. :) Let's see what others have to say. — gusbrs, Mar 20 '18 at 11:16
Oh, actually the script is running! I just wanted to say the solution could rely on extra tools, not only (lua)latex, biblatex, biber tricks… — NBur, Mar 20 '18 at 11:30
Mmh, I suppose your python script can be set up to generate a bib file that contains "all and only" the entries which will be cited in vols 1 and 2. In this case, it would fall back on my initial suggestion. And, if I understand correctly what you want, you could simply drop the category=cited from your \printbibliographys to get two equal bibliographies in each volume (with backrefs for the volume). — gusbrs, Mar 20 '18 at 11:30
Oh, NBur, I didn't mean to sound grumpy in my previous comment. It was just I was out of ideas. — gusbrs, Mar 20 '18 at 11:33
If this is not what you want to achieve, please be more specific as to what you mean with "taking into account what happens in every volumes". — gusbrs, Mar 20 '18 at 13:04
I mean I want not only to print references cited in every volumes, but also the backref. For instance if refA is in vol1 and refB in vol2, I want them both displayed in bibliography (your solution can do this) but also with the mention cited p. x (in vol1) and p. y (in vol2) — NBur, Mar 20 '18 at 17:09
I see. I believe these specifications deserve to be included in the question itself. Regarding how to achieve it, does your python script allows you to either: i) add a field in the entry; or ii) partition the entries generating one bib file per volume? By the way, approximately how many volumes are we speaking of? — gusbrs, Mar 20 '18 at 17:19
Mmh, no. This idea would not be enough. I thought of something (based on either of those possibilities) to add the information regarding the volume. But the page backref would still be missing. This information would have to be made available somehow, either in the bibentry itself or in a "common aux file". But I'm afraid this is indeed out of my reach. — gusbrs, Mar 20 '18 at 17:25
One more question. I'm I right to assume that the pagination of the volumes will not start from 1 in all of them but would rather be sequential? (that is, first page in vol 2 = last page in volume 1 +1) — gusbrs, Mar 20 '18 at 17:35
And to answer previous comment, it should be possible to add the requested feature in the python script — NBur, Mar 21 '18 at 08:04

gusbrs · Accepted Answer · 2018-03-21T18:06:01.247

Considering you mentioned you have the possibility of either adding a field in the entry or partitioning the entries generating one bib file per volume based on you python script, I think I may suggest one possible approach, which, however, possibly does not achieve the desired degree of automation.

Both possibilities essentially can be handled by the same procedure. By partitioning the bib files per volume, we can use this information to add the volume number with a \DeclareSourcemap to an arbitrary field (here I use userd, it can be any, as long as it is not used somewhere else). If you can add a field to the entries, that information can be inserted directly.

\DeclareSourcemap{
    \maps[datatype=bibtex]{
        \map[overwrite]{
            \perdatasource{Extraction_Hal_2017_vol1.bib}
            \step[fieldset=userd, fieldvalue={1}]
        }
        \map[overwrite]{
            \perdatasource{Extraction_Hal_2017_vol2.bib}
            \step[fieldset=userd, fieldvalue={2}]
        }
    }
}

However, to get the backreferences right working with separate documents, the information regarding the backref would have to be stored per entry and then somehow fed back to biblatex. That might be possible (it usually is...), but I don't know how to do it. I thus suggest you to use a single main document with volumes added with \include which will allow to gather all the relevant info in the main document, and also print them selectively with \includeonly.

So, the structure in full would be:

main.tex:

% !TeX document-id = {59748d60-1764-4a68-aa7d-506dedf42c2a}
% !TeX TS-encoding = utf8
% !TeX TS-spellcheck = fr_FR
% !BIB TS-program = biber
% !TeX TS-program = lualatex

\documentclass[12pt, a4paper]{memoir}

\usepackage{fontspec}
\defaultfontfeatures{Ligatures=TeX}%,Numbers=OldStyle}% ,Scale=MatchLowercase} bug in current Biolinum
\setmainfont{Linux Libertine O}
\setsansfont{Linux Biolinum O}

\usepackage{polyglossia}
\setmainlanguage{french}
\setotherlanguage{english}
\usepackage[hyphens,spaces,obeyspaces]{url}
\urlstyle{same}
\newcommand{\murl}[2]{\href{#2}{#1}\footnote{\url{#2}}}
\let\oldurl\url
\renewcommand{\url}[1]{\begin{english}\oldurl{#1}\end{english}}

\usepackage{hyperref}
\hypersetup{
    pdfauthor={Bur Nicolas},
    pdftitle={Recueil des publications ESTIA},
    pdfencoding=auto,
    pdffitwindow = false,
    pdfstartview = {FitH},
    bookmarksopen = true,
    bookmarksnumbered = true,
    bookmarksopen = true,
    bookmarksopenlevel = 3,
    bookmarksdepth = 4,
    colorlinks = true,    %colorise les liens
    breaklinks = true,    %permet le retour à la ligne dans les liens trop longs
    urlcolor = black,        %couleur des hyperliens
    linkcolor = black,        %couleur des liens internes
    citecolor = black,
    backref=page,
    pdfpagelabels,
    pdfencoding=auto,
    unicode,
    naturalnames,
}

\usepackage{csquotes}
\usepackage[%
    language=french,
    backend=biber,
    sorting=nyt,
    backref=true,
    indexing=true,
    maxnames=99,
    style=authoryear-icomp,
    url=false,
]{biblatex}

\DeclareBibliographyCategory{cited}
\AtEveryCitekey{\addtocategory{cited}{\thefield{entrykey}}}

\newcommand{\Publi}[1]{%
    \clearpage
    \phantomsection
    \addcontentsline{toc}{chapter}{\fullcite{#1}}
    \markboth{\footnotesize\citetitle{#1}}{\footnotesize\citetitle{#1}}
    \null
    \vfill
    {\Large \fullcite{#1}}
    \vfill
    \clearpage
}

\renewbibmacro*{pageref}{%
    \iflistundef{pageref}
    {}
    {\printtext[parens]{%
        \ifnumgreater{\value{pageref}}{1}
        {\bibstring{backrefpages}\ppspace}
        {\bibstring{backrefpage}\ppspace}%
        \printlist[pageref][-\value{listtotal}]{pageref}%
        \setunit{\addcomma\space}%
        \printtext{in\space vol\adddot\space}%
        \printfield{userd}}}}

\addbibresource{Extraction_Hal_2017_vol1.bib}
\addbibresource{Extraction_Hal_2017_vol2.bib}


\DeclareSourcemap{
    \maps[datatype=bibtex]{
        \map[overwrite]{
            \perdatasource{Extraction_Hal_2017_vol1.bib}
            \step[fieldset=userd, fieldvalue={1}]
        }
        \map[overwrite]{
            \perdatasource{Extraction_Hal_2017_vol2.bib}
            \step[fieldset=userd, fieldvalue={2}]
        }
    }
}

%\includeonly{vol1}
%\includeonly{vol2}

\begin{document}

\include{vol1}

\include{vol2}

\renewcommand{\thepage}{}
\printbibliography[heading=bibintoc, category=cited]

\end{document}

vol1.tex:

\Publi{abiakle:hal-01438790}
\Publi{abiakle:hal-01664371}
\Publi{abiakle:hal-01664375}

vol2.tex:

\Publi{ambrosino:hal-01534697}
\Publi{badets:hal-01655230}
\Publi{barquissau:hal-01655914}

Extraction_Hal_2017_vol1.bib (the partitioning of the bib files being handled by your python script):

@inproceedings{abiakle:hal-01664375,
    TITLE = {{Helping inhabitants in energy saving and getting inputs from usage for eco-design: Cooking case study}},
    AUTHOR = {Abi Akle, Audrey and Lizarralde, Iban},
    URL = {https://hal.archives-ouvertes.fr/hal-01664375},
    BOOKTITLE = {{The 21st International Conference on Engineering Design (ICED17 )}},
    ADDRESS = {Vancouver, Canada},
    YEAR = {2017},
    MONTH = Aug,
    HAL_ID = {hal-01664375},
    HAL_VERSION = {v1},
    USERA = {COMCL},
}
@article{abiakle:hal-01438790,
    TITLE = {{Information visualization for selection in Design by Shopping}},
    AUTHOR = {Abi Akle, Audrey and Minel, St{\'e}phanie and Yannou, Bernard},
    URL = {https://hal.archives-ouvertes.fr/hal-01438790},
    JOURNAL = {{Research in Engineering Design}},
    PUBLISHER = {{Springer Verlag}},
    VOLUME = {28},
    NUMBER = {1},
    PAGES = {99-117},
    YEAR = {2017},
    MONTH = Jan,
    DOI = {10.1007/s00163-016-0235-2},
    KEYWORDS = { Computer aided design (CAD) ;  Decision making ; Visual Design ; Computer Aided Design (CAD) ; Decision Making},
    PDF = {https://hal.archives-ouvertes.fr/hal-01438790/file/RiED%202016%20-%20AbiAkle%20Minel%20Yannou.pdf},
    HAL_ID = {hal-01438790},
    HAL_VERSION = {v1},
    USERA = {ART},
}
@inproceedings{abiakle:hal-01664371,
    TITLE = {{Design space visualization for efficiency in knowledge discovery leading to an informed decision}},
    AUTHOR = {Abi Akle, Audrey and Minel, St{\'e}phanie and YANNOU, Bernard},
    URL = {https://hal.archives-ouvertes.fr/hal-01664371},
    BOOKTITLE = {{The 21st International Conference on Engineering Design (ICED17 )}},
    ADDRESS = {Vancouver, Canada},
    YEAR = {2017},
    MONTH = Aug,
    KEYWORDS = {Visualization ; Computer aided design (CAD) ; Decision making ; Design by shopping},
    PDF = {https://hal.archives-ouvertes.fr/hal-01664371/file/ICED%202017%20-%20Abi%20Akle%20et%20al.pdf},
    HAL_ID = {hal-01664371},
    HAL_VERSION = {v1},
    USERA = {COMCL},
}

and Extraction_Hal_2017_vol2.bib:

@inproceedings{ambrosino:hal-01534697,
    TITLE = {{IdeaBulb: A Smart and Tangible User Interface for Monitoring Ideation During Creative Sessions}},
    AUTHOR = {Ambrosino, Julien and Daniel, Maxime and Masson, Dimitri and Legardeur, J{\'e}r{\'e}my},
    URL = {https://hal.archives-ouvertes.fr/hal-01534697},
    BOOKTITLE = {{Sixth International Conference on Smart Cities, Systems, Devices and Technologies (SMART 2017)}},
    ADDRESS = {Venice, Italy},
    ORGANIZATION = {{IARIA}},
    EDITOR = {XPS (Xpert Publishing Services)},
    PUBLISHER = {{Berntzen, L. et al.}},
    SERIES = {SMART INTERFACES 2017, The Symposium for Empowering and Smart Interfaces in Engineering},
    PAGES = {pp. 70-74},
    YEAR = {2017},
    MONTH = Jun,
    KEYWORDS = {idea generation ; ideation ; tangible interface ; creativity},
    PDF = {https://hal.archives-ouvertes.fr/hal-01534697/file/Smart%20Interface%20IdeaBulb_v2.1.pdf},
    HAL_ID = {hal-01534697},
    HAL_VERSION = {v2},
    USERA = {ACTCL},
}
@inproceedings{badets:hal-01655230,
    TITLE = {{Human efficiency for reducing Lean limits }},
    AUTHOR = {Badets, Patrick and Merlo, Christophe and Pilniere, V{\'e}ronique},
    URL = {https://hal.archives-ouvertes.fr/hal-01655230},
    BOOKTITLE = {{12e congr{\`e}s international de g{\'e}nie industriel}},
    ADDRESS = {Compi{\`e}gne, France},
    YEAR = {2017},
    MONTH = May,
    KEYWORDS = {Lean approach ; D{\'e}marche Lean ; mod{\`e}le de performance ; mod{\'e}lisation d'entreprise ; -- Lean approach ; performance model ; enterprise modelling ; GRAI},
    PDF = {https://hal.archives-ouvertes.fr/hal-01655230/file/CIGI2017_paper48-6finale.pdf},
    HAL_ID = {hal-01655230},
    HAL_VERSION = {v1},
    USERA = {COMM},
}
@inproceedings{barquissau:hal-01655914,
    TITLE = {{Les forums d'actionnaires individuels : un outil de r{\'e}duction du risque ?}},
    AUTHOR = {Barquissau, {\'E}ric and Darras-Barquissau, Nathalie},
    URL = {https://hal.archives-ouvertes.fr/hal-01655914},
    BOOKTITLE = {{Oriane - 15{\`e}me colloque francophone sur le risque}},
    ADDRESS = {Bayonne, France},
    ORGANIZATION = {{IUT Bayonne Pays Basque and Universit{\'e} de Pau et des Pays de l'Adour}},
    YEAR = {2017},
    MONTH = Sep,
    KEYWORDS = {Actionnaires individuels ; R{\'e}seaux sociaux ; Risque financier ; Risque per{\c c}u ; Netnographie ; Forum boursier},
    PDF = {https://hal.archives-ouvertes.fr/hal-01655914/file/Oriane%202017-Barquissau-Darras.pdf},
    HAL_ID = {hal-01655914},
    HAL_VERSION = {v1},
    USERA = {COMM},
}

To achieve correct backreferences, you should first compile the full main document (that is, without the \includeonly) until it is stable. Then you can use the \includeonlys to generate each of the volumes.

The bibliography for each volume would look like:

Thank you very much for this solution.
Since I know the citation keys I use bib2bib to generate the per volume bib files.

I took time before accepting answer because I tried some tricks to keep the bibliography as you did whilst updating table of contents, bookmarks… But with the \includeonly we have to compile only once. — NBur, Mar 23 '18 at 08:00
@NBur Yes, only once. The alternative, I guess, would be to generate the whole bunch of volumes with LaTeX and then break it into the volumes with an outside tool, such as pdftk or pdfsam (that might not be a bad idea, actually). And I'm glad to know this was useful to you after all, this has been an interesting problem. — gusbrs, Mar 23 '18 at 09:46

Biblatex get cited references from multiple documents

1 Answers1