Macro for mass hyper-reference?

Question

Context

Suppose I have a large document split across many subfiles. Further, suppose I have distinctive chapter, or section names (e.g. a name such that when it appears anywhere in the text it unambiguously refers to chapter, or section, etc).

Note

While there is find and replace, my editor does not have global find and replace. Further as stated above I am working with many subfiles.
While the terminal is always an option, one that I am familiar with, this question is about a LaTeX native way of making the change.

Question

How would one write a macro in LaTeX (included before right before \end{document} or in the preamble), that would hyper-reference any verbatim string match with a given link e.g.

\hyperrefALL[ch:my-unique-chapter-name]{My Unique Chapter Name}.

and then any instance of My Unique Chapter Name is hyper referenced to ch:my-unique-chapter-name

latex really has no way of reliably applying textual replaces like this, also why specify it at the end, it would be much more reliable just to do a text replace in your editor to change My Unique Chapter Name to \hyperef{ch:my-unique-chapter-name}{My Unique Chapter Name} or even just \ref{ch:my-unique-chapter-name} — David Carlisle, Dec 05 '16 at 16:59
Besides the technical difficulty of implementing this in LaTeX without breaking things, the payoff is so minimal and it is also much more error prone (what if you also match text that should not be hyperlinked in that particular context?). It is much easier and safer to define a macro \newcommand{\MyChap}{\hyperef{ch:my-unique-chapter-name}{My Unique Chapter Name}} and use it instead of the plain text version — Bordaigorl, Dec 05 '16 at 17:11
My editor doesn't allow global search and replace, hence the specification as multiple subfiles.... — SumNeuron, Dec 05 '16 at 17:32
You should consider using LuaLaTeX for this. You can tap into the line-by-line processing and replace any piece of text with something else, almost like a search-and-replace mid-processing. — Werner, Dec 05 '16 at 18:27
@DavidCarlisle it has search and replace, just not global search and replace, so I would have to go through like 100 subfiles :/ — SumNeuron, Dec 06 '16 at 03:47
@DavidCarlisle while I appreciate your insight into the reliability of making such a macro, I am still interested in how it could be achieved. — SumNeuron, Dec 06 '16 at 06:16
@Bordaigorl the payoff is minimal if the is global find and replace or if you keep a very large document all in one TeX file rather than using subfile or a different package for pooling together multiple tex files. As stated in the question, this is the case, thus the pay off is worth while rather than doing find and replace in about 100 files — SumNeuron, Dec 06 '16 at 06:18
@SumNeuron: Try this code in SharLaTeX under the LuaLaTeX engine. You'll see that it produces a two-chapter document, with the chapter names being unique. The last paragraph contains these two unique chapter names as part of the sentence without any \hyperrefs. However, LuaLaTeX intercepts this and does a string replacement, replacing the matched unique chapter name X with \hyperref[link]{X}. Is this what you're after? — Werner, Dec 06 '16 at 06:18
@Werner this does work (with the exception of changing documentation type in ShareLatex for whatever reason). If you write it as an answer I will up vote, but I am still looking for a LaTeX native way of doing the same. — SumNeuron, Dec 06 '16 at 06:31
To a good first approximation, it's the Lua route or nothing at all (I can imagine a TeX-based approach but it would likely break everything!). I assume @Werner will post a Lua-based answer soon enough: do you want a separate one on why its extremely tricky in TeX? (This won't actually add a solution, so ...) — Joseph Wright, Dec 06 '16 at 07:07
@JosephWright while it may not be a solution, I am honestly interested to know why it would be so difficult and so devastating to LaTeX. It is a macro based language yes? — SumNeuron, Dec 06 '16 at 07:10
@SumNeuron regarding Search&Replace a decent editor like Sublime Text is easily able to perform it on multiple documents at once — Bordaigorl, Dec 06 '16 at 07:55
@SumNeuron regarding the complication of implementing this: LaTeX is a macro based language, yes. This only means that if you DO use macros you are good to go, and this is the suggestion I was making. On the other hand if you do not, you need to have a macro parse everything you typed in your doc, and make sure everything that does not match is reproduced with the original catcodes and meaning. This is extremely tricky because, intuitively, each package introduces special cases to this process — Bordaigorl, Dec 06 '16 at 07:57
It's not "difficult" to do in classical tex it is, to all intents impossible. You could cite Turing completeness and claim anything is possible but you would have to completely rewrite latex. — David Carlisle, Dec 06 '16 at 07:58
lack of reasonable multi-file editing facilities sounds like something that you should be putting in as a feature request to the editing system rather than trying to do editing in latex which is completely unsuited for that task. — David Carlisle, Dec 06 '16 at 09:02
As others have pointed out, aside from the 'academic' interest this question has, in the real world there are many much better options such as sed or awk. Someone once used TeX to control Mars Rover, which makes for a fascinating read. But I'm not recommending it as a general solution.... — jon, Dec 07 '16 at 03:34

score 10 · Accepted Answer · edited Dec 06 '16 at 08:56

You can use LuaLaTeX to intercept the processing of your input via its process_input_buffer data processing callback. This callback allows you to change the contents of the line input buffer just before LuaTeX actually starts looking at it.

My use of Lua is one pixel above non-existent, so there's probably room for improvement here:

\documentclass{report}

\usepackage{luacode,hyperref}
\usepackage{lipsum}

\begin{luacode}
  function inserthref ( s )
    if string.find ( s , '\\chapter' ) == nil then
      s = string.gsub ( s , 'My unique chapter name', 
        '\\hyperref[ch:my-unique-chapter-name]{My unique chapter name}' )
      s = string.gsub ( s , 'Introduction',
        '\\hyperref[ch:introduction]{Introduction}' )
    end
    return ( s )
  end
\end{luacode}
\AtBeginDocument{\directlua{luatexbase.add_to_callback (
   "process_input_buffer", inserthref, "inserthref" ) }}

\begin{document}

\chapter{Introduction}
\label{ch:introduction}

\chapter{My unique chapter name}
\label{ch:my-unique-chapter-name}
\lipsum[1-10]

See My unique chapter name as well as introduction, Introduction.

\end{document}

The callback code essentially checks whether you've called a \chapter or not. If this is not the case, it performs a string substitution based on your unique chapter names. Each unique chapter name found is replaced by its hyperlinked counterpart. Note that the string substition is case-sensitive.

You can add more chapters to this list of callback substitutions, one for each chapter or section.

For a 'real' use case, you could consider a lookup table in Lua. One approach trying to minimise repetition would be

\begin{luacode}
  local mylabels =  {
      ["My unique chapter name"] = "my-unique-chapter-name" ,
      ["Introduction"]           = "introduction" 
  }

  function inserthref (s)
    for k,v in pairs(mylabels) do
      if string.match(s, "\\chapter{" .. k .. "}") then
        return string.gsub(s, k, "\\hyperref[ch:" .. v .. "]{" .. s .. "}")
      end
    end
    return s
  end
\end{luacode}

Alternatively, you might want to specify the 'full' search string so that you can use one function for different types of substitution. For example

\begin{luacode}
  local mylabels =  {
      ["\\chapter{My unique chapter name}"] = 
        "\\chapter{\\hyperref[ch:my-unique-chapter-name]{My unique chapter name}}" ,
      ["\\chapter{Introduction}"]           =
        "\\chapter{\\hyperref[ch:introduction]{Introduction}}" ,
  }

  function inserthref (s)
    for k,v in pairs(mylabels) do
      if string.match(s, k) then
        return string.gsub(s, k, v)
      end
    end
    return s
  end
\end{luacode}

This approach can be made simpler if we assume the string will be a full line only:

\begin{luacode}
  local mylabels =  {
      ["\\chapter{My unique chapter name}"] = 
        "\\chapter{\\hyperref[ch:my-unique-chapter-name]{My unique chapter name}}" ,
      ["\\chapter{Introduction}"]           =
        "\\chapter{\\hyperref[ch:introduction]{Introduction}}" ,
  }

  function inserthref (s)
    if mylabels[s] then
        return mylabels[s]
    end
    return s
  end
\end{luacode}

Probably for a real use case you'd store the search strings in a table and do a lookup: scales better than lots of single tests! — Joseph Wright, Dec 06 '16 at 07:29
@JosephWright: Agreed. Look-ups would work here, but I'm unfamiliar with such things. Could you guide me somewhere? — Werner, Dec 06 '16 at 07:40
I've edited some suggestions in: feel free to drop, modify, etc. (seems easier than posting elsewhere ...) — Joseph Wright, Dec 06 '16 at 08:56

Ulrich Diez · Answer 2 · 2016-12-07T13:23:29.220

This answer does not provide a solution to the problem that caused the question.
Please take this answer for a moot thing / an "academical" thing intended to exhibit the complexity of the problem.

Your request seems to include the task of properly detecting occurrences of text phrases in (La)TeX input files.

It was proposed to leave the handling of this "detection task" to editing programs that are used for creating the (La)TeX input files at creation time instead of leaving the handling of this detection task to the (La)TeX compiler which processes the (La)TeX input files at compilation time in order to produce a .pdf output file.

I am going to focus on this.

Example 1:

You can make it hard for a nowadays editing program to find the phrase in question in the (La)TeX input file by interspersing that phrase with characters that at reading time get ignored by (La)TeX and/or by interspersing that phrase with characters that introduce a comment or belong to a comment. Be aware that in (La)TeX any character can be turned into a character that gets ignored at reading time. Be aware that in (La)TeX any character can be turned into a character that introduces a comment. (Such characters can still be names of control symbols.)

\documentclass{article}
\begin{document}
\catcode`\A=14
\catcode`\B=14
\catcode`\D=14
\catcode`\X=9
\catcode`\Y=9
\catcode`\Z=9

My Unique Chapter Name =
My UA 129(dec) is not a prime number yet.
nXi%
que CD Father Christmas has Santa Claus' reindeer under 
haYpteB the vegan sledgehammer. Isn't this nice?!
r NaZme
\end{document}

Example 1 delivers:

Example 2:

You can apply catcode trickery for making the phrase in question which occurs in the (La)TeX input file deliver something totally different in the .pdf output file. If you do this, you probably don't want it to be replaced.

\documentclass{article}

\begin{document}
\ttfamily\selectfont
{%
  \newcommand\activedef[2]{\begingroup\lccode`\~=`#1\relax
  \lowercase{\endgroup\def~}{#2}\catcode`#1=13\relax}%
  \csname @firstofone\endcsname{\activedef{e}{\activedef{e}{%
  \activedef{e}{\activedef{e}{\activedef{e}{\activedef{e}{\string
  s}\string n}\string h}\string s}\string n}\string h}\activedef
  {M}{\string S}\activedef{y}{\string p}\activedef{ }{\activedef
  { }{\activedef{ }{\activedef{ }{\activedef{ }{\activedef{ }{ }%
  \string a}\string i} }\string a}\string i}\activedef{U}{\string
  d}\activedef{n}{\string e}\activedef{i}{\string r}\activedef{q}%
  {\string s}\activedef{u}{ }\activedef{t}{\string a}\activedef
  {p}{\string m}\activedef{r}{\string y}\activedef{N}{\string e}%
  \activedef{m}{\string e}\activedef{C}{\string v}\activedef{h}{%
  \string e}\activedef{a}{\activedef{a}{\activedef{a}{\activedef
  {a}{\string y} }\string y} }}%
%
1.My Unique Chapter Name%

2.My Unique Chapter Name%
}%

3.My Unique Chapter Name%
\end{document}

Example 2 delivers:

There is no phrase "Spiders have many eyes" in the LaTeX input file. But there are two of them in the .pdf output file.
There is only one phrase "My Unique Chapter Name" in the .pdf output file although there are three of them in the LaTeX input file.
How can this be? Which one in the LaTeX input file is to be replaced by the replacement routines of an editing program for text files? Which ones in the LaTeX input file are to be kept in order to preserve the infos about spiders?

Example 3:

You can have the phrase in question occur in the .pdf output file without having it occur in the (La)TeX input file at all. If a phrase does not occur in the (La)TeX input file, it cannot be replaced in the (La)TeX input file by a program for editing text files.

%%   The following piece of code requires LaTeX with eTeX-support.    %%

\documentclass{article}

\begin{document}

\noindent My homework is that I must write ten times a phrase
about a unique chapter name.

\bigskip

\noindent Here we go:

\bigskip

\begingroup\endlinechar-1\let\+\let\+\a\def\a\c{\j\o}\+\m\relax\+\g\else
\a\b{\c7 \f\o>93 \f\o<101\c1 \h\h\f\o>125\c-93 \h}\+\j\advance\+\f\ifnum
\a\d{\catcode\o12 \uccode\o\p}\+\n\endgroup\+\l\noexpand\+\o\fam\+\p\mag
\a\q#1{\f`#1=13\k~\h\f`#1=32\k!\h\f`#1=36\k##\h\f`#1=94\k"\h}\+\k\string
\+\e\expandafter\+\h\fi\a\i{\j\p1 }\a~#1{\f\p<126\d\b\i\f\p=94\i\h\g\f\p
=126\b\b\b\b\c127 \g\d\c7 \f\o>255\c-129 \h\h\i\h\f\p>255\edef~{\a\l~\l~
####1 ####2\k E\k T}~{\+~\q\uppercase{\edef~{##1}}\e\n\e\begingroup\e
\newlinechar\e1\e3\e\relax\e\scantokens\e{\e\endgroup~}}\h#1#1}\o"41
\begingroup\a~{\n\p13 ~\m\p32 ~\m\p94 ~\m\p36 ~~}~
]b{H`4&Jm4IuHgPHW`.WHEH&_PWHmIgWJHWJ-HWg&JPHW`JH;`I.PJH~Ob{H=-gB_JHx`.;W
JIHi.&J~O@A]A]>JIJHmJHX48A]ANU8Hb{H=-gB_JHx`.;WJIHi.&JAAN\8Hb{H=-gB_JHx`
.;WJIHi.&JAANd8Hb{H=-gB_JHx`.;WJIHi.&JAANk8Hb{H=-gB_JHx`.;WJIHi.&JAANr8H
b{H=-gB_JHx`.;WJIHi.&JAANy8Hb{H=-gB_JHx`.;WJIHi.&JAAN#8Hb{H=-gB_JHx`.;WJ
IHi.&JAAN*8Hb{H=-gB_JHx`.;WJIHi.&JAAN18Hb{H=-gB_JHx`.;WJIHi.&JAAUN8Hb{H=
-gB_JHx`.;WJIHi.&J %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% END OF SNIPPET

\end{document}

Example 3 delivers:

Ten instances of the phrase "My Unique Chapter Name" can be found in the .pdf output file although no instance of that phrase can be found in the LaTeX input file.

(By the way: I don't know if "name of chapter" or "name of section" is good terminology. For one thing a chapter is also a section of a document. For another thing sections often have headings consisting of sectioning numbers and/or section titles and sometimes a section title also is a unique identifier for the section in question.)

Thank you for such an insightful look at this problem :) – SumNeuron Dec 07 '16 at 10:08 — SumNeuron, Dec 07 '16 at 10:08

score 6 · Answer 3 · edited Apr 13 '17 at 12:35

When TeX is processing text, it is simply collecting up material to typeset: there is no sense in which it 'examines' the meaning of the text. Commands either start \ or use 'active' characters (more on that below): in both cases we are talking about macros. As such, if we have a block of plain text there is no 'further processing' unless something is done to enable that. In particular we don't have access to the 'text collected for the current paragraph' from TeX (we can arrange to look ahead the upcoming tokens).

The obvious example of processing material is the argument to a command: in

\foo{text}

the implementation of \foo can for example check if text matches one of several special cases. However, in the question we are talking about 'general' text, so this does not apply.

It is possible to grab the content of an environment to allow similar work, so one could imagine grabbing the entire document body and doing search-and-replace. However, that's likely to be a lot of text to grab in one go and would not work for anything at all 'hidden'

\newcommand\awkward{My special text}
Here I talk about `\awkward' stuff.

(\input is a bigger real-world risk here I guess.) It would also be very painful to do the searching: essentially one would have to examine the entire document body one character at a time and branch for each possible hit! There are also issues with anything that changes the interpretation of input (category codes): for example, \verb would break. As such, this really isn't a general (or recommended!) approach.

Extending this idea, it is notable that listings does do search-and-replace on text within it's environment by grabbing the contents verbatim, writing to a file and reading back in a controlled way. However, this is generally for short snippets rather than an entire document. The listings package is also at the advantage that it does not need to do expansion of the 'rest' of the text it grabs: everything is verbatim except for the specific search strings. Going from 'a controlled environment' to 'the entire document' is therefore not necessarily impossible but extremely challenging, certainly compared with editing the sources.

An alternative approach is to make all characters 'active', so each letter is a command rather than 'text'. Most of the time these would simply output the letter itself but one could attach a look-ahead first, so that each letter looks at the next token to see if it matches the 'special' text. However, this is again (extremely) tedious and error prone, and likely to break pretty much everything else (as most code expects letters to be text!).

As detailed in Werner's answer, LuaTeX offers a way around this as at the 'Lua end' there is the concept of seeing what text is 'about' (in the input buffer) and doing search-and-replace there.

score 3 · Answer 4 · answered Dec 05 '16 at 17:08

3

In a UNIX-oid environment, such as Linux, you could use sed or some similar tool to perform some sort of pre-processing run. In a build-script for your large document, e.g. a thesis, you do the following tasks:

Find all tex-files and apply your find&replace rule
Run your latex build tool of your choice

This, however, involves some regular expressions, you might want to distinguish between My Chapter and \myRef[chap01]{My Chapter}.

answered Dec 05 '16 at 17:08

Dohn Joe

1,912

While I appreciate your answer, it does not answer the question. I am familiar with regular expression and, no I do not wish to distinguish between My Chapter and \myRef[chap01]{My Chapter} as stated in the OP. The question inquires about a LaTeX native way of doing this. sed is always a good option though. – SumNeuron Dec 06 '16 at 06:10

Ulrich Diez · Answer 5 · 2016-12-08T08:59:47.937

What you can easily do to a certain degree with (La)TeX is copying a source text file to a target text file in a way where (La)TeX does under verbatim-category-code-régime read the source file line by line and write the target file line by line. Hereby you can have (La)TeX perform some replacements within each line before writing it to the target file.

With the approach outlined by me replacement is done only within lines but not across linebreaks.

With the approach outlined by me replacement-copying does not take place across several \input-files.

But you can probably use it for writing a little script in LaTeX for processing/replace-copying all your input files.

I decided to implement this by "hacking" the verbatimcopy package (Version 2008/11/17 v0.06) by Lars Madsen and me.

That "hack" yielded another small package, called verbatimreplacementcopy.sty :

% verbatimreplacementcopy.sty (C) 2016 by Ulrich Diez.
% Licence: LPPL.
\ProvidesPackage{verbatimreplacementcopy}[2016/12/08 v0.02 beta by Ulrich Diez]
\RequirePackage{verbatimcopy}[2008/11/17]
\begingroup
\newcommand*\VerbatimCopyB[2]{% {from file}{to file}
  \@bsphack
  \expandafter\def\expandafter\VC@target\expandafter{\VC@outputdir#2}%
  \IfFileExists{\VC@@quote#1\VC@@quote}%
  {%
    \bgroup
    \def\@verbatim{%
      \obeylines
      \let\do\@makeother
      \dospecials
    }%
    \let\endtrivlist\relax
    \def\verbatim@processline{%
      \begingroup
      \edef\verbatim@line{\the\verbatim@line}%
      \@onelevel@sanitize{\verbatim@line}%
      \verbatim@replacementhook
      \expandafter\endgroup
      \expandafter\verbatim@line\expandafter{\verbatim@line}%
      \immediate\write\verbatim@out{\the\verbatim@line}%
    }%
    \immediate\openout\verbatim@out\VC@@quote\VC@target\VC@@quote\relax
    \verbatiminput{\VC@@quote#1\VC@@quote}%
    \immediate\closeout\verbatim@out%
    \egroup%
  }%
  {%
    \PackageError{verbatimcopy}%
    {Source-file cannot be found}%
    {%
      For copying source-file to target-file it would be nice to
      have the source-file available.%
    }%
  }%
  \@esphack
}%
\ifx\VerbatimCopy\OldVerbatimCopy
  \expandafter\global\expandafter\let\csname VerbatimCopy\endcsname\VerbatimCopyB
\fi
\global\let\OldVerbatimCopy\VerbatimCopyB
\endgroup
\newcommand\verbatim@replacementhook{}%
\newcounter{verbatim@replacements}%
\newcommand\Replace{}%
\outer\def\Replace{%
  \stepcounter{verbatim@replacements}%
  \VCverbaction{\VCverbaction{\expandafter\VC@repldef\@firstofone}}{}%
}%
\newcommand\Noreplacements{}%
\outer\def\Noreplacements{\def\verbatim@replacementhook{}}%
\newcommand*\VC@repldef[2]{%
  \expandafter\VC@@repldef
  \csname VC@@repl\number\value{verbatim@replacements}\expandafter\endcsname
  \csname VC@repl\number\value{verbatim@replacements}\endcsname{#1}{#2}%
}%
\newcommand*\VC@@repldef[4]{%
  \@ifdefinable#1{%
    \def#1##1#3##2\@nil{%
      \ifx\@nil##2\@nil
        \expandafter\@firstoftwo
      \else
        \expandafter\@secondoftwo
      \fi
      {##1}{##1#4#1##2\@nil}%
    }%
  }%
  \newcommand*#2[1]{#1##1#3\@nil}%
  \expandafter\def\expandafter\verbatim@replacementhook\expandafter{%
    \verbatim@replacementhook
    \edef\verbatim@line{\expandafter#2\expandafter{\verbatim@line}}%
  }%
}%
\endinput

As verbatimreplacementcopy.sty is based on the verbatimcopy package, I strongly recommend reading the manual of that package. The manual explains how to change the target directory etc. It also points out that the \VerbatimCopy-macro will overwrite and thus destroy the target file without warnings!

You can call verbatimreplacementcopy.sty from within a script for copy-replacing text files. Let's call such a script Replacer.tex:

\documentclass{minimal}
\usepackage{verbatimreplacementcopy}

\Replace{My Unique Chapter Name}{ch:my-unique-chapter-name}
\Replace{My other Unique Chapter Name}{ch:my-other-unique-chapter-name}
\VerbatimCopy{Original.tex}{CopyA.tex}%
\Noreplacements
\Replace{My Unique Chapter Name}{ch:my/unique/chapter/name}
\Replace{My other Unique Chapter Name}{ch:my/other/unique/chapter/name}
\VerbatimCopy{Original.tex}{CopyB.tex}%

\stop

\Replace will add another replacement directive for \VerbatimCopy.

Replacement directives will be applied successively by \VerbatimCopy whenever \VerbatimCopy is copying a text file.

\Noreplacements will clear all replacement directives so that no replacement directive will be applied by \VerbatimCopy when \VerbatimCopy is copying a text file.

Therefore this script will now copy the file Original.tex to the file CopyA.tex.
Hereby with each line of Original.tex the following will be done:

Each phrase "My Unique Chapter Name" will be replaced by "ch:my-unique-chapter-name".
In the result of that replacement each phrase "My other Unique Chapter Name" will be replaced by "ch:my-other-unique-chapter-name".

Then the \Noreplacements-directive does clear all replacement directives.

Then this script will copy the file Original.tex to the file CopyB.tex.
Hereby with each line of Original.tex the following will be done:

Each phrase "My Unique Chapter Name" will be replaced by "ch:my/unique/chapter/name".
In the result of that replacement each phrase "My other Unique Chapter Name" will be replaced by "ch:my/other/unique/chapter/name".

If the content of Original.tex was

My Unique Chapter Name. Some text. My other Unique Chapter Name Some
text text. My other Unique Chapter Name. Phrases. Text.
Chapter Name. My Unique Chapter Name text text phrase

, the content of CopyA.tex will be:

ch:my-unique-chapter-name. Some text. ch:my-other-unique-chapter-name Some
text text. ch:my-other-unique-chapter-name. Phrases. Text.
Chapter Name. ch:my-unique-chapter-name text text phrase

and the content of CopyB.tex will be:

ch:my/unique/chapter/name. Some text. ch:my/other/unique/chapter/name Some
text text. ch:my/other/unique/chapter/name. Phrases. Text.
Chapter Name. ch:my/unique/chapter/name text text phrase

U s a g e i s a t y o u r o w n r i s k !
I give no warranties.
If something breaks I am not interested in the pieces.
(I am interested in bug reports.)

Macro for mass hyper-reference?

Context

Note

Question

5 Answers5

Linked