links and reference links in latex converted from markdown

Question

I am trying to convert markdown to latex using a self made script. One of the issues I am running against are hyperlinks.

Within markdown there are numerous ways to place links (external hyperlinks), like this:

My favorite search engine is Duck Duck Go.
or as a quick link: https://www.markdownguide.org.

The two options above I have no problem with converting to latex (other than a package clash between hyperref and url, but I'll fix that).
My favorite search engine is \href{https://duckduckgo.com}{Duck Duck Go}.
or as a quick link: \url{https://www.markdownguide.org}.

The issues start with markdown's referenced links though, which look like this:

In a hole in the ground there lived a hobbit. Not a nasty, dirty, wet hole, filled with the ends of worms and an oozy smell, nor yet a dry, bare, sandy hole with nothing in it to sit down on or to eat: it was a [hobbit-hole][1], and that means comfort.

the tag [1] is then later in the page/document referred as:

[1]: https://en.wikipedia.org/wiki/Hobbit#Lifestyle
[1]: https://en.wikipedia.org/wiki/Hobbit#Lifestyle "Hobbit lifestyles"
[1]: https://en.wikipedia.org/wiki/Hobbit#Lifestyle 'Hobbit lifestyles'
[1]: https://en.wikipedia.org/wiki/Hobbit#Lifestyle (Hobbit lifestyles)
[1]: <https://en.wikipedia.org/wiki/Hobbit#Lifestyle> "Hobbit lifestyles"
[1]: <https://en.wikipedia.org/wiki/Hobbit#Lifestyle> 'Hobbit lifestyles'
[1]: <https://en.wikipedia.org/wiki/Hobbit#Lifestyle> (Hobbit lifestyles)

which will result in the references not showing and the [hobbit-hole] becoming a hyperlink.

Is a similar logic for referenced links available in latex, or in the hyperref package?

Answer 1

A problem is that URLs should be read verbatim.

If you are interested I can offer an interface

\urllabel{⟨referencing-label⟩}⟨verbatim-argument denoting url⟩

and

\urllink{⟨link text⟩}{⟨referencing-label⟩}

where \urllabel reads the ⟨verbatim-argument denoting url⟩ under verbatim-catcode-régime and copies it to .aux-file so that it can be referenced by \urlref in any place of the document. At the time of referencing \urllink passes the ⟨verbatim-argument denoting url⟩ to \href.

The mechanism is based on the kernel-macro \@newl@bel, but with some wrappers for copying/reading arguments under verbatim-catcode-régime. This way you get warnings about multiply defined labels and about labels being changed. (\@setref cannot be used but needs to be implemented analogously as that is redefined by hyperref.)

(This could easily be implemented by means of xparse if xparse provided means for getting hold of the verbatim-delimiter in use when reading a v/+v-argument.)

Caveats:

• In the first LaTeX-run/while no .aux-file does exist yet cross-referencing-labels/verb-labels are undefined and you thus you don't get hyperlinks.

• You need at least two compilations until everything matches out.

• Like \verb and other macros that expect their argument to be tokenized in non-standard-catcode-régime, \urllabel cannot be used within the arguments or definition-texts of other macros and the like.

• The interface uses the .aux-file. Therefore TeX-distributions where some characters are written to .aux-file in ^^-notation, e.g., due to character-translation/tcx-files, might cause problems.

\makeatletter
%%======================Code for \UDcollectverbarg=============================
%% \UDcollectverbarg{<non-optional 1>}{<non-optional 2>}|<verbatim arg>|
%% 
%% reads <verbatim arg> under verbatim-catcode-regime and delivers:
%%
%%    <non-optional 1>{<non-optional 2>{<verbatim arg>}}
%%-----------------------------------------------------------------------------
%% \UDcollectverbarg*{<non-optional 1>}{<non-optional 2>}|<verbatim arg>|
%% 
%% reads <verbatim arg> under verbatim-catcode-regime and delivers:
%%
%%    <non-optional 1>{<non-optional 2>{|<verbatim arg>|}}
%%-----------------------------------------------------------------------------
%% Instead of using verbatim-delimiter | or the like the <verbatim arg> can 
%% be nested in braces.
%%
%% You cannot use percent or spaces or horizontal tab as verbatim-delimiter.
%%
%% You can use <non-optional 1> for nesting calls to \UDcollectverbarg.
%%
%% The unstarred variant removes the verbatim-delimiters/braces that surround
%% <verbatim arg>.
%% The starred variant keeps the verbatim-delimiters/braces that surround
%% <verbatim arg>.
%% Reason: When you feed things to \scantokens you don't need the verbatim-
%%         delimiters.
%%         When you use things for writing to temporary files and reading back
%%         verbatimized, you may need them.
%%=============================================================================
\@ifdefinable\UDcollectverbarg{%
  \DeclareRobustCommand\UDcollectverbarg{%
    \@ifstar{\UD@collectverbarg{\@secondoftwo}}{\UD@collectverbarg{\@firstoftwo}}%
  }%
}%
\newcommand\UD@collectverbarg[3]{%
  % #1 - indicator whether to remove/keep the verb-delimiter (\@firstoftwo/\@secondoftwo)
  % #2 - non-optional 1
  % #3 - non-optional 2
  \begingroup
  \let\do\@makeother % <- this and the next line switch to
  \dospecials        %    verbatim-category-code-régime.
  \catcode`\{=1      % <- give opening curly brace the usual catcode so a 
                     %    curly-brace-balanced argument can be gathered in
                     %    case of the first thing of the verbatimized-argument 
                     %    being a curly opening brace.
  \catcode`\ =10     % <- give space and horizontal tab the usual catcode so \UD@@collectverbarg
  \catcode`\^^I=10   %    cannot catch a space or a horizontal tab as its 4th undelimited argument.
                     %    (Its 4th undelimited argument denotes the verbatim-
                     %     syntax-delimiter in case of not gathering a
                     %     curly-brace-nested argument.)
  \catcode`\%=14     % <- make percent comment.
  \kernel@ifnextchar\bgroup
  {% seems a curly-brace-nested argument is to be picked:
    \catcode`\}=2    % <- give closing curly brace the usual catcode also.
    \UD@@collectverbarg{#1}{\@firstoftwo}{#2}{#3}{}%
  }{% seems an argument with verbatim-syntax-delimiter is to be picked:
    \do\{% <- give opening curly brace the verbatim-catcode again.
    \UD@@collectverbarg{#1}{\@secondoftwo}{#2}{#3}%
  }%
}%
\newcommand\UD@@collectverbarg[5]{%
  % #1 - indicator whether to remove/keep the verb-delimiter (\@firstoftwo/\@secondoftwo)
  % #2 - indicator whether braces or verb-delimiter (\@firstoftwo/\@secondoftwo)
  % #3 - non-optional 1
  % #4 - non-optional 2
  % #5 - verb-delimiter or emptiness
  \do\ %   <- Now that \UD@@collectverbarg has the delimiter or
  \do\^^I%    emptiness in its 4th arg, give space and horizontal tab
         %    the verbatim-catcode again.
  \do\^^M% <- Give the carriage-return-character the verbatim-catcode.
  \do\%%   <- Give the percent-character the verbatim-catcode.
  \long\def\@tempb##1#5{%
    #1{\def\@tempb{##1}}{#2{\def\@tempb{{##1}}}{\def\@tempb{#5##1#5}}}%
    \@onelevel@sanitize\@tempb % <- Turn characters into their "12/other"-pendants.
                               %    This may be important with things like the 
                               %    inputenc-package which may make characters 
                               %    active/which give them catcode 13(active).
    \expandafter\UD@@@collectverbarg\expandafter{\@tempb}{#3}{#4}% <- this "spits out the result.
  }%
  \@tempb
}%
\newcommand\UD@@@collectverbarg[3]{\endgroup#2{#3{#1}}}%
%%================= End of code for \UDcollectverbarg =========================
%%
\DeclareRobustCommand\urllabel[1]{%
  \@bsphack\UDcollectverbarg*{\@firstofone}{\UD@writeverblabel{#1}}%
}%
\newcommand\UD@writeverblabel[2]{%
  \begingroup
  \newlinechar=\endlinechar
  \immediate\write\@auxout{\string\newverblabel{#1}#2\@percentchar}%
  \endgroup
  \@esphack
}%
\newcommand\newverblabel[1]{%
  \UDcollectverbarg{\@firstofone}{\UD@defineverblabel{#1}}%
}%
\newcommand\UD@defineverblabel[2]{\@newl@bel{vrblbl}{#1}{#2}}%
\DeclareRobustCommand\urlref[2]{%
  \expandafter\ifx\csname vrblbl@#2\endcsname\relax 
    \protect\G@refundefinedtrue
    \@latex@warning {Reference `#2' on page \thepage\space undefined}%
    #1%
  \else 
    \expandafter\expandafter\expandafter\href
    \expandafter\expandafter\expandafter{\csname vrblbl@#2\expandafter\endcsname}{#1}%
  \fi
}%
\makeatother
\documentclass{article}
\usepackage{hyperref}
\begin{document}
Some text.
\urlref{CTAN}{2} is an important resource for users of \TeX.
\urlref{TeX LaTeX StackExchange is a question-answer-platform.}{TeXLaTeXStackExchange}
Some more text.
%-------------------------------------------------------------------
\urllabel{TeXLaTeXStackExchange}|https://tex.stackexchange.com/|
\urllabel{2}|https://ctan.org/|
\end{document}