Underline part of a word while preserving kerning

Question

This question led to a new package:
lua-ul

The usual ways of underlining text (or at least the ones I know of: be it with \underline or e.g. the soul package) break kerning when used with part of a word. My current solution consists of a two-step process

1. I type the letters to be underlined (this takes care of the kerning with the preceding letter)
2. I type the underline
3. I grab the following letter and add the kerning manually

MWE with picture of the result

\documentclass{article}

\makeatletter
\def\foo#1{%
   #1%
   \setbox0=\hbox{#1}%
   \hb@xt@\z@{\hss\vrule height -1.2pt depth 1.6pt width\wd0}%
   \addkerncorrection{#1}%
}

\def\addkerncorrection#1#2{%
   \setbox0=\hbox{{#1}{#2}}%
   \setbox1=\hbox{#1#2}%
   \dimen@=\wd1
   \advance\dimen@ by -\wd0
   \kern\dimen@ #2%
}
\makeatother

\begin{document}

V\foo{A}V

VAV

V\underline{A}V

\end{document}

The problem is that this kind of fails if the underlined part is the last piece of a word. Of course I can correct this with an empty group but I'd like to avoid that. I have a working trick (which I refuse to post here :-)) based on \new@ifnextchar from amsgen to test for a following space character but I'm starting to grow uncomfortable with the level of hacking. Furthermore, when utf8 encoding is used it breaks down for stuff like l\foo{ie}ß, which should be thus input as l\foo{ie}\ss.

So the question is: Is there a (better) way to underline part of a word while preserving the correct kerning and not swallowing spaces?

For those who might think this is an XY problem: this is about typesetting psalms in the different tones, like

Answer 1

If you are using LuaTeX, you can use the new lua-ul package which also allows e.g. nested underlines:

\documentclass{article}
\usepackage{lua-ul,luacolor}
\usepackage{tikzducks,pict2e}

\newunderlinetype\myunderduck{\cleaders\hbox{%
    \begin{tikzpicture}[x=.5ex,y=.5ex,baseline=.8ex]%
      \duck
    \end{tikzpicture}%
}}
\newunderlinetype\myunderwavy{\cleaders\hbox{%
    \setlength\unitlength{.3ex}%
    \begin{picture}(4,0)(0,1)
      \thicklines
      \color{red}%
      \qbezier(0,0)(1,1)(2,0)
      \qbezier(2,0)(3,-1)(4,0)
    \end{picture}%
}}
\newcommand\underDuck[1]{{\myunderduck#1}}
\newcommand\underWavy[1]{{\myunderwavy#1}}
\begin{document}
V\underLine{A}V

\underDuck{VAV}

\underDuck{V}\underLine{AV}

\underDuck{These are \underWavy{ucks}}

\strikeThrough{Dinner is ready!}
\end{document}

Old answer (this later became lua-ul)

Using LuaTeX, you can use an attribute to mark the characters you want to underline. This does not interfere with kerning/ligaturing/line breaking because it only acts after all this is finished.

I added some extra flexibility for customized line thickness, placing, duck underlines, ... but the callback basically just iterates over the node tree and draws leaders whenever it finds a marked node:

\documentclass{article}
\usepackage{luacode}
\usepackage{tikzducks,pict2e}

\newattribute\underlineattr
\begin{luacode*}
  local underlineattr = token.create'underlineattr'.index
  local underline_types = {}
  function new_underline_type()
    table.insert(underline_types, tex.box[0].head)
    tex.box[0].head = nil
    tex.sprint(#underline_types)
  end
  local add_underline_h
  local function add_underline_v(head)
    for n in node.traverse(head) do
      if head.id == node.id'hlist' then
        add_underline_h(n)
      elseif head.id == node.id'vlist' then
        add_underline_v(n.head)
      end
    end
  end
  function add_underline_h(head)
    node.slide(head.head)
    local last_value
    local first
    for n in node.traverse(head.head) do
      local new_value = node.has_attribute(n, underlineattr)
      if n.id == node.id'hlist' then
        new_value = nil
        add_underline_h(n)
      elseif n.id == node.id'vlist' then
        new_value = nil
        add_underline_v(n.head)
      elseif n.id == node.id'kern' and n.subtype == 0 then
        if n.next and not node.has_attribute(n.next, underlineattr) then
          new_value = nil
        else
          new_value = last_value
        end
      elseif n.id == node.id'glue' and (
          n.subtype == 8 or
          n.subtype == 9 or
          n.subtype == 15 or
      false) then
        new_value = nil
      end
      if last_value ~= new_value then
        if last_value then
          local width = node.rangedimensions(head, first, n)
          local kern = node.new'kern'
          kern.kern = -width
          kern.next = node.copy(underline_types[last_value])
          kern.next.width = width
          kern.next.next = n
          n.prev.next = kern
        end
        if new_value then
          first = n
        end
        last_value = new_value
      end
    end
    if last_value then
      local width = node.rangedimensions(head, first)
      local kern = node.new'kern'
      kern.kern = -width
      kern.next = node.copy(underline_types[last_value])
      kern.next.width = width
      node.tail(head.head).next = kern
    end
  end
  local function filter(b, loc, prev, mirror)
    add_underline_v(b)
    local new_prev = mirror and b.height or b.depth
    if prev > -65536000 then
      local lineglue = tex.baselineskip.width - prev - (mirror and b.depth or b.height)
      local skip
      if lineglue < tex.lineskiplimit then
        skip = node.new('glue', 1)
        node.setglue(skip, node.getglue(tex.lineskip))
      else
        skip = node.new('glue', 2)
        node.setglue(skip, node.getglue(tex.baselineskip))
        skip.width = lineglue
      end
      skip.next = b
      b = skip
    end
    return b, new_prev
    -- return node.prepend_prevdepth(b)
  end
  luatexbase.callbacktypes.append_to_vlist_filter = 3 -- This should not be necessary
  luatexbase.add_to_callback('append_to_vlist_filter', filter, 'add underlines to list')
\end{luacode*}

\newcommand\newunderlinetype[2]{%
  \setbox0\hbox{#2\hskip0pt}%
  \chardef#1=\directlua{new_underline_type()}\relax
}
\newunderlinetype\myunderline{\leaders\vrule height-1ptdepth1.5pt}
\newunderlinetype\mystrikethrough{\leaders\vrule height2.5ptdepth-2pt}
\newunderlinetype\myunderduck{\cleaders\hbox{%
    \begin{tikzpicture}[baseline=3,scale=0.05]%
      \duck
    \end{tikzpicture}%
}}
\newunderlinetype\myunderwavy{\leaders\hbox{%
    \setlength\unitlength{.3mm}%
    \begin{picture}(4,0)(0,1)
      \thicklines
      \color{red}%
      \qbezier(0,0)(1,1)(2,0)
      \qbezier(2,0)(3,-1)(4,0)
    \end{picture}%
}}
\newcommand\underLine[1]{{\underlineattr=\myunderline#1}}
\newcommand\underDuck[1]{{\underlineattr=\myunderduck#1}}
\newcommand\underWavy[1]{{\underlineattr=\myunderwavy#1}}
\newcommand\strikeThrough[1]{{\underlineattr=\mystrikethrough#1}}
\begin{document}
V\underLine{A}V

\underDuck{VAV}

\underDuck{V}\underLine{AV}

\underDuck{These are \underWavy{ucks}}

\strikeThrough{Dinner is ready!}
\end{document}