Paragraph callback to help with widows/orphans hand tuning

Question

It is often hard to tell when adding \looseness=-1 to a paragraph would actually succeed in producing one less row. How can I add a callback to the paragraph builder so that every paragraph is annotated to show where it would work?

I'm thinking something like this: for every paragraph, build it normally, and build it again with the decreased looseness to compare the row count. Then output the "normal" paragraph, but with the last row in red if the decreased-looseness paragraph would be shorter.

The reason I'm asking this: I'm hand-tuning a book to get rid of widows and orphans, and I spend a lot of time jumping around pages looking for places where a +1 or -1 looseness would work and thinking how several adjustments would work together. Currently I have to tentatively add \looseness=-1, recompile and check the result, so the proposed callback would be a huge help.

Answer 1

Edit: The code (update 2) now uses different colors to give more information:

• Red: paragraph can be shrunk
• Blue: paragraph can be expanded
• Magenta: paragraph can be either expanded or shrunk
• Dull cyan: paragraph can be expanded using the current emergency stretch value
• Dull pink: paragraph can be either expanded with emergency stretch or shrunk

Here's an example of the result:

Old: Here's a first attempt at a solution:

\documentclass[twocolumn]{article}
\usepackage[margin=2cm]{geometry}
\usepackage{luatexbase}
\usepackage{lipsum}

% This doesn't seem to work when using tex.linebreak
\setlength\parskip{0pt}

\begin{document}
\directlua{
    % Use method aliases for better performance
    local nodeid = node.id
    local nodenew = node.new
    local nodecopy = node.copy
    local nodetail = node.tail
    local nodeinsertbefore = node.insert_before
    local nodeinsertafter = node.insert_after
    local nodetraverseid = node.traverse_id

    % Get node ids from their names
    local HLIST = nodeid("hlist")
    local WHAT = nodeid("whatsit")
    local COL = node.subtype("pdf_colorstack")

    % Make nodes for beginning and end of colored regions
    local color_push = nodenew(WHAT,COL)
    local color_pop = nodenew(WHAT,COL)
    color_push.stack = 0
    color_pop.stack = 0
    color_push.data = "1 0 0 rg" % PDF code for RGB red
    color_push.command = 1
    color_pop.command = 2

    % Function to color the last line in the given list
    local color_last_line = function (n)
        % Get the last hlist in the given list
        local lastLine
        for line in nodetraverseid(HLIST, n) do
            lastLine = line
        end

        % Surround it with color start/stop
        lastLine.head = nodeinsertbefore(lastLine.head, lastLine.head, nodecopy(color_push))
        nodeinsertafter(lastLine.head, nodetail(lastLine.head), nodecopy(color_pop))
    end

    % Callback to color the last line wherever a decreased looseness would work
    local linebreak_filter_test_looseness = function (head, is_display)
        % Build a copy of the paragraph with decreased looseness
        local nM1, iM1 = tex.linebreak(node.copy_list(head), {looseness=tex.looseness-1})

        % Build the paragraph normally
        local n, i = tex.linebreak(head)

        % If decreasing the looseness does decrease the line count, color the last line
        if iM1.prevgraf < i.prevgraf then
            color_last_line(n)
        end

        return n
    end

    % Register callback
    luatexbase.add_to_callback("linebreak_filter", linebreak_filter_test_looseness, "linebreak_filter_test_looseness")
}

\lipsum[1]
\lipsum[6]
\lipsum[71]
\lipsum[3]
\lipsum[35]
\lipsum[76]
\lipsum[7-8]

\end{document}

(Thanks to this message and the chickenize documentation.)

You can see the result here.

The problem is that the parskip setting no longer works. For a comparison, see here for the output when the callback is not used: the lines in the left and right columns are perfectly aligned.

It seems to be a general problem when building paragraphs with a callback calling tex.linebreak... Just doing \directlua{ luatexbase.add_to_callback("linebreak_filter", tex.linebreak, "myfilter") } will reproduce the problem.

Update: Here is a workaround for the tex.linebreak issue: a pre-linebreak filter tests if \looseness=-1 would work, and sets a variable that is checked by the post-linebreak filter to do the coloring. It's a bit dirty to communicate the information through an external variable but this way I let LuaTeX build the paragraph internally, so tex.linebreak doesn't get a chance to mess with the baselineskip glues.

\documentclass[twocolumn]{article}
\usepackage[margin=2cm]{geometry}
\usepackage{luatexbase}
\usepackage{lipsum}

\usepackage[callback=]{nodetree}

% This doesn't seem to work when using tex.linebreak
\setlength\parskip{0pt}

\begin{document}
\directlua{
    % Use method aliases for better performance
    local nodeid = node.id
    local nodenew = node.new
    local nodecopy = node.copy
    local nodetail = node.tail
    local nodeinsertbefore = node.insert_before
    local nodeinsertafter = node.insert_after
    local nodetraverseid = node.traverse_id

    % Get node ids from their names
    local HLIST = nodeid("hlist")
    local WHAT = nodeid("whatsit")
    local COL = node.subtype("pdf_colorstack")

    % Make nodes for beginning and end of colored regions
    local color_push = nodenew(WHAT,COL)
    local color_pop = nodenew(WHAT,COL)
    color_push.stack = 0
    color_pop.stack = 0
    color_push.data = "1 0 0 rg" % PDF code for RGB red
    color_push.command = 1
    color_pop.command = 2

    % Set to true when the next post-linebreak filter should color the last line
    local ColorLastLine = false

    % Function to color the last line in the given list
    local color_last_line = function (n)
        % Get the last hlist in the given list
        local lastLine
        for line in nodetraverseid(HLIST, n) do
            lastLine = line
        end

        % Surround it with color start/stop
        lastLine.head = nodeinsertbefore(lastLine.head, lastLine.head, nodecopy(color_push))
        nodeinsertafter(lastLine.head, nodetail(lastLine.head), nodecopy(color_pop))
    end

    % Callback to check if decreasing the looseness would decrease the line count
    local pre_linebreak_test_looseness = function (head, groupeCode)
        % Build a copy of the paragraph with decreased looseness
        local nM1, iM1 = tex.linebreak(node.copy_list(head), {looseness=tex.looseness-1})

        % Build a copy of the paragraph normally
        local n, i = tex.linebreak(node.copy_list(head))

        % Store whether decreasing the looseness does decrease the line count, for "post" callback
        ColorLastLine = iM1.prevgraf < i.prevgraf

        return true
    end

    % Callback to colorize the last line of the paragraph when ColorLastLine is true
    local post_linebreak_color_last_line = function (head, groupcode)
        if ColorLastLine then
            color_last_line(head)
        end
        return true
    end

    % Register callbacks
    luatexbase.add_to_callback("pre_linebreak_filter", pre_linebreak_test_looseness, "pre_linebreak_test_looseness")
    luatexbase.add_to_callback("post_linebreak_filter", post_linebreak_color_last_line, "post_linebreak_color_last_line")
}

\lipsum[1]
\lipsum[6]
\lipsum[71]
\lipsum[3]
\lipsum[35]
\lipsum[76]
\lipsum[7-8]

\end{document}

Update 2: Here is a standalone Lua file that uses different colors for the last line of the paragraph, depending on whether it can be shrunk, expanded or both (as described at the top of this answer).

File widow-assist.lua:

-- Use method aliases for better performance
local nodecopy = node.copy
local nodecopylist = node.copy_list
local nodetail = node.tail
local nodeinsertbefore = node.insert_before
local nodeinsertafter = node.insert_after
local nodetraverseid = node.traverse_id

-- Get node ids from their names
local HLIST = node.id("hlist")
local WHAT = node.id("whatsit")
local COL = node.subtype("pdf_colorstack")

-- Make nodes for beginning and end of colored regions
local color_p1 = node.new(WHAT,COL)
local color_p1s = node.new(WHAT,COL)
local color_m1 = node.new(WHAT,COL)
local color_pm1 = node.new(WHAT,COL)
local color_pm1s = node.new(WHAT,COL)
local color_pop = node.new(WHAT,COL)
color_p1.stack = 0
color_p1.command = 1
color_p1.data = "0 0 1 rg" -- PDF code for RGB blue
color_p1s.stack = 0
color_p1s.command = 1
color_p1s.data = "0 0.7 0.7 rg" -- PDF code for RGB dark cyan
color_m1.stack = 0
color_m1.command = 1
color_m1.data = "1 0 0 rg" -- PDF code for RGB red
color_pm1.stack = 0
color_pm1.command = 1
color_pm1.data = "1 0 1 rg" -- PDF code for RGB magenta
color_pm1s.stack = 0
color_pm1s.command = 1
color_pm1s.data = "1 .5 .5 rg" -- PDF code for RGB pink
color_pop.stack = 0
color_pop.command = 2

-- Color to use for last line in the next post-linebreak filter call (nil = no color)
local LastLineColor = nil

-- Function to color the last line in the given list
local color_last_line = function (n)
    -- Get the last hlist in the given list
    local lastLine
    for line in nodetraverseid(HLIST, n) do
        lastLine = line
    end

    -- Surround it with color start/stop
    lastLine.head = nodeinsertbefore(lastLine.head, lastLine.head, nodecopy(LastLineColor))
    nodeinsertafter(lastLine.head, nodetail(lastLine.head), nodecopy(color_pop))
end

-- Callback to check if changing the looseness by +-1 would affect the line count
local pre_linebreak_test_looseness = function (head, groupeCode)
    -- Disable underfull and overfull boxes reporting
    luatexbase.add_to_callback("hpack_quality", function() end, "hpqfilter")
    luatexbase.add_to_callback("vpack_quality", function() end, "vpqfilter")

    -- Build a copy of the paragraph normally
    local n, i = tex.linebreak(nodecopylist(head))

    -- Build a copy of the paragraph with increased looseness and default emergency stretch
    local nP1s, iP1s = tex.linebreak(nodecopylist(head), {looseness=tex.looseness+1})

    local nP1, iP1
    if iP1s.prevgraf > i.prevgraf then
        -- It worked with the default emergency stretch, let's try without
        nP1, iP1 = tex.linebreak(nodecopylist(head), {looseness=tex.looseness+1, emergencystretch=0})
    else
        -- Didn't work with emergency stretch, no point to try without
        nP1, iP1 = n, i
    end

    -- Build a copy of the paragraph with decreased looseness
    local nM1, iM1 = tex.linebreak(nodecopylist(head), {looseness=tex.looseness-1})

    -- Reenable underfull and overfull boxes reporting
    luatexbase.remove_from_callback("hpack_quality", "hpqfilter")
    luatexbase.remove_from_callback("vpack_quality", "vpqfilter")

    -- Set color to use in the post-linebreak callback
    if iP1.prevgraf > i.prevgraf and iM1.prevgraf < i.prevgraf then
        -- Both +1 and -1 looseness would work
        LastLineColor = color_pm1
    elseif iP1s.prevgraf > i.prevgraf and iM1.prevgraf < i.prevgraf then
        -- Both +1 and -1 looseness would work, but +1 only with emergency stretch
        LastLineColor = color_pm1s
    elseif iP1.prevgraf > i.prevgraf then
        -- Only +1 looseness would work
        LastLineColor = color_p1
    elseif iP1s.prevgraf > i.prevgraf then
        -- Only +1 looseness would work and only thanks to the emergency stretch
        LastLineColor = color_p1s
    elseif iM1.prevgraf < i.prevgraf then
        -- Only -1 looseness would work
        LastLineColor = color_m1
    else
        LastLineColor = nil
    end

    return true
end

-- Callback to colorize the last line of the paragraph when ColorLastLine is true
local post_linebreak_color_last_line = function (head, groupcode)
    if LastLineColor then
        color_last_line(head)
    end
    return true
end

-- Register callbacks
luatexbase.add_to_callback("pre_linebreak_filter", pre_linebreak_test_looseness, "pre_linebreak_test_looseness")
luatexbase.add_to_callback("post_linebreak_filter", post_linebreak_color_last_line, "post_linebreak_color_last_line")

Usage is then as follows:

\documentclass{article}
\usepackage{lipsum}
\usepackage{luatexbase}
\directlua{dofile("widow-assist.lua")}
\begin{document}
\lipsum[1-5]
\end{document}