Can LaTeX support multiple classes of hyphenation break point?

Question

I've just come across an interesting snippet in New Hart's Rules (p.59):

The New Oxford Spelling Dictionary  therefore uses two levels of word division -- 'preferred' divisions (marked |), which are acceptable under almost any circumstances, and 'permitted' divisions (marked ¦), which are not as good, given a choice. Thus unhelpful is shown as un|help¦ful.

This question extends LaTeX Hyphenation.

This seems like an immensely good idea for improving hyphenation; is there a TeX format or package that provides an interface to this type of function? In particular, is there any way I can provide hyphenation priorities à la Liang for an arbitrary word of my choice?

I couldn't find anything helpful in TeX by Topic, though Appendix H of The TeXbook seems to mention hyphenation priorities in relation to Liang's algorithm (1994, p.449)

Answer 1

TeX (and all of its variants including LuaTeX) implement Liang's hyphenation algorithm unchanged (pTeX might be an exception as Japanese requires a completely different approach, but probably Liang's algorithm is available there too).

This algorithm does not implement any hyphenation priorities! It is true that different numbers appear in the patterns, but they are there only to enable overwriting decisions made by other patterns: by the end of the day a hyphenation point is found if the maxium number is "odd".

If I remember the correctly how the patterns are derived, then this is a multi-pass generation: first one generates pattern with only values 0 and 1 from a set of hyphenated words. Then one applies these patterns and looks at all false hyphenations resulting from them. For those false positives patterns with value 2 (forbidden) are generated. Then one looks at what one is missing with the new set and generates patterns with value 3 (overwriting the 2 in places), then ... so in theory those numbers could go up as high as you like, but usually the results are pretty good already after a few iterations. Anyway, bottom line is the algorithm produces a simple yes/no for each place and no weights whatsoever.

To do so one would need to do some research on how this could be best captured, stored and used and as Joseph remarked this is one of the issues with TeX that I already high-lighted in 1990 in E-TeX: Guidelines for future TeX Extensions. Nothing has happened since then unfortunately.

David's suggestion of using two set of hyphenation patterns and first apply only the "preferred" ones and only then the "full" set is an interesting idea. It might work with LuaTeX but there is the complication text containing words in several languages. There is also the question if this is the best possible approach or if it would be better to use different demerits in the linebreaking (i.e., lower ones for hyphenation in preferred places). My assumption is that on the whole this would result in better solutions---but again, this really needs research which so far nobody has undertaken.

There is also the interesting question of what is more desirable as a hyphenation point. In my opinion very undesirable ones are thos where you are likely to pick up the wrong meaning from a word, e.g., in German

Nonnen-kloster  = nuns abbey
Nonnenklo-ster  = nuns toilet + <no word>

Spar-gelder     = (bank) savings
Spargel-der     = asparagus + the

But for the same reason I would disagree with the Oxford Dictionary's choice of "preferred" hyphenation point as

un-helpful      

means that the word "helpful" shows up at the beginning of the line/page and that may result in a reader picking up the opposite meaning from what was intended, if he starts for some reason to skim/read at this point.