How can I limit how much LaTeX can stretch the vertical spacing between definitions, theorems, lemmas, corollaries, proofs, etc.?

Question

I am working on a long document that includes numerous definitions, theorems, lemmas, corollaries, and proofs. These are defined in the preamble using the amsthm package as follows:

\usepackage{amsthm}
\theoremstyle{definition}
\newtheorem{thm}{Theorem}
\newtheorem{lem}{Lemma}
\newtheorem{cor}{Corollary}

On several pages, LaTeX stretches the vertical spacing between these elements (and between them and text above and below) more than I would like. The pages are roughly 5" x 8" and don't allow widow or orphan lines, so this aggressive vertical stretching is probably just done to stretch the content downward to be flush with the bottom of the page.

How can I limit how much LaTeX can stretch these vertical spaces throughout the document?

---

Notes:

These "oversized white spaces" is also described in an answer here. However the solution there, of changing \flushbottom to \raggedbottom, or the related solution of giving individual pages a ragged bottom as described here, is too extreme for my application -- I would like to allow some stretch but also limit this stretch.

I'm finding that making an MWE is challenging due to the occasional nature of the problem, but I'll post one if I succeed.

Answer 1

Page breaking is LaTeX's achilles heel¹. For better or for worse, it is capable of doing an awful job of page breaking.

So, you're going to need to do some manual adjustment to get the desired results. No matter how small you make the stretch on the theorem style, LaTeX will want to stretch something to get a flush bottom.

First off, for the love of all that his holy,² do not make \baselineskip stretchable.

Next off, what you need to do is to get a bit more text on the page that's affected. This means you're going to need to go through the whole document and look for places to adjust the page breaks. When I was publishing Serif back in the 90s, I was often looking for opportunities to lengthen or shorten page lengths to get the best page breaks (further complicated by the fact that it was a two column layout so I needed to have four perfect column breaks in a row). Figures or tables are a great place to add a little extra space (or trim the space a little) so that you can move a line backwards or forwards.

You can adjust an individual page's length using the command

\enlargethispage{\baselineskip}

to lengthen the page or

\enlargethispage{-\baselineskip}

to shorten the page.

You'll want to adjust facing pairs of pages by identical amounts to keep the bottom of the page from getting too long.

Longer paragraphs can sometimes be stretched out, as mentioned in the comments by adding

\looseness=±1

to the paragraph to tighten or stretch it by one line. I wouldn't expect good results on shorter paragraphs.

You've also indicated that you have forbidden widows and orphans in your document. You may find yourself needing to relax this prohibition to get better page breaks. Personally, I wouldn't allow a hyphenation across a page break, but I'd be ok with a widow line or a longer orphan if it can't be helped.

\mbox is helpful for eliminating bad hyphenations at page breaks.³ You may also find it helpful to change some end-of-paragraph spaces to ~ to keep a last line from getting too short at a page break.

With those measures in place, you can relax the widow and orphan rules by defining the following command:

\NewDocumentCommand{\okpar}{}
   {{\widowpenalty=0 \clubpenalty=0 \par}}

and putting \okpar at the end of the paragraph that you want to allow widows and/or orphans.⁵

It's probably not a bad idea to wrap up any commands for \looseness in macros as well, although those, at least don't need the grouping trick that \okpar uses to restrict their effect to a single paragraph.

---

1. To be fair, I've not seen good automatic page breaking from anything. I recently had to give some specific guidance to the designer of a journal that published a story of mine to get good page breaks. The original page breaks were atrocious.

2. 1 Cor 10:23.

3. In typical short-sighted mode, I often spent ages fiddling with adjusting page sizes for Serif instead of just writing \mbox{polysyllabic}⁴ in my document.

4. Actually \hbox since Serif was typeset with a custom format. Among other things, it enabled a simple mechanism to adjust page size on the fly. The hanging punctuation code, however, broke line breaks after – and — so I had to insert a manual penalty after each instance of -- and --- in my source code to allow line breaks there. I might have needed it after - as well—it's 20 years since I last looked at those files and the hard drive which they're on crashed.

5. Knuth's definition of widow is what I was taught was an orphan with his club being a widow.⁶^,⁷

6. In Knuth's defense, googling this to verify what I remember being taught, I've found all sorts of conflicting definitions of widow and orphan in typographic terms.

7. Late addition Checking with Bringhurst's Elements of Typographic Style, Bringhurst has the orphan at the bottom of the page and the widow at the top of the page, so Knuth has one of the chief arbiters of typographic taste on his side on this one.

Answer 2

To try to answer the original question directly, it appears that spacing above and below definitions, theorems, etc. defined by the amsthm package can at least be somewhat constrained following Werner's answer to this question. For example:

\newtheoremstyle{resultstyle}
  {1em} % Space above
  {1em} % Space below
  {} % Body font
  {} % Indent amount
  {\bfseries} % Theorem head font
  {.} % Punctuation after theorem head
  {.5em} % Space after theorem head
  {} % Theorem head spec (can be left empty, meaning `normal')
\theoremstyle{resultstyle}
\newtheorem{thm}{Theorem}
\newtheorem{lem}{Lemma}
\newtheorem{cor}{Corollary}

However, as Fran's longer comment above suggests, this may expand vertical spaces elsewhere as LaTeX tries to achieve a \flushbottom.

For example, perhaps surprisingly, the vertical space between a theorem / lemma / corollary and a proof that follows may still stretch considerably. It may be tempting to insert a \vspace{-\topsep} into this stretched vertical space or use a related global approach, such as those described here. In particular, we might do so on the hope that LaTeX would preferentially satisfy the skip and sep constraints, ignoring the contradicting \flushbottom constraint if necessary on each relevant page. However, in my experience, LaTeX (understandably) does not behave this way, and the output of this approach is sometimes poor.

Generally, a better output is obtained by either

1. relinquishing the demand of \flushbottom by switching to \raggedbottom when the publisher allows this, or
2. reversing course: allowing all relevant skips and seps to stretch by some conservative amount. This can include flexibility in \parskip and even (hypothetically and highly uncommonly) in \baselineskip as described in this very recent question.

---

In my own case, for what it is worth, the variation in \parskip and around theorems, proofs, etc. for an inflexible \baselineskip was too visually noticeable on some pages (compared to similar books on my shelf), even after page-by-page manual adjustments (see Don Hosek's answer for an excellent summary of these and the comments below the main question for related links). Rather than finishing with a slightly flexible \parskip and slightly flexible \baselineskip to achieve \flushbottom, which appears to be unusual in professional practice, I opted just for a \raggedbottom.