How to manage spacing in a document so there are no large gaps

Question

When I'm typing my own documents, it's often the case that an awkwardly placed theorem environment near the end of the page will jump to the next because it's too large to fit in one page. When this happens, a large blank space is created at the end of the first; of course, this looks remarkably ugly.

In particular, consider the following image:

As a stylistic choice, I don't want to allow the theorem boxes to split across pages, which causes the following problem.

On the other hand, I noticed that no author has this problem in their books; they somehow always manage to fit their environments and text in such a way that every page is filled to the brim (so to speak) and no theorem environment is broken across a page. My question is: how do these authors do this? Is there a way to remedy the situation above automatically, or do I have to manually fill in the gap with extra content?

Thanks.

Edit: An example as suggested in the comments:

\documentclass{article}
\usepackage{lipsum}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage[margin=4cm,includefoot]{geometry}

\usepackage{newpxtext}
\usepackage{newpxmath}

% theorem styles
\usepackage{mdframed}
\usepackage{xcolor}

\mdfsetup{skipabove=5pt,skipbelow=5pt,nobreak}

\definecolor{pale}{HTML}{FFB5CE}
\definecolor{redbrown}{HTML}{A52A2A}

\mdfdefinestyle{paleStyle}{%
    linecolor=pale,
    backgroundcolor=pale,
    innertopmargin=0pt,
    innerbottommargin=10pt,
    innerrightmargin=10pt,
    innerleftmargin=10pt,
    frametitleaboveskip=5pt,
    font=\itshape
}

\mdtheorem[style=paleStyle, frametitleaboveskip = 10pt, frametitlebelowskip = 10pt, frametitlefont = \normalfont\large\fontfamily{pag}\selectfont, theoremseparator = {\quad}]{MDtheorem}{Theorem}


\newenvironment{theorem}[1][]{
    \ifstrempty{#1}{\begin{MDtheorem}}{\begin{MDtheorem}[#1]}
}{%
    \end{MDtheorem}%
}%
\newenvironment{theorem*}[1][]{
    \ifstrempty{#1}{\begin{MDtheorem*}}{\begin{MDtheorem*}[\normalfont#1]}%
}{%
    \end{MDtheorem*}%
}%


\begin{document}

\lipsum[1-4]

Another strip of text another strip of text Another strip of text another strip of text Another strip of text another strip of text another strip of text another strip of text another strip of text another strip of text 

\begin{theorem}[Riemann Hypothesis]
\lipsum[2]
\end{theorem}

\end{document}

In this case, I have the "nobreak" setting on, so the theorem environment is forced to the next page, leaving a large white space below the text.