This seems to be a problem with polyglossia. I found a solution here but it seemed incomplete. It adds font switching commands to the language switching command, but it didn't do that when you end a language environment, e.g. at \end{english}. I added that code, so that it sets up the language outside of the english environment, which normally should be hebrew. The scheme will probably not work with more complicated nested environments.
\documentclass[openany,a5paper]{book}
\usepackage{polyglossia,xcolor}
\setdefaultlanguage{hebrew}
\setotherlanguage{english}
\newfontfamily\englishfont{Latin Modern Roman}[
Script=Latin,
Language=English,
Ligatures=TeX,
Color=red]
\newfontfamily\hebrewfont{Arial}[
Script=Hebrew,
Color=blue]
% Add correction code, inspired by https://tex.stackexchange.com/a/325833/113546
\makeatletter
\def\xpg@set@normalfont#1{%
\letcs{\rmfamily}{#1@font@rm}%
\letcs{\sffamily}{#1@font@sf}%
\letcs{\ttfamily}{#1@font@tt}%
\def\normalfont{\protect\xpg@select@fontfamily{#1}}%def instead of gdef
\def\reset@font{\protect\normalfont}%
}
\addto\inlineextras@english{\xpg@set@normalfont{english}}
\addto\blockextras@english{\xpg@set@normalfont{english}}
% Switch back explicitely to previous language
% This should normally be 'hebrew' outside 'english'.
\addto\endenglish{\selectlanguage{\babelname}}
\addto\inlineextras@hebrew{\xpg@set@normalfont{hebrew}}
\addto\blockextras@hebrew{\xpg@set@normalfont{hebrew}}
\makeatother
\begin{document}
\chapter{פרק ראשון}
טקסט טקסט טקסט.
\begin{english}
\chapter{this should be in Latin Modern}
Like this one is.
\chapter{Another test}
Some other text.
\end{english}
\chapter{פרק ארבע}
טקסט טקסט טקסט.
\end{document}
with thebidiandlayout=sectioning.tabularcommands should be able to handle this. If you addbabeltags, your document should be backward-compatible with\begin{english}or\texthebrew`. – Davislor Feb 06 '22 at 20:21