We can define a command to switch to a fallback font using fontspec.
Then we can use newunicodechar to map the missing Unicode characters to a command that switched to the fallback font for those characters.
\documentclass{article}
\usepackage{fontspec, newunicodechar}
\setmainfont{PT Serif}
\newfontfamily{\fallbackfont}{Linux Libertine O}[Scale=MatchLowercase]
\DeclareTextFontCommand{\textfallback}{\fallbackfont}
\newunicodechar{ɔ}{\textfallback{ɔ}}
\newunicodechar{ϱ}{\textfallback{ϱ}}
\begin{document}
Hellɔ woϱld.
\end{document}

Output of pdffonts showing that both fonts are used:
name type emb sub uni object ID
------------------------------------ ----------------- --- --- --- ---------
NUOAQT+PTSerif-Regular CID TrueType yes yes yes 4 0
UAEGUX+LinLibertineO CID Type 0C yes yes yes 5 0
EDIT:
We could also define a general command for setting up missing characters.
It takes an optional argument which defaults to \textfallback, but a different font could be inserted in case an additional fallback font was needed.
\newcommand{\fallbackchar}[2][\textfallback]{%
\newunicodechar{#2}{#1{#2}}%
}
\fallbackchar{ɔ}
\fallbackchar{ϱ}
\DeclareTextFontCommand\textfallback{\fallbackfont}is the official way of defining\textfallback. – Manuel Jan 23 '15 at 15:08\DeclareTextFontCommandused twice in thefontspecmanual, but not explained. – musarithmia Jan 23 '15 at 15:10latex.ltx. I edited the answer to use this. – musarithmia Jan 23 '15 at 15:26fontspecstumbles upon a character that is not in the font, it prints that character with\textfallbackinstead. – Gaussler Jan 23 '15 at 19:50