This looks like quite a useful feature for the tqft library. I've had a go at implementing it, I'd like to know if it works "in the wild" before pushing it to CTAN. You can get the package from github, download the file tqft.dtx and run tex tqft.dtx to generate the package files. Put them somewhere that TeX can find them, and try the following code.
\documentclass{standalone}
%\url{https://tex.stackexchange.com/q/312405/86}
\usepackage{tikz}
\usetikzlibrary{tqft}
\begin{document}
\begin{tikzpicture}[
tqft,
every outgoing boundary component/.style={fill=blue!50},
outgoing boundary component 3/.style={fill=none,draw=red},
every incoming boundary component/.style={fill=green!50},
every lower boundary component/.style={draw,ultra thick, dashed},
every upper boundary component/.style={draw,purple},
cobordism/.style={fill=red!50},
cobordism edge/.style={draw},
genus=3,
hole 2/.style={ultra thick, blue},
view from=incoming,
anchor=between incoming 1 and 2
]
\pic[name=a,tqft,
incoming boundary components=5,
skip incoming boundary components={2,4},
outgoing boundary components=7,
skip outgoing boundary components={2,3,5},
offset=-.5];
\begin{scope}[every pin edge/.style={<-}]
\foreach \anchor/\ang in {
hole 1/-90,
hole 2/90,
hole 3/-90,
incoming boundary 3/90,
outgoing boundary 4/-90,
between last incoming and last outgoing/0,
between first incoming and first outgoing/180,
between incoming 1 and 3/90,
between outgoing 1 and 4/-90,
between outgoing 4 and 6/-90
} {
\node[pin=\ang:\anchor,at=(a-\anchor),inner sep=0pt] {};
}
\draw[<-] (0,0) -- ++(0,3);
\end{scope}
\end{tikzpicture}
\end{document}
There are two new keys demonstrated in the above:
skip incoming boundary components={2,4}
skip outgoing boundary components={2,3,5}
For both sets of boundary components, the number of components (set by the incoming boundary components=5 and outgoing boundary components=7) sets the total number of boundary slots that are available. The new keys then say which of those slots to ignore. So in the above cobordism, there are actually only 3 incoming boundary components as the 2nd and 4th are ignored. Notice that the anchors are defined with reference to the gaps, so that the boundary components are labelled as 1, 3, and 5.
The above code produces the following diagram.

Comments, bugs, and feature requests welcome - either here or on github.
For completeness, here's my rendering of your original picture with all the boundary components aligned.
\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{shapes.geometric,tqft}
\usepackage{tqft}
\tikzset{
tqft/use nodes=false,
}
\begin{document}
\begin{tikzpicture}[
tqft,
every outgoing boundary component/.style={fill=blue!50},
outgoing boundary component 3/.style={fill=none,draw=red},
every incoming boundary component/.style={fill=green!50},
every lower boundary component/.style={draw,ultra thick, dashed},
every upper boundary component/.style={draw,purple},
cobordism/.style={fill=red!50},
cobordism edge/.style={draw},
view from=incoming,
cobordism height=4cm,
]
\begin{scope}[every node/.style={rotate=90}]
\pic[name=a,
tqft,
incoming boundary components=3,
skip incoming boundary components=2,
outgoing boundary components=0
];
\pic[name=b,
tqft,
incoming boundary components=3,
skip incoming boundary components=2,
outgoing boundary components=3,
skip outgoing boundary components=2,
offset=1,
anchor=incoming boundary 1,
at=(a-incoming boundary 2)
];
\pic[name=c,
tqft,
incoming boundary components=2,
outgoing boundary components=0,
anchor={(0,0)},
at=(b-incoming boundary 3)
];
\pic[name=d,
tqft,
incoming boundary components=0,
outgoing boundary components=4,
skip outgoing boundary components=3,
at=(a-incoming boundary 1)
];
\pic[name=e,
tqft,
incoming boundary components=0,
outgoing boundary components=1,
at=(c-incoming boundary 2)
];
\end{scope}
\end{tikzpicture}
\end{document}
