I ran into this same issue when trying to define an inner product operator. (The other answers seem to require scaling "by hand," whereas this one is more automatic in a sense, hence why I am posting it.) I settled on the following simplistic, but to my taste, satisfactory, solution:
\documentclass{article}
\usepackage{mathtools}
\newcommand{\bilinear}[2]{\left(\left.#1, \middle|, #2\right.\right)}
\begin{document}
[
\lambda_{l}
= \lim_{q \to 0^{+}}P \bilinear{X_{2} \leq F_{2}^{(-1)}(q)} {X_{1} \leq F_{1}^{(-1)}(q)}
]
\end{document}

The command here is called "bilinear", but you could call it what you like of course. The idea here was to first enclose the stuff on either side of the middle | character by (empty) \left. and \right. so that the | isn't too big (since it doesn't match the | to enclosing parentheses), then enclose the ensemble in a \left( and \right). Now the | isn't larger than the parentheses at least.
I should add, a lot of this was inspired by the comment by @Dan above.
amsmathloaded, the issue does not present for me. – cslstr May 29 '14 at 16:04Computer Modern, please be sure to indicate which font-related package(s) you load. – Mico May 29 '14 at 16:39