I'd like to use the syntax of the mathtools package to generate brackets larger than \Bigg.
One can create larger brackets than \Bigg of specified size using
About big parenthesis larger than Bigg
However, that solution doesn't work with the \DeclarePairedDelimiter construct of mathtools.
See the MWE below. I suspect that what's needed is \Vastl, \Vastr, and \Vastm commands.
\documentclass{article}
\usepackage{mathtools}
\makeatletter
\newcommand{\vast}{\bBigg@{4}}
\newcommand{\Vast}{\bBigg@{5}}
\makeatother
\DeclarePairedDelimiter{\parens}()
\begin{document}
\[
\parens[\Bigg]{\dfrac{\dfrac{y}{x}}{\dfrac{a}{b}}}
\]
\[
\Vast({\dfrac{\dfrac{y}{x}}{\dfrac{a}{b}}} \Vast)\qquad
\parens[\Vast]{\dfrac{\dfrac{y}{x}}{\dfrac{a}{b}}} % does not work
\]
\end{document}

% does not workon the last example. your output shows that it does work (and i have just reproduced that result). – barbara beeton Aug 19 '15 at 13:22