RE-REVISED ANSWER taking insightful suggestion from barbara's comment. Here, as in the immediately prior answer, I use Knuth's \underbrace rotated to make the vertical brace. However, I took barbara's suggestion to make the short brace look better by first making an \underbrace twice as wide as needed, and then squeezing it in half. The net effect is to make the points of the brace more vertically compressed.
The controlling parameters are \hsqz to control the horizontal compression of the brace, and \sqz and its reciprocal \sqzinverse, to control the vertical compression of the brace points. And, as before, bls is empirically matched to the BMAT line spacing, so that the 1st argument to \scalebrace can be given as an integer number of lines, rather than as a length.
\documentclass{article}
\usepackage{easybmat}
\usepackage{bm}
\usepackage{graphicx}
\def\sqz{.5}
\def\sqzinverse{2}
\def\hsqz{.7}
\newcommand{\smsub}[1]{\scriptscriptstyle\mathrm{#1}}
\newlength\bls
\bls=1.5\baselineskip\relax
\newlength\tmplength
\def\scalebrace#1#2{\tmplength=#1\bls\relax%
\scalebox{\hsqz}[\sqz]{\rotatebox{90}{$\underbrace{\hspace{\sqzinverse\tmplength}}$}}%
\raisebox{\dimexpr+.5\tmplength+.5\dp\strutbox-.5\ht\strutbox}{$\scriptstyle \; #2$}}
\begin{document}
\begin{equation}
\hat{\bm{P}}=
\left(
\begin{BMAT}(b){c}{ccc.c}
P_{\smsub{1}} \\
\vdots \\
P_{\smsub{K-1}}\\
P_{\smsub{K}}
\end{BMAT}
\right)=
\left(
\begin{BMAT}(b){c}{ccc.c}
\vphantom{\vdots} \\
\bm{\breve{P}} \\
\vphantom{\vdots} \\
P_{\smsub{K}}
\end{BMAT}
\right)
\begin{BMAT}(@)[1pt,10pt,0pt]{l}{cc}
\scalebrace{3}{(K-1)\times 1}\\
\scalebrace{1}{1 \times 1}\\
\end{BMAT}
\end{equation}
\end{document}

REVISED ANSWER (replacing stretched scalerel brace with a rotated \underbrace). The user syntax will be the same as the original answer (e.g., \scalebrace{1}{1 \times 1}), though the resulting brace will look different (and better).
\documentclass{article}
\usepackage{easybmat}
\usepackage{bm}
\usepackage{graphicx}
\newcommand{\smsub}[1]{\scriptscriptstyle\mathrm{#1}}
\newlength\bls
\bls=1.5\baselineskip\relax
\newlength\tmplength
\def\scalebrace#1#2{\tmplength=#1\bls\relax%
\scalebox{.7}[1]{\rotatebox{90}{$\underbrace{\hspace{\tmplength}}$}}%
\raisebox{\dimexpr+.5\tmplength+.5\dp\strutbox-.5\ht\strutbox}{$\scriptstyle \; #2$}}
\begin{document}
\begin{equation}
\hat{\bm{P}}=
\left(
\begin{BMAT}(b){c}{ccc.c}
P_{\smsub{1}} \\
\vdots \\
P_{\smsub{K-1}}\\
P_{\smsub{K}}
\end{BMAT}
\right)=
\left(
\begin{BMAT}(b){c}{ccc.c}
\vphantom{\vdots} \\
\bm{\breve{P}} \\
\vphantom{\vdots} \\
P_{\smsub{K}}
\end{BMAT}
\right)
\begin{BMAT}(@)[1pt,10pt,0pt]{l}{cc}
\scalebrace{3}{(K-1)\times 1}\\
\scalebrace{1}{1 \times 1}\\
\end{BMAT}
\end{equation}
\end{document}

The width of the brace is controlled by the argument to \scalebox in the definition of \scalebrace, currently set to .7. Changing that to 1 will give just a plain rotated \underbrace.
ORIGINAL ANSWER:
Here's one way. The scalerel package allows width-limited scales. Thus, when I scale the top right brace to 3 row heights and the bottom right brace to 1 row height, I limit the scaled width of each brace to 1.2ex. In this case, I use the syntax
\scaleleftright[1.2ex]{.}{phantom rule}{\}} ,
where I create the phantom rule with the newly defined \irule that takes an integer number of rows as an argument.
I wrap up the whole shebang inside a macro
\scalebrace{rows}{aftertext}
FYI: The "row height" inside the matrix was empirically determined as 1.5\baselineskip, set in the length \bls.
\documentclass{article}
\usepackage{easybmat}
\usepackage{bm}
\usepackage{scalerel}
\newcommand{\smsub}[1]{\scriptscriptstyle\mathrm{#1}}
\newlength\bls
\bls=1.5\baselineskip\relax
\newlength\tmplength
\def\irule#1{\tmplength=#1\bls\relax%
\rule[-.5\tmplength-.5\dp\strutbox+.5\ht\strutbox]{0ex}{\tmplength}}
\def\scalebrace#1#2{%
\scaleleftright[1.2ex]{.}{\irule{#1}}{\}}{\scriptstyle \; #2}}
\begin{document}
\begin{equation}
\hat{\bm{P}}=
\left(
\begin{BMAT}(b){c}{ccc.c}
P_{\smsub{1}} \\
\vdots \\
P_{\smsub{K-1}}\\
P_{\smsub{K}}
\end{BMAT}
\right)=
\left(
\begin{BMAT}(b){c}{ccc.c}
\vphantom{\vdots} \\
\bm{\breve{P}} \\
\vphantom{\vdots} \\
P_{\smsub{K}}
\end{BMAT}
\right)
\begin{BMAT}(@)[1pt,10pt,0pt]{l}{cc}
\scalebrace{3}{(K-1)\times 1}\\
\scalebrace{1}{1 \times 1}\\
\end{BMAT}
\end{equation}
\end{document}

scalerelproduces its result by taking a text glyph (in this case, the regular}character) and scaling (and or stretching) it. This is different than the way LaTeX normally would build a brace. Since both instances of}are the same width, but different height, one will look slightly stretched. There is one other possibility I could try, and if I get a chance, I'll see if it pans out. – Steven B. Segletes Jun 30 '14 at 09:56scalerelapproach. But as to the revised answer, it is the way Knuth created the\underbracecommand. Trying to make something fixed in one dimension, but stretchable in the other dimension, necessarily involves compromises. Also, all the braces, in each respective solution, are of the same width. I guarantee it, though it may appear otherwise. – Steven B. Segletes Jun 30 '14 at 15:18