Writing in your style:
\documentclass[reqno,12pt]{amsart}
\usepackage{amsmath,mathtools,amssymb}
\usepackage[top=2cm,left=2cm,right=2cm]{geometry}
\author{something}
\title{something}
\begin{document}
\maketitle
\section{something}
\begin{equation}
\newcommand\yourmatrix{\begin{bmatrix}
0&0&0&0&0\\
0&0&0&0&0\\
0.271&0&0&0&0\\
0.498&0.271&0&0&0\\
0.687&0.498&0.271&0&0\\
0.845&0.687&0.498&0.271&0\\
0.977&0.845&0.687&0.498&0.271\\
1.087&0.977&0.845&0.687&0.498\\
1.179&1.087&0.977&0.845&0.687\\
1.256&1.179&1.087&0.977&0.845
\end{bmatrix}}
\mathbf{G}=\overbrace{\yourmatrix}^{\text{something 1}}\left.\vphantom{\yourmatrix}\right\}{\scriptstyle\text{something 2}}
\end{equation}
\end{document}
In this way, the vertical brace is good, but it is a little painful to have it. The horizontal brace is not good because it groups the whole matrix (with brackets). Consider some methods used in braces over matrix and similar questions.

This is a good modification of it, but this is a bit overkill. Literally you can't avoid defining a macro if you follow this way.
\documentclass[varwidth]{standalone}
\usepackage{amsmath,mathtools,amssymb}
\begin{document}
\begin{equation}
\newcommand\yourmatrix{\begin{matrix}
0&0&0&0&0\\
0&0&0&0&0\\
0.271&0&0&0&0\\
0.498&0.271&0&0&0\\
0.687&0.498&0.271&0&0\\
0.845&0.687&0.498&0.271&0\\
0.977&0.845&0.687&0.498&0.271\\
1.087&0.977&0.845&0.687&0.498\\
1.179&1.087&0.977&0.845&0.687\\
1.256&1.179&1.087&0.977&0.845
\end{matrix}}
\mathbf{G}=\left[\vphantom{\yourmatrix}\right.\overbrace{\yourmatrix}^{\text{something 1}}\left.\vphantom{\yourmatrix}\right]\left.\vphantom{\yourmatrix}\right\}{\scriptstyle\text{something 2}}
\end{equation}
\end{document}

Using savebox (hope I get it right – please correct me if not)
\documentclass[varwidth]{standalone}
\usepackage{amsmath,mathtools,amssymb}
\newsavebox{\yourmatrix}
\begin{document}
\begin{equation}
\sbox{\yourmatrix}{$\begin{matrix}
0&0&0&0&0\\
0&0&0&0&0\\
0.271&0&0&0&0\\
0.498&0.271&0&0&0\\
0.687&0.498&0.271&0&0\\
0.845&0.687&0.498&0.271&0\\
0.977&0.845&0.687&0.498&0.271\\
1.087&0.977&0.845&0.687&0.498\\
1.179&1.087&0.977&0.845&0.687\\
1.256&1.179&1.087&0.977&0.845
\end{matrix}$}
\mathbf{G}=\left[\vphantom{\usebox{\yourmatrix}}\right.\overbrace{\usebox{\yourmatrix}}^{\text{something 1}}\left.\vphantom{\usebox{\yourmatrix}}\right]\left.\vphantom{\usebox{\yourmatrix}}\right\}{\scriptstyle\text{something 2}}
\end{equation}
\end{document}
(same output as above)
\left.\right\}? – Apr 22 '19 at 15:30\left.doesn't produce anything. However, honestly this way is very bad. Consider something like https://tex.stackexchange.com/q/130798/156344 – Apr 22 '19 at 15:34\left \righta bit more carefully. The problem is the horizontal brace. – Apr 22 '19 at 15:38