\documentclass{book}
\usepackage{showframe}
\makeatletter
\newcommand\myvspace[1]{\if@minipage\else\vspace{#1}\fi}
\makeatother
\begin{document}
\clearpage
\vspace{1cm}
test
\clearpage
\noindent\begin{minipage}{\textwidth}
\myvspace{1cm} %should be ignored
test
\myvspace{1cm} %should be kept
test
\end{minipage}
\end{document}
I left it as you had it but did you intend a paragraph break so there was vertical space between the test ? You can not see the second vspace as it comes after the text.
The \if@minpage is true at the start of a minipage and set false at the first paragraph, it is used.
As a perhaps better alternative to defining your own command you could use the standard LaTeX \addvspace command which already includes this test (you would then have to have a paragraph break though as \addvspace in horizontal mode is an error.
\documentclass{book}
\usepackage{showframe}
\begin{document}
\clearpage
\addvspace{1cm}
test
\clearpage
\noindent\begin{minipage}{\textwidth}
\addvspace{1cm} %should be ignored
test
\addvspace{1cm} %should be kept
test
\end{minipage}
\end{document}
