It looks like you want to use independent counters for algorithms and protocols. As far as I can see the algorithm package does not allow this and only works with the algorithm counter. So, what you need to do is "trick" the algorithmic environment to use separate counters. Do do this you should set up two new counters: one to keep track of the number of protocols and one to keep track of the algorithms. In the MWE below I have defined them as:
\newcounter{protocol}% counter for protocols
\newcounter{algorithm saved}% the real counter for algorithms
The algorithmic environment will happily use the algorithm counter. When you switch to your protocol environment you need to switch to using the protocol counter -- and at the same time save the current value of the algorithm counter. If you "swap" back to using the algorithm counter at the end of the protocol environment then everything will remain in sync.
The following MWE does what you want:
\documentclass{article}
\usepackage{algorithm}
\usepackage{algpseudocode}
\newcounter{protocol}% counter for protocols
\newcounter{algorithm saved}% the real counter for algorithms
\makeatletter
\newenvironment{protocol}[1][htb]{%
\renewcommand{\ALG@name}{Protocol}% Update algorithm name
\setcounter{algorithm saved}{\value{algorithm}} % switch to using the protocol counter
\setcounter{algorithm}{\value{protocol}}% save the current number of algorithms
\begin{algorithm}[#1]%
}{\end{algorithm}
\setcounter{protocol}{\value{algorithm}}% save the current number of protocols
\setcounter{algorithm}{\value{algorithm saved}}% restore the algorithm counter
}
\makeatother
\begin{document}
\begin{algorithm}
\caption{Algo Test}
\label{algo:test}
\begin{algorithmic}
\State $x \gets y + 1$
\end{algorithmic}
\end{algorithm}
\begin{protocol}
\caption{Prot Test}
\label{prot:test}
\begin{algorithmic}
\State $x \gets y + 1$
\end{algorithmic}
\end{protocol}
\begin{algorithm}
\caption{Algo Test}
\label{algo:test}
\begin{algorithmic}
\State $x \gets y + 1$
\end{algorithmic}
\end{algorithm}
\begin{protocol}
\caption{Prot Test}
\label{prot:test}
\begin{algorithmic}
\State $x \gets y + 1$
\end{algorithmic}
\end{protocol}
\end{document}
Here's the output:
