I cannot give a deep answer on why that is but the rectangle construction is AFAIK a simple (point1) -| (point1) -| cycle. And the |- does not get transformed normally. However, there are various possibilities to achieve what you seem to want. You see that the rotate around key is not as "efficient" as the canvas is yx plane key that comes with the 3d library. Notice that there is a small bug in the 3d library that has been fixed here. Of course, with the 3d library it is not straightforward to implement arbitrary rotations, however, that can be fixed by using the tikz-3dplot package. If you want to access all possible rotations, you may be interested in the answers to this question. There you can also find a way of achieving arbitrary rotations without even loading the tikz-3dplot package, but there is IMHO no reason not to load that package.
\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{3d}
\begin{document}
\begin{tikzpicture}[rotate around x=90,transform shape]
\draw[red] (0,0) rectangle (1,1);
\draw[yshift=-20,purple] (0,0) -| (1,1) -| cycle;
\draw[yshift=-40,blue] (0,0) -- (1,0) -- (1,1) -- (0,1) -- cycle;
\node[anchor=north west,minimum width=1cm,minimum
height=1cm,draw,green!50!black] at (0,-2){};
\begin{scope}[xshift=8cm,canvas is yx plane at z=0,transform shape]
\draw[red] (0,0) rectangle (1,1);
\draw[yshift=-40,blue] (0,0) -- (1,0) -- (1,1) -- (0,1) -- cycle;
\node[anchor=north west,minimum width=1cm,minimum height=1cm,draw,green!50!black] at (0,-2){};
\end{scope}
\end{tikzpicture}
\end{document}

3dlibrary for that and then usetransform shapein somecanvas is yz plane at x=0. – Aug 30 '18 at 17:16