You need different column widths for each column. With an up-to-date TeX distribution you can use the w column type provided by array. But I'm afraid you have to decide the widest entry of each column.
I provide both the right and the center aligned versions; I much prefer the latter.
Avoid trailing \\ as a general rule (unless your alignment has a bottom rule). They do no harm inside array or matrix, they do in align.
\documentclass[10pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{mathtools}
\usepackage{array,calc}
\begin{document}
\begin{align}
\Phi_{1} &=
\begin{bmatrix}
\begin{array}{
w{r}{\widthof{$\pi$}}
w{r}{\widthof{$-\pi$}}
w{r}{\widthof{$-\pi$}}
w{r}{\widthof{$-\pi$}}
}
0 & -\pi & -\pi & 0 \
\pi & 0 & 0 & \pi \
\pi & 0 & 0 & \pi \
0 & -\pi & -\pi & 0
\end{array}
\end{bmatrix}
&
\Phi_{2} &=
\begin{bmatrix}
\begin{array}{
w{r}{\widthof{$\frac{3\pi}{2}$}}
w{r}{\widthof{$-\pi$}}
w{r}{\widthof{$-\pi$}}
w{r}{\widthof{$-\frac{3\pi}{2}$}}
}
0 & 0 & -\pi & -\pi \
0 & 0 & -\pi & -\pi \
\pi & \pi & 0 & 0 \
\pi & \pi & 0 & 0
\end{array}
\end{bmatrix}
\
\Phi_{3} &=
\begin{bmatrix}
\begin{array}{
w{r}{\widthof{$\pi$}}
w{r}{\widthof{$-\pi$}}
w{r}{\widthof{$-\pi$}}
w{r}{\widthof{$-\pi$}}
}
0 & -\pi & 0 & -\pi \
\pi & 0 & \pi & \pi \
0 & -\pi & 0 & -\pi \
\pi & -\pi & \pi & 0
\end{array}
\end{bmatrix}
&
\Phi_{4} &=
\begin{bmatrix}
\begin{array}{
w{r}{\widthof{$\frac{3\pi}{2}$}}
w{r}{\widthof{$-\pi$}}
w{r}{\widthof{$-\pi$}}
w{r}{\widthof{$-\frac{3\pi}{2}$}}
}
0 & -\frac{\pi}{2} & -\pi & -\frac{3\pi}{2} \
\frac{\pi}{2} & 0 & -\frac{\pi}{2} & -\pi \
\pi & \frac{\pi}{2} & 0 & -\frac{\pi}{2} \
\frac{3\pi}{2} & \pi & \frac{\pi}{2} & 0
\end{array}
\end{bmatrix}
\end{align}
\begin{align}
\Phi_{1} &=
\begin{bmatrix}
\begin{array}{
w{c}{\widthof{$\pi$}}
w{c}{\widthof{$-\pi$}}
w{c}{\widthof{$-\pi$}}
w{c}{\widthof{$-\pi$}}
}
0 & -\pi & -\pi & 0 \
\pi & 0 & 0 & \pi \
\pi & 0 & 0 & \pi \
0 & -\pi & -\pi & 0
\end{array}
\end{bmatrix}
&
\Phi_{2} &=
\begin{bmatrix}
\begin{array}{
w{c}{\widthof{$\frac{3\pi}{2}$}}
w{c}{\widthof{$-\pi$}}
w{c}{\widthof{$-\pi$}}
w{c}{\widthof{$-\frac{3\pi}{2}$}}
}
0 & 0 & -\pi & -\pi \
0 & 0 & -\pi & -\pi \
\pi & \pi & 0 & 0 \
\pi & \pi & 0 & 0
\end{array}
\end{bmatrix}
\
\Phi_{3} &=
\begin{bmatrix}
\begin{array}{
w{c}{\widthof{$\pi$}}
w{c}{\widthof{$-\pi$}}
w{c}{\widthof{$-\pi$}}
w{c}{\widthof{$-\pi$}}
}
0 & -\pi & 0 & -\pi \
\pi & 0 & \pi & \pi \
0 & -\pi & 0 & -\pi \
\pi & -\pi & \pi & 0
\end{array}
\end{bmatrix}
&
\Phi_{4} &=
\begin{bmatrix}
\begin{array}{
w{c}{\widthof{$\frac{3\pi}{2}$}}
w{c}{\widthof{$-\pi$}}
w{c}{\widthof{$-\pi$}}
w{c}{\widthof{$-\frac{3\pi}{2}$}}
}
0 & -\frac{\pi}{2} & -\pi & -\frac{3\pi}{2} \
\frac{\pi}{2} & 0 & -\frac{\pi}{2} & -\pi \
\pi & \frac{\pi}{2} & 0 & -\frac{\pi}{2} \
\frac{3\pi}{2} & \pi & \frac{\pi}{2} & 0
\end{array}
\end{bmatrix}
\end{align}
\end{document}

This is LuaHBTeX, Version 1.12.0 (MiKTeX 20.7) (format=lualatex 2020.11.13) 20 NOV 2020 12:54 system commands enabled.
LaTeX2e <2020-02-02> patch level 5
Not new enough?
– Simon Dispa Nov 20 '20 at 16:37\piand-\pidistracting, Except for the fractions, I think the right-alignment both most pleasing and most logical. (Zero and pi are close enough in width that no adjustment is needed.) I'd tinker with the fractions to center them on the horizontal midpoint of the zeros, and\llapa minus if one is there. – barbara beeton Oct 02 '22 at 20:44