Note that the integers modulo 41 (or any prime) form a field.
41 isn't too many numbers: you can just do as you did and check by hand. But you may also compute this by exploiting Fermat's little theorem. I am not sure that's a whole lot better than just checking but here is a type of logical approach: We will do our best to create the expression $x^{40}$ as we know by Fermat's little theorem $x^{p-1}\equiv1 \mod p$.
$$\begin{align}
&x^3& \equiv &20 \mod 41 \\
&x^3&\equiv &2^2 \times 5 \mod 41 \\
&{(x^3)}^{13}&\equiv &2^{26} \times 5^{13} \mod 41
\end{align}
$$
But note that $(2^3 \times 5)^8\equiv (-1)^8 \mod 41$ so then
$$\begin{align}
&{(x^3)}^{13}&\equiv & 2^{2} \times 5^{5} \mod 41 \\
&2^25^5&=&10^25^3=100\times 125\equiv 18\times 2\mod 41
\end{align}
$$
Then we have
$${(x^3)}^{13}\equiv 36 \mod 41$$
but then we also know by Fermat's little theorem that $x^{40} \cong 1 \mod 41$ therefore $x\equiv 36^{-1} $
But it's actually easy to spot the multiplicative inverse of 36 mod 41. It must be $7^2$ because $6 \times 7 =42$
$6^2 \times 7^2 =42^2\equiv 1\mod 41$ and therefore $x \equiv 7^2\equiv 8 \mod 41. $