After reading this article I tried some examples and it looks like mod $4$ there are only two cases: $S_k\equiv 0$ if $k\equiv 0\lor k\equiv-1$ and $S_k\equiv 1$ otherwise.
For even $i=2m$ we have $(2m)^{2m}=(4m^2)^m=4^m\,m^{2m}\equiv 0\;\mod {4}$,
$(4i+1)^{4i+1}\equiv 1\;\mod{4}$ for the binomial expansion
$(4i-1)^{4i-1}\equiv -1\;\mod{4}$ for the same reason
thus adding together $S_k=1^1+2^2+3^3+4^4+5^5+\ldots$ we get $\mod{4}$ the following $1+0-1+0+1+\ldots$ therefore the property is proved.
Do you think the proof is correct? Any generalization?
Edit. for $k=3+16n$ we have $S_k=\sum _{i=1}^k i^i\equiv 0 \mod 16$
the proof is a bit boring if I don't find a shortcut