Hilbert's 14th question roughly asks when the ring of invariants (under the rational action of an algebraic group $G$) of a finitely generated $k$-algebra $A$ ($k$ a field) is finitely generated. Nagata answered this affirmatively if $G$ is geometrically reductive and gave a counter-example if $G$ is not reductive (Popov showed that if $A^G$ is finitely generated for all $A$ then $G$ had to be reductive). Haboush resolved Mumford's conjecture and showed reductive groups (unipotent radical is trivial) and geometrically reductive groups are equivalent. This allowed for GIT to be be extended from characteristic 0 to arbitrary characteristic. Here is the basic idea behind GIT. First, use the ring of invariants in the affine setting to construct an affine algebraic quotient (good categorical quotient). Then, in the quasi-projective case one chooses an ample $G$-linearized line bundle $L$ (essentially a choice of coordinate system) and with respect to $L$ define semistable points (essentially points that can be distinguished by invariants in terms of the coordinates). Then one covers the semistable locus with affine opens and uses the affine quotient construction to obtain a local quotients. Finally one glues together the local affine pieces to obtain a quasi-projective GIT quotient. The construction all comes down to Hilbert's 14th being true for reductive groups and potentially false otherwise.
References:
To learn about GIT, where this summary mostly comes from, I recommend Lectures on Invariant Theory by Dolgachev.
There is non-reductive GIT too, but it is much more complicated. See Towards non-reductive geometric invariant theory by Brent Doran, Frances Kirwan.
Now, that speaks to the practical reality of why reductive groups are important, but does not address the "philosophy" that you ask for.
Here opinions might vary. I find the following philosophy comforting: "reductive affine algebraic groups are the algebro-geometric analogue of compact Lie groups."
There are a number of reasons for this:
- Instead of integrating over compact groups, one has the Reynolds operator for reductive groups.
- A complex affine algebraic group $G$ is reductive if and only if it is complexification of a compact Lie group.
- Compact quotients have Mostow Slices and Complex Reductive quotients have Luna Slices.
- Through Kempf-Ness Theory one can actually replace (up to homeomorphism) affine GIT quotients $X/G$ with the analytic topology with quotients $Y/K$ where $Y\subset X$ and $K$ is a maximal compact subgroup of $G$.
- Via 4. Oliver's proof of the Conner Conjecture extends to affine GIT quotients with the analytic topology.
Reference: Schwarz, Gerald W., The topology of algebraic quotients. Topological methods in algebraic transformation groups (New Brunswick, NJ, 1988), 135–151, Progr. Math., 80, Birkhäuser Boston, Boston, MA, 1989.