One definition of conjugate arises from the factoring of
$a^2 - b^2$ into $(a + b)(a - b)$.
But that does not answer the question of why we can obtain the complex conjugate of a complex number only by negating the imaginary part
and never by negating the real part.
(For that matter, it does not explain why we separate the number into
real and imaginary parts in order to obtain a conjugate in the first place.)
But there is another, somewhat different notion of conjugate.
Quoting one writer:
Two elements $\alpha, \beta$ of a field $K$, which is an extension field of a field $F$, are called conjugate (over $F$) if they are both algebraic over $F$ and have the same minimal polynomial.
(Barile, Margherita. "Conjugate Elements." From MathWorld--A Wolfram Web Resource, created by Eric W. Weisstein. http://mathworld.wolfram.com/ConjugateElements.html)
If we take $K$ to be the complex numbers, and $F$ to be the real numbers,
then we can verify that $a + bi$ and $a - bi$ ($a, b$ both real)
are the two roots of a certain polynomial over $z$,
specifically, the solutions for $z$ in the equation
$$ z^2 - 2az + a^2 + b^2 = 0, $$
in which the left-hand side is a polynomial
over the real numbers (that is, over $F$).
That is, $a + bi$ and $a - bi$ fit perfectly the definition of
conjugate elements of $\mathbb C$,
viewed as an extension field of $\mathbb R$.
What about $a + bi$ and $-a + bi$? These are solutions of the
equation $(z - a - bi)(z + a - bi) = 0$; multiplying out the
left-hand side, we see that $a + bi$ and $-a + bi$
are solutions for $z$ in
$$ z^2 - (2bi)z - a^2 - b^2 = 0. $$
The coefficients of the polynomial on the left-hand side are not
all real numbers unless $b = 0$, so it seems that $-a + bi$ cannot be
a conjugate of $a + bi$ in the same interesting way that $a - bi$ can.
Historically, complex numbers arose in the process of trying to
solve polynomials with real-valued coefficients.
Eventually, people decided that complex numbers were actually acceptable
roots of such a polynomial.
When you have a polynomial with integer coefficients, it always has
a factorization into polynomials of the form $(ax + b)$
or $(ax^2 + bx + c)$, where all the coefficients $a, b, c$ are real numbers.
Of course $(ax + b)$ has just one real root (and no other roots),
but the roots of $(ax^2 + bx + c)$ are precisely a pair of
complex conjugates.