A relation between two elements

Question

Let $G $ be a group and $ a, b\in G $ such that $a^{-1}b^2a=b^3$ and $b^{-1}a^2b=a^3$.

How can we show that $a=b$$\,$?

Answer 1

$\bbox[5px,border:2px solid]{\begin{array}{c}b^2a=ab^3\\a^2b=ba^3\end{array}}\ $ these relations are equivalent to OP's since elements in a group are invertible.

If $G$ is commutative or if $a,b$ commute

We have $ab=ba$.

$b^3=a^{-1}b^2a=a^{-1}ab^2=b^2\Rightarrow b=1$

$a^3=b^{-1}a^2b=b^{-1}ba^2=a^2\Rightarrow a=1$

We get $a=b=1$.

G any group, but either $a$ or $b$ is of order $\le$ 4

For instance $a=1$

$b^3=a^{-1}b^2a=b^2\Rightarrow b=1$

For instance $a^2=1$

$a=a^3=b^{-1}a^2b=b^{-1}b=1$ and we conclude like previously

For instance $a^3=1$

$b=ba^3=a^2b\Rightarrow a^2=1$ and we conclude like previously

For instance $a^4=1$

$(ba)a^2=ba^3=a^2b=a^2ba^4=a^2(ba^3)a=a^2(a^2b)a=a^4(ba)=ba\Rightarrow a^2=1$ and we conclude like previously

Everything being symetrical in $a,b$, the same conclusion arises if we make the hypothesis on $b$.

So here also we get $a=b=1$

G any group, but either $a$ or $b$ order is finite

Let assume a^n=1

$a^2(ba)=(a^2b)a=(ba^3)a=(ba)a^3\Rightarrow \bbox[5px,border:2px solid]{\forall k\in\mathbb N,\ a^{2k}(ba)=(ba)a^{3k}}$

If n even then n=2p

$a^{2p}(ba)=a^n(ba)=(ba)=(ba)a^{3p}=(ba)a^na^{p}=(ba)a^{p}\Rightarrow a^{p}=1$ and since $p<n$ we can conclude by recurrence.

If n odd then n=2p+1  

$a^{2p}(ba)=a^{n-1}(ba)=a^{-1}(ba)=(ba)a^{3p}=(ba)a^na^{p-1}=(ba)a^{p-1}\Rightarrow (ba)=(ab)a^p$

$b=(ab)a^{p-1}$

$(ab)b^2=ab^3=b^2a=(ab)a^{p-1}\,(ab)a^{p-1}\ a=(ab)a^{p-2}(a^2b)a^p=(ab)a^{p-2}(ba^3)a^p$

$b^2=a^{p-2}ba^{p+3}=a^{p-4}(a^2b)a^{p+3}=a^{p-4}(ba^3)a^{p+3}=a^{p-4}ba^{p+6}$

$\bbox[5px,border:2px solid]{\forall k\in\mathbb N,\ b^2=a^{p-2k}\,b\,a^{p+3k}}$

If p even

For $p=2k$ we have $b^2=ba^{5k}\Rightarrow b=a^{5k}$ then $a,b$ commute and we can conclude.

If p odd

For $p=2k+1$ we have $b^2=aba^{5k+1}\Rightarrow (ab)b^2=ab^3=b^2a=aba^{5k+2}=(ab)a^{5k+2}$

$b^2=a^{5k+2}\Rightarrow ab^3=b^2a=a^{5k+3}\Rightarrow b^3=a^{5k+2}=b^2\Rightarrow b=1$ and we can also conclude.

Everything being symetrical in $a,b$ the same conclusion arises if we make the hypothesis $b^n=1$.

Thus we have proved it in all cases either directly if $n$ odd, either by recurrence if $n$ is even.

G infinite non-commutative group, order of $a,b$ infinite

I have no idea... :-(