I recently took an algebra test, and one of the questions was like this (apologise upfront because I don't remember the exact matrix elements, but the idea is essentially the same):
Let $A$ and $B$ be two square matrices. We know that: $$AB=\begin{bmatrix}1 & 2 \\ 3&4\end{bmatrix}, \quad BA=\begin{bmatrix}3 & a \\b &2\end{bmatrix}$$ What can be said about values $a$ and $b$? What possible values can they have?
The real exam problem gave me only 4 options and only 1 was correct (it was a multiple-choice test question). The first 3 of the answers suggested possible values for $a$ and $b$, whereas the last answer stated $\text{(d)Nothing can be said about a and b}$.
How would you proceed? I spent like 10 minutes trying several things to no avail. No further conditions were given on A nor B. We only know they are square and 2x2.
What I tried:
I stablished: $$AB=X, \quad BA=Y$$ Then I multiplied by the inverse of $B$ (assuming it has an inverse), to reach: $$A=XB^{-1}=B^{-1}Y$$ $$B=A^{-1}X=YA^{-1}$$ But I really don't know where this will lead me.