Well, first of all, if your equation is intended to be inline you should be using single $s only - i.e. $ ... $.
In fact, in general, $$ ... $$ is deprecated in LaTeX:
Why is \[ ... \] preferable to $$ ... $$?
Now, onto the main point. I think what you're doing is enclosing the entire mathematical equation in $ ... $ (this is correct) and individual mathematical objects within the equation in $ ... $ as well. This is incorrect.
Try it like this:
$n_{\text{Alice}} = a \times b$
And:
$\gcd(n_{\text{Alice}}, n_{\text{Bob}}) =
\gcd(n_{\text{Alice}} \mod n_{\text{Bob}}, n_{\text{Bob}})$
You may like to play around a bit with the spacing, here. For example, I prefer it like this:
$\gcd(n_{\text{Alice}}, n_{\text{Bob}}) =
\gcd(n_{\text{Alice}} \!\! \mod n_{\text{Bob}}, n_{\text{Bob}})$
Although, I'll be honest, I'm not sure what the optimal result is here.
Just FYI: \! inserts a thin negative space.
Final big word of advice. Don't continue to muddle along. Get yourself a good beginner's guide:
What are good learning resources for a LaTeX beginner?
And nail the basics first. Then, absolutely, the best way to learn is to get your hands dirty and try it out and just use LaTeX. Learn by doing and look up how to do things as and when you need them, and ask here if you need help. But you've got to get first principles out of the way first, and many of those guides are great ways to get information about more tricky stuff as well.
