In asymmetric cryptography, everyone has the public keys and only the entity being verified by the public key has the private key. The public key and private key are related in a way that having the public key doesn't reveal anything about the private key, but that the holder of the private key can prove their identity by having someone verify a challenge using the public key.
Basically, something that is encrypted with a public key can only be decrypted by the private key and something that is encrypted with the private key can only be decrypted with the public key.
To prove the server is who they claim to be, they encrypt a challenge with their private key (which only they have) and the public key is used to verify that the challenge can be decrypted, thus the other party has the private key.
To talk to the server securely, the public key is used to encrypt a symmetric session key. That session key is encrypted with the public key and sent to the server. Only the valid server (that has the private key) can decrypt the session key, so only the client (who created the session key) and the server with the private key can communicate.
The public key is trusted by the client because it is similarly signed by a root certificate authority which is included with the browser.
Update: I misread the question as SSL, not SSH. The basic idea is still roughly the same, but the server certificate often won't be signed by a CA, in which case the certificate needs to be manually verified the first time after which your client should keep track of who you connected to last time. The key exchange also differs in terms of how a key is agreed upon, but I don't know the SSH handshakes as well as I know SSL's, so see Thomas Pornin's answer for that.
Aconnects toGOOD, he usesGOOD's public key to encrypt his messages toGOOD. IfBADintercepts encrypted message intended forGOOD,BADcan't read them, because of the encryption. AssumingAalready knowsGOOD's public key, he will not useBAD's public key by mistake when trying to talk toGOOD(or if he does, the system will give him a stern warning that it's a bad idea). Thus,A's outgoing messages will be encrypted withGOOD's public key, and messages fromGOODtoAare encrypted withA's public key. – apsillers Jul 26 '13 at 16:46GOODdoesn't have a keypair, thenAcan't send encrypted messages toGOOD. In an asymmetric key system, the recipient of encrypted messages must have a private key (and then senders use the associated public key to send messages).A's public key is used for encrypting messages intended forA(i.e., to be decrypted byA's private key). To have two-way communication and authentication, you need a keypair for each party. – apsillers Jul 26 '13 at 16:50~/.ssh/authorized_keysto allow me to SSH from my notebook to my server. This is precisely the scenario I was concerned with, connecting to a server out there that is actually a MITM pretending to be my server (also having the public key of course). – orokusaki Jul 26 '13 at 17:37authorized_keysto verify your identity, and you store a fingerprint of the server's public (in your laptop's~/.ssh/known_hosts) key to verify the server. See also What is the difference between authorized_key and known_host file for SSH? – apsillers Jul 26 '13 at 17:59