My guess would be that the client announce the key pair he's using, giving the public key (of course!). But indeed, the key given with that message is a private key, which I don't fully understand why. I tried with a ssh -vvv with a server I use:
debug1: Offering RSA public key: /home/user/.ssh/id_rsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /home/user/.ssh/id_dsa
debug3: no such identity: /home/user/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /home/user/.ssh/id_ecdsa
debug3: no such identity: /home/user/.ssh/id_ecdsa: No such file or directory
debug1: Offering ED25519 public key: /home/user/.ssh/id_ed25519
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug1: Server accepts key: pkalg ssh-ed25519 blen 51
As I have 2 keys on this computer: the RSA one and the ECDSA one, it looks like the server rejected my RSA key. That's ok, I did not provide the RSA key to this server. Then, he tries to find a DSA and a ECDSA key, but I haven't any of these, so the "no such identity" is understandable. Then, I own a ED25519 key, and it tries this one. And that goes well, since I registered this key pair to that server. Note that it accepts my public key now!
Then, the protocol keeps running through a challenge where the server now knows which public key use to cypher the challenge, and the client (my machine) knows which private key to use for this challenge.