How it HTTPS secure? Can't anyone decrypt the responses?
cryptography, https, security
Solution
Your analysis of how the SSL connection is established is missing an important part of the initialization, the key exchange.
As part of the setup of the SSL connection, the client and server agree on a symmetric cipher to be used, and the key to be used with that cipher. The security is based on the strength of the cipher, and the secrecy of the symmetric key, which is known only to the client and the server.
The mechanics of generating and sharing the symmetric key is complicated.
In an oversimplified scenario (sufficient to answer the question that was asked) we can consider that the client generates the symmetric key to be used. The client sends the key to the server, the trick is that it sends it in a message that is encrypted with an asymmetric cipher, using the public key of the server.
Since the message (containing the "secret" key) is encrypted using the public key of the server, only the server can decrypt the message, the symmetric key is "secret" in that only the server and the client know the symmetric key.
Once the client and server have established the symmetric cipher and key to be used for encryption, the rest of the session uses that cipher and key to encrypt and decrypt messages.
Q: Can't anyone decrypt the responses?
It would be possible for any message that is "encrypted" with the server's private key to be decrypted by anyone that has the server's public key. This is however not how SSL works - messages should never be encrypted with a private key.
The "trick" (if you will) is that second key and cipher. The client generates a second "secret" key, known only to itself, and then securely sends that second key to the server. (It does this securely by encrypting the second "secret" key in a message, using the public key of the server. Either that or the client and server establish a secret key by performing a key agreement protocol called Diffie-Hellman to generate the secret key.
After the key establishment, the remainder of the session uses the second "secret" key, which is now known by both the client (that generated the key), and the server.
As @MarteenBodewes points out, the mechanism that the client and server use for agreeing on a symmetric key, including validating the authenticity of the server, et al. is more complicated that the overly simplified explanation I gave in the answer above. That is to say, it's not correct to say that the client generates the symmetric key and sends it to the server.
No matter the actual mechanics and how complicated the key exchange is, the main idea is that the client and server agree on symmetric cipher to use, and agree on a secret key to use. The security comes from the strength of the symmetric cipher and on the secrecy of the key.
Problem
I understand the concept of public-key cryptography, and HTTPS makes perfect sense to me in terms of proving identity. What I don't understand is how it makes the user's requests and the site's responses unreadable. The site's public key is, well, public -- so can't anyone record the network traffic, capture the responses, and decode them using the public key available to everyone? They wouldn't be able to execute a man in the middle attack without the private key, but why can't they read the responses the user is getting? And vice versa: the user presumably sends their public key to the server upon the initial connection, so can't someone recording the user's network traffic decrypt all the requests? I suppose what I'm asking is: how can two-way public key cryptography provide anything but proof of ID?