How can Alice and Bob agree on a secret, without having any prior knowledge of each other, over a public medium (such as the Internet or a crowded room) and with Eve (an eavesdropper) listening in on the conversation?
This is the exact problem that the Diffie-Hellman Algorithm tries to solve. The trick is that you are not sharing information during the key exchange, but rather you're creating a key, together.
Why is it Useful?
Since with the Diffie-Hellman Key Exchange you are just creating a key, it is not suitable for sharing information, however using this technique you can generate an encryption key and then start encrypting the traffic with that key.
Perfect Forward Secrecy
In cryptography, perfect forward secrecy is a property of secure communication protocols in which compromise of long-term keys does not compromise past session keys. Forward secrecy protects past sessions against future compromises of secret keys or passwords. - Wikipedia: Forward Secrecy
Using the Diffie-Hellman Key Exchange, even if the traffic is recorded and later analyzed, there's absolutely no way to figure out what the key was, even though the key exchange conversation would have been recorded. Nobody analyzing the traffic at a later date can break in because the key was never saved, never transmitted, and never made visible anywhere.
Visualizing the Diffie-Hellman Key Exchange
The best way to visualize the Diffie-Hellman Key Exchange is with the famous paint color mixing method.
This diagram from the Wikipedia Article explains the process in the video:
Limitations
The Diffie-Hellman Key Exchange does not provide authentication of any kind. It only allow 2 anonymous parties to share a common secret. But for all Alice knows, she could be shaking hands with Eve instead of Bob. This is why we need at least one party to be authenticated. This is why SSL (HTTPS) uses the Public Key Infrastructure (PKI) to authenticate. The Handshake/Negotiation of an SSL Connection is expensive in terms of overhead; using the Diffie-Hellman Key Exchange, both parties can securely negotiate a symmetric key for encryption/decryption which is much more efficient.