Once in google buzz you have a prompt with the following "You're already set up to follow the people you email and chat with." So by simply emailing someone you will now be "following" them and they will have access to you contact list.
One of the main issues being discussed about Google buzz is the automatic opt-in, in a sense forcing users into using the service. Then publicly disclosing your email and contact list, leaving your email open to spammers. All and all a bad move from Google. They seem to be taking a quick and serious response to the issues, with a couple fixes being brought out already.
However in the mean time google is asking for feedback, and you can give yours here: http://mail.google.com/support/bin/request.py?contact_type=buzz
In other news our submission wasn't accepted for the local security summit, yet the talk has been internationally accepted. We can understand and wouldn't want to side track the vendor talks and actually get some technical talks in there anyway. ;)
UPDATE: Well, Google has listened to everyones feedback and already fixed a number of the issues. For more info please read here.