Yes, logging in once logs you in to all sites on the network*
There are a couple ways about this.
1) http://wordpress.org/extend/plugins/sidebar-login/
2) I wrote this a while back and posted it just now for ya: http://halfelf.org/hacks/sidebar-login-widget/
(* - Caveat: if you use subdomains and/or mapped domains, logging in will not log you in for all sites, as you are protected from cross-domain cookies for your own good.)