One additional point to this-

If a Group is created with both Access Control and Mailing list checked.
This group will not be available to select if the logged in user doesn't have the required permission.

If the non-permitted user has the ACL created for it to "view Groups" or related, this group will be listed in CiviMail. (introduced in 4.6.7 https://issues.civicrm.org/jira/browse/CRM-13644)