If you're on Drupal you can do this via views (where you have full control over which field types you pull in).

Here's a good example:
https://civicrm.stackexchange.com/questions/224/views-proximity-search-on-latitude-longitude