I want to add a new variable to the variables.php file. Should I add it at the bottom of the file, or where it logically sits in the order of the variables, ie under the variable that will be above it on the form?
I assume you mean CRM/Core/Config/Variables.php
If you're adding something to that file that you're going to submit as a pull request or patch back to core, put your changes where they fit most logically. It'll make more sense to others in the future.
If you're doing a site-specific customization and want to edit the file, don't. You'll have trouble when it comes time to upgrade. Instead, create an extension.
Neither place is appropriate for my use. CRM/Core/Config/Variables.php is 'deprecated'. (I didn't know that when I asked my question.)
New variables for settings should be added to the appropriate file in the settings folder.
See http://wiki.civicrm.org/confluence/display/CRMDOC/Settings+Reference for more details.
If you want to make local changes to core files there's also the Custom PHP directory. This won't get overwritten on upgrade but you'll need to review your patches to make sure the underlying core file hasn't significantly changed.