Some clarification about 'getting smart groups to refresh'
I assume you are using the button 'update smartgroup counts' from the Manage groups screen. This could perhaps be better called 'calculate missing smartgroup counts' (although that wouldn't fit on the button.....) - that button causes the count to be calculated for all groups that the cache has not been built for.
Groups are only recalculated if they have aged out - by default the smart group timeout is 5 minutes. Pretty much whenever you save a contact it attempts to delete the cache for any smart groups that are more than those 5 minutes old (you can set the cache to less than 5 minutes or 0 for more caching). It will then re-fill the cache for that group next time you access it.
If you barely ever edit contacts I guess the cache might not get cleared as often as it 'should' - you can schedule a job (Job.group_cache_flush) to run however often you want - if you do this it makes sense to set the setting 'smart_group_cache_refresh_mode' to 'deterministic' to stop it from also clearing on contact edit (note that this is often good to do for performance reasons)