I've been getting complaints from my invitees that there's no way to indicate that they won't be attending. Is there someway to activate or build this option?
You can create a custom "Participant Status" at civicrm/admin/participant_status (via the menu at Administer > CiviEvent > Participant Statuses).
I don't think there is a default way to expose the participant status on a core event registration form (they are typically for admins in the backend only), so you may consider using a separate workflow with a tag or group for those that are not attending, as noted by @Nicholai below. You might even be able to use Rules to automatically change their status to the new one you created based on the tag/group they are in.
However, if you are using Drupal with Webform-CiviCRM for the registration, it would be a nice clean solution to expose this new participant status on the form (allowing "Registered" or "Not Attending" as choices), and then use Webform's Conditional functionality to show or hide event registration fields as appropriate. This way they can indicate they are not attending without having to see/fill out any custom registration fields on the form.
This seems an inventive workaround but would imply the user who doesn't want to attend the event still has to fill in all required fields of the registration form (most of those very specifically and only relevant for people who will attend).
The main use case for declining an 'invitation' would be in case someone is invited with a link to the registration form and a checksum token. A simple button/link to inform the event host with either a maybe, a no or a registration (a yes) would definitely mature the event registration of CiviCRM! A special profile with fields for people who choose maybe/no would preferably then be selectable from the even settings page.
Consider keeping the invitation and the registration as different concepts in your workflow, as invitees aren't really event participants if they don't attend. When you email invitees, link them to a Profile page containing options for two groups (I will attend / I will not attend). They can check the desired group, and you can track and contact them more easily.
You could maybe even play with the CiviMail checksum links to have your options be direct links within the email, which would add the contact to the appropriate group in one click (depending on whether you use the double opt-in feature of groups).