Is there a way to version the configuration so we can just roll it back if its a bit broken? Or is the only option full filesystem and db backups?
There is the CiviConfigure extension but I'm not sure how current it is.
There is also still bin/csv/export.php which you can use for option values and tags and such.
You can also use the api to retrieve and set various entities, e.g. using the cv tool.
For CiviCase, one of the original arguments for using external files was for versioning and managing dev/production. You can still use xml files for its config.