If you prefer editing a file to GUI clicking, you could try the node_export module: export your existing webform as a text file, edit it very carefully, and import it as a new webform.
On a simple 2 contact POC, I removed the components with form_keys starting civicrm_1, then renamed the civicrm_2 form_keys to civicrm_1. In the webform_civicrm contacts array, I removed the first contact. The imported form seems ok.
You might need to do more on a complex form.
Some features depend on the order of contacts (eg relationships are only possible with earlier contacts) so shuffling the order could cause problems.