Karen's answer above certainly showed me what is wrong. It seems that its edits the label and not the name but the check on import is to name and not label. Is this a bug? I am on CiviCRM 5.15.1 at present.
I sorted things out for me by deleting the Payment Method that was causing the problem and then recreating it. This was no problem for me as there were no existing contributions. Might have side effects in a mature site.
Update: deleting and re-inputting won't work for the "Check" Payment method as this is reserved and can't be deleted. So for this one I had to go in and change the database.
It looks like its fixed in 5.17.0 though... see https://lab.civicrm.org/dev/core/issues/1133