There is maybe a simpler path but this worked for me. Documenting this here since I don't know where else to put it.
Start creating a new D8+civi site by using the installer at https://gitlab.com/roundearth/drupal-civicrm-project.
When doing the D8 install part, just install a minimal site and don't further configure it - just what's needed to get D8 installed.
Follow the standard drupal instructions to upgrade. https://www.drupal.org/docs/8/upgrade/upgrading-from-drupal-6-or-7-to-drupal-8. The short version is enable the Migrate UI module on the new site and then go to /upgrade.
Do not enable civicrm yet.
From your old site, export the civicrm_xxx tables.
There is probably a better way to do this step but it worked for me to manually update any urls and paths in the export file (taking care with php-serialized strings) and then import the sql file into the new database. Note that any references to sites/all/modules/civicrm are now under vendor/civicrm/civicrm-core. URLs for sites/default/files/... can stay as is, but note that filepaths will now be web/sites/default/files. Also userFrameworkResourceURL is now libraries/civicrm.
Copy civicrm.settings.php to the new site under web/sites/default. Edit the file:
Change CIVICRM_UF from "Drupal" to "Drupal8". Change the mysql connection strings. Change paths and urls as above. Again note civicrm_root is now under vendor/civicrm/civicrm-core, and you need to put "web/" in front of filepaths to "sites/XXX".
Enable the civicrm module.
Clear caches at admin/config/development/performance and also the civicrm caches at civicrm/admin/setting/updateConfigBackend?reset=1. I also ran /civicrm/menu/rebuild?reset=1 but I'm not sure it's necessary here.
The UI will look all messed up because it was a minimal drupal install but you can change the theme and set things as you normally would.
You may also need to set these settings in civicrm.settings.php:
$civicrm_setting['URL Preferences']['userFrameworkResourceURL'] = CIVICRM_UF_BASEURL . '/libraries/civicrm/'; $civicrm_paths['civicrm.root']['url'] = CIVICRM_UF_BASEURL . '/libraries/civicrm/'; $civicrm_setting['domain']['userFrameworkResourceURL'] = CIVICRM_UF_BASEURL . '/libraries/civicrm/'; $civicrm_paths['cms.root']['path'] = '/full/path/to/web'; // the "web" folder at the root of the install