Migrate 4.x data to 5.x

The Continuous Delivery 5.x series uses Bolt for installation and management rather than Puppet Application Manager, so you must migrate your 4.x series data to the new installation.

What is migrated?

The migration task copies and migrates to 5.x:

• The database, which contains all your Continuous Delivery 4.x installation's information, users, integrations, history, and artifacts (except job commands and job logs).
• The object store, which contains all of your 4.x installation's job commands and job logs.
• Any of these configuration settings, if in use on 4.x:

  CD4PE_REPO_CACHE_RETRIEVAL_TIMEOUT_MINUTES
  CD4PE_LDAP_GROUP_SEARCH_SIZE_LIMIT
  CD4PE_ENABLE_REPO_CACHING
  CD4PE_HTTP_CONNECTION_TIMEOUT_SEC
  CD4PE_HTTP_READ_TIMEOUT_SEC
  CD4PE_HTTP_WRITE_TIMEOUT_SEC
  CD4PE_HTTP_REQUEST_TIMEOUT_SEC
  CD4PE_INCLUDE_GIT_HISTORY_FOR_CD4PE_JOBS
  CD4PE_JOB_GLOBAL_TIMEOUT_MINUTES
  CD4PE_JOB_HTTP_READ_TIMEOUT_MINUTES

The migration task does not copy or move to 5.x

• The 4.x root user password, you are prompted to create a new root user password when installing 5.x.
• The JVM_ARGS configuration setting, if in use on 4.x.
• These backup tables created for you by Continuous Delivery during the 4.x series:
  • app-pipelines-3-0-backup.pfi, which was created during the upgrade from 3.x to 4.x
  • pe-ia-node-results-cve-2020-7944-backup.pfi and pe-ia-resource-results-cve-2020-7944-backup.pfi, which were created as part of the CVE 2020-7944 remediation in version 3.4.0

Migration cautions

You must have upgraded Continuous Delivery to version 4.26.0 or later before you can migrate to version 5.x.

A super user or the root user must perform this process.

Migrations from installations with external databases are not supported.

Run the migration task

The migration task uses the cd4peadm::install_from_v4 plan to migrate data from existing 4.x PAM-based installation to a new 5.x install.

Once you've completed the above preparations, you're ready to run the migration task:

1. Install Bolt version 3.27.4 or later on a Linux jumphost. This can be the intended Continuous Delivery 5.x host, or any other system.
2. Create the Continuous Delivery Bolt project and switch to that directory.

   mkdir cd4pe-bolt-project 
   cd cd4pe-bolt-project
   bolt project init cd4pe_bolt_project

3. Edit the bolt-project.yaml file’s modules and module-install sections. Replace the version_requirement with the version you wish to install, for example you would use 5.7.0 for the latest version. Add your Forge API token in authorization_token.

   # bolt-project.yaml
   modules:
     - name: puppetlabs/cd4peadm
       version_requirement: <version to install>
   
   module-install:
     forge:
        authorization_token: 'Bearer <your API token>'
        baseurl: https://forgeapi.puppet.com
   

4. Optional: Enable log append mode for Bolt logging to simplify debugging efforts in the case of problems. Add the following to bolt-project.yaml:

   # bolt-project.yaml
   log:
     bolt-debug.log:
       append: true
       level: debug			

   Enabling logging causes bolt-debug.log to grow over time. You can safely delete this file to free space as needed.
5. Install the cd4peadm module using the command: bolt module install.
6. Create an inventory.yaml file with two targets: One for the host on which Continuous Delivery 5.x is to be installed and another with kubectl access to your Continuous Delivery 4.x install. If your 4.x instance is not installed in the default namespace, add a var to the target called kubernetes_namespace and set it to the namespace you are using. For example:

   ---
   targets:
   - name: cd4pe-5-node
     uri: <IP address or DNS name>
     config:
       transport: ssh
       ssh:
         host-key-check: false
         native-ssh: true
         ssh-command: <Path to SSH command>
         user: root
   - name: cd4pe-4-node
     uri: <IP address or DNS name>
     config:
       transport: ssh
       ssh:
         host-key-check: false
         native-ssh: true
         ssh-command: <Path to SSH command>
         user: root
       vars:
         kubernetes_namespace: 'cd4pe'

   If you are installing on a localhost, use the following connection information instead:

   - name: local-install-host
     uri: localhost
     config:
       transport: local
       local:
         bundled-ruby: false
   

   If you need to change your resolvable hostname for some reason, see Update your resolvable hostname.
7. Optional: If you have a license file, create a directory named files and copy the license file to files/license.lic. A license file is required to access the premium features of Continuous Delivery.
8. Run the migration plan using: bolt plan run cd4peadm::install_from_v4. The plan extracts your 4.x settings via kubectl, installs Continuous Delivery 5.x with those settings, and migrates data from the 4.x database to the new 5.x database.
   Migrating the database data creates a database dump in /tmp on the host that is running kubectl. Please ensure that there is sufficient space for that data on the host. You can check the size of your database using the command:

   kubectl exec -it postgres-0 -- du -skh /var/lib/postgresql/data/pgdata

Refresh ADO OAuth integration

Migrating Continuous Delivery from 4.x to 5.x typically requires installing the app on a new node. If this results in the URL of the application changing, OAuth integrations with ADO Cloud need to be recreated, both within Continuous Delivery and in ADO. This does not affect any control repos or modules you have set up; it refreshes the credentials and supplies the new application URL to ADO.

To recreate the integration, take the following steps:

1. In Continuous Delivery’s root console, go to Settings → Integrations and delete the ADO integration.

2. In any workspace that has been integrated with ADO, go to Settings → Source Control and delete any ADO integrations.

3. Follow the instructions to integrate Azure Devops to create a new application and register it with Continuous Delivery, using the new authorization callback URL from the root console of your migrated instance.

4. Re-add the workspace integrations.

5. If you are not using your old ADO OAuth app (for instance, if you have shut down your 4.x instance that it was previously integrated with), it is safe to delete it now from the ADO console.

Test the new 5.x installation

You must thoroughly test the 5.x installation to make sure the data was properly migrated and all functionality is working as expected.

1. Test the data migration to make sure your 4.x installation's data is present in your 5.x installation. For each workspace in your 5.x installation:
   1. Review event logs for all control repos and modules.
   2. Check that all jobs and hardware connections are present.
   3. Check that all users and user groups are present.
2. If any data is missing or looks incorrect, rerun the migration task.
3. Test manual pipeline runs to make sure your 5.x installation is functioning properly. For each workspace in your 5.x installation, trigger a manual pipeline run for all control repos and modules. To do this, select Trigger pipeline from the Manual actions menu above each pipeline.

Update webhooks

As the final step in the 4.x to 5.x data migration process, update the webhooks that connect Continuous Delivery to your source control system.

1. In the Continuous Delivery 4.x root console, navigate to Config > Optional Configuration, select View options for using a proxy or external load balancer, and copy the backend service endpoint. Use the default value if a custom value has not been defined.
2. In the Continuous Delivery 5.x root console, click Settings > Webhooks, and enter the backend service endpoint you copied from the 4.x root console.
3. Click Update webhooks.

Post-migration guidance and troubleshooting

After migrating Continuous Delivery data from 4.x to 5.x, thouroughly testing your 5.x installation, and updating webhooks, you're ready to begin using 5.x with your team. However, do not immediately decommission your 4.x installation. You may need to roll back to 4.x if issues arise.

Maintaining your dormant 4.x installation for an extended test period allows you to roll back to 4.x and prevent production environment disruptions if an issue arises with your 5.x migration.

Rolling back to 4.x

If necessary, you can temporarily roll back to your 4.x installation by switching the active webhooks from the 5.x installation to the 4.x installation and generating new access tokens for all PE instances connected with the 4.x installation.

1. In your Continuous Delivery 4.x installation's web UI, navigate to a control repo's pipeline and click Manage pipelines.
2. Select Manage in the web UI and locate the Automation webhook section where the webhook (including token) for your control repo is shown.
3. Copy the webhook and navigate to the corresponding repository in your source control system. Add the 4.x webhook to the repository and remove the 5.x webhook.
4. Repeat this process for each control repo and module repo in your 4.x installation.
5. Generate a new access token for each PE instance connected to the 4.x installation. In the Continuous Delivery web UI, navigate to Settings > Puppet Enterprise and click More actions > Edit integration.
6. Select Basic authorization or API token and enter the required information:
   • For Basic authorization, enter the username and password for your "Continuous Delivery" user. Continuous Delivery uses this information to generate an API token for you. The username and password are not saved.
   • For API token, generate a PE access token for your "Continuous Delivery" user using puppet-access or the RBAC API v1 service, and paste it into the API token field. For information about generating PE access tokens and the RBAC API v1 service, see Token-based authentication in the PE documentation.
     To avoid unintended service interruptions, create access tokens with a multi-year lifespans.

Once the 4.x webhooks and new PE access tokens are in place, you can resume using the 4.x installation.

When you're ready to return to 5.x, repeat the migration process, starting with Run the migration task.