Restore your infrastructure

Use the restore process when you migrate your primary server to a new operating system or to a new host. You can also use the restore process to recover your installation after a system failure.

Before you begin:
  • Carefully review the instructions below before carrying out the steps.

  • You must have created backup files, as described in Back up your infrastructure.

  • You must import the GPG key pair (both the public and private keys) that you used for encryption to your new primary server. The GPG keys are required to decrypt an encrypted backup.

  1. If you are restoring the primary server to a node that is currently hosting an active primary server or Puppet agent, you must first purge the existing installation and install PE. To do this:

    1. On the primary server, run this command as root to uninstall PE:
      /opt/puppetlabs/bin/puppet-enterprise-uninstaller -p -d
      
    2. Ensure that /opt/puppetlabs/ and /etc/puppetlabs/ are no longer present on the system:

      rm -rf /opt/puppetlabs/
      rm -rf /etc/puppetlabs/
      
  2. Install PE on the primary server you want to restore to. You must install the same PE version used to create your backup files.

    1. If you don't have the PE installer script on the machine you want to restore to, download the installer tarball and unpack it by running:

      tar -xf <TARBALL_FILENAME>
    2. Navigate to the directory containing the install script. Normally, the installer script is located in the PE directory that is created when you unpacked the tarball.

    3. To install PE, run as root:

      ./puppet-enterprise-installer
  3. To ensure that the pg_repack extension doesn't run, stop the pe_databases module timers:
    systemctl stop pe_databases-*.timer
  4. On your primary server, run the puppet-backup restore command as root user to restore your PE infrastructure. The default command is:
    puppet-backup restore <BACKUP-FILENAME>

    You must use the following parameter to specify the backup file you want to restore from:

    You can customize your restore by specifying the following optional parameters:

    • --pe-environment=<ENVIRONMENT>: Specify an environment to restore. The default value is production.
    • --scope=<SCOPE_LIST>: Specify the data you want to restore. This is used for Customize scope of backup and restore. The default scope is all. If you only want to restore specific data, specify one or more of: certs, code, config, or puppetdb.
    • If you are restoring for a PE migration or recovering an installation, you must restore all four data sets at once with --scope=all.
    • --force: Specify this parameter if you want to bypass validation checks and ignore warnings.

  5. Restore your secret key backups. These keys are used to encrypt and decrypt sensitive data in the inventory service, orchestrator, and the LDAP service (if enabled).
    • The location of the Inventory service and orchestrator keys directory is: /etc/puppetlabs/orchestration-services/conf.d/secrets/
    • The location of the LDAP service key file is: /etc/puppetlabs/console-services/conf.d/secrets/keys.json
    The puppet-backup restore command does not include secret keys. You must restore this data separately.
  6. Make sure the inventory service's secret key ownership is configured as: chown -R pe-orchestration-services:pe-orchestration-services  /etc/puppetlabs/orchestration-services/conf.d/secrets/
  7. If the LDAP service is enabled, make sure the LDAP service's secret key ownership is configured as: chown pe-console-services:pe-console-services /etc/puppetlabs/console-services/conf.d/secrets/keys.json
  8. Restart pe-orchestration-services and pe-console-services on the primary server:
    1. puppet resource service pe-orchestration-services ensure=stopped
    2. puppet resource service pe-console-services ensure=stopped
    3. puppet resource service pe-orchestration-services ensure=running
    4. puppet resource service pe-console-services ensure=running
  9. Run Puppet to apply changes:
    1. puppet agent -t --no-use_cached_catalog
    2. puppet agent -t --no-use_cached_catalog

    3. You must run the command twice.
  10. If your backup source PE installation includes compilers and you are restoring your primary server on the same operating system that was in use when the backup was created, migrate your compilers to the new primary server. Run the following command on all compilers: puppet agent -t --server_list <RESTORED_DESTINATION_PRIMARY_CERTNAME>
  11.  If your backup source PE installation includes compilers and you are restoring your primary server after changing or upgrading your operating system, you must forget the previous compilers and add new compilers on the same OS major version, platform, and architecture as the restored (destination) primary server. Complete the following steps on the restored primary server:
    1. On the primary server, run the following command for each compiler: puppet node purge <COMPILER_CERTNAME>. If any of the compilers included in your installation were legacy compilers (compilers without the PuppetDB service), you must unpin them from the PE Master node group.
    2. On the primary server, run Puppet to apply the configuration changes: puppet agent -t

    3. Install and configure new compilers.
  12.  If your backup source PE installation includes a PE replica, you must complete the following steps:
    1. Run the following command to generate a new admin token: puppet-access login
    2. On the primary server, run the following command: puppet infrastructure forget <REPLICA_CERTNAME>

    3. Provision and enable a new replica.

  13. If Code Manager was enabled when you created your backup file, complete the following steps on the newly restored primary server:

    1. If necessary, run the following command to generate a new Puppet token for deploying code: puppet-access login
    2. For each code environment you want to deploy, run: puppet code deploy --wait <ENVIRONMENT_NAME>
    3. Alternatively, if you want to deploy all of your PE code environments, you can run the following command: puppet code deploy --wait --all

  14. Run Puppet to apply changes:
    puppet agent -t --no-use_cached_catalog
  15. If you restored PE onto a primary server with a different hostname than the original installation, and you have not configured the dns_alt_names setting in the pe.conf file, you need to redirect your agents to the new primary server. If your agents have a load balancer address in puppet.conf, this step is unnecessary and you can simply make load balancer changes to redirect agent Puppet traffic. If you are not using load balancers, we recommend updating the server_list configuration setting to the new infrastructure using either the puppet_enteprise::profile::agent::server_list class parameter or with the puppet_conf task.
  16. Once your agents are migrated to the new infrastructure, restart pe-orchestration-services on the primary server. This resets all pxp-agent connections.
    1. puppet resource service pe-orchestration-services ensure=stopped
    2. puppet resource service pe-orchestration-services ensure=running

Related information