Rotate the inventory service secret key

Rotate the secret key every 90 days to reduce the probability of an attacker compromising the secret key.

1. Stop the inventory service on the primary server. You can use the command puppet resource service pe-orchestration-services ensure=stopped, where the pe-orchestration-services service contains both the orchestrator and inventory services.
2. Stop the Puppet service to ensure that a periodic Puppet run does not accidentally start the inventory service while you are rotating the secret key.
3. Use this command to download the key_rotation.rb script:

   curl https://puppet.com/docs/pe/latest/files/key_rotation.rb -L --output key_rotation.rb

4. Run the key_rotation.rb script on the primary server. You must log in as root or use sudo to run the script with elevated privileges.

   The key_rotation.rb script:

   • Calculates the secret key directory and database URL by reading the inventory service's config file.
   • Generates the new key and writes it to <SECRET_KEY_DIR>/new_key.json.
   • Uses psql to re-encrypt the old data with the new key.
   • Moves the new key to the old key's location (<SECRET_KEY_DIR>/keys.json).

   If the inventory service's database is on a different host than the primary server, you must specify the URL using the DATABASE_URL environment variable. This must be a valid PostgreSQL URL. For example, the following invocation connects to the inventory_service database as the inventory_user with the password inventory_password on host remote_db_host:

   DATABASE_URL=postgres://inventory_user:inventory_password@remote_db_host/inventory_service key_rotation.rb

   If re-encryption fails, you can re-run the script. The script does not generate another new key; instead, it detects the previously-created new key and skips to reattempt re-encryption.

   If moving the new key to old key's location fails, you must manually move the new key to the old key’s location. To do this, you can run:

   mv <SECRET_KEY_DIR>/new_key.json <SECRET_KEY_DIR>/keys.json

   For example:

   mv etc/puppetlabs/orchestration-services/conf.d/secrets/new_key.json etc/puppetlabs/orchestration-services/conf.d/secrets/keys.json

5. Delete the key_rotation.rb script to prevent unintentional secret key rotations.
6. Restart the inventory service on the primary server by running: puppet resource service pe-orchestration-services ensure=running
7. Restart the Puppet service.