Convert a version 3 hiera.yaml to version 5

Hiera 5 supports three versions of the hiera.yaml file: version 3, version 4, and version 5. If you've been using Hiera 3, your existing configuration is a version 3 hiera.yaml file at the global layer.

There are two migration tasks that involve translating a version 3 config to a version 5:

  • Creating new v5 hiera.yaml files for environments.

  • Updating your global configuration to support Hiera 5 backends.

These are essentially the same process, although the global hierarchy has a few special capabilities.

Consider this example hiera.yaml version 3 file: 

:backends:
  - mongodb
  - eyaml
  - yaml
:yaml:
  :datadir: "/etc/puppetlabs/code/environments/%{environment}/hieradata"
:mongodb:
  :connections:
    :dbname: hdata
    :collection: config
    :host: localhost
:eyaml:
  :datadir: "/etc/puppetlabs/code/environments/%{environment}/hieradata"
  :pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
  :pkcs7_public_key:  /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem
:hierarchy:
  - "nodes/%{trusted.certname}"
  - "location/%{facts.whereami}/%{facts.group}"
  - "groups/%{facts.group}"
  - "os/%{facts.os.family}"
  - "common"
:logger: console
:merge_behavior: native
:deep_merge_options: {}
To convert this version 3 file to version 5:

  1. Use strings instead of symbols for keys.

    Hiera 3 required you to use Ruby symbols as keys. Symbols are short strings that start with a colon, for example, :hierarchy. The version 5 config format lets you use regular strings as keys, although symbols won’t (yet) cause errors. You can remove the leading colons on keys.

  2. Remove settings that aren’t used anymore. In this example, remove everything except the :hierarchy setting:
    1. Delete the following settings completely, which are no longer needed:

      • :logger

      • :merge_behavior

      • :deep_merge_options

      For information on how Hiera 5 supports deep hash merging, see Merging data from multiple sources.

    2. Delete the following settings, but paste them into a temporary file for later reference:

      • :backends

      • Any backend-specific setting sections, like :yaml or :mongodb

  3. Add a version key, with a value of 5: 

    version: 5
hierarchy:
  # ...

  4. Set a default backend and data directory.

    If you use one backend for the majority of your data, for example YAML or JSON, set a defaults key, with values for datadir and one of the backend keys.

    The names of the backends have changed for Hiera 5, and the backend setting itself has been split into three settings:

    Hiera 3 backendHiera 5 backend setting
    yamldata_hash: yaml_data
    jsondata_hash: json_data
    eyamllookup_key: eyaml_lookup_key

  5. Translate the hierarchy.

    The version 5 and version 3 hierarchies work differently:

    • In version 3, hierarchy levels don’t have a backend assigned to them, and Hiera loops through the entire hierarchy for each backend.

    • In version 5, each hierarchy level has one designated backend, as well as its own independent configuration for that backend.

    Consult the previous values for the :backends key and any backend-specific settings.

    In the example above, we used yaml, eyaml, and mongodb backends. Your business only uses Mongo for per-node data, and uses eyaml for per-group data. The rest of the hierarchy is irrelevant to these backends. You need one Mongo level and one eyaml level, but still want all five levels in YAML. This means Hiera consults multiple backends for per-node and per-group data. You want the YAML version of per-node data to be authoritative, so put it before the Mongo version. The eyaml data does not overlap with the unencrypted per-group data, so it doesn’t matter where you put it. Put it before the YAML levels. When you translate your hierarchy, you have to make the same kinds of investigations and decisions.

  6. Remove hierarchy levels that use calling_module, calling_class, and calling_class_path, which were allowed pseudo-variables in Hiera 3. Anything you were doing with these variables is better accomplished by using the module data layer, or by using the glob pattern (if the reason for using them was to enable splitting up data into multiple files, and not knowing in advance what they names of those would be)

    Hiera.yaml version 5 does not support these. Remove hierarchy levels that interpolate them.

  7. Translate built-in backends to the version 5 config, where the hierarchy is written as an array of hashes. For hierarchy levels that use the built-in backends, for example YAML and JSON, use the data_hash key to set the backend. See Configuring a hierarchy level in the hiera.yaml v5 reference for more information.

    Set the following keys:

    • name - A human-readable name.

    • path or paths - The path you used in your version 3 hiera.yaml hierarchy, but with a file extension appended.

    • data_hash - The backend to use yaml_data for YAML, json_data for JSON.

    • datadir - The data directory. In version 5, it’s relative to the hiera.yaml file’s directory.

    If you have set default values for data_hash and datadir, you can omit them.

    version: 5
defaults:
  datadir: data
  data_hash: yaml_data
hierarchy:
  - name: "Per-node data (yaml version)"
    path: "nodes/%{trusted.certname}.yaml" # Add file extension.
    # Omitting datadir and data_hash to use defaults.

  - name: "Other YAML hierarchy levels"
    paths: # Can specify an array of paths instead of one.
      - "location/%{facts.whereami}/%{facts.group}.yaml"
      - "groups/%{facts.group}.yaml"
      - "os/%{facts.os.family}.yaml"
      - "common.yaml"

  8. Translate hiera-eyaml backends, which work in a similar way to the other built-in backends.

    The differences are:

    • The hiera-eyaml gem has to be installed, and you need a different backend setting. Instead of data_hash: yaml, use lookup_key: eyaml_lookup_key. Each hierarchy level needs an options key with paths to the public and private keys. You cannot set a global default for this.

      - name: "Per-group secrets"
   path: "groups/%{facts.group}.eyaml"
   lookup_key: eyaml_lookup_key
   options:
     pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
     pkcs7_public_key:  /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem

  9. Translate custom Hiera 3 backends.

    Check to see if the backend’s author has published a Hiera 5 update for it. If so, use that; see its documentation for details on how to configure hierarchy levels for it.

    If there is no update, use the version 3 backend in a version 5 hierarchy at the global layer — it does not work in the environment layer. Find a Hiera 5 compatible replacement, or write Hiera 5 backends yourself.

    For details on how to configure a legacy backend, see Configuring a hierarchy level (legacy Hiera 3 backends) in the hiera.yaml (version 5) reference.

    When configuring a legacy backend, use the previous value for its backend-specific settings. In the example, the version 3 config had the following settings for MongoDB:

    :mongodb:
  :connections:
    :dbname: hdata
    :collection: config
    :host: localhost
    So, write the following for a per-node MongoDB hierarchy level:
    - name: "Per-node data (MongoDB version)"
   path: "nodes/%{trusted.certname}"      # No file extension
   hiera3_backend: mongodb
   options:    # Use old backend-specific options, changing keys to plain strings
     connections:
       dbname: hdata
       collection: config
       host: localhost

Results

After following these steps, you’ve translated the example configuration into the following v5 config:

version: 5
defaults:
  datadir: data
  data_hash: yaml_data
hierarchy:
  - name: "Per-node data (yaml version)"
    path: "nodes/%{trusted.certname}.yaml" # Add file extension
    # Omitting datadir and data_hash to use defaults.

  - name: "Per-node data (MongoDB version)"
    path: "nodes/%{trusted.certname}"      # No file extension
    hiera3_backend: mongodb
    options:    # Use old backend-specific options, changing keys to plain strings
      connections:
        dbname: hdata
        collection: config
        host: localhost

  - name: "Per-group secrets"
    path: "groups/%{facts.group}.eyaml"
    lookup_key: eyaml_lookup_key
    options:
      pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
      pkcs7_public_key:  /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem

  - name: "Other YAML hierarchy levels"
    paths: # Can specify an array of paths instead of a single one.
      - "location/%{facts.whereami}/%{facts.group}.yaml"
      - "groups/%{facts.group}.yaml"
      - "os/%{facts.os.family}.yaml"
      - "common.yaml"

Related information