Using load balancers with compilers

When using more than one compiler, a load balancer can help distribute the load between the compilers and provide a level of redundancy.

Specifics on how to configure a load balancer infrastructure falls outside the scope of this document, but examples of how to leverage haproxy for this purpose can be found on the HAproxy module Forge page.

Calculating load balancing

For load balancing between the Puppet agent and the Puppet primary server, implement a load balancing algorithm that distributes traffic among compilers based on the number of open connections. Traffic is directed to the compiler with the smallest number of open connections. This strategy is known as “balancing by least connections.”

For load balancing between PCP brokers and PXP agents, a different method is used. PXP agents establish TCP connections to PCP brokers over port 8142. PCP brokers are built on web sockets and require many persistent connections. PCP brokers depend on maintaining connectivity to the Puppet orchestrator, but if the brokers become disconnected from the orchestrator, the brokers can fail at the HTTP level while still accepting TCP connections. Follow these guidelines:

  • If you are using HTTP health checks, use a "least connections" algorithm to distribute load evenly.
  • If you are not using HTTP health checks, use a round robin or random load balancing algorithm to avoid directing all traffic to an unhealthy PCP broker. You can check connections for possible errors by using the /status/v1/simple endpoint.

Using health checks

The Puppet REST API exposes a status endpoint that can be used for load balancer health checks to ensure that unhealthy hosts don’t receive agent requests.

To check the health of your hosts, issue HTTP GET requests to the following endpoints:

  • For Puppet agent traffic on port 8140, use https://<hostname>:8140/status/v1/simple/server. This endpoint checks the health of the pe-puppetserver service and returns an HTTP 200 status when the server is fully operational. It does not check the health of other services such as the broker-service.
  • For pxp-agent traffic on port 8142, use https://<hostname>:8140/status/v1/simple/broker-service. This endpoint checks the health of the broker-service and returns an HTTP 200 status code when the service is fully operational.

If your load balancer doesn't support HTTP health checks, you can use a TCP connection tests on port 8140 to check whether a host is listening. This test is useful for verifying that the port is open, but does not confirm whether the host is running a Puppet service.

Related information