Dynamic configuration (filesystem)

This example walks through configuring Envoy using filesystem-based dynamic configuration.

It demonstrates how configuration provided to Envoy dynamically can be updated without restarting the server.

Step 1: Start the proxy container

Change directory to examples/dynamic-config-fs in the Envoy repository.

Build and start the containers.

This should also start two upstream HTTP echo servers, service1 and service2.

$ pwd
envoy/examples/dynamic-config-fs
$ docker compose pull
$ docker compose up --build -d
$ docker compose ps

       Name                            Command                State                     Ports
------------------------------------------------------------------------------------------------------------------------
dynamic-config-fs_proxy_1      /docker-entrypoint.sh /usr ... Up      0.0.0.0:10000->10000/tcp, 0.0.0.0:19000->19000/tcp
dynamic-config-fs_service1_1   /bin/echo-server               Up      8080/tcp
dynamic-config-fs_service2_1   /bin/echo-server               Up      8080/tcp

Step 2: Check web response

You should be able to make a request to port 10000, which will be served by service1.

$ curl -s http://localhost:10000
Request served by service1

HTTP/2.0 GET /

Host: localhost:10000
User-Agent: curl/7.72.0
Accept: */*
X-Forwarded-Proto: http
X-Request-Id: 6672902d-56ca-456c-be6a-992a603cab9a
X-Envoy-Expected-Rq-Timeout-Ms: 15000

Step 3: Dump Envoy’s `dynamic_active_clusters` config

If you now dump the proxy’s dynamic_active_clusters configuration, you should see it is configured with the example_proxy_cluster pointing to service1.

$ curl -s http://localhost:19000/config_dump | jq -r '.configs[1].dynamic_active_clusters'

[
  {
    "cluster": {
      "@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
      "name": "example_proxy_cluster",
      "type": "LOGICAL_DNS",
      "connect_timeout": "5s",
      "dns_lookup_family": "V4_ONLY",
      "load_assignment": {
        "cluster_name": "example_proxy_cluster",
        "endpoints": [
          {
            "lb_endpoints": [
              {
                "endpoint": {
                  "address": {
                    "socket_address": {
                      "address": "service1",
                      "port_value": 8080
                    }
                  }
                }
              }
            ]
          }
        ]
      }
    },
    "last_updated": "2020-10-25T20:37:05.838Z"
  }
]

Step 4: Replace `cds.yaml` inside the container to update upstream cluster

The example setup provides Envoy with two dynamic configuration files:

configs/cds.yaml to provide a Cluster discovery service (CDS).
configs/lds.yaml to provide a Listener discovery service (LDS).

Edit cds.yaml inside the container and change the cluster address from service1 to service2:

    cluster_name: example_proxy_cluster
    endpoints:
    - lb_endpoints:
      - endpoint:
          address:
            socket_address:
              address: service1
              port_value: 8080

You can do this using sed inside the container:

docker compose exec -T proxy sed -i s/service1/service2/ /var/lib/envoy/cds.yaml

Note

The above example uses sed -i, which works as an inplace edit as sed does copy, edit and move in order to do this.

Step 5: Check Envoy uses updated configuration

Checking the web response again, the request should now be handled by service2:

$ curl http://localhost:10000 | grep "served by"
Request served by service2

Dumping the dynamic_active_clusters, the example_proxy_cluster should now be configured to proxy to service2:

$ curl -s http://localhost:19000/config_dump | jq -r '.configs[1].dynamic_active_clusters'

[
  {
    "cluster": {
      "@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
      "name": "example_proxy_cluster",
      "type": "LOGICAL_DNS",
      "connect_timeout": "5s",
      "dns_lookup_family": "V4_ONLY",
      "load_assignment": {
        "cluster_name": "example_proxy_cluster",
        "endpoints": [
          {
            "lb_endpoints": [
              {
                "endpoint": {
                  "address": {
                    "socket_address": {
                      "address": "service2",
                      "port_value": 8080
                    }
                  }
                }
              }
            ]
          }
        ]
      }
    },
    "last_updated": "2020-10-25T20:37:05.838Z"
  }
]