Draining
In a few different scenarios, Envoy will attempt to gracefully shed connections. For instance, during server shutdown, existing requests can be discouraged and listeners set to stop accepting, to reduce the number of open connections when the server shuts down. Draining behaviour is defined by the server options in addition to individual listener configs.
Draining occurs at the following times:
The server is being hot restarted.
The server begins the graceful drain sequence via the drain_listeners?graceful admin endpoint.
The server has been manually health check failed via the healthcheck/fail admin endpoint. See the health check filter architecture overview for more information.
Individual listeners are being modified or removed via LDS.
By default, the Envoy server will close listeners immediately on server shutdown. To drain listeners for some duration of time prior to server shutdown, use drain_listeners before shutting down the server. The listeners will be directly stopped without any graceful draining behaviour, and cease accepting new connections immediately.
To add a graceful drain period prior to listeners being closed, use the query
parameter drain_listeners?graceful.
By default, Envoy will discourage requests for some period of time (as
determined by --drain-time-s) but continue accepting new connections
until the drain timeout. The behaviour of request discouraging is determined by
the drain manager.
Note that although draining is a per-listener concept, it must be supported at the network filter
level. Currently the only filters that support graceful draining are
Redis,
Mongo,
Thrift
(if the envoy.reloadable_features.thrift_connection_draining runtime feature is enabled),
and HTTP connection manager.
By default, the HTTP connection manager filter will add “Connection: close” to HTTP1 requests, send HTTP2 GOAWAY, and terminate connections on request completion (after the delayed close period).
Each configured listener has a drain_type setting which controls when draining takes place. The currently supported values are:
- default
Envoy will drain listeners in response to all three cases above (admin health fail, hot restart, and LDS update/remove). This is the default setting.
- modify_only
Envoy will drain listeners only in response to the 2nd and 3rd cases above (hot restart and LDS update/remove). This setting is useful if Envoy is hosting both ingress and egress listeners. It may be desirable to set modify_only on egress listeners so they only drain during modifications while relying on ingress listener draining to perform full server draining when attempting to do a controlled shutdown.