.. _version_history_1.32.0: 1.32.0 (October 15, 2024) ========================== Incompatible behavior changes ----------------------------- *Changes that are expected to cause an incompatibility if applicable; deployment changes are likely required* * **eds**: Enabling caching of EDS assignments when used with ADS by default (introduced in Envoy v1.28). Prior to this change, Envoy required that EDS assignments were sent after an EDS cluster was updated. If no EDS assignment was received for the cluster, it ended up with an empty assignment. Following this change, after a cluster update, Envoy waits for an EDS assignment until :ref:`initial_fetch_timeout ` times out, and will then apply the cached assignment and finish updating the warmed cluster. This change can be temporarily disabled by setting the runtime flag ``envoy.restart_features.use_eds_cache_for_ads`` to ``false``. * **golang**: Change ``OnLogDownstreamStart``, ``OnLogDownstreamPeriodic`` and ``OnLog`` methods so that user can get the request/response's headers and trailers when producing access log. * **http**: Access log handlers added by filters will now be evaluated before access log handlers that configured in the :ref:`access_log configuration `. This change can be disabled by setting the runtime guard flag ``envoy.reloadable_features.filter_access_loggers_first`` to ``false``. * **http**: Added HTTP/1-safe option for :ref:`max_connection_duration ` in HttpConnectionManager. When enabled, ``max_connection_duration`` will only drain downstream HTTP/1 connections by adding the ``Connection: close`` response header; it will never cause the ``HttpConnectionManager`` to close the connection itself. Defaults to off (allows "unsafe" connection closing) but is configurable via :ref:`http1_safe_max_connection_duration `. * **http**: Allow HTTP/2 and HTTP/3 upstream servers to half close the stream before the downstream. This enables bidirectional gRPC streams where server completes streaming before the client. Behavior of HTTP/1 or TCP proxy upstream servers is unchanged and the stream is reset if the upstream server completes response before the downstream. The stream is also reset if the upstream server responds with an error status before the downstream. This behavior is disabled by default and can be enabled by setting the ``envoy.reloadable_features.allow_multiplexed_upstream_half_close`` runtime key to true. * **http**: The default configuration of Envoy will continue to trust internal addresses while in the future it will not trust them by default. If you have tooling such as probes on your private network which need to be treated as trusted (e.g. changing arbitrary ``x-envoy`` headers) please explictily include those addresses or CIDR ranges into :ref:`internal_address_config ` See the config examples from the above ``internal_address_config`` link. You can turn up or test the upcoming internal address defaults by setting runtime guard ``envoy.reloadable_features.explicit_internal_address_config`` to ``true``. * **monitoring**: Removed runtime feature flag ``envoy.restart_features.enable_execution_context``. The execution context feature can instead be enabled by setting compile option ``--define=execution_context=enabled``. * **stats scoped_rds**: Added new tag extraction so that scoped rds stats have their ``scope_route_config_name`` and stat prefix extracted. * **tracing**: Removed support for (long deprecated) opentracing. See `issue 27401 `_ for details. Minor behavior changes ---------------------- *Changes that may cause incompatibilities for some users, but should not for most* * **access_log**: Sanitize SNI for potential log injection. The invalid character will be replaced by ``_`` with an ``invalid:`` marker. This behavioral change can be temporarily reverted by setting runtime guard ``envoy.reloadable_features.sanitize_sni_in_access_log`` to ``false``. * **aws**: AWS request signing common code now uses the HTTP async client by default, moving curl to the deprecation path. This behavior change can be reverted by setting the ``envoy_reloadable_features_use_http_client_to_fetch_aws_credentials`` runtime flag to ``false``. * **command line options**: :option:`--enable-fine-grain-logging` and :option:`--component-log-level` were incompatible in that one would make the other ineffective. Setting both options at once is now an error, to reduce potential confusion. * **conn_handler**: Enhanced listener filter chain execution to handle the case that listener filter has ``maxReadBytes()`` of 0, but may return ``StopIteration`` in ``onAccept`` to wait for asynchronous callback. * **ext_proc**: Added support for :ref:`send_body_without_waiting_for_header_response `. * **http**: Modified the authority header value validator to allow the same characters as oghttp2 plus the ``@`` character. This is compliant with nghttp2, and supports the HTTP/1 use-cases that allow ``user-info@`` as part of the authority. This behavior can be reverted by setting the runtime guard ``envoy.reloadable_features.internal_authority_header_validator`` to ``false``. * **http2**: Changed the default value of ``envoy.reloadable_features.http2_use_oghttp2`` to ``false``. This changes the codec used for HTTP/2 requests and responses to address to address stability concerns. This behavior can be reverted by setting the feature to ``true``. * **http3**: HTTP/3 ``alt-svc`` headers will now be respected from IP-address-based hostnames. This change is guarded by runtime guard ``envoy.reloadable_features.allow_alt_svc_for_ips``. * **http3**: The ``ACCEPT_UNTRUSTED`` option now works more consistently for HTTP/3 requests. This change is guarded by ``envoy.reloadable_features.extend_h3_accept_untrusted``. * **http_11_proxy**: Made the inner ``transport_socket`` field optional in the proto configuration. * **lua**: When Lua scripts execute ``httpCall``, backpressure is now exercised when receiving body from downstream client. This behavior can be reverted by setting the runtime guard ``envoy.reloadable_features.lua_flow_control_while_http_call`` to ``false``. * **quic**: UDP client connection sockets are now connected before use and sockets will only bind if the local address is specified. This behavior change can be reverted by setting the ``envoy_reloadable_features_quic_connect_client_udp_sockets`` runtime flag to ``false``. * **sni**: When computing SNI and SAN value for the auto-sni and auto-san verification feature, route host manipulations are now taken into account. This behavior can be reverted by setting the runtime guard ``envoy_reloadable_features_use_route_host_mutation_for_auto_sni_san`` to ``false``. * **tcp**: Added support for :ref:`connection_pool_per_downstream_connection ` flag in tcp connection pool. * **tracers**: Set status code based on gRPC status code for OpenTelemetry tracers (previously unset). * **udp**: Envoy now sets the Don't Fragment (``DF``) flag bit on IP packet header on UDP listener sockets and QUIC upstream connection sockets. This behavior can be reverted by setting ``envoy.reloadable_features.udp_set_do_not_fragment`` to ``false``. * **xds-failover**: Add the ability to stick with either the primary or the failover xDS sources once Envoy connects to one of them. To use the prior behavior, set the runtime flag ``envoy.reloadable_features.xds_failover_to_primary_enabled`` to ``false``. Bug fixes --------- *Changes expected to improve the state of the world and are unlikely to have negative effects* * **c-ares**: Applying a C-ares patch to fix DNS resoultion by the Google gRPC library. * **dispatcher**: Update approximate now after polling instead of before polling. This is only used by QUIC. The behavior can be reverted by setting ``envoy.restart_features.fix_dispatcher_approximate_now`` to ``false``. * **dns**: The DNS filter no longer returns ``FORMERR`` if a message has an ``ID`` of 0. * **dynamic_forward_proxy**: Fixed a bug where DFP sub-cluster gets removed due to CDS update and doesn't gets recreated. This behavior can be reverted by setting the runtime guard ``envoy.reloadable_features.avoid_dfp_cluster_removal_on_cds_update`` to ``false``. * **ext_proc**: Add runtime guard for timeout error code ``504 Gateway Timeout`` that is returned to downstream. If runtime flag ``envoy.reloadable_features.ext_proc_timeout_error`` is set to ``false``, old error code ``500 Internal Server Error`` will be returned. * **http**: Fixed a bug where an incomplete request (missing body or trailers) may be proxied to the upstream when the limit on the number of requests per I/O cycle is configured and an HTTP decoder filter that pauses filter chain is present. This behavior can be reverted by setting the runtime guard ``envoy.reloadable_features.use_filter_manager_state_for_downstream_end_stream`` to ``false``. * **http2**: Fixed bug where an upstream that sent a ``GOAWAY`` and gracefully closed a connection would result in an increment of the cluster stat ``upstream_cx_protocol_error`` and setting the ``UpstreamProtocolError`` response flag. This behavior can be reverted by setting the runtime guard ``envoy.reloadable_features.http2_no_protocol_error_upon_clean_close`` to ``false``. * **http3**: Fixed a bug in the ``CONNECT-UDP`` forwarding mode where Envoy reset the upstream stream when it received HTTP/3 datagrams before receiving the SETTINGS frame from the upstream peer. Envoy now drops the datagrams in this case instead of resetting the stream. * **http3**: Fixed a bug where an empty trailers block could be sent. This would occur if a filter removed the last trailer - a likely occurrence with the ``grpc_web_filter``. This change makes HTTP/3 behave the same way HTTP/2 does, converting an empty trailers block to no trailers. This behavior can be reverted by setting the runtime guard ``envoy.reloadable_features.http3_remove_empty_trailers`` to ``false``. * **http_async_client**: Fixed the local reply and destroy order crashes when using the HTTP async client for websocket handshake. * **jwt**: Fixed a bug where using ``clear_route_cache`` with remote JWKs works incorrectly and may cause a crash when the modified request does not match any route. * **oauth**: Set token cookies in response regardless of :ref:`forward_bearer_token ` config option. * **proxy_filter**: Fixed a bug in the ``CONNECT`` implementation that would cause the ``CONNECT`` request created to be invalid when the hostname contains a port number. When the port number is not specified, the port 443 will be automatically added. This behavior can be reverted by setting the runtime guard ``envoy.reloadable_features.proxy_ssl_port`` to ``false``. * **quic**: Fixes access log formatter ``%CONNECTION_ID%`` for QUIC connections. * **rbac**: RBAC will now allow stat prefixes configured in per-route config to override the base config's stat prefix. * **runtime**: Fixed an inconsistency in how boolean values are loaded in RTDS, where they were previously converted to ``1``/``0`` instead of ``true``/``false``. The correct string representation (``true``/``false``) will now be used. This change can be reverted by setting the runtime guard ``envoy.reloadable_features.boolean_to_string_fix`` to ``false``. * **tracing**: Fixed a bug where the OpenTelemetry tracer exports the OTLP request even when no spans are present. * **upstream**: Fixed a bug using hard coded drop category when reporting drop_overload stats to the load report service. It is changed to use drop category that is set in :ref:`category `. * **upstream**: Use previously calculated healthcheck when endpoints move priority levels repeatedly. * **wasm**: Fixed a bug where a body received in chunks is not correctly dumped, resulting in an incomplete dump and loss of the last chunk. This issue is manifested in HTTP/2. * **websocket**: Fixed a bug where the websocket upgrade filter would not take into account per-filter configs. Removed config or runtime ------------------------- *Normally occurs at the end of the* :ref:`deprecation period ` * **DNS**: Removed ``envoy.reloadable_features.dns_cache_set_first_resolve_complete`` runtime flag and legacy code paths. * **dynamic forward proxy**: Removed ``envoy.reloadable_features.normalize_host_for_preresolve_dfp_dns`` runtime flag and legacy code paths. * **dynamic_foward_proxy**: Removed runtime flag ``envoy.reloadable_features.dfp_mixed_scheme`` and legacy code path. * **ext_proc**: Removed runtime flag ``envoy_reloadable_features_immediate_response_use_filter_mutation_rule`` and legacy code path. * **ext_proc**: Removed runtime flag ``envoy_reloadable_features_send_header_raw_value`` and legacy code path. * **grpc reverse bridge**: Removed ``envoy.reloadable_features.grpc_http1_reverse_bridge_change_http_status`` runtime flag and legacy code paths. * **grpc reverse bridge**: Removed ``envoy.reloadable_features.grpc_http1_reverse_bridge_handle_empty_response`` runtime flag and legacy code paths. * **http**: Removed ``envoy.reloadable_features.use_http3_header_normalisation`` runtime flag and legacy code paths. * **http**: Removed runtime flag ``envoy.reloadable_features.abort_filter_chain_on_stream_reset`` and legacy code path. * **http**: Removed runtime flag ``envoy.reloadable_features.http1_connection_close_header_in_redirect`` and legacy code paths. * **http**: Removed runtime flag ``envoy.reloadable_features.no_downgrade_to_canonical_name`` and legacy code path. * **http**: Removed the ``envoy.reloadable_features.http2_validate_authority_with_quiche`` runtime flag and its legacy code paths. * **quic**: Removed ``envoy.reloadable_features.quic_fix_filter_manager_uaf`` runtime flag and legacy code paths. * **stateful_session**: Removed ``envoy.reloadable_features.stateful_session_encode_ttl_in_cookie`` runtime flag and legacy code paths. * **tls**: Removed runtime flag ``envoy.reloadable_features.ssl_transport_failure_reason_format``. * **udp**: Removed ``envoy.restart_features.udp_read_normalize_addresses`` runtime flag and legacy code paths. * **upstream**: Removed runtime flag ``envoy.reloadable_features.avoid_zombie_streams`` and legacy code paths. * **upstream**: Removed runtime flag ``envoy.reloadable_features.upstream_allow_connect_with_2xx`` and legacy code paths. * **upstream flow control**: Removed ``envoy.reloadable_features.upstream_wait_for_response_headers_before_disabling_read`` runtime flag and legacy code paths. New features ------------ * **access log**: Added support for :ref:`%DOWNSTREAM_PEER_CHAIN_FINGERPRINTS_1% `, ``%DOWNSTREAM_PEER_CHAIN_FINGERPRINTS_256%``, and ``%DOWNSTREAM_PEER_CHAIN_SERIALS%``, as access log formatters. * **access_log**: Added new JSON access log formatter implementation which provides 16-25x performance improvement over the old implementation. The legacy implementation will be enabled by default for now but will be deprecated and removed in a future release. The :ref:`sort_properties ` field will be ignored in the new implementation because the new implementation always sorts properties. And the new implementation will always keep the value type in the JSON output. For example, the duration field will always be rendered as a number instead of a string. The new JSON formatter could be enabled by setting the runtime guard ``envoy.reloadable_features.logging_with_fast_json_formatter`` to true. * **access_log**: Added new access log command operators ``%START_TIME_LOCAL%`` and ``%EMIT_TIME_LOCAL%``, similar to ``%START_TIME%`` and ``%EMIT_TIME%``, but use local time zone. * **access_log**: added ``%UPSTREAM_CLUSTER_RAW%`` access log formatter to log the original upstream cluster name, regardless of whether ``alt_stat_name`` is set. * **access_logs**: Added support for ``%UPSTREAM_CONNECTION_ID%`` access log substitution string in TCP and UDP tunneling flows. * **apple_dns_impl**: For the apple DNS resolver, prefer using IPv6 address when addresses from both families are available. Can be reverted by setting ``envoy.reloadable_features.prefer_ipv6_dns_on_macos`` to false. * **basic_auth**: Added support to provide an override :ref:`authentication_header ` to load Basic Auth credential. * **cluster**: Allows customizing the happy eyeballs algorithm for an upstream cluster by configuring :ref:`happy_eyeballs_config `. A default configuration will be used if not provided. This behavior can be reverted by setting the runtime guard ``envoy.reloadable_features.use_config_in_happy_eyeballs`` to false. * **dns**: for the :ref:`strict DNS ` and :ref:`logical DNS ` cluster types, the new :ref:`dns_jitter ` field, if provided, will causes the cluster to refresh DNS entries later by a random amount of time as to avoid stampedes of DNS requests. This field sets the upper bound (exclusive) for the random amount. * **ext_authz**: Added :ref:`emit_filter_state_stats ` which enables filter state stats for access logging. * **ext_authz**: Added config field :ref:`filter_metadata ` for injecting arbitrary data to the filter state for logging. * **ext_proc**: Added allow list :ref:`allowed_override_modes ` for :ref:`mode_override `. * **extension_discovery_service**: added ECDS support for :ref:`UDP session filters `. * **formatter**: Added full feature absl::FormatTime() support to the DateFormatter. This allows the timepoint formatters (like ``%START_TIME%``) to use ``%E#S``, ``%E*S``, ``%E#f`` and ``%E*f`` to format the subsecond part of the timepoint. * **geoip**: Added ``envoy.reloadable_features.mmdb_files_reload_enabled`` runtime flag that enables reload of mmdb files by default. * **getaddrinfo**: Added :ref:`num_retries` to configure the number of DNS resolution retries. If this field is not provided, the ``getaddrinfo`` resolver will retry indefinitely until it succeeds or the DNS query times out. This behavior can be reverted by setting the runtime guard ``envoy.reloadable_features.getaddrinfo_num_retries`` to false. * **getaddrinfo**: Added ``envoy.enable_dfp_dns_trace`` runtime flag to enable DNS query trace in the DNS resolution details. * **grpc_field_extraction**: Added ``map`` support: Target fields of type ``map`` can be extracted and added to dynamic metadata. * **http**: Added configuration setting for the :ref:`maximum size of response headers ` in responses. The default can be overridden with runtime key ``envoy.reloadable_features.max_response_headers_size_kb``. * **http3**: ``http3_protocol_options`` in ``HttpConnectionManager`` has been upgraded to general access. * **http_11_proxy**: Added the option to configure the transport socket via locality or endpoint metadata. * **jwt_authn**: Added missing implementation to jwt_authn matchers to allow glob pattern matching. * **load_balancing**: Added a new WIP implementation of :ref:`client_side_weighted_round_robin ` load balancing policy that uses ``OrcaLoadReport`` provided by the upstream host to calculate host load balancing weight. * **local_ratelimit**: Add the :ref:`rate_limits ` field to generate rate limit descriptors. If this field is set, the :ref:`VirtualHost.rate_limits ` or :ref:`RouteAction.rate_limits ` fields will be ignored. * **lua**: Added two new methods ``oidsPeerCertificate()`` and ``oidsLocalCertificate()`` to SSL connection object API :ref:`SSL connection info object `. * **matching**: Added dynamic metadata matcher support :ref:`Dynamic metadata input ` and :ref:`Dynamic metadata input matcher `. * **oauth**: Added :ref:`disable_access_token_set_cookie ` to disable setting the Access Token cookie. * **oauth**: Added :ref:`disable_refresh_token_set_cookie ` to disable setting the Refresh Token cookie. * **oauth**: Added :ref:`retry_policy ` to reconnect to the auth server when a connection fails to be established. Added :ref:`cookie_domain ` field to OAuth2 filter to allow setting the domain of cookies. Added a nonce to the state parameter in the authorization request to mitigate CSRF attacks. The nonce is generated by the OAuth2 filter and stored in a cookie. This feature is enabled by defaut starting from this release. * **original_ip_detection extension**: The :ref:`xff ` original IP detection method now supports using a list of trusted CIDRs when parsing ``x-forwarded-for``. * **quic**: Added QUIC protocol option :ref:`reject_new_connections ` to reject connection-establishing packets at the QUIC layer. * **quic**: Added QUIC protocol option :ref:`save_cmsg_config ` to optionally specify a CMSG header type to be propagated from the first packet on the connection to QuicListenerFilter. * **quic**: Added ``max_packet_length`` to the QUIC protocol options to allow users to change the largest size of a QUIC packet that can be transmitted over the QUIC connection. * **quic**: QUIC server and client support certificate compression, which can in some cases reduce the number of round trips required to setup a connection. This change temporarily disabled by setting the runtime flag ``envoy.reloadable_features.quic_support_certificate_compression`` to ``false``. * **ratelimit**: Added the ability to modify :ref:`hits_addend ` by setting by setting filter state value ``envoy.ratelimit.hits_addend`` to the desired value. * **rbac**: Added :ref:`delay_deny ` to support deny connection after the configured duration. * **redis**: Added support for publish. * **redis_proxy**: Added :ref:`external_auth_provider ` to support external authentication for redis proxy. * **resource_monitors**: Added possibility to monitor CPU utilization in Linux based systems via :ref:`cpu utilization monitor ` in overload manager. * **sockets**: Added socket ``type`` field for specifying a socket type to apply the socket option to under :ref:`SocketOption `. If not specified, the socket option will be applied to all socket types. * **tls**: Added :ref:`prefer_client_ciphers ` to support enabling client cipher preference instead of server's for TLS handshakes. * **tls**: Added an extension point :ref:`custom_tls_certificate_selector ` to allow overriding TLS certificate selection behavior. An extension can select certificate base on the incoming SNI, in both sync and async mode. * **udp_access_logs**: Added support for ``%BYTES_RECEIVED%``, ``%BYTES_SENT%``, ``%UPSTREAM_HEADER_BYTES_SENT%``, ``%UPSTREAM_HEADER_BYTES_RECEIVED%``, ``%UPSTREAM_WIRE_BYTES_SENT%``, ``%UPSTREAM_WIRE_BYTES_RECEIVED%`` access log substitution strings for UDP tunneling flows.