https://www.envoyproxy.io/docs/envoy/latest/operations/performance

Performance¶

Envoy is architected to optimize scalability and resource utilization by running an event loop on a small number of threads. The “main” thread is responsible for control plane processing, and each “worker” thread handles a portion of the data plane processing. Envoy exposes two statistics to monitor performance of the event loops on all these threads.

• Loop duration: Some amount of processing is done on each iteration of the event loop. This amount will naturally vary with changes in load. However, if one or more threads have an unusually long-tailed loop duration, it may indicate a performance issue. For example, work might not be distributed fairly across the worker threads, or there may be a long blocking operation in an extension that’s impeding progress.

Kernel

https://lwn.net/Articles/542629/
One of the features merged in the 3.9 development cycle was TCP and UDP support for the SO_REUSEPORT socket option; that support was implemented in a series of patches by Tom Herbert. The new socket option allows multiple sockets on the same host to bind to the same port, and is intended to improve the performance of multithreaded network server applications running on top of multicore systems.
The basic concept of SO_REUSEPORT is simple enough. Multiple servers (processes or threads) can bind to the same port if they each set the option as follows:

https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/connection_pooling#automatic-protocol-selection

Connection pooling

For HTTP traffic, Envoy supports abstract connection pools that are layered on top of the underlying wire protocol (HTTP/1.1, HTTP/2, HTTP/3). The utilizing filter code does not need to be aware of whether the underlying protocol supports true multiplexing or not. In practice the underlying implementations have the following high level properties:

HTTP/1.1

The HTTP/1.1 connection pool acquires connections as needed to an upstream host (up to the circuit breaking limit). Requests are bound to connections as they become available, either because a connection is done processing a previous request or because a new connection is ready to receive its first request. The HTTP/1.1 connection pool does not make use of pipelining so that only a single downstream request must be reset if the upstream connection is severed.

TLS ALPN

Force HTTP 1.1

https://www.tetrate.io/blog/envoy-and-istio-security-releases-june-2020/

Envoy versions can mitigate those vulnerabilities by disabling HTTP2 and allowing only HTTP/1.1 by setting http_connection_manager.codec_type to “HTTP1” and removing “h2” from common_tls_context.alpn_protocols.

For Istio:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: disable-ingress-h2
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  - applyTo: NETWORK_FILTER # http connection manager is a filter in Envoy
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: "envoy.http_connection_manager"
    patch:
      operation: MERGE
      value:
        typed_config:
          "@type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
          codec_type: HTTP1

HTTP 1.1 Upgrade header

https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/http/upgrades#http-upgrades

HTTP Protocol Options

https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/upstreams/http/v3/http_protocol_options.proto#extensions-upstreams-http-v3-httpprotocoloptions-autohttpconfig

This extension may be referenced by the qualified name envoy.upstreams.http.http_protocol_options

extensions.upstreams.http.v3.HttpProtocolOptions

HttpProtocolOptions specifies Http upstream protocol options. This object is used in typed_extension_protocol_options, keyed by the name envoy.extensions.upstreams.http.v3.HttpProtocolOptions.

This controls what protocol(s) should be used for upstream and how said protocol(s) are configured.

This replaces the prior pattern of explicit protocol configuration directly in the cluster. So a configuration like this, explicitly configuring the use of HTTP/2 upstream:

https://helm.sh/docs/topics/charts/

Chart

A Chart is a Helm package. It contains all of the resource definitions necessary to run an application, tool, or service inside of a Kubernetes cluster. Think of it like the Kubernetes equivalent of a Homebrew formula, an Apt dpkg, or a Yum RPM file.

Directory structure

wordpress/
  Chart.yaml          # A YAML file containing information about the chart
  LICENSE             # OPTIONAL: A plain text file containing the license for the chart
  README.md           # OPTIONAL: A human-readable README file
  values.yaml         # The default configuration values for this chart
  values.schema.json  # OPTIONAL: A JSON Schema for imposing a structure on the values.yaml file
  charts/             # A directory containing any charts upon which this chart depends.
  crds/               # Custom Resource Definitions
  templates/          # A directory of templates that, when combined with values,
                      # will generate valid Kubernetes manifest files.
  templates/NOTES.txt # OPTIONAL: A plain text file containing short usage notes

Pull:

As you edit your chart, you can validate that it is well-formed by running helm lint.

When it’s time to package the chart up for distribution, you can run the helm package command:

1
2

$ helm package deis-workflow
deis-workflow-0.1.0.tgz

Helm installs charts into Kubernetes, creating a new release for each installation. And to find new charts, you can search Helm chart repositories.

Helm Chart Repository

A Repository is the place where charts can be collected and shared. It’s like Perl’s CPAN archive or the Fedora Package Database, but for Kubernetes packages.

1

helm repo add bitnami https://charts.bitnami.com/bitnami

$ helm search repo bitnami
NAME                             	CHART VERSION	APP VERSION  	DESCRIPTION
bitnami/bitnami-common           	0.0.9        	0.0.9        	DEPRECATED Chart with custom templates used in ...
bitnami/airflow                  	8.0.2        	2.0.0        	Apache Airflow is a platform to programmaticall...
bitnami/apache                   	8.2.3        	2.4.46       	Chart for Apache HTTP Server
bitnami/aspnet-core              	1.2.3        	3.1.9        	ASP.NET Core is an open-source framework create...

Chart

A Chart is a Helm package. It contains all of the resource definitions necessary to run an application, tool, or service inside of a Kubernetes cluster. Think of it like the Kubernetes equivalent of a Homebrew formula, an Apt dpkg, or a Yum RPM file.

Helm installs resources in the following order:

• Namespace
• NetworkPolicy
• ResourceQuota
• LimitRange
• PodSecurityPolicy
• PodDisruptionBudget
• ServiceAccount
• Secret
• SecretList
• ConfigMap
• StorageClass
• PersistentVolume
• PersistentVolumeClaim
• CustomResourceDefinition
• ClusterRole
• ClusterRoleList
• ClusterRoleBinding
• ClusterRoleBindingList
• Role
• RoleList
• RoleBinding
• RoleBindingList
• Service
• DaemonSet
• Pod
• ReplicationController
• ReplicaSet
• Deployment
• HorizontalPodAutoscaler
• StatefulSet
• Job
• CronJob
• Ingress
• APIService

https://helm.sh/docs/howto/charts_tips_and_tricks/

Helm uses Go templates for templating your resource files. While Go ships several built-in functions, we have added many others.

First, we added all of the functions in the Sprig library, except env and expandenv, for security reasons.

We also added two special template functions: include and required. The include function allows you to bring in another template, and then pass the results to other template functions.

Using the ‘include’ Function

For example, this template snippet includes a template called mytpl, then lowercases the result, then wraps that in double quotes.

Values

https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing

To see what options are configurable on a chart, use helm show values:

$ helm show values bitnami/wordpress
## Global Docker image parameters
## Please, note that this will override the image parameters, including dependencies, configured to use the global value
## Current available global Docker image parameters: imageRegistry and imagePullSecrets
##
# global:
#   imageRegistry: myRegistryName
#   imagePullSecrets:
#     - myRegistryKeySecretName
#   storageClass: myStorageClass

## Bitnami WordPress image version
## ref: https://hub.docker.com/r/bitnami/wordpress/tags/
##
image:
  registry: docker.io
  repository: bitnami/wordpress
  tag: 5.6.0-debian-10-r35
  [..]

You can then override any of these settings in a YAML formatted file, and then pass that file during installation.

k8s headless svc

k8s headless and dns

https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
A/AAAA records

“Normal” (not headless) Services are assigned a DNS A or AAAA record, depending on the IP family of the service, for a name of the form my-svc.my-namespace.svc.cluster-domain.example. This resolves to the cluster IP of the Service.

“Headless” (without a cluster IP) Services are also assigned a DNS A or AAAA record, depending on the IP family of the service, for a name of the form my-svc.my-namespace.svc.cluster-domain.example. Unlike normal Services, this resolves to the set of IPs of the pods selected by the Service. Clients are expected to consume the set or else use standard round-robin selection from the set.