https://helm.sh/docs/howto/charts_tips_and_tricks/

Helm uses Go templates for templating your resource files. While Go ships several built-in functions, we have added many others.

First, we added all of the functions in the Sprig library, except env and expandenv, for security reasons.

We also added two special template functions: include and required. The include function allows you to bring in another template, and then pass the results to other template functions.

Using the ‘include’ Function

For example, this template snippet includes a template called mytpl, then lowercases the result, then wraps that in double quotes.

Values

https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing

To see what options are configurable on a chart, use helm show values:

$ helm show values bitnami/wordpress
## Global Docker image parameters
## Please, note that this will override the image parameters, including dependencies, configured to use the global value
## Current available global Docker image parameters: imageRegistry and imagePullSecrets
##
# global:
#   imageRegistry: myRegistryName
#   imagePullSecrets:
#     - myRegistryKeySecretName
#   storageClass: myStorageClass

## Bitnami WordPress image version
## ref: https://hub.docker.com/r/bitnami/wordpress/tags/
##
image:
  registry: docker.io
  repository: bitnami/wordpress
  tag: 5.6.0-debian-10-r35
  [..]

You can then override any of these settings in a YAML formatted file, and then pass that file during installation.

k8s headless svc

k8s headless and dns

https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
A/AAAA records

“Normal” (not headless) Services are assigned a DNS A or AAAA record, depending on the IP family of the service, for a name of the form my-svc.my-namespace.svc.cluster-domain.example. This resolves to the cluster IP of the Service.

“Headless” (without a cluster IP) Services are also assigned a DNS A or AAAA record, depending on the IP family of the service, for a name of the form my-svc.my-namespace.svc.cluster-domain.example. Unlike normal Services, this resolves to the set of IPs of the pods selected by the Service. Clients are expected to consume the set or else use standard round-robin selection from the set.

https://banzaicloud.com/blog/istio-mixerless-telemetry/

Because Istio Telemetry V2 lacks a central component (Mixer) with access to K8s metadata, the proxies themselves require the metadata necessary to provide rich metrics. Additionally, features provided by Mixer had to be added to the Envoy proxies to replace the Mixer-based telemetry. Istio Telemetry V2 uses two custom Envoy plugins to achieve just that.

In-proxy service-level metrics in Telemetry V2 are provided by two custom plugins, metadata-exchange and stats.

https://istio.io/latest/docs/tasks/observability/metrics/tcp-metrics/

Understanding TCP telemetry collection

In this task, you used Istio configuration to automatically generate and report metrics for all traffic to a TCP service within the mesh. TCP Metrics for all active connections are recorded every 15s by default and this timer is configurable via tcpReportingDuration. Metrics for a connection are also recorded at the end of the connection.

TCP attributes

Several TCP-specific attributes enable TCP policy and control within Istio. These attributes are generated by Envoy Proxies and obtained from Istio using Envoy’s Node Metadata. Envoy forwards Node Metadata to Peer Envoys using ALPN based tunneling and a prefix based protocol. We define a new protocol istio-peer-exchange, that is advertised and prioritized by the client and the server sidecars in the mesh. ALPN negotiation resolves the protocol to istio-peer-exchange for connections between Istio enabled proxies, but not between an Istio enabled proxy and any other proxy. This protocol extends TCP as follows:

https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/

from worknode

no cert check TLS

1
2
3
4
5
6

export DOMAIN=fortio-server.idm-mark.svc.cluster.local
export INGRESS_IP=10.97.117.127
export SECURE_INGRESS_PORT=8080

curl -v -HHost:$DOMAIN --resolve "$DOMAIN:$SECURE_INGRESS_PORT:$INGRESS_IP" \
-k "https://$DOMAIN:$SECURE_INGRESS_PORT/fortio/"

simple TLS

1
2

curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \
--cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"

through gateway

no cert check TLS

1
2
3
4
5
6

export DOMAIN=fortio-server.idm-mark.svc.cluster.local
export INGRESS_IP=10.100.122.140
export SECURE_INGRESS_PORT=80

curl -v -HHost:$DOMAIN --resolve "$DOMAIN:$SECURE_INGRESS_PORT:$INGRESS_IP" \
-k "https://$DOMAIN:$SECURE_INGRESS_PORT/fortio/"

no cert check TLS

1
2
3
4
5
6

export DOMAIN=fortio-server.idm-mark.svc.cluster.local
export INGRESS_IP=10.100.122.140
export SECURE_INGRESS_PORT=80

curl -v -HHost:$DOMAIN --resolve "$DOMAIN:$SECURE_INGRESS_PORT:$INGRESS_IP" \
-k "https://$DOMAIN:$SECURE_INGRESS_PORT/fortio/"

https://istio.io/latest/docs/ops/configuration/traffic-management/tls-configuration/

Sidecars

Sidecar traffic has a variety of associated connections. Let’s break them down one at a time.

Sidecar proxy network connections

1. External inbound traffic This is traffic coming from an outside client that is captured by the sidecar. If the client is inside the mesh, this traffic may be encrypted with Istio mutual TLS. By default, the sidecar will be configured to accept both mTLS and non-mTLS traffic, known as PERMISSIVE mode. The mode can alternatively be configured to STRICT, where traffic must be mTLS, or DISABLE, where traffic must be plaintext. The mTLS mode is configured using a PeerAuthentication resource.
2. Local inbound traffic This is traffic going to your application service, from the sidecar. This traffic will always be forwarded as-is. Note that this does not mean it’s always plaintext; the sidecar may pass a TLS connection through. It just means that a new TLS connection will never be originated from the sidecar.
3. Local outbound traffic This is outgoing traffic from your application service that is intercepted by the sidecar. Your application may be sending plaintext or TLS traffic. If automatic protocol selection is enabled, Istio will automatically detect the protocol. Otherwise you should use the port name in the destination service to manually specify the protocol.
4. External outbound traffic This is traffic leaving the sidecar to some external destination. Traffic can be forwarded as is, or a TLS connection can be initiated (mTLS or standard TLS). This is controlled using the TLS mode setting in the trafficPolicy of a DestinationRule resource. A mode setting of DISABLE will send plaintext, while SIMPLE, MUTUAL, and ISTIO_MUTUAL will originate a TLS connection.

The key takeaways are:

https://istio.io/v1.4/docs/tasks/security/authentication/mtls-migration/

Ensure that your cluster is in PERMISSIVE mode before migrating to mutual TLS. Run the following command to check:

1
2
3
4
5
6

$ kubectl get meshpolicy default -o yaml
...
spec:
  peers:
  - mtls:
      mode: PERMISSIVE

In PERMISSIVE mode, the Envoy sidecar relies on the ALPN value istio to decide whether to terminate the mutual TLS traffic. If your workloads (without Envoy sidecar) have enabled mutual TLS directly to the services with Envoy sidecars, enabling PERMISSIVE mode may cause these connections to fail.

SPIFFE

https://spiffe.io/docs/latest/spiffe-about/overview/

https://spiffe.io/docs/latest/spiffe-about/spiffe-concepts/

old school Official SPIFFE method:

https://blog.envoyproxy.io/securing-the-service-mesh-with-spire-0-3-abb45cd79810

Workload

A workload is a single piece of software, deployed with a particular configuration for a single purpose; it may comprise multiple running instances of software, all of which perform the same task. The term “workload” may encompass a range of different definitions of a software system, including:

• A web server running a Python web application, running on a cluster of virtual machines with a load-balancer in front of it.
• An instance of a MySQL database.
• A worker program processing items on a queue.
• A collection of independently deployed systems that work together, such as a web application that uses a database service. The web application and database could also individually be considered workloads.

SPIFFE ID

A SPIFFE ID is a string that uniquely and specifically identifies a workload. SPIFFE IDs may also be assigned to intermediate systems that a workload runs on (such as a group of virtual machines). For example, spiffe://acme.com/billing/payments is a valid SPIFFE ID.

x-forwarded-client-cert

https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_conn_man/headers#x-forwarded-client-cert

x-forwarded-client-cert (XFCC) is a proxy header which indicates certificate information of part or all of the clients or proxies that a request has flowed through, on its way from the client to the server. A proxy may choose to sanitize/append/forward the XFCC header before proxying the request.

The XFCC header value is a comma (",") separated string. Each substring is an XFCC element, which holds information added by a single proxy. A proxy can append the current client certificate information as an XFCC element, to the end of the request’s XFCC header after a comma.

https://istio.io/latest/docs/ops/common-problems/network-issues/#double-tls

Double TLS (TLS origination for a TLS request)

When configuring Istio to perform TLS origination, you need to make sure that the application sends plaintext requests to the sidecar, which will then originate the TLS.

TLS Origination

TLS origination occurs when an Istio proxy (sidecar or egress gateway) is configured to accept unencrypted internal HTTP connections, encrypt the requests, and then forward them to HTTPS servers that are secured using simple or mutual TLS. This is the opposite of TLS termination where an ingress proxy accepts incoming TLS connections, decrypts the TLS, and passes unencrypted requests on to internal mesh services.

How Does the CPU Manager Work?

When CPU manager is enabled with the “static” policy, it manages a shared pool of CPUs. Initially this shared pool contains all the CPUs in the compute node. When a container with integer CPU request in a Guaranteed pod is created by the Kubelet, CPUs for that container are removed from the shared pool and assigned exclusively for the lifetime of the container. Other containers are migrated off these exclusively allocated CPUs.