Memory Manager Goals

• 保证最少 NUMA Node 去满足 POD 的内存需求: Offer guaranteed memory (and hugepages) allocation over a minimum number of NUMA nodes for containers (within a pod).

• 长远是让pod中的所有 container 运行在尽量少的 NUMA NODE 中： Guaranteeing the affinity of memory and hugepages to the same NUMA node for the whole group of containers (within a pod). This is a long-term goal which will be achieved along with PR #1752 and the implementation of hintprovider.GetPodLevelTopologyHints() API in the Memory Manager.- Offer guaranteed memory (and hugepages) allocation over a minimum number of NUMA nodes for containers (within a pod).

Design Overview

Once kubelet requests a guaranteed QoS pod admission, as shown in the figure above, Topology Manager queries Memory Manager about the preferred NUMA affinity for memory and hugepages, for all containers in a pod. For each container in a pod, Memory Manager calculates the affinity using its internal database, namely the Node Map. The Node Map is an object which is responsible for tracking the utilization of memory (and hugepages) for all containers in Guaranteed QoS class. Once Memory Manager completes the computations, it returns the hints to Topology Manager so that Topology Manager can figure out which NUMA node or a group of NUMA nodes are the best fit for memory pinning, for a container. The overall calculation is performed for all containers in the pod, and if none of containers is rejected, the pod becomes finally admitted and deployed(只有所有 Container 都通过时，才能通过 Admit).

Design Details

1
2
3
4
5
6
7
8

// MemoryTable contains memory information
type MemoryTable struct {
	TotalMemSize   uint64 `json:"total"`
	SystemReserved uint64 `json:"systemReserved"` // kubelet 中指定的系统预留
	Allocatable    uint64 `json:"allocatable"`
	Reserved       uint64 `json:"reserved"` //container limit汇总
	Free           uint64 `json:"free"`
}

每个 NUMA_Node + pageSize 组合对应一个 MemoryTable 。数值之间的关系：

Allocatable = TotalMemSize - SystemReserved
Free + Reserved = Allocatable

例如：
--reserved-memory={numa-node=0, type=memory, limit=500MB}, {numa-node=1,type=memory, limit=100MB}

Ref

https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/1769-memory-manager#the-concept-of-node-map-and-memory-maps