SQL

Understanding Node Selector and Node Affinity in Kubernetes

July 5, 2020 comments off

The Kubernetes scheduler can be constrained to place a pod on particular nodes using few different options. In your POD specification, there are many ways in which you declare a POD should be dedicated to specific nodes.

  • Node Selector
  • Node Affinity
  • Pod Affinity aka inter-pod affinity/anti-affinity
  • Taints and Tolerations

In this post, I am going to discuss on Node Selector and Node Affinity.

nodeSelector
nodeSelector is the simplest recommended form of node selection constraint. nodeSelector is a field of PodSpec. It specifies a map of key-value pairs. For the pod to be eligible to run on a node, the node must have each of the indicated key-value pairs as labels (it can have additional labels as well).

Interlude: built-in node labels

In addition to labels you attach, nodes come pre-populated with a standard set of labels. These labels are

kubernetes.io/hostname
failure-domain.beta.kubernetes.io/zone
failure-domain.beta.kubernetes.io/region
topology.kubernetes.io/zone
topology.kubernetes.io/region
beta.kubernetes.io/instance-type
node.kubernetes.io/instance-type
kubernetes.io/os
kubernetes.io/arch

The affinity feature consists of two types of affinity

  • node affinity” and
  • inter-pod affinity/anti-affinity”

nodeAffinity

Kubernetes also has a more nuanced way of setting affinity called nodeAffinity and podAffinity.

nodeSelector provides a very simple way to constrain pods to nodes with particular labels. Node affinity is conceptually similar to nodeSelector but nodeAffinity allows users to more expressive way pods to nodes with particular labels.

The affinity/anti-affinity feature, greatly expands the types of constraints you can express. The key enhancements are

  1. The affinity/anti-affinity language is more expressive. The language offers more matching rules besides exact matches created with a logical AND operation;
  2. you can indicate that the rule is “soft”/”preference” rather than a hard requirement, so if the scheduler can’t satisfy it, the pod will still be scheduled;
  3. you can constrain against labels on other pods running on the node (or other topological domain), rather than against labels on the node itself, which allows rules about which pods can and cannot be co-located

The affinity feature consists of two types of affinity

  • node affinity – Node affinity is like the existing nodeSelector (but with the first two benefits listed above)
  • inter-pod affinity/anti-affinity – inter-pod affinity/anti-affinity constrains against pod labels rather than node labels, as described in the all three item listed above.

These are fields in under Pod metadata and take automatic or user-defined metadata to dictate where to schedule pods. affinity differs from nodeSelector in the following ways:

  • Schedule a pod based on which other pods are or are not running on a node.
  • Request without requiring that a pod be run on a node.
  • Specify a set of allowable values instead of a single value requirement.

Types of node affinity

There are currently two types of node affinity, called

requiredDuringSchedulingIgnoredDuringExecution

  • hard rule

preferredDuringSchedulingIgnoredDuringExecution

  • soft rule

nodeAffinity example

Lets take the above example of deploying a nginx and a httpd pod, except we have a more complicated set of requirements:

  • nginx cannot run on the same node as httpd
  • httpd should run on a node with the x-web:yes label, but can run anywhere.
  • nginx must run on a node with y-web:yes label and should fail if not.

inter-pod affinity/anti-affinity
Inter-pod affinity and anti-affinity allow you to constrain which nodes your pod is eligible to be scheduled based on labels on pods that are already running on the node rather than based on labels on nodes.

An example of a pod that uses pod affinity

Reference

https://coreos.com/fleet/docs/latest/affinity.html

Rajesh Kumar
Follow me
Latest posts by Rajesh Kumar (see all)