{"id":8005,"date":"2019-12-08T00:54:39","date_gmt":"2019-12-08T00:54:39","guid":{"rendered":"https:\/\/www.devopsschool.com\/blog\/?p=8005"},"modified":"2021-11-15T07:13:08","modified_gmt":"2021-11-15T07:13:08","slug":"deep-dive-into-kubernetes-taint-with-node","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/blog\/deep-dive-into-kubernetes-taint-with-node\/","title":{"rendered":"Update the taints on one or more nodes in Kubernetes"},"content":{"rendered":"\n<p><strong>What is meaning of taint?<\/strong><\/p>\n\n\n\n<p>a trace of a bad or undesirable substance or quality.<\/p>\n\n\n\n<p><strong>What is tainted?<\/strong><\/p>\n\n\n\n<p>spoiled; damaged in quality, taste, or value:<\/p>\n\n\n\n<p><strong>Follwing are workload which run in a cluster&#8217;s node\u2026.<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>DaemonSet<\/li><li>Deployment <\/li><li>Pod <\/li><li>ReplicaSet<\/li><li>ReplicationController<\/li><li>StatefulSet <\/li><\/ul>\n\n\n\n<p>When you submit a workload, The scheduler determines where to place the Pods associated with the workload. The scheduler is free to place a Pod on any node that satisfies the Pod&#8217;s CPU, memory, and custom resource requirements.<\/p>\n\n\n\n<p>If your cluster runs a variety of workloads, you might want to exercise some control over which workloads can run on a particular pool of nodes.<\/p>\n\n\n\n<p>A node taint lets you mark a node so that the scheduler avoids or prevents using it for certain Pods. A complementary feature, tolerations, lets you designate Pods that can be used on &#8220;tainted&#8221; nodes.<\/p>\n\n\n\n<p>A taint consists of a key, value, and effect. As an argument here, it is expressed as key=value:effect.<\/p>\n\n\n\n<p>The key must begin with a letter or number, and may contain letters, numbers, hyphens, dots, and underscores, up to 253 characters. The value is optional. If given, it must begin with a letter or number, and may contain letters, numbers, hyphens, dots, and underscores, up to  63 characters.<\/p>\n\n\n\n<p>The effect must be <strong>NoSchedule<\/strong>, <strong>PreferNoSchedule <\/strong>or <strong>NoExecute<\/strong>. <strong>Currently taint can only apply to node. Here are the available effects: <\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>NoSchedule<\/strong>: Pods that do not tolerate this taint are not scheduled on the node.<\/li><li><strong>PreferNoSchedule<\/strong>: Kubernetes avoids scheduling Pods that do not tolerate this taint onto the node.<\/li><li><strong>NoExecute<\/strong>: Pod is evicted from the node if it is already running on the node, and is not scheduled onto the node if it is not yet running on the node.<\/li><\/ul>\n\n\n\n<p><strong>Adding \/ Inspecting \/ Removing a taint to an existing node using NoSchedule<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-1\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">\n<span class=\"hljs-comment\"># Update node 'node1' with a taint with key 'dedicated' and value 'special-user' and effect 'NoSchedule'.<\/span>\n<span class=\"hljs-comment\"># Adding a taint to an existing node using NoSchedule<\/span>\n$ kubectl taint nodes node1 dedicated=special-user:NoSchedule<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-1\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Inspecting a node&#8217;s taints<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-2\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\">\n$ kubectl describe nodes node1 | grep -i taint\n$ kubectl run nginx --image=nginx --replicas=<span class=\"hljs-number\">8<\/span>\n$ kubectl <span class=\"hljs-keyword\">get<\/span> pod -o wide<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-2\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Removing a taint from a node. Remove from node &#8216;node1&#8217; the taint with key &#8216;dedicated&#8217; and effect &#8216;NoSchedule&#8217; if one exists.<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">\n$ kubectl taint nodes node1 dedicated:NoSchedule-\n$ kubectl taint nodes ip-172-31-24-84.ap-south-1.compute.internal node-role.kubernetes.io\/master:NoSchedule-<\/code><\/span><\/pre>\n\n\n<p><strong>Adding \/ Inspecting \/ Removing a taint to an existing node using PreferNoSchedule<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-3\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">\nAdd a taint with key <span class=\"hljs-string\">'dedicated'<\/span> on nodes having label mylabel=X\n$ kubectl taint node -l myLabel=X  dedicated=foo:PreferNoSchedule\n$ kubectl taint node node1 dedicated=foo:PreferNoSchedule\n\nInspecting a node<span class=\"hljs-string\">'s taints\n$ kubectl describe nodes node1 | grep -i taint\n$ kubectl run nginx --image=nginx --replicas=8\n$ kubectl get pod -o wide\n\nRemoving a taint from a node\n$ kubectl taint nodes node1 dedicated:PreferNoSchedule-<\/span><\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-3\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Adding \/ Inspecting \/ Removing a taint to an existing node using NoExecute<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-4\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">\n$ kubectl taint nodes node2 node2=DoNotSchedulePods:NoExecute\n\nInspecting a node<span class=\"hljs-string\">'s taints\n$ kubectl describe nodes node2 | grep Taint\n# Because of that, all the pods on node2 are terminated and created on node3.\n$ kubectl run nginx --image=nginx --replicas=8\n$ kubectl get pod -o wide\n\nRemoving a taint from a node\n$ kubectl taint nodes node2 node2:NoExecute-\n$ kubectl describe nodes node2 | grep Taint\n\n# Add to node '<\/span>foo<span class=\"hljs-string\">' a taint with key '<\/span>bar<span class=\"hljs-string\">' and no value\n$ kubectl taint nodes foo bar:NoSchedule\n\nLet\u2019s verify the status of Taints on Master Node.\n$ kubectl describe nodes node1 | grep Taints\nTaints:             node-role.kubernetes.io\/master:NoSchedule<\/span><\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-4\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Untaint the node and verify as below:<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-5\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\">\n$ kubectl taint nodes --all node-role.kubernetes.io\/master-\n$ kubectl describe nodes node1 | grep -i taint\n\nRun Test deployment <span class=\"hljs-keyword\">in<\/span> Untaint Node\n$ kubectl run testsvr --image=nginx --replicas=<span class=\"hljs-number\">7<\/span>\n$ kubectl <span class=\"hljs-keyword\">get<\/span> pods -o wide | grep testsvr<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-5\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Untaint the setting on node2<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">\n$ kubectl taint nodes node3 node3=DoNotSchedulePods:NoSchedule\n$ kubectl describe nodes node3 | grep Taint\n$ kubectl run nginx --image=nginx --replicas=8<\/code><\/span><\/pre>\n\n\n<p>By default, kubernetes cluster will not schedule pods on the master node for security reasons. But if we would like to be able to schedule pods on the master node, e.g: for a single-node kubernetes cluster for testing and development purposes, we can run following commands.<\/p>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">$ kubectl taint nodes ip-172-31-24-84.ap-south-1.compute.internal node-role.kubernetes.io\/master:NoSchedule-<\/code><\/span><\/pre>\n\n<div class=\"epyt-gallery\" data-currpage=\"1\" id=\"epyt_gallery_86932\"><figure class=\"wp-block-embed wp-block-embed-youtube is-type-video is-provider-youtube epyt-figure\"><div class=\"wp-block-embed__wrapper\"><iframe loading=\"lazy\"  id=\"_ytid_44141\"  width=\"760\" height=\"427\"  data-origwidth=\"760\" data-origheight=\"427\" src=\"https:\/\/www.youtube.com\/embed\/?enablejsapi=1&#038;autoplay=0&#038;cc_load_policy=0&#038;cc_lang_pref=&#038;iv_load_policy=1&#038;loop=0&#038;rel=1&#038;fs=1&#038;playsinline=0&#038;autohide=2&#038;theme=dark&#038;color=red&#038;controls=1&#038;disablekb=0&#038;\" class=\"__youtube_prefs__  no-lazyload\" title=\"YouTube player\"  data-epytgalleryid=\"epyt_gallery_86932\"  allow=\"fullscreen; accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen data-no-lazy=\"1\" data-skipgform_ajax_framebjll=\"\"><\/iframe><\/div><\/figure><div class=\"epyt-gallery-list\"><div>Sorry, there was a YouTube error.<\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>What is meaning of taint? a trace of a bad or undesirable substance or quality. What is tainted? spoiled; damaged in quality, taste, or value: Follwing are workload which run in a cluster&#8217;s node\u2026. DaemonSet Deployment Pod ReplicaSet ReplicationController StatefulSet When you submit a workload, The scheduler determines where to place the Pods associated with&#8230;<\/p>\n","protected":false},"author":1,"featured_media":8008,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_joinchat":[],"footnotes":""},"categories":[4859],"tags":[5591,4860,1543,5543,5590],"class_list":["post-8005","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-kubernetes","tag-deep","tag-kubernetes","tag-node","tag-pod","tag-taint"],"_links":{"self":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/8005","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=8005"}],"version-history":[{"count":8,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/8005\/revisions"}],"predecessor-version":[{"id":25285,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/8005\/revisions\/25285"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/media\/8008"}],"wp:attachment":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=8005"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=8005"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=8005"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}