{"id":6265,"date":"2019-07-26T18:11:16","date_gmt":"2019-07-26T18:11:16","guid":{"rendered":"https:\/\/www.devopsschool.com\/blog\/?p=6265"},"modified":"2021-11-16T11:14:27","modified_gmt":"2021-11-16T11:14:27","slug":"setup-kubernetes-cluster-using-kubeadm-calico-and-rbac-in-ubuntu","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/blog\/setup-kubernetes-cluster-using-kubeadm-calico-and-rbac-in-ubuntu\/","title":{"rendered":"Setup Kubernetes Cluster using kubeadm, calico and rbac in Ubuntu"},"content":{"rendered":"\n

Prerequisite Setup<\/strong><\/h3>\n\n\n
#Setup <\/span>\n#   1. 3 VMs Ubuntu 16.04.5 or 18.04.1.0, 1 master, 2 nodes.<\/span>\n#   2. Static IPs on individual VMs<\/span>\n#   3. \/etc\/hosts hosts file includes name to IP mappings for VMs<\/span>\n#   4. Swap is disabled<\/span>\n#   5. Take snapshots prior to installations, this way you can install <\/span>\n#       and revert to snapshot if needed<\/span>\n\n#Disable swap, swapoff then edit your fstab removing any entry for swap partitions<\/span>\n#You can recover the space with fdisk. You may want to reboot to ensure your config is ok. <\/span>\nswapoff -a\nvi \/etc\/fstab\n\n#Add Google's apt repository gpg key<\/span>\ncurl -s https:\/\/packages.cloud.google.com\/apt\/doc\/apt-key.gpg | sudo apt-key add -<\/span>\n\n#Add the Kubernetes apt repository<\/span>\nsudo bash -c 'cat <<EOF >\/etc\/apt\/sources.list.d\/kubernetes.list\ndeb https:\/\/apt.kubernetes.io\/ kubernetes-xenial main\nEOF'<\/span>\n\n#Update the package list and use apt-cache to inspect versions available in the repository<\/span>\nsudo apt-get update\napt-cache policy kubelet | head -n 20<\/span> \napt-cache policy docker.io | head -n 20<\/span> \n\n#Install the required packages, if needed we can request a specific version<\/span>\nsudo apt-get install -y docker.io kubelet kubeadm kubectl\nsudo apt-mark hold docker.io kubelet kubeadm kubectl\n\n#Check the status of our kubelet and our container runtime, docker.<\/span>\n#The kubelet will enter a crashloop until it's joined. <\/span>\nsudo systemctl status kubelet.service \nsudo systemctl status docker.service \n\n#Ensure both are set to start when the system starts up.<\/span>\nsudo systemctl enable kubelet.service\nsudo systemctl enable docker.service<\/code><\/span>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
Master Node Setup aka Kubernetes Control Panel Setup<\/h3>\n\n\n
#Only on the master, download the yaml files for the pod network<\/span>\nwget https:\/\/docs.projectcalico.org\/v3.3\/getting-started\/kubernetes\/installation\/hosted\/rbac-kdd.yaml<\/span>\nwget https:\/\/docs.projectcalico.org\/v3.3\/getting-started\/kubernetes\/installation\/hosted\/kubernetes-datastore\/calico-networking\/1.7\/calico.yaml<\/span>\n\n#Look inside calico.yaml and find the network range, adjust if needed.<\/span>\nvi calico.yaml\n\n#Create our kubernetes cluster, specifying a pod network range matching that in calico.yaml!<\/span>\nsudo kubeadm init --pod-network-cidr=192.168<\/span>.0<\/span>.0<\/span>\/16<\/span>\n\n#Configure our account on the master to have admin access to the API server from a non-privileged account.<\/span>\nmkdir -p $HOME\/.kube\nsudo cp -i \/etc\/kubernetes\/admin.conf $HOME\/.kube\/config\nsudo chown $(id -u):$(id -g) $HOME\/.kube\/config\n\n#Download yaml files for your pod network<\/span>\nkubectl apply -f rbac-kdd.yaml\nkubectl apply -f calico.yaml\n\n#Look for the all the system pods and calico pod to change to Running. <\/span>\n#The DNS pod won't start until the Pod network is deployed and Running.<\/span>\nkubectl get pods --all-namespaces\n\n#Gives you output over time, rather than repainting the screen on each iteration.<\/span>\nkubectl get pods --all-namespaces --watch\n\n#All system pods should be Running<\/span>\nkubectl get pods --all-namespaces\n\n#Get a list of our current nodes, just the master.<\/span>\nkubectl get nodes \n\n#Check out the systemd unit, and examine 10-kubeadm.conf<\/span>\n#Remeber the kubelet starts static pod manifests, and thus the core cluster pods<\/span>\nsudo systemctl status kubelet.service \n\n#check out the directory where the kubeconfig files live<\/span>\nls \/etc\/kubernetes\n\n#let's check out the manifests on the master<\/span>\nls \/etc\/kubernetes\/manifests\n\n#And look more closely at API server and etcd's manifest.<\/span>\nsudo more \/etc\/kubernetes\/manifests\/etcd.yaml\nsudo more \/etc\/kubernetes\/manifests\/kube-apiserver.yaml\n<\/code><\/span>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
Worker Node Setup <\/h3>\n\n\n
#For this demo ssh into c1-node1<\/span>\nssh aen@c1-node1\n\n#Disable swap, swapoff then edit your fstab removing any entry for swap partitions<\/span>\n#You can recover the space with fdisk. You may want to reboot to ensure your config is ok. <\/span>\nswapoff -a\nvi \/etc\/fstab\n\n#Add the Google's apt repository gpg key<\/span>\ncurl -s https:\/\/packages.cloud.google.com\/apt\/doc\/apt-key.gpg | sudo apt-key add -<\/span>\n\n#Add the kuberentes apt repository<\/span>\nsudo bash -c 'cat <<EOF >\/etc\/apt\/sources.list.d\/kubernetes.list\ndeb https:\/\/apt.kubernetes.io\/ kubernetes-xenial main\nEOF'<\/span>\n\n#Update the package list <\/span>\nsudo apt-get update\napt-cache policy kubelet | head -n 20<\/span> \n\n#Install the required packages, if needed we can request a specific version<\/span>\nsudo apt-get install -y docker.io kubelet kubeadm kubectl\nsudo apt-mark hold docker.io kubelet kubeadm kubectl\n\n#Check the status of our kubelet and our container runtime, docker.<\/span>\n#The kubelet will enter a crashloop until it's joined<\/span>\nsudo systemctl status kubelet.service \nsudo systemctl status docker.service \n\n#Ensure both are set to start when the system starts up.<\/span>\nsudo systemctl enable kubelet.service\nsudo systemctl enable docker.service\n\n#If you didn't keep the output, on the master, you can get the token.<\/span>\nkubeadm token list<\/span>\n\n#If you need to generate a new token, perhaps the old one timed out\/expired.<\/span>\nkubeadm token create\n\n#On the master, you can find the ca cert hash.<\/span>\nopenssl x509 -pubkey -in \/etc\/kubernetes\/pki\/ca.crt | openssl rsa -pubin -outform der 2<\/span>>\/dev\/null<\/span> | openssl dgst -sha256 -hex | sed 's\/^.* \/\/'<\/span>\n\n#Using the master (API Server) IP address or name, the token and the cert has, let's join this Node to our cluster.<\/span>\nsudo kubeadm join 172.16<\/span>.94<\/span>.10<\/span>:6443<\/span> \\\n    --token 9<\/span>woi9e.gmuuxnbzd8anltdg \\\n    --discovery-token-ca-cert-hash sha256:f9cb1e56fecaf9989b5e882f54bb4a27d56e1e92ef9d56ef19a6634b507d76a9\n\n#Back on master, this will say NotReady until the networking pod is created on the new node. Has to schedule the pod, then pull the container.<\/span>\nkubectl get nodes \n\n#On the master, watch for the calico pod and the kube-proxy to change to Running on the newly added nodes.<\/span>\nkubectl get pods --all-namespaces --watch\n\n#Still on the master, look for this added node's status as ready.<\/span>\nkubectl get nodes\n\n#GO BACK TO THE TOP AND DO THE SAME FOR c1-node2.<\/span>\n#Just SSH into c1-node2 and run the commands again.<\/span>\nssh aen@c1-node2\n#You can skip the token re-creation if you have one that's still valid.<\/span><\/code><\/span>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n