8.0 KiB
Install instructions for a 3-Node Kubernetes Cluster
Prerequisites
Minimal Installed Debian 12/11
2 CPU / vCPU, 2 GB RAM
20 GB free disk space
Sudo User with Admin rights
Stable Internet Connectivity
Ensure that each node can communicate with the others via a reliable network connection.
Lab Setup
For the demonstration, I am using three Debian 12 systems.
Master Node (vmcon-master.his.de) – 192.168.202.10
Worker Node 1 (vmcon-node1.his.de) – 192.168.202.11
Worker Node 2 (vmcon-node2.his.de) – 192.168.202.12
Installation
Set Host Name and Update Hosts File
Login to each node (master & woker nodes) and set their hostname using hostnamectl command.
$ hostnamectl set-hostname "vmcon-master.his.de" // Run on master node
$ hostnamectl set-hostname "vmcon-node1.his.de" // Run on 1st worker node
$ hostnamectl set-hostname "vmcon-node2.his.de" // Run on 2nd worker node
Also add the following entries in /etc/hosts file on all the nodes,
192.168.202.10 vmcon-master.his.de vmcon-master.his.de
192.168.202.11 vmcon-node1.his.de vmcon-node1
192.168.202.12 vmcon-node2.his.de vmcon-node2
Disable Swap on All Nodes
For kubelet to work smoothly, it is recommended to disable swap. Run following commands on master and worker nodes to turn off swap.
$ swapoff -a
$ sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
Add Firewall Rules for Kubernetes Cluster
In case, OS firewall is enabled on your Debian systems then allow following ports on master and worker nodes respectively.
On Master node, run
$ ufw allow 6443/tcp
$ ufw allow 2379/tcp
$ ufw allow 2380/tcp
$ ufw allow 10250/tcp
$ ufw allow 10251/tcp
$ ufw allow 10252/tcp
$ ufw allow 10255/tcp
$ ufw reload
On Worker Nodes,
$ ufw allow 10250/tcp
$ ufw allow 30000:32767/tcp
$ ufw reload
Note: If firewall is disabled on your Debian 12/11 systems, then you can skip this step.
Install Containerd Run time on All Nodes
Containerd is the industry standard container run time and supported by Kubernetes. So, install containerd on all master and worker nodes.
Before installing containerd, set the following kernel parameters on all the nodes.
$ cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf \
overlay \
br_netfilter \
EOF \
$ modprobe overlay \
$ modprobe br_netfilter \
$ cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-k8s.conf \
net.bridge.bridge-nf-call-iptables = 1 \
net.ipv4.ip_forward = 1 \
net.bridge.bridge-nf-call-ip6tables = 1 \
EOF
To make above changes into the effect, run
$ sysctl --system
Now, install conatinerd by running following apt command on all the nodes.
$ apt update
$ apt -y install containerd
Next, configure containerd so that it works with Kubernetes, run beneath command on all the nodes
$ containerd config default | sudo tee /etc/containerd/config.toml >/dev/null 2>&1
Edit the file ‘/etc/containerd/config.toml’ and look for the section ‘[plugins.”io.containerd.grpc.v1.cri”.containerd.runtimes.runc.options]’ and change ‘SystemdCgroup = false’ to ‘SystemdCgroup = true‘
$ nano /etc/containerd/config.toml
Save and exit the file.
Restart and enable containerd service on all the nodes,
$ systemctl restart containerd
$ systemctl enable containerd
Add Kubernetes Apt Repository
In Debian 12/11, Kubernetes related packages are not available in the default package repositories. We have to add additional Kubernetes apt repository on all the nodes, run
$ echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list
$ curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
Install Kubernetes Tools
Next, install the Kubernetes tools, including kubeadm, kubelet, and kubectl on all the nodes.
$ apt update
$ apt install kubelet kubeadm kubectl -y
$ apt-mark hold kubelet kubeadm kubectl
Install Kubernetes Cluster with Kubeadm
kubelet doesn’t appreciate the command-line options anymore (these are deprecated). Instead, I suggest to create a configuration file, say ‘kubelet.yaml’ with following content.
$ nano kubelet.yaml
apiVersion: kubeadm.k8s.io/v1beta3 \
kind: InitConfiguration \
--- \
apiVersion: kubeadm.k8s.io/v1beta3 \
kind: ClusterConfiguration \
kubernetesVersion: "1.28.0" # Replace with your desired version \
controlPlaneEndpoint: "k8s-master" \
--- \
apiVersion: kubelet.config.k8s.io/v1beta1 \
kind: KubeletConfiguration
Now, we are all set to initialize Kubernetes cluster, run following command only from master node,
$ kubeadm init --config kubelet.yaml
Above output confirms that control plane has been initialized successfully. In the output, we have commands for regular user for interacting with the cluster and also the command to join any worker node to this cluster.
To start interacting with cluster, run following commands on master node,
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
Run following kubectl command to get nodes and cluster information
$ kubectl get nodes \
$ kubectl cluster-info
On your worker nodes, join them to the cluster by running the command that was displayed when you initialized the master node. It will look something like this ‘Kubeadm join’
Note: Copy the exact command from the output of ‘kubeadm init’ command. In my case, following is the command
$ sudo kubeadm join k8s-master:6443 --token 21nm87.x1lgd4jf0lqiiiau \
--discovery-token-ca-cert-hash sha256:28b503f1f2a2592678724c482776f04b445c5f99d76915552f14e68a24b78009
Check the nodes status by running following command from master node,
$ kubectl get nodes
To make nodes status ready, we must install POD network addons like Calico or flannel.
Setup Pod Network Using Calico
On the master node, run beneath command to install calico,
$ kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/calico.yaml
Allow Calico ports in OS firewall, run beneath ufw commands on all the nodes,
$ sudo ufw allow 179/tcp \
$ sudo ufw allow 4789/udp \
$ sudo ufw allow 51820/udp \
$ sudo ufw allow 51821/udp \
$ sudo ufw reload \
Verify the status of Calico pods, run
$ kubectl get pods -n kube-system
Confirm, that master and worker nodes are in ready staate. Now, we can say our cluster ready handle workload.
Install MetalLB
$ kubectl create ns metallb-system
$ helm upgrade --install -n metallb-system metallb \
oci://registry-1.docker.io/bitnamicharts/metallb
Next create L2-range-allocation.yaml and define your IP range pool.
# L2-range-allocation.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: main-pool
namespace: metallb-system
spec:
addresses:
- 192.168.202.160/27
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: empty
namespace: metallb-system
Apply with
$ kubectl apply -f L2-range-allocation.yaml
Check if all speakers are running
$ kubectl get pods -n metallb-system
Note: https://itnext.io/kubernetes-loadbalancer-service-for-on-premises-6b7f75187be8
Test Kubernetes Cluster Installation
In order validate and test Kubernetes cluster installation, let’s try to deploy nginx based application via deployment. Run beneath commands,
$ kubectl create deployment nginx-app --image=nginx --replicas 2
$ kubectl expose deployment nginx-app --name=nginx-web-svc --type NodePort --port 80 --target-port 80
$ kubectl describe svc nginx-web-svc
Try to access the nginx based application using following curl command along with the nodeport 32283.
Note : In the curl command we can use either of worker node’s hostname.