Installing K8S

                                  Installing K8S

Follow below process to install K8S on all nodes for K8S cluster.

 1) Install docker and start on all the node in cluster .

yum install -y docker

systemctl enable docker && systemctl start docker

2)  Setup repo for K8S

[root@tssperf10 ~]# cat /etc/yum.repos.d/kubernetes.repo

[kubernetes]

name=Kubernetes

baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64

enabled=1

gpgcheck=1

repo_gpgcheck=1

gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

[root@tssperf10 ~]# 

Note:- Turn off SE Linux .

setenforce 0

3) Install kubernetes services and start kubelet services .

yum install -y kubelet kubeadm kubectl

systemctl enable kubelet && systemctl start kubelet

4) To avoid issues with traffic being routed incorrectly due to iptables add below configs to k8s.conf  and reload system config from files . 

[root@tssperf10 ~]# cat /etc/sysctl.d/k8s.conf

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

[root@tssperf10 ~]#

sysctl --system

5) Start Kubenetes master on first node .

[root@tssperf09 ~]# kubeadm init

[init] Using Kubernetes version: v1.9.2

[init] Using Authorization modes: [Node RBAC]

[preflight] Running pre-flight checks.

[WARNING FileExisting-crictl]: crictl not found in system path

[certificates] Generated ca certificate and key.

[certificates] Generated apiserver certificate and key.

[certificates] apiserver serving cert is signed for DNS names [tssperf09.lab kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.10.72.249]

[certificates] Generated apiserver-kubelet-client certificate and key.

[certificates] Generated sa key and public key.

[certificates] Generated front-proxy-ca certificate and key.

[certificates] Generated front-proxy-client certificate and key.

[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"

[kubeconfig] Wrote KubeConfig file to disk: "admin.conf"

[kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf"

[kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf"

[kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf"

[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"

[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"

[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"

[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"

[init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests".

[init] This might take a minute or longer if the control plane images have to be pulled.

[apiclient] All control plane components are healthy after 47.002868 seconds

[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace

[markmaster] Will mark node tssperf09.lab as master by adding a label and a taint

[markmaster] Master tssperf09.lab tainted and labelled with key/value: node-role.kubernetes.io/master=""

[bootstraptoken] Using token: f663f7.9e62cd14e10df69f

[bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials

[bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token

[bootstraptoken] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster

[bootstraptoken] Creating the "cluster-info" ConfigMap in the "kube-public" namespace

[addons] Applied essential addon: kube-dns

[addons] Applied essential addon: kube-proxy

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube

  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.

Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node

as root:

  kubeadm join --token f663f7.9e62cd14e10df69f 10.10.72.249:6443 --discovery-token-ca-cert-hash sha256:9b7f89467f5fcb50a9007ba2f44fd6b2f4f9d25d62a97ff0902e81e9da894ba5

[root@tssperf09 ~]#

Note :- For testing we will run below command . Just to export kubeconfig on the shell itself.

 export KUBECONFIG=/etc/kubernetes/admin.conf

6) Install a pod network for Pods to communicate with each other.

[root@tssperf09 ~]# kubectl apply -f https://docs.projectcalico.org/v2.6/getting-started/kubernetes/installation/hosted/kubeadm/1.6/calico.yaml

configmap "calico-config" created

daemonset "calico-etcd" created

service "calico-etcd" created

daemonset "calico-node" created

deployment "calico-kube-controllers" created

deployment "calico-policy-controller" created

clusterrolebinding "calico-cni-plugin" created

clusterrole "calico-cni-plugin" created

serviceaccount "calico-cni-plugin" created

clusterrolebinding "calico-kube-controllers" created

clusterrole "calico-kube-controllers" created

serviceaccount "calico-kube-controllers" created

[root@tssperf09 ~]#

7) Now we can see Kubernetes master is up

[root@tssperf09 ~]# kubectl cluster-info

Kubernetes master is running at https://10.10.72.249:6443

KubeDNS is running at https://10.10.72.249:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

[root@tssperf09 ~]# kubectl get all

NAME             TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE

svc/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   7m

[root@tssperf09 ~]# kubectl get nodes

NAME            STATUS    ROLES     AGE       VERSION

tssperf09.lab   Ready     master    7m        v1.9.2

[root@tssperf09 ~]# kubectl get pods --all-namespaces

NAMESPACE     NAME                                      READY     STATUS    RESTARTS   AGE

kube-system   calico-etcd-rvxmg                         1/1       Running   0          5m

kube-system   calico-kube-controllers-d554689d5-6j5b5   1/1       Running   0          5m

kube-system   calico-node-qrw75                         2/2       Running   0          5m

kube-system   etcd-tssperf09.lab                        1/1       Running   0          8m

kube-system   kube-apiserver-tssperf09.lab              1/1       Running   0          7m

kube-system   kube-controller-manager-tssperf09.lab     1/1       Running   0          8m

kube-system   kube-dns-6f4fd4bdf-jndmv                  3/3       Running   0          8m

kube-system   kube-proxy-2ztzj                          1/1       Running   0          8m

kube-system   kube-scheduler-tssperf09.lab              1/1       Running   0          7m

[root@tssperf09 ~]# kubectl taint nodes --all node-role.kubernetes.io/master-

node "tssperf09.lab" untainted

[root@tssperf09 ~]# 

8) Now allow other 2 nodes to join the Kubernetes cluster .

i)  [root@tssperf10 ~]# kubeadm join --token f663f7.9e62cd14e10df69f 10.10.72.249:6443 --discovery-token-ca-cert-hash sha256:9b7f89467f5fcb50a9007ba2f44fd6b2f4f9d25d62a97ff0902e81e9da894ba5

[preflight] Running pre-flight checks.

[WARNING FileExisting-crictl]: crictl not found in system path

[discovery] Trying to connect to API Server "10.10.72.249:6443"

[discovery] Created cluster-info discovery client, requesting info from "https://10.10.72.249:6443"

[discovery] Requesting info from "https://10.10.72.249:6443" again to validate TLS against the pinned public key

[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "10.10.72.249:6443"

[discovery] Successfully established connection with API Server "10.10.72.249:6443"

This node has joined the cluster:

* Certificate signing request was sent to master and a response

  was received.

* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the master to see this node join the cluster.

ii)  [root@tssperf11 ~]# kubeadm join --token f663f7.9e62cd14e10df69f 10.10.72.249:6443 --discovery-token-ca-cert-hash sha256:9b7f89467f5fcb50a9007ba2f44fd6b2f4f9d25d62a97ff0902e81e9da894ba5

[preflight] Running pre-flight checks.

[WARNING FileExisting-crictl]: crictl not found in system path

[discovery] Trying to connect to API Server "10.10.72.249:6443"

[discovery] Created cluster-info discovery client, requesting info from "https://10.10.72.249:6443"

[discovery] Requesting info from "https://10.10.72.249:6443" again to validate TLS against the pinned public key

[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "10.10.72.249:6443"

[discovery] Successfully established connection with API Server "10.10.72.249:6443"

This node has joined the cluster:

* Certificate signing request was sent to master and a response

  was received.

* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the master to see this node join the cluster.

[root@tssperf11 ~]# 

9 )  Now the Kubernetes cluster is up and it has one master and 2 worker nodes.

[root@tssperf09 ~]# kubectl get nodes

NAME            STATUS    ROLES     AGE       VERSION

tssperf09.lab   Ready     master    17m       v1.9.2

tssperf10.lab   Ready     <none>    2m        v1.9.2

tssperf11.lab   Ready     <none>    1m        v1.9.2

[root@tssperf09 ~]#