Deploy kubernetes cluster on 2 or more servers ; k8s on-premise

DRAFT

https://dzone.com/articles/kubespray-10-simple-steps-for-installing-a-product

get all yamls backup kubectl get po,deployment,rc,rs,ds,no,job -o yaml?

PLEAE note that swap should be disable on the k8s nodes. This should be done during install with ansible kubespray, however remember to check it in case of issues.

all actions under root.

make servers able to communicate cvia ssh with keys ,m like:

on master as root

ssh-keygen -t rsa -b 4096
ssh-copy-id root@<slaveIP>

on slave as root

ssh-keygen -t rsa -b 4096 
ssh-copy-id root@<masterIP>

On master rememebr to add /root/.ssh/id_rsa.pub content into /root/.ssh/authorized_keys this will allow to connect master on himself, yeah this need if we start ansible from master

master+slabve: yum upgrade disable selinux:

 setenforce 0
 sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

stop and disable firewaal

systemctl stop firewalld
systemctl disable firewalld

ON master:

 #modprobe br_netfilter
 #echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
 #sysctl -w net.ipv4.ip_forward=1

On Slave:

# echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
# sysctl -w net.ipv4.ip_forward=1

next slave and master again:

 sudo yum install epel-release
 sudo yum install ansible

  dnf install python3

//// easy_install pip

pip3 install jinja2 --upgrade

on master install kubespay

git clone https://github.com/kubernetes-incubator/kubespray.git

cd kubespray
sudo pip install -r requirements.txt

Copy inventory/sample as inventory/ndicluster (change “ndisluter” to any name you want for the cluster)

cp -avr inventory/sample  inventory/ndicluster

Update the Ansible inventory file with inventory builder

declare -a IPS=( 213.108.199.13* 213.108.199.14*)
CONFIG_FILE=inventory/ndicluster/hosts.ini python3 contrib/inventory_builder/inventory.py ${IPS[@]}

check the hosts.ini ... it may be generation in differ variants but I;'ve put my to the next view:

[all]
node 1 ansible_host=213.108.199.13*  ip=213.108.199.13*  
node2 ansible_host=213.108.199.14*  ip=213.108.199.14*  

[kube-master]
node1 

[kube-node]
node1#should not be there!??
node2 

[etcd]
node1 

[k8s-cluster:children]
kube-master
kube-node 

[calico-rr]

[vault] #probably add!??
node1
node2

BETTER USE CENTOS7 !

ansible-playbook -i inventory/mycluster/hosts.ini cluster.yml

check

kubectl get pods -A

NOTE: if you like to reinstall you cluster you may do it with reset.yaml:

ansible-playbook -i inventory/ndicluster/hosts.ini reset.yml 

and install back after

---dashoboard----

references: #https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md

           #https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/

Insecure way but will allow you to access dashboard quickly w/o proxy: allow dashboard access

kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous

create user

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kube-system
EOF

apply permissions

cat <<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system
EOF

get token:

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')

Login with token:

https://213.108.19*.*:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#/overview?namespace=default

---HELM---

we will install helm exactly on our master node; but you may easily confifgure the kubectl and helm work from your PC

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

Init

helm init

By default RBAC enabled so you need to add proper permissions to helm tiller

kubectl create serviceaccount tiller --namespace kube-system

Create tiller-perm.yaml:

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: tiller-clusterrolebinding
subjects:
- kind: ServiceAccount
  name: tiller
  namespace: kube-system
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: ""

Apply

kubectl create -f tiller-perm.yaml 
 clusterrolebinding.rbac.authorization.k8s.io/tiller-clusterrolebinding created

Update tiller deployment:

helm init --service-account tiller --upgrade

wait few mins, test:

helm ls #should be no errors

Example install elasticsearch:

• optional you may like to swicth current context to some another for example:

 kubectl config set-context --current --namespace=jx-production

Add repo

helm repo add elastic https://helm.elastic.co
"elastic" has been added to your repositories

#optional: you may donwload helm chart and modify iot before apply or apply default from repo. whatever
#curl -O https://raw.githubusercontent.com/elastic/Helm-charts/master/elasticsearch/examples/minikube/values.yaml
#helm install --name elasticsearch elastic/elasticsearch -f ./values.yaml 

helm install --name elasticsearch elastic/elasticsearch