Deploy kubernetes cluster on 2 or more servers ; k8s on-premise
DRAFT
https://dzone.com/articles/kubespray-10-simple-steps-for-installing-a-product
get all yamls backup kubectl get po,deployment,rc,rs,ds,no,job -o yaml?
PLEAE note that swap should be disable on the k8s nodes. This should be done during install with ansible kubespray, however remember to check it in case of issues.
all actions under root.
make servers able to communicate cvia ssh with keys ,m like:
on master as root
ssh-keygen -t rsa -b 4096 ssh-copy-id root@<slaveIP>
on slave as root
ssh-keygen -t rsa -b 4096 ssh-copy-id root@<masterIP>
On master rememebr to add /root/.ssh/id_rsa.pub content into /root/.ssh/authorized_keys this will allow to connect master on himself, yeah this need if we start ansible from master
master+slabve:
yum upgrade
disable selinux:
setenforce 0 sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
stop and disable firewaal
systemctl stop firewalld systemctl disable firewalld
ON master:
#modprobe br_netfilter #echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables #sysctl -w net.ipv4.ip_forward=1
On Slave:
# echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables # sysctl -w net.ipv4.ip_forward=1
next slave and master again:
sudo yum install epel-release sudo yum install ansible
dnf install python3
//// easy_install pip
pip3 install jinja2 --upgrade
on master install kubespay
git clone https://github.com/kubernetes-incubator/kubespray.git
cd kubespray sudo pip install -r requirements.txt
Copy inventory/sample as inventory/ndicluster (change “ndisluter” to any name you want for the cluster)
cp -avr inventory/sample inventory/ndicluster
Update the Ansible inventory file with inventory builder
declare -a IPS=( 213.108.199.13* 213.108.199.14*) CONFIG_FILE=inventory/ndicluster/hosts.ini python3 contrib/inventory_builder/inventory.py ${IPS[@]}
check the hosts.ini ... it may be generation in differ variants but I;'ve put my to the next view:
[all] node 1 ansible_host=213.108.199.13* ip=213.108.199.13* node2 ansible_host=213.108.199.14* ip=213.108.199.14* [kube-master] node1 [kube-node] node1#should not be there!?? node2 [etcd] node1 [k8s-cluster:children] kube-master kube-node [calico-rr] [vault] #probably add!?? node1 node2
BETTER USE CENTOS7 !
ansible-playbook -i inventory/mycluster/hosts.ini cluster.yml
check
kubectl get pods -A
NOTE: if you like to reinstall you cluster you may do it with reset.yaml:
ansible-playbook -i inventory/ndicluster/hosts.ini reset.yml
and install back after
---dashoboard----
references: #https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md
#https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
Insecure way but will allow you to access dashboard quickly w/o proxy: allow dashboard access
kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous
create user
cat <<EOF | kubectl apply -f - apiVersion: v1 kind: ServiceAccount metadata: name: admin-user namespace: kube-system EOF
apply permissions
cat <<EOF | kubectl apply -f - apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: admin-user roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: admin-user namespace: kube-system EOF
get token:
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
Login with token:
https://213.108.19*.*:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#/overview?namespace=default
---HELM---
we will install helm exactly on our master node; but you may easily confifgure the kubectl and helm work from your PC
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 chmod 700 get_helm.sh ./get_helm.sh
Init
helm init
By default RBAC enabled so you need to add proper permissions to helm tiller
kubectl create serviceaccount tiller --namespace kube-system
Create tiller-perm.yaml:
kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: tiller-clusterrolebinding subjects: - kind: ServiceAccount name: tiller namespace: kube-system roleRef: kind: ClusterRole name: cluster-admin apiGroup: ""
Apply
kubectl create -f tiller-perm.yaml clusterrolebinding.rbac.authorization.k8s.io/tiller-clusterrolebinding created
Update tiller deployment:
helm init --service-account tiller --upgrade
wait few mins, test:
helm ls #should be no errors
Example install elasticsearch:
- optional you may like to swicth current context to some another for example:
kubectl config set-context --current --namespace=jx-production
Add repo
helm repo add elastic https://helm.elastic.co "elastic" has been added to your repositories #optional: you may donwload helm chart and modify iot before apply or apply default from repo. whatever #curl -O https://raw.githubusercontent.com/elastic/Helm-charts/master/elasticsearch/examples/minikube/values.yaml #helm install --name elasticsearch elastic/elasticsearch -f ./values.yaml
helm install --name elasticsearch elastic/elasticsearch
""