keepalived 负载均衡 ingress-nginx

官方原始文件使用的是 deployment，replicate 为 1，这样将会在某一台节点上启动对应的 nginx-ingress-controller pod。外部流量访问至该节点，由该节点负载分担至内部的service。考虑到单点故障的问题，改为 DaemonSet，配合亲和性部署在指定节点上启动 nginx-ingress-controller pod，确保有多个节点启动nginx-ingress-controller pod，生产环境中建议 ingress 节点打上污点不允许业务pod进行调度，以避免业务应用与 Ingress 服务发生资源争抢。后续将这些节点加入到外部硬件负载均衡组实现高可用性。

部署 Ingress-nginx (此处略)

参考文章: Kubernetes-Ingress

部署 keepalived

1. 分别在 k8s-node01 节点以及 k8s-node02 节点安装 keepalived

   yum install -y keepalived

2. 配置 keepalived 服务配置文件 /etc/keepalived/keepalived.conf，每个节点上除了 priority 的值不同，其他都一样

   mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived_ori.conf
   
   export INTERFACE=$(ip route show |grep default |cut -d ' ' -f5)
   export IPADDR=$(ifconfig |grep -A1 $INTERFACE |grep inet |awk '{print $2}')
   
   cat > /etc/keepalived/keepalived.conf <<EOF
   ! Configuration File for keepalived
   
   global_defs {
       notification_email {
         root@localhost
       }
       notification_email_from kaadmin@localhost
       smtp_server 127.0.0.1
       smtp_connect_timeout 30
       router_id LVS_DEVEL
   }
   
   vrrp_script chk_nginxserver {
       script "/etc/keepalived/check_nginx.sh"
       interval    2
       weight      -5
       fall    3
       rise    2
   }
   
   vrrp_instance VI_2 {
       state MASTER
       interface ${INTERFACE}
       mcast_src_ip ${IPADDR}
       virtual_router_id 27
       priority 100
       advert_int 1
       authentication {
           auth_type PASS
           auth_pass 1111
       }
       virtual_ipaddress {
           10.10.246.27
       }
       track_script {
           chk_nginxserver
       }
   }
   
   virtual_server 10.10.246.27 80{
       delay_loop 6
       lb_algo loadbalance
       lb_kind DR
       nat_mask 255.255.255.0
       persistence_timeout 0
       protocol TCP
   
       real_server 10.10.246.39 80{
           weight 1
           TCP_CHECK {
           connect_timeout 3
           }
       }
       real_server 10.10.246.40 80{
           weight 1
           TCP_CHECK {
           connect_timeout 3
           }
       }
   }
   virtual_server 10.10.246.27 443{
       delay_loop 6
       lb_algo loadbalance
       lb_kind DR
       nat_mask 255.255.255.0
       persistence_timeout 0
       protocol TCP
   
       real_server 10.10.246.39 443{
           weight 1
           TCP_CHECK {
           connect_timeout 3
           }
       }
       real_server 10.10.246.40 443{
           weight 1
           TCP_CHECK {
           connect_timeout 3
           }
       }
   }
   EOF

3. 创建服务状态检测脚本 /etc/keepalived/check_nginx.sh

   $ cat > /etc/keepalived/check_nginx.sh <<EOF
   #/bin/bash
   
   err=0
   for k in \$(seq 1 3)
   do
       check_code=\$(pgrep nginx)
       if [[ \$check_code == "" ]]; then
           err=\$(expr \$err + 1)
           sleep 1
           continue
       else
           err=0
           break
       fi
   done
   
   if [[ \$err != "0" ]]; then
       echo "systemctl stop keepalived"
       /usr/bin/systemctl stop keepalived
       exit 1
   else
       exit 0
   fi
   EOF
   
   chmod +x /etc/keepalived/check_nginx.sh 

4. 启动 keepalived 服务

   systemctl enable --now keepalived