Kind 简介

  • Kind 是 Kubernetes In Docker 的缩写，顾名思义是使用 Docker 容器作为 Node 并将 Kubernetes 部署至其中的一个工具。
  • Kind 可以作为一种本地集群搭建的工具进
  • 由于便于快速搭建环境，经常被用来做 k8s 相关测试准备

流程

  1. 构建环境
  2. 测试

手动搭建单节点环境测试

  • 关闭 kind 自带的 kindnet 网络插件,并将其设置为 Calico 的默认子网，编写一个 kind-calico.yaml 文件
kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
networking:
  disableDefaultCNI: true # disable kindnet
  podSubnet: 192.168.0.0/16 # set to Calico's default subnet
  • 使用该文件创建 不带网络插件的集群
[root@k8s-master01 kind]# kind create cluster --config kind-calico.yaml
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.16.3) 🖼
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️️ 
 ✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Thanks for using kind! 😊
  • 导入 calico 镜像 并部署
# 将calico 镜像打包
docker save calico/kube-controllers:v3.10.1 calico/node:v3.10.1 calico/pod2daemon-flexvol:v3.10.1  calico/cni:v3.10.1 -o calico.tar

# 导入 calico 相关镜像
kind load image-archive calico.tar

# 部署 calico 网络
wget https://docs.projectcalico.org/v3.10/manifests/calico.yaml
kubectl create -f calico.yaml

看到 calico 相关的所有 pod 正常启动运行

NAMESPACE        NAME                                         READY   STATUS             RESTARTS   AGE   IP             NODE                 NOMINATED NODE   READINESS GATES
kube-system      calico-kube-controllers-6b64bcd855-xmvfb     1/1     Running            0          22m   192.168.82.1   kind-control-plane   <none>           <none>
kube-system      calico-node-5lk9m                            1/1     Running            0          22m   172.17.0.2     kind-control-plane   <none>           <none>
kube-system      coredns-5644d7b6d9-6l2w5                     1/1     Running            0          28m   192.168.82.2   kind-control-plane   <none>           <none>
kube-system      coredns-5644d7b6d9-rzqfn                     1/1     Running            0          28m   192.168.82.3   kind-control-plane   <none>           <none>
kube-system      etcd-kind-control-plane                      1/1     Running            0          27m   172.17.0.2     kind-control-plane   <none>           <none>
kube-system      kube-apiserver-kind-control-plane            1/1     Running            0          27m   172.17.0.2     kind-control-plane   <none>           <none>
kube-system      kube-controller-manager-kind-control-plane   1/1     Running            0          27m   172.17.0.2     kind-control-plane   <none>           <none>
kube-system      kube-proxy-srbjr                             1/1     Running            0          28m   172.17.0.2     kind-control-plane   <none>           <none>
kube-system      kube-scheduler-kind-control-plane            1/1     Running            0          27m   172.17.0.2     kind-control-plane   <none>           <none>
  • 导入 e2e 测试所需镜像
# server 镜像
kind load docker-image beyond.io:5000/kubernetes-e2e-test-images/agnhost:2.6

# client 镜像
kind load docker-image beyond.io:5000/busybox:1.29
  • 使用准备好的 e2e 二进制进行测试
./e2e.test --ginkgo.focus="NetworkPolicy"

手动搭建多节点环境测试

  • 多节点配置 muilt-nodes.yaml
kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
nodes:
- role: control-plane
- role: worker
- role: worker
networking:
  disableDefaultCNI: true
  serviceSubnet: "10.96.0.1/12"
  podSubnet: "192.168.0.0/16"

后续安装 calico 及 测试和单节点一致

构建脚本测试

脚本 模板

#!/usr/bin/env bash

NAME="test-cluster"

echo "+++ Cleaning up old test artifacts"
kind delete cluster --name "${NAME}" > /dev/null 2>&1 || true
rm sample-controller > /dev/null 2>&1 || true

echo "+++ Creating test cluster"
kind create cluster --name "${NAME}" --config config.yaml
export KUBECONFIG="$(kind get kubeconfig-path --name="{$NAME}")"

echo "+++ Building sample-controller"
CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -o sample-controller

echo "+++ Building Docker image"
docker build -t sample-controller:test .

echo "+++ Loading image into kind container"
kind load docker-image --name "{$NAME}" sample-controller:test

echo "+++ Deploying sample-controller"
kubectl apply -f sample-manifest.yaml

echo "+++ Deploying example Foo resource"
kubectl apply -f example-foo.yaml

高可用配置

kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
kubeadmConfigPatches:
- |
  apiVersion: kubeadm.k8s.io/v1beta1
  kind: ClusterConfiguration
  metadata:
    name: config
  networking:
    serviceSubnet: 10.0.0.0/16
  imageRepository: registry.aliyuncs.com/google_containers
  nodeRegistration:
    kubeletExtraArgs:
      pod-infra-container-image: registry.aliyuncs.com/google_containers/pause:3.1  
- |
  apiVersion: kubeadm.k8s.io/v1beta1
  kind: InitConfiguration
  metadata:
    name: config
  networking:
    serviceSubnet: 10.0.0.0/16
  imageRepository: registry.aliyuncs.com/google_containers  
nodes:
- role: control-plane
- role: control-plane
- role: control-plane
- role: worker
- role: worker
- role: worker