在AWS使用EKS中

　　本文链接地址:  在AWS使用EKS - 慢慢的回味
　　使用AWS的EKS来托管Kubernetes是比较复杂，按照如下的方法可以创建出一个满足大部分使用环境的EKS。
　　Content: 5 设置EKS的存储EFS5.1 创建接入EFS的策略(Root用户操作)5.2 创建访问EFS的角色(Root用户操作)5.3 为OpenID Connect创建Identity Provider(Root用户操作)5.4 在EKS中创建服务账户(IAM用户)5.5 创建EFS CSI 插件(IAM用户)5.6 创建EFS文件系统(Root用户操作)5.7 创建Kubernetes里面的存储类(IAM用户) 6 部署Jenkins来测试(IAM用户)6.1 部署Jenkins6.2 验证结果
　　5 设置EKS的存储EFS5.1 创建接入EFS的策略(Root用户操作)
　　自定义一策略：＂TestEKSAccessEFSPolicy＂ {     ＂Version＂: ＂2012-10-17＂,     ＂Statement＂: [         {             ＂Effect＂: ＂Allow＂,             ＂Action＂: [                 ＂elasticfilesystem:DescribeAccessPoints＂,                 ＂elasticfilesystem:DescribeFileSystems＂             ],             ＂Resource＂: ＂*＂         },         {             ＂Effect＂: ＂Allow＂,             ＂Action＂: [                 ＂elasticfilesystem:CreateAccessPoint＂             ],             ＂Resource＂: ＂*＂,             ＂Condition＂: {                 ＂StringLike＂: {                     ＂aws:RequestTag/efs.csi.aws.com/cluster＂: ＂true＂                 }             }         },         {             ＂Effect＂: ＂Allow＂,             ＂Action＂: ＂elasticfilesystem:DeleteAccessPoint＂,             ＂Resource＂: ＂*＂,             ＂Condition＂: {                 ＂StringEquals＂: {                     ＂aws:ResourceTag/efs.csi.aws.com/cluster＂: ＂true＂                 }             }         }     ] }5.2 创建访问EFS的角色(Root用户操作)
　　创建角色＂TestEKSAccessEFSRole＂并分配策略＂TestEKSAccessEFSPolicy＂。
　　在信任关系＂Trust relationships＂中，修改如下内容。
　　替换＂oidc.eks.us-east-1.amazonaws.com/id/98F61019E9B399FA9B7A43A19B56DF14″为你EKS的＂OpenID Connect provider URL＂。
　　{     ＂Version＂: ＂2012-10-17＂,     ＂Statement＂: [         {             ＂Effect＂: ＂Allow＂,             ＂Principal＂: {                 ＂Federated＂: ＂arn:aws:iam::675892200046:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/98F61019E9B399FA9B7A43A19B56DF14＂             },             ＂Action＂: ＂sts:AssumeRoleWithWebIdentity＂,             ＂Condition＂: {                 ＂StringEquals＂: {                     ＂oidc.eks.us-east-1.amazonaws.com/id/98F61019E9B399FA9B7A43A19B56DF14:sub＂: ＂system:serviceaccount:kube-system:efs-csi-controller-sa＂                 }             }         }     ] }
　　5.3 为OpenID Connect创建Identity Provider(Root用户操作)
　　填入提供URL和审计URL ＂sts.amazonaws.com＂, 点击＂Get thumbprint＂, 然后单击＂Add provider＂。
　　5.4 在EKS中创建服务账户(IAM用户)
　　创建文件＂efs-service-account.yaml＂，包含如下内容，然后＂kubectl apply -f efs-service-account.yaml＂创建账户，注意修改account id。 apiVersion: v1 kind: ServiceAccount metadata:   name: efs-csi-controller-sa   namespace: kube-system   labels:     app.kubernetes.io/name: aws-efs-csi-driver   annotations:     eks.amazonaws.com/role-arn: arn:aws:iam::675892200046:role/TestEKSAccessEFSRole5.5 创建EFS CSI 插件(IAM用户)
　　执行如下命令获取EFS插件的安装yaml文件：driver.yaml kubectl kustomize ＂github.com/kubernetes-sigs/aws-efs-csi-driver/deploy/kubernetes/overlays/stable/ecr?ref=release-1.3＂ > driver.yaml
　　上面已经创建了服务账号，所以driver.yaml文件里面的＂efs-csi-controller-sa＂段可以去掉。
　　接着运行命令 ＂kubectl apply -f driver.yaml＂创建CSI插件。 apiVersion: v1 kind: ServiceAccount metadata:   name: efs-csi-controller-sa   namespace: kube-system   labels:     app.kubernetes.io/name: aws-efs-csi-driver   annotations:     eks.amazonaws.com/role-arn: arn:aws:iam::675892200046:role/TestEKSAccessEFSRole --- apiVersion: v1 kind: ServiceAccount metadata:   labels:     app.kubernetes.io/name: aws-efs-csi-driver   name: efs-csi-node-sa   namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata:   labels:     app.kubernetes.io/name: aws-efs-csi-driver   name: efs-csi-external-provisioner-role rules: - apiGroups:   - ＂＂   resources:   - persistentvolumes   verbs:   - get   - list   - watch   - create   - delete - apiGroups:   - ＂＂   resources:   - persistentvolumeclaims   verbs:   - get   - list   - watch   - update - apiGroups:   - storage.k8s.io   resources:   - storageclasses   verbs:   - get   - list   - watch - apiGroups:   - ＂＂   resources:   - events   verbs:   - list   - watch   - create   - patch - apiGroups:   - storage.k8s.io   resources:   - csinodes   verbs:   - get   - list   - watch - apiGroups:   - ＂＂   resources:   - nodes   verbs:   - get   - list   - watch - apiGroups:   - coordination.k8s.io   resources:   - leases   verbs:   - get   - watch   - list   - delete   - update   - create --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata:   labels:     app.kubernetes.io/name: aws-efs-csi-driver   name: efs-csi-provisioner-binding roleRef:   apiGroup: rbac.authorization.k8s.io   kind: ClusterRole   name: efs-csi-external-provisioner-role subjects: - kind: ServiceAccount   name: efs-csi-controller-sa   namespace: kube-system --- apiVersion: apps/v1 kind: Deployment metadata:   labels:     app.kubernetes.io/name: aws-efs-csi-driver   name: efs-csi-controller   namespace: kube-system spec:   replicas: 2   selector:     matchLabels:       app: efs-csi-controller       app.kubernetes.io/instance: kustomize       app.kubernetes.io/name: aws-efs-csi-driver   template:     metadata:       labels:         app: efs-csi-controller         app.kubernetes.io/instance: kustomize         app.kubernetes.io/name: aws-efs-csi-driver     spec:       containers:       - args:         - --endpoint=$(CSI_ENDPOINT)         - --logtostderr         - --v=2         - --delete-access-point-root-dir=false         env:         - name: CSI_ENDPOINT           value: unix:///var/lib/csi/sockets/pluginproxy/csi.sock         image: 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/aws-efs-csi-driver:v1.3.8         imagePullPolicy: IfNotPresent         livenessProbe:           failureThreshold: 5           httpGet:             path: /healthz             port: healthz           initialDelaySeconds: 10           periodSeconds: 10           timeoutSeconds: 3         name: efs-plugin         ports:         - containerPort: 9909           name: healthz           protocol: TCP         securityContext:           privileged: true         volumeMounts:         - mountPath: /var/lib/csi/sockets/pluginproxy/           name: socket-dir       - args:         - --csi-address=$(ADDRESS)         - --v=2         - --feature-gates=Topology=true         - --extra-create-metadata         - --leader-election         env:         - name: ADDRESS           value: /var/lib/csi/sockets/pluginproxy/csi.sock         image: 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/csi-provisioner:v2.1.1         imagePullPolicy: IfNotPresent         name: csi-provisioner         volumeMounts:         - mountPath: /var/lib/csi/sockets/pluginproxy/           name: socket-dir       - args:         - --csi-address=/csi/csi.sock         - --health-port=9909         image: 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/livenessprobe:v2.2.0         imagePullPolicy: IfNotPresent         name: liveness-probe         volumeMounts:         - mountPath: /csi           name: socket-dir       hostNetwork: true       nodeSelector:         kubernetes.io/os: linux       priorityClassName: system-cluster-critical       priorityClassName: system-cluster-critical       serviceAccountName: efs-csi-controller-sa       volumes:       - emptyDir: {}         name: socket-dir --- apiVersion: apps/v1 kind: DaemonSet metadata:   labels:     app.kubernetes.io/name: aws-efs-csi-driver   name: efs-csi-node   namespace: kube-system spec:   selector:     matchLabels:       app: efs-csi-node       app.kubernetes.io/instance: kustomize       app.kubernetes.io/name: aws-efs-csi-driver   template:     metadata:       labels:         app: efs-csi-node         app.kubernetes.io/instance: kustomize         app.kubernetes.io/name: aws-efs-csi-driver     spec:       affinity:         nodeAffinity:           requiredDuringSchedulingIgnoredDuringExecution:             nodeSelectorTerms:             - matchExpressions:               - key: eks.amazonaws.com/compute-type                 operator: NotIn                 values:                 - fargate       containers:       - args:         - --endpoint=$(CSI_ENDPOINT)         - --logtostderr         - --v=2         env:         - name: CSI_ENDPOINT           value: unix:/csi/csi.sock         image: 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/aws-efs-csi-driver:v1.3.8         imagePullPolicy: IfNotPresent         livenessProbe:           failureThreshold: 5           httpGet:             path: /healthz             port: healthz           initialDelaySeconds: 10           periodSeconds: 2           timeoutSeconds: 3         name: efs-plugin         ports:         - containerPort: 9809           name: healthz           protocol: TCP         securityContext:           privileged: true         volumeMounts:         - mountPath: /var/lib/kubelet           mountPropagation: Bidirectional           name: kubelet-dir         - mountPath: /csi           name: plugin-dir         - mountPath: /var/run/efs           name: efs-state-dir         - mountPath: /var/amazon/efs           name: efs-utils-config         - mountPath: /etc/amazon/efs-legacy           name: efs-utils-config-legacy       - args:         - --csi-address=$(ADDRESS)         - --kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)         - --v=2         env:         - name: ADDRESS           value: /csi/csi.sock         - name: DRIVER_REG_SOCK_PATH           value: /var/lib/kubelet/plugins/efs.csi.aws.com/csi.sock         - name: KUBE_NODE_NAME           valueFrom:             fieldRef:               fieldPath: spec.nodeName         image: 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/csi-node-driver-registrar:v2.1.0         imagePullPolicy: IfNotPresent         name: csi-driver-registrar         volumeMounts:         - mountPath: /csi           name: plugin-dir         - mountPath: /registration           name: registration-dir       - args:         - --csi-address=/csi/csi.sock         - --health-port=9809         - --v=2         image: 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/livenessprobe:v2.2.0         imagePullPolicy: IfNotPresent         name: liveness-probe         volumeMounts:         - mountPath: /csi           name: plugin-dir       dnsPolicy: ClusterFirst       hostNetwork: true       nodeSelector:         beta.kubernetes.io/os: linux       priorityClassName: system-node-critical       serviceAccountName: efs-csi-node-sa       tolerations:       - operator: Exists       volumes:       - hostPath:           path: /var/lib/kubelet           type: Directory         name: kubelet-dir       - hostPath:           path: /var/lib/kubelet/plugins/efs.csi.aws.com/           type: DirectoryOrCreate         name: plugin-dir       - hostPath:           path: /var/lib/kubelet/plugins_registry/           type: Directory         name: registration-dir       - hostPath:           path: /var/run/efs           type: DirectoryOrCreate         name: efs-state-dir       - hostPath:           path: /var/amazon/efs           type: DirectoryOrCreate         name: efs-utils-config       - hostPath:           path: /etc/amazon/efs           type: DirectoryOrCreate         name: efs-utils-config-legacy --- apiVersion: storage.k8s.io/v1 kind: CSIDriver metadata:   annotations:     helm.sh/hook: pre-install, pre-upgrade     helm.sh/hook-delete-policy: before-hook-creation     helm.sh/resource-policy: keep   name: efs.csi.aws.com spec:   attachRequired: false
　　等一会，＂efs-csi-controller*＂应该就绪了。
　　5.6 创建EFS文件系统(Root用户操作)
　　在Amazon EFS产品中，点击＂Create file system＂开始创建：
　　选择＂Standard＂作为存储类，这样可用区里面的所有节点都可以访问。
　　创建完成后，等待＂Network＂可用，然后点击＂Manage＂按钮添加集群安全组。
　　5.7 创建Kubernetes里面的存储类(IAM用户)
　　安装如下内容创建＂storageclass.yaml＂，并运行＂kubectl apply -f storageclass.yaml＂来创建。
　　注意修改＂fileSystemId＂成你自己的，通过如下图查询。
　　kind: StorageClass apiVersion: storage.k8s.io/v1 metadata:   name: efs-sc provisioner: efs.csi.aws.com parameters:   provisioningMode: efs-ap   fileSystemId: fs-04470c1ed1eab275c   directoryPerms: ＂700＂   gidRangeStart: ＂1000＂ # optional   gidRangeEnd: ＂2000＂ # optional   basePath: ＂/dynamic_provisioning＂ # optional6 部署Jenkins来测试(IAM用户)6.1 部署Jenkins
　　注意设置存储类为efs-sc。
　　helm repo add jenkinsci https://charts.jenkins.io/
　　helm install my-jenkins jenkinsci/jenkins –version 4.1.17 –set persistence.storageClass=efs-sc 6.2 验证结果
　　等Jenkins启动后，可以采用端口转发来临时访问。 [awscli@bogon ~]$ kubectl port-forward svc/my-jenkins --address=0.0.0.0 8081:8080 Forwarding from 0.0.0.0:8081 -> 8080 Handling connection for 8081 Handling connection for 8081 Handling connection for 8081 Handling connection for 8081 Handling connection for 8081 Handling connection for 8081 Handling connection for 8081 Handling connection for 8081 Handling connection for 8081 Handling connection for 8081 Handling connection for 8081 Handling connection for 8081 Handling connection for 8081