资源管理核心概念
K8s 的设计理念-分层架构
- Kubernetes 架构：http://docs.kubernetes.org.cn/251.html
核心层：Kubernetes最核心的功能，对外提供API构建高层的应用，对内提供插件式应用执行环境
应用层：部署（无状态应用（无集群关系）、有状态应用（数据库主从 Rredis集群）、批处理任务、集群应用等）和路由（服务发现、DNS解析等）。有状态应用一般是跑在物理机上面
管理层：系统度量（如基础设施、容器和网络的度量），自动化（如自动扩展、动态Provision等）以及策略管理（RBAC、Quota、PSP、NetworkPolicy等）
接口层：kubectl命令行工具、客户端SDK以及集群联邦
生态系统：在接口层之上的庞大容器集群管理调度的生态系统，可以划分为两个范畴
1.Kubernetes 外部：日志、监控、配置管理、CI、CD、Workflow、FaaS、OTS应用、ChatOps等
2.Kubernetes 内部：CRI、CNI、CVI、镜像仓库、Cloud Provider、集群自身的配置和管理等K8s 的设计理念-API设计原则
Kubernetes 设计理念与分布式系统
- 分析和理解Kubernetes的设计理念可以使我们更深入地了解Kubernetes系统，更好地利用它管理分布式部署的云原生应用，另一方面也可以让我们借鉴其在分布式系统设计方面的经验。
API设计原则
-
对于云计算系统，系统API实际上处于系统设计的统领地位，正如本文前面所说，K8s集群系统每支持一项新功能，引入一项新技术，一定会新引入对应的API对象，支持对该功能的管理操作，理解掌握的API，就好比抓住了K8s系统的牛鼻子。K8s系统API的设计有以下几条原则：
-
所有API应该是声明式的。正如前文所说，声明式的操作，相对于命令式操作，对于重复操作的效果是稳定的，这对于容易出现数据丢失或重复的分布式环境来说是很重要的。另外，声明式操作更容易被用户使用，可以使系统向用户隐藏实现的细节，隐藏实现的细节的同时，也就保留了系统未来持续优化的可能性。此外，声明式的API，同时隐含了所有的API对象都是名词性质的，例如Service、Volume这些API都是名词，这些名词描述了用户所期望得到的一个目标分布式对象。
-
API对象是彼此互补而且可组合的。这里面实际是鼓励API对象尽量实现面向对象设计时的要求，即“高内聚，松耦合”，对业务相关的概念有一个合适的分解，提高分解出来的对象的可重用性。事实上，K8s这种分布式系统管理平台，也是一种业务系统，只不过它的业务就是调度和管理容器服务。
-
高层API以操作意图为基础设计。如何能够设计好API，跟如何能用面向对象的方法设计好应用系统有相通的地方，高层设计一定是从业务出发，而不是过早的从技术实现出发。因此，针对K8s的高层API设计，一定是以K8s的业务为基础出发，也就是以系统调度管理容器的操作意图为基础设计。
-
低层API根据高层API的控制需要设计。设计实现低层API的目的，是为了被高层API使用，考虑减少冗余、提高重用性的目的，低层API的设计也要以需求为基础，要尽量抵抗受技术实现影响的诱惑
-
尽量避免简单封装，不要有在外部API无法显式知道的内部隐藏的机制。简单的封装，实际没有提供新的功能，反而增加了对所封装API的依赖性。内部隐藏的机制也是非常不利于系统维护的设计方式，例如PetSet和ReplicaSet，本来就是两种Pod集合，那么K8s就用不同API对象来定义它们，而不会说只用同一个ReplicaSet，内部通过特殊的算法再来区分这个ReplicaSet是有状态的还是无状态。
-
API操作复杂度与对象数量成正比。这一条主要是从系统性能角度考虑，要保证整个系统随着系统规模的扩大，性能不会迅速变慢到无法使用，那么最低的限定就是API的操作复杂度不能超过O(N)，N是对象的数量，否则系统就不具备水平伸缩性了
-
API对象状态不能依赖于网络连接状态。由于众所周知，在分布式环境下，网络连接断开是经常发生的事情，因此要保证API对象状态能应对网络的不稳定，API对象的状态就不能依赖于网络连接状态
-
尽量避免让操作机制依赖于全局状态，因为在分布式系统中要保证全局状态的同步是非常困难的。
-
API-对象
- 是 K8s 集群中的管理操作单元
| 类别 | 名称 |
|---|---|
| 工作负载型资源对象 | Pod Replicaset ReplicationController Deployments StatefulSets Daemonset Job CronJob |
| 服务发现及负载均衡 | Service Ingress |
| 配置与存储 | Volume Persistent Volume CSl configmap secret |
| 集群资源 | Namespace Node Role ClusterRole RoleBinding ClusterRoleBinding |
| 元数据资源 | HPA PodTemplate LimitRang |
K8s 命令使用
Kubectl 概述
-
Kubectl 命令概述：https://kubernetes.io/zh/docs/reference/kubectl/overview/
-
Kubectl 命令行接口
使用以下语法 kubectl 从终端窗口运行命令：
kubectl [command] [TYPE] [NAME] [flags]
其中 command、TYPE、NAME 和 flags 分别是：
-
command：指定要对一个或多个资源执行的操作，例如create、get、describe、delete。 -
TYPE：指定资源类型。资源类型不区分大小写，可以指定单数、复数或缩写形式。例如，以下命令输出相同的结果。
kubectl get pod pod1
kubectl get pods pod1
kubectl get po pod1root@k8s-master1:~# kubectl get po
NAME READY STATUS RESTARTS AGE
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d4h
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d4h
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d4h
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d4h
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 46h
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 4d19h
root@k8s-master1:~# kubectl get pod
NAME READY STATUS RESTARTS AGE
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d4h
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d4h
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d4h
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d4h
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 46h
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 4d19h
root@k8s-master1:~# kubectl get pods
NAME READY STATUS RESTARTS AGE
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d4h
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d4h
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d4h
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d4h
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 46h
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 4d19hNAME：指定资源的名称。名称区分大小写。如果省略名称，则显示所有资源的详细信息kubectl get pods。
在对多个资源执行操作时，您可以按类型和名称指定每个资源，或指定一个或多个文件：
-
要按类型和名称指定资源：
- 要对所有类型相同的资源进行分组，请执行以下操作：
TYPE1 name1 name2 name<#>。例子：kubectl get pod example-pod1 example-pod2 - 分别指定多个资源类型：
TYPE1/name1 TYPE1/name2 TYPE2/name3 TYPE<#>/name<#>。例子：kubectl get pod/example-pod1 replicationcontroller/example-rc1
- 要对所有类型相同的资源进行分组，请执行以下操作：
-
用一个或多个文件指定资源：-f file1 -f file2 -f file<#>
- 使用 YAML 而不是 JSON 因为 YAML 更容易使用，特别是用于配置文件时。例子：
kubectl get pod -f ./pod.yaml
- 使用 YAML 而不是 JSON 因为 YAML 更容易使用，特别是用于配置文件时。例子：
-
flags: 指定可选的参数。例如，可以使用-s或-server参数指定 Kubernetes API 服务器的地址和端口。
操作命令和语法
root@k8s-master1:~# kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 192.168.0.1 <none> 443/TCP 5d7h
magedu-nginx-service NodePort 192.168.3.55 <none> 80:30004/TCP 2d2h
magedu-tomcat-service NodePort 192.168.1.200 <none> 80:30005/TCP 4d19h
root@k8s-master1:~# kubectl describe service magedu-tomcat-service #查看 service 中 service magedu-tomcat-service 的详细信息
Name: magedu-tomcat-service
Namespace: default
Labels: app=magedu-tomcat-service-label
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"magedu-tomcat-service-label"},"name":"magedu-tomcat-serv...
Selector: app=tomcat
Type: NodePort
IP: 192.168.1.200
Port: http 80/TCP
TargetPort: 8080/TCP
NodePort: http 30005/TCP
Endpoints: 10.10.5.17:8080
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>root@k8s-master1:~# kubectl explain pod #查看某个对象的方法 层层嵌套
root@k8s-master1:~# kubectl explain pod.apiVersion
root@k8s-master1:~# kubectl explain deployment.spec.selector- Create 和 apply 的区别
- apply 支持对 yaml 文件的多次修改和动态生效，修改完成重建执行
apply -f file.yaml。create 单次创建资源后期如果修改 yaml 文件想生效，那么需要删除之前的资源再重新创建。而且文件创建过程需要在删除之前修改，然后再重新创建(删除-修改-再创建)。
- apply 支持对 yaml 文件的多次修改和动态生效，修改完成重建执行
root@k8s-master1:~# kubectl cluster-info
Kubernetes master is running at https://192.168.26.248:6443
KubeDNS is running at https://192.168.26.248:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.root@k8s-master1:~# kubectl cordon --help
Mark node as unschedulable.
Examples:
# Mark node "foo" as unschedulable.
kubectl cordon foo
Options:
--dry-run=false: If true, only print the object that would be sent, without sending it.
-l, --selector='': Selector (label query) to filter on
Usage:
kubectl cordon NODE [options]
Use "kubectl options" for a list of global command-line options (applies to all commands).root@k8s-master1:~# kubectl cordon k8s-master1 #设置该节点不被调度
node/k8s-master1 cordoned
root@k8s-master1:~# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master1 Ready,SchedulingDisabled master 5d8h v1.17.4
k8s-master2 Ready master 5d8h v1.17.4
k8s-master3 Ready master 5d7h v1.17.4
node-1 Ready <none> 5d7h v1.17.4
node-2 Ready <none> 5d7h v1.17.4
node-3 Ready <none> 5d7h v1.17.4root@k8s-master1:~# kubectl uncordon k8s-master1 #取消该节点不被调度
node/k8s-master1 uncordoned
root@k8s-master1:~# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master1 Ready master 5d8h v1.17.4
k8s-master2 Ready master 5d8h v1.17.4
k8s-master3 Ready master 5d7h v1.17.4
node-1 Ready <none> 5d7h v1.17.4
node-2 Ready <none> 5d7h v1.17.4
node-3 Ready <none> 5d7h v1.17.4root@k8s-master1:~# kubectl drain --help #驱逐无状态服务，用于 node 节点紧急下线
Drain node in preparation for maintenance.
...省略...
Usage:
kubectl drain NODE [options]
Use "kubectl options" for a list of global command-line options (applies to all commands).root@k8s-master1:~# kubectl api-resources
NAME SHORTNAMES APIGROUP NAMESPACED KIND
bindings true Binding
componentstatuses cs false ComponentStatus
configmaps cm true ConfigMap
endpoints ep true Endpoints
events ev true Event
limitranges limits true LimitRange
namespaces ns false Namespace
nodes no false Node
persistentvolumeclaims pvc true PersistentVolumeClaim
persistentvolumes pv false PersistentVolume
pods po true Pod
podtemplates true PodTemplate
replicationcontrollers rc true ReplicationController
resourcequotas quota true ResourceQuota
secrets true Secret
serviceaccounts sa true ServiceAccount
services svc true Service
mutatingwebhookconfigurations admissionregistration.k8s.io false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io false ValidatingWebhookConfiguration
customresourcedefinitions crd,crds apiextensions.k8s.io false CustomResourceDefinition
apiservices apiregistration.k8s.io false APIService
controllerrevisions apps true ControllerRevision
daemonsets ds apps true DaemonSet
deployments deploy apps true Deployment
replicasets rs apps true ReplicaSet
statefulsets sts apps true StatefulSet
tokenreviews authentication.k8s.io false TokenReview
localsubjectaccessreviews authorization.k8s.io true LocalSubjectAccessReview
selfsubjectaccessreviews authorization.k8s.io false SelfSubjectAccessReview
selfsubjectrulesreviews authorization.k8s.io false SelfSubjectRulesReview
subjectaccessreviews authorization.k8s.io false SubjectAccessReview
horizontalpodautoscalers hpa autoscaling true HorizontalPodAutoscaler
cronjobs cj batch true CronJob
jobs batch true Job
certificatesigningrequests csr certificates.k8s.io false CertificateSigningRequest
leases coordination.k8s.io true Lease
endpointslices discovery.k8s.io true EndpointSlice
events ev events.k8s.io true Event
ingresses ing extensions true Ingress
ingresses ing networking.k8s.io true Ingress
networkpolicies netpol networking.k8s.io true NetworkPolicy
runtimeclasses node.k8s.io false RuntimeClass
poddisruptionbudgets pdb policy true PodDisruptionBudget
podsecuritypolicies psp policy false PodSecurityPolicy
clusterrolebindings rbac.authorization.k8s.io false ClusterRoleBinding
clusterroles rbac.authorization.k8s.io false ClusterRole
rolebindings rbac.authorization.k8s.io true RoleBinding
roles rbac.authorization.k8s.io true Role
priorityclasses pc scheduling.k8s.io false PriorityClass
csidrivers storage.k8s.io false CSIDriver
csinodes storage.k8s.io false CSINode
storageclasses sc storage.k8s.io false StorageClass
volumeattachments storage.k8s.io false VolumeAttachment输出选项和语法
root@k8s-master1:~# kubectl get pod
NAME READY STATUS RESTARTS AGE
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d6h
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d6h
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d6h
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d6h
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 2d
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 4d21hroot@k8s-master1:~# kubectl get pod -o wide #输出更多信息
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d6h 10.10.5.18 node-3 <none> <none>
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d6h 10.10.4.14 node-2 <none> <none>
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d6h 10.10.6.16 node-1 <none> <none>
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d6h 10.10.4.15 node-2 <none> <none>
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 2d 10.10.4.16 node-2 <none> <none>
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 4d21h 10.10.5.17 node-3 <none> <none>root@k8s-master1:~# kubectl get pod -o json #输出为 json 格式显示
{
"apiVersion": "v1",
"items": [
{
"apiVersion": "v1",
"kind": "Pod",
"metadata": {
"creationTimestamp": "2020-03-29T09:12:07Z",
"generateName": "net-test1-5fcc69db59-",
"labels": {
"pod-template-hash": "5fcc69db59",
"run": "net-test1"
},
"name": "net-test1-5fcc69db59-c2plf",
...省略...root@k8s-master1:~# kubectl get pod -o yaml #输出为 yaml 格式显示
apiVersion: v1
items:
- apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2020-03-29T09:12:07Z"
generateName: net-test1-5fcc69db59-
labels:
pod-template-hash: 5fcc69db59
run: net-test1
name: net-test1-5fcc69db59-c2plf
namespace: default
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: trueK8s 的牛鼻子--API
K8s 的几个重要概念
- 对象调用 K8s 是和什么打交道的？ 通过声明式 API
- 怎么调用 K8s 的声明式 API？ 通过 yaml 文件
apiVersion: apps/v1 #创建该对象所使用的 Kubernetes API 的版本
kind: Deployment #想要创建的对象的类型
metadata: #帮助识别对象唯一性的元数据，包括一个 name 名称，可选的 namespace
name: nginx-deployment
labels:
app: nginx
spec: #定义 deployment 中容器的详细信息，可以定义多个容器，名称不能冲突
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: k8s.harbor.com/base-images/nginx:1.14.2
ports:
- containerPort: 80
---
kind: Service
apiVersion: v1
metadata:
labels:
app: magedu-nginx-service-label
name: magedu-nginx-service
namespace: default
spec:
type: NodePort
ports:
- name: http
port: 80 #service 端口，通过 service 端口的 80 转发到容器 (targetPort) pod 端口 80
protocol: TCP
targetPort: 80 #pod 端口
nodePort: 30004 #宿主机端口，通过宿主机端口 300004 访问到 service 端口 80
selector:
app: nginx- 必须字段怎么申明？
- 1.apiVersion-创建该对象所使用的 Kubernetes API 的版本
- 2.kind-想要创建的对象的类型
- 3.metadata-帮助识别对象唯一性的数据，包括一个 name 名称、可选的 namespace
- 4.spec-定义 deployment 中容器的详细信息
- 5.status-Pod 创建完成后 K8s 自动生成 status 状态
root@k8s-master1:~# kubectl get pod
NAME READY STATUS RESTARTS AGE
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d6h
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d6h
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d6h
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d6h
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 2d
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 4d21h
root@k8s-master1:~# kubectl describe pod tomcat-deployment-5cd65b4d74-d4x7s
Name: tomcat-deployment-5cd65b4d74-d4x7s
Namespace: default
Priority: 0
Node: node-3/192.168.26.164
Start Time: Mon, 30 Mar 2020 02:24:23 +0800
Labels: app=tomcat
pod-template-hash=5cd65b4d74
Annotations: <none>
Status: Running #状态是否 runing
IP: 10.10.5.17
IPs:
IP: 10.10.5.17
Controlled By: ReplicaSet/tomcat-deployment-5cd65b4d74
Containers:
tomcat:
Container ID: docker://1cd04d9b73e80ea8cbd849f62ba2c19a4ef83ccc44ddbb168d278fb6f4c07a5d
Image: k8s.harbor.com/bokebi/tomcat:app
Image ID: docker-pullable://k8s.harbor.com/bokebi/tomcat@sha256:76e6f187899ec9d58c865c71458bbd8a780d6e019e093b0513e886ef4d3f0aeb
Port: 8080/TCP
Host Port: 0/TCP
State: Running #状态是否 runing
Started: Wed, 01 Apr 2020 18:33:57 +0800
Last State: Terminated
Reason: Error
Exit Code: 143
Started: Wed, 01 Apr 2020 15:44:35 +0800
Finished: Wed, 01 Apr 2020 18:33:18 +0800
Ready: True
Restart Count: 2
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-qnr5k (ro)
Conditions:
Type Status #状态的查询
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-qnr5k:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-qnr5k
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>yaml 文件及语法基础
- 需要提前创建好 yaml 文件，并创建好 pod 运行所需要的 namespace、yaml 文件等资源
root@k8s-master1:~# cd /opt/
root@k8s-master1:/opt# mkdir file-yaml
root@k8s-master1:/opt#cd file-yaml/
root@k8s-master1:/opt/file-yaml# vim linux39-ns.yml
apiVersion: v1 #API 版本
kind: Namespace #类型为 namespac
metadata: #定义元数据
name: linux39 #namespace 名称
root@k8s-master1:/opt/file-yaml# kubectl get ns
NAME STATUS AGE
default Active 5d9h
kube-node-lease Active 5d9h
kube-public Active 5d9h
kube-system Active 5d9h
kubernetes-dashboard Active 5d1h
test Active 2d
root@k8s-master1:/opt/file-yaml# kubectl apply -f linux39-ns.yml
namespace/linux39 created
root@k8s-master1:/opt/file-yaml# kubectl get ns
NAME STATUS AGE
default Active 5d9h
kube-node-lease Active 5d9h
kube-public Active 5d9h
kube-system Active 5d9h
kubernetes-dashboard Active 5d1h
linux39 Active 12s
test Active 2d- 浏览器访问：https://192.168.26.144:30002/
- 查看 linux39 的 namespace 已经存储到 etcd 但是现在还没有任何数据
- 在线 yaml 与 json 编辑器：http://www.bejson.com/validators/yaml_editor/
大小写敏感
使用缩进表示层级关系
缩进时不允许使用 Tal 键，只允许使用空格
缩进的空格数目不重要，只要相同层级的元素左侧对齐即可
使用 "#" 表示注释，从这个字符一直到行尾，都会被解析器忽略
比 json 更适用于配置文件- yaml 文件主要特性
上下级关系
列表
键值对 (也称为 maps，即 key:value 格式的键值对数据)Nginx 业务 yaml 文件详解
# pwd
/opt/k8s-data/yaml/bokebi
# mkdir nginx tomcat-app1 tomcat-app2
# cd nginx
# pwd
/opt/k8s-data/yaml/bokebi/nginx
# cat nginx.yaml
---
kind: Deployment #类型，是 deployment 控制器，kubectl explain Deployment
apiversion: extensions/vlbetal #API 版本，kubectl explain Deployment.apiversion
metadata: #pod 的元数据信息，kubectl explain Deployment.metadata
labels: #自定义 pod 的标签，kubectl explain Deployment.metadata.labels
app: bokebi-nginx-deployment-label #标签名称为 app 值为 bokebi-nginx-deployment-label，后面会用到此标签
name: bokebi-nginx-deployment #pod 的名称
namespace: bokebi #pod 的 namespace，默认是 default
spec: #定义 deployment 中容器的详细信息，kubectl explain Deployment.spec
replicas: 1 #创建出的 pod 的副本数，即多少个 pod，默认值为 1
selsctor: #定义标签选择器
matchLabels: #定义匹配的标签，必须要设置
app: #bokebi-nginx-selector #匹配的目标标签
template: #定义模板，必须定义，模板是起到描述要创建的 pod 的作用
metadata: #定义模板元数据
labels: #定义模板 label，kubectl explain Deployment.spec.template.metadata.labels
app: bokebi-nginx-selector # 定义标签，等于 Deployment.spec.seletor.matchLabels.app 的值
spec: #定义 pod 信息
containners #定义 pod 中容器列表，可以多个至少一个，pod 不能动态增减容器
- name: bokebi-nginx-container #容器名称
image: k8s.harbor.com/base-images/nginx:1.14.2 #镜像地址
#command: ["/apps/tomcat/bin/run_tomcat.sh"] #容器启动
#imagePullPolicy: IfNotPresent
imagePullPolicy: Always #拉取镜像策略
port: #定义容器端口列表
- containerPort: 80 #定义一个端口
protocol: TCP #端口协议
name: http #端口名称
- containerPort: 443 #定义一个端口
protocol: TCP #端口协议
name: https #端口名称
env: #配置环境变量
- name: "password" #变量名称，必须要用引号引起来
value: "123456" #当前变量的值
- name: "age" #另一个变量名称
value: "18" #另一个变量的值
resources: #对资源的请求设置和限制设置
limits: #资源限制设置，上限
cpu: 2 #cpu 的限制，单位为 core 数，可以写为 2 或者 2000m 等 CPU 的压缩值，2000 毫核
memory: 2Gi #内存限制，单位可以为 Mib/Gib，将用于 docker run --memory 参数
requests: #资源请求的设置
cpu:1 #cpu 请求数，容器启动的初始可用数量，可以写 1 或者 1000m 等 CPU 的压缩值，1000 毫核
memory: 512Mi #内存请求大小，容器启动的初始可用数量，用于调度 pod 的时候使用
---
kind: Service #类型为 service
apiVersion: v1 #service API 版本，kubectl explain Service.apiVersion
metadata: #定义 service 元数据，kubectl explain Service.metadata
labels: #自定义标签，kubectl explain Service.metadata.labels
app: bokebi-nginx #定义 service 标签的内容
name: bokebi-nginx-spec #定义 service 的名称，此名称会被 DNS 解析
namespace: bokebi #该 service 隶属于的 namespaces 名称，即把 service 创建到哪个 namespace 里面
spec: #定义 service 的详细信息，kubectl explain Service.spec
type: NodePort #service 的类型，定义服务的访问方式，默认为 ClusterIP，kubectl explain Service.spec.type
ports: #定义访问端口，kubectl explain Service.spec.ports
- name: #定义一个端口名称
port: 80 #service 80 端口
protocol: TCP #协议类型
targetPort: 80 #目标 pod 的端口
nodePort: 30001 #node 节点暴漏的端口
- name: https #SSL 端口
port: 443 #service 443 端口
protocol: TCP #端口协议
targetPort: 443 #目标 pod 端口
modePort: 300043 #node 节点暴露的 SSL 端口
selector: #service 的标签选择器，定义要访问的目标 pod
app: bokebi-nginx #将流量路由到选择的 pod 上，需等于 kubectl explain Deployment.spec.selector.matchLabelsspec 和 status 的区别
spec 是期望状态 (你想让 K8s 做到那些事情)
status 是实际状态 (K8s 实际上能做到的事情)Pod 概述：
1.Pod 是 k8s 中的最小单元
2.一个 Pod 中可以运行一个容器，也可以运行多个容器
3.运行多个容器的话，这些容器是一起被调度的
4.Pod 的生命周期是短暂的，不会自愈，是用完就销毁的实体
5.一般我们是通过 Controller 来创建和管理pod的Controller:控制器
- 标签选择器： https://kubernetes.io/zh/docs/concepts/overview/working-with-objects/labels/
- ReplicationController： https://kubernetes.io/zh/docs/concepts/workloads/controllers/replicationcontroller/
实验示例
root@k8s-master1:/opt/k8s-data/yaml/namespace# pwd
/opt/k8s-data/yaml/namespace
root@k8s-master1:/opt/k8s-data/yaml/namespace# ll
total 16
drwxr-xr-x 2 root root 4096 Apr 4 00:55 ./
drwxr-xr-x 4 root root 4096 Apr 1 23:48 ../
-rw-r--r-- 1 root root 60 Apr 4 00:55 linux39-ns.yml
-rw-r--r-- 1 root root 60 Apr 4 00:55 linux40-ns.yml
root@k8s-master1:/opt/k8s-data/yaml/namespace# cat linux*
apiVersion: v1 #API 版本
kind: Namespace #类型为 namespace
metadata: #定义元数据
name: linux39 #namespace 名称
apiVersion: v1 #API 版本
kind: Namespace #类型为 namespace
metadata: #定义元数据
name: linux40 #namespace 名称
root@k8s-master1:/opt/k8s-data/yaml/namespace# kubectl apply -f linux39-ns.yml
namespace/linux39 created
root@k8s-master1:/opt/k8s-data/yaml/namespace# kubectl apply -f linux40-ns.yml
namespace/linux40 created
root@k8s-master1:/opt/k8s-data/yaml/namespace# kubectl get ns
NAME STATUS AGE
default Active 5d10h
kube-node-lease Active 5d10h
kube-public Active 5d10h
kube-system Active 5d10h
kubernetes-dashboard Active 5d2h
linux39 Active 17s
linux40 Active 12s
test Active 2d1hcase1:pod 的控制器类型
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl apply -f deployment.yml
deployment.apps/nginx-deployment created
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl get pod -n linux39
NAME READY STATUS RESTARTS AGE
nginx-deployment-6997c89dfb-p5t2v 0/1 ContainerCreating 0 13s #处于创建状态，正在拉取镜像
nginx-deployment-6997c89dfb-zgb7f 0/1 ContainerCreating 0 13s
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl get pod -n linux39
NAME READY STATUS RESTARTS AGE
nginx-deployment-6997c89dfb-6jjm7 1/1 Running 0 65s #已经正常运行
nginx-deployment-6997c89dfb-p5t2v 1/1 Running 0 4m3sReplicaSet
root@master-1:/opt/k8s-data/yaml/linux39/case1# vim rs.yml #ReplicaSet 控制器
apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: ReplicaSet #ReplicaSet 控制器
metadata:
name: frontend
namespace: linux39
spec:
replicas: 3
selector:
#matchLabels:
# app: ng-rs-80
matchExpressions:
- {key: app, operator: In, values: [ng-rs-80,ng-rs-81]} #正则匹配多个
template:
metadata:
labels:
app: ng-rs-80
spec:
containers:
- name: ng-rs-80
image: nginx
ports:
- containerPort: 80- 先把之前创建的资源删除
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl delete -f deployment.yml
deployment.apps "nginx-deployment" deleted
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl get pod -n linux39
No resources found in linux39 namespace.- 用 ReplicaSet 创建新的资源
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl get pod -n linux39
NAME READY STATUS RESTARTS AGE
frontend-7fhcv 1/1 ContainerCreating 0 89s
frontend-c8sgg 1/1 ContainerCreating 0 89s
frontend-msr4g 1/1 ContainerCreating 0 89s
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl get pod -n linux39
NAME READY STATUS RESTARTS AGE
frontend-7fhcv 1/1 Running 0 89s
frontend-c8sgg 1/1 Running 0 89s
frontend-msr4g 1/1 Running 0 89s- ReplicaSet 需要先删除之后才能再次修改，除非加一些参数选项
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl delete -f rs.yml
replicaset.apps "frontend" deleted
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl create --help
Create a resource from a file or from stdin.
JSON and YAML formats are accepted.
Examples:
# Create a pod using the data in pod.json.
kubectl create -f ./pod.json
...省略...
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl create -f rs.yml --save-config=true #再次创建添加的参数
replicaset.apps/frontend created
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# vim rs.yml
...省略...
spec:
replicas: 2 #把副本的数量改为2个
...省略...
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl apply -f rs.yml #再次创建还是依赖apply
replicaset.apps/frontend configured
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl get pod -n linux39
NAME READY STATUS RESTARTS AGE
frontend-fxqkf 1/1 Running 0 47s
frontend-vs9hz 1/1 Running 0 47sReplicationController (快淘汰)
apiVersion: v1
kind: ReplicationController #ReplicationController 控制器
metadata:
name: ng-rc
namespace: linux39
spec:
replicas: 2
selector:
app: ng-rc-80
#app1: ng-rc-81
template:
metadata:
labels:
app: ng-rc-80
#app1: ng-rc-81
spec:
containers:
- name: ng-rc-80
image: nginx
ports:
- containerPort: 80```c
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl delete -f rs.yml
replicaset.apps "frontend" deleted
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl get pod -n linux39
No resources found in linux39 namespace.```c
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl apply -f rc.yml
replicationcontroller/ng-rc created
root@k8s-master1:/opt/k8s-data/yaml/linux39/case1# kubectl get pod -n linux39
NAME READY STATUS RESTARTS AGE
ng-rc-mv6rh 1/1 Running 0 16s
ng-rc-zpzlr 1/1 Running 0 16s- Rc、Rs 和 Deployment
- Deployment 概念： https://kubernetes.io/zh/docs/concepts/workloads/controllers/deployment/
- ReplicaSet 概念： https://kubernetes.io/zh/docs/concepts/workloads/controllers/replicaset/
- ReplicationController 概念： https://kubernetes.io/zh/docs/concepts/workloads/controllers/replicationcontroller/
- ReplicationController：副本控制器（selector= 或者!=）（等于或者不等于）
- ReplicaSet：副本控制集，和副本控制器的区别是：对选择器的支持（selector还支持innotin）
- Deployment：比 rs 更高一级的控制器，除了有rs的功能之外，还有很多高级功能,，比如说最重要的：滚动升级、回滚等
Service
Why：pod重启之后ip就变了，pod之间直接访问会有问题
What：解耦了服务和应用。简化服务的调用
How：声明一个service对象一般常用的有两种：
k8s集群内的service：selector 指定 pod，自动创建 Endpoints
k8s集群外的service：手动创建 Endpoints，指定外部服务的 ip，端口和协议kube-proxy 和 service 的关系：
kube-proxy-----------> k8s-apiserver
watch
kube-proxy 监听着 k8s-apiserver，一旦 service 资源发生变化（调 k8s-api 修改 service 信息），kube-proxy 就会生成对应的负载调度的调整，这样就保证 service 的最新状态。kube-proxy有三种调度模型：
userspace：k8s 1.1 之前
iptables：k8s 1.10 之前
ipvs：k8s 1.11 之后，如果没有开启 ipvs，则自动降级为 iptablesService 和 deployment 实现一个 nginx
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# pwd
/opt/k8s-data/yaml/linux39/case2
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# ll
total 3916
drwxr-xr-x 2 root root 4096 Apr 4 01:47 ./
drwxr-xr-x 4 root root 4096 Apr 4 01:46 ../
-rw-r--r-- 1 root root 542 Mar 30 18:33 1-deploy_node.yml
-rw-r--r-- 1 root root 214 Mar 30 18:33 2-svc_service.yml
-rw-r--r-- 1 root root 233 Mar 30 18:33 3-svc_NodePort.yml
-rw-r--r-- 1 root root 3983872 Mar 30 18:33 busybox-online.tar.gz
-rw-r--r-- 1 root root 277 Mar 30 18:33 busybox.yamlroot@k8s-master1:/opt/k8s-data/yaml/linux39/case2# vim 1-deploy_node.yml
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: linux39
spec:
replicas: 1
selector:
#matchLabels: #rs or deployment
# app: ng-deploy3-80
matchExpressions:
- {key: app, operator: In, values: [ng-deploy-80,ng-rs-81]}
template:
metadata:
labels:
app: ng-deploy-80
spec:
containers:
- name: ng-deploy-80
image: nginx:1.17.5
ports:
- containerPort: 80
#nodeSelector:
# env: group1root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# kubectl apply -f 1-deploy_node.yml
deployment.apps/nginx-deployment created
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# kubectl get pod -n linux39
NAME READY STATUS RESTARTS AGE
nginx-deployment-6997c89dfb-727wx 1/1 Running 0 10sClusterIP 内部访问
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# pwd
/opt/k8s-data/yaml/linux39/case2
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# vim 2-svc_service.yml
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# cat 2-svc_service.yml
apiVersion: v1
kind: Service
metadata:
name: ng-deploy-80
namespace: linux39
spec:
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
type: ClusterIP
selector:
app: ng-deploy-80root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# kubectl get svc -n linux39 #获取 linux39 的 service 集群地址
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ng-deploy-80 ClusterIP 192.168.1.206 <none> 80/TCP 23s- 进入到 pod 服务器
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# kubectl exec -it net-test1-5fcc69db59-xzvrl sh
/ # apk add curl #安装 curl 命令
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/4) Installing ca-certificates (20191127-r1)
(2/4) Installing nghttp2-libs (1.40.0-r0)
(3/4) Installing libcurl (7.67.0-r0)
(4/4) Installing curl (7.67.0-r0)
Executing busybox-1.31.1-r9.trigger
Executing ca-certificates-20191127-r1.trigger
OK: 7 MiB in 18 packages
/ # curl 192.168.1.206 #访问svc集权地址 看是否能够联通
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title> #nginx 默认页面
...省略...
</body>
</html>
/ # ping ng-deploy-80 #ping 一下 service NAME 不通
ping: bad address 'ng-deploy-80'busybox 镜像创建
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# docker pull busybox
...省略...
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# docker images
busybox latest 83aa35aa1c79 3 weeks ago 1.22MB
...省略...
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# docker tag busybox:latest k8s.harbor.com/base-images/busybox
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# docker push k8s.harbor.com/base-images/busybox
...省略...- 浏览器访问：http://k8s.harbor.com/
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# cat busybox.yaml
apiVersion: v1
kind: Pod
metadata:
name: busybox
namespace: linux39 #default namespace 的 DNS
spec:
containers:
- image: k8s.harbor.com/base-images/busybox
command:
- sleep
- "3600"
imagePullPolicy: Always
name: busybox
restartPolicy: Alwaysroot@k8s-master1:/opt/k8s-data/yaml/linux39/case2# kubectl apply -f busybox.yaml
pod/busybox created
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# kubectl get pod -n linux39
NAME READY STATUS RESTARTS AGE
busybox 1/1 Running 0 8s
nginx-deployment-6997c89dfb-9vblx 1/1 Running 0 28m- 测试在同一个 namespace 中能够 ping 通
- 这里进入到 busybox 的 pod 中进行测试：进行 ping 之前
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# kubectl exec -it busybox sh -n linux39
/ # ping ng-deploy-80 #访问 service 的 NAME
PING ng-deploy-80 (192.168.1.206): 56 data bytes #虽然ping不通但是能解析DNS 也说明能够访问
^C
--- ng-deploy-80 ping statistics ---
6 packets transmitted, 0 packets received, 100% packet loss
/ # wget ng-deploy-80 #下载 index.html 页面
Connecting to ng-deploy-80 (192.168.1.206:80)
saving to 'index.html'
index.html 100% |****************************************************| 612 0:00:00 ETA
'index.html' saved
/ # cat index.html #验证 index.html 页面
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...省略...
/ # - 结论：在同一个 namespace 中 可以直接通过 service 的 NAME 进行访问 在 yaml 文件中直接指定 service-name 即可以实现微服务对外不访问
service对外访问
root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# cat 3-svc_NodePort.yml
apiVersion: v1
kind: Service
metadata:
name: ng-deploy-80
namespace: linux39
spec:
ports:
- name: http
port: 81
targetPort: 80 #pod中的服务端口 一定要写对
nodePort: 30012 #宿主机暴露端口
protocol: TCP
type: NodePort
selector:
app: ng-deploy-80root@k8s-master1:/opt/k8s-data/yaml/linux39/case2# kubectl apply -f 3-svc_NodePort.yml
service/ng-deploy-80 configured- 30012 这个自定义端口在K8S集群中每一个宿主机都会监听的一个端口,只要有这个端口就能通过这个端口访问到 service 地址进而 service 端口转发到内部的 pod 节点
root@node-1:~# ss -ntl
root@node-2:~# ss -ntl
root@node-3:~# ss -ntl- 在任何一个 K8S 集群节点都能访问
- 浏览器访问： http://192.168.26.164:30012/
Volume
- 容器中的文件在磁盘上是临时存放的，这给容器中运行的特殊应用程序到来了一些问题。首先，当容器崩溃时，kubelet 将重启容器，容器中的文件将会丢失————因为容器会以干净的状态重建。其次，当在一个 Pod 中同时运行多个容器时，常常需要在这些容器之间共享文件。Kubernetes 抽象出 Volume 对象来解决这两个问题。
数据和镜像解耦，以及容器间的数据共享。
k8s抽象出的一个对象，用来保存数据，做存储用。
常用的几种卷：
emptyDir：本地临时卷
hostPath：本地卷
nfs等：共享卷
configmap: 配置文件CASE3：emptyDir
-
emptyDir： https://kubernetes.io/zh/docs/concepts/storage/volumes/#emptydir
-
Pod 示例
apiVersion: v1
kind: Pod
metadata:
name: test-pd
spec:
containers:
- image: k8s.gcr.io/test-webserver
name: test-container
volumeMounts:
- mountPath: /cache
name: cache-volume
volumes:
- name: cache-volume
emptyDir: {}root@k8s-master1:/opt/k8s-data/yaml/linux39/case3# pwd
/opt/k8s-data/yaml/linux39/case3
root@k8s-master1:/opt/k8s-data/yaml/linux39/case3# cat deploy_empty.yml
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: linux39
spec:
replicas: 1
selector:
matchLabels: #rs or deployment
app: ng-deploy-80
template:
metadata:
labels:
app: ng-deploy-80
spec:
containers:
- name: ng-deploy-80
image: nginx
ports:
- containerPort: 80
volumeMounts:
- mountPath: /cache
name: cache-volume
volumes:
- name: cache-volume
emptyDir: {}root@k8s-master1:/opt/k8s-data/yaml/linux39/case3# pwd
/opt/k8s-data/yaml/linux39/case3
root@k8s-master1:/opt/k8s-data/yaml/linux39/case3# kubectl apply -f deploy_empty.yml
deployment.apps/nginx-deployment created
root@k8s-master1:/opt/k8s-data/yaml/linux39/case3# kubectl get pod -n linux39
NAME READY STATUS RESTARTS AGE
busybox 1/1 Running 0 28m
nginx-deployment-7cc86d98d5-mk95q 1/1 Running 0 12s
root@k8s-master1:/opt/k8s-data/yaml/linux39/case3# kubectl exec -it nginx-deployment-7cc86d98d5-mk95q bash -n linux39
root@nginx-deployment-7cc86d98d5-mk95q:/# cd /cache/
root@nginx-deployment-7cc86d98d5-mk95q:/cache# echo 112233 > linux39.txt
root@nginx-deployment-7cc86d98d5-mk95q:/cache# exit
exit
root@k8s-master1:/opt/k8s-data/yaml/linux39/case3# kubectl get pod -n linux39 -o wide #查看创建的数据在哪一个节点 发现在 node-3 中
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deployment-7cc86d98d5-mk95q 1/1 Running 0 2m1s 10.10.6.53 node-1 <none> <none>- 在 node-1 中查找数据的存储目录
root@node-1:~# find / -name linux39.txt
/var/lib/kubelet/pods/daad624e-23a0-4850-a523-376b9a4c6969/volumes/kubernetes.io~empty-dir/cache-volume/linux39.txt- 上面的路径是固定的。但是 pod 的编号 ID(daad624e-23a0-4850-a523-376b9a4c6969) 不是固定的，这里可以用 * 进行匹配。emptyDir 的数据在 pod 被删除的同时数据也会被删除清空!
CASE4:hostPath
- hostPath： https://kubernetes.io/zh/docs/concepts/storage/volumes/#hostpath
- hostPath 卷能将主机节点文件系统上的文件或目录挂载到您的 Pod 中。 虽然这不是大多数 Pod 需要的，但是它为一些应用程序提供了强大的逃生舱。
- hostPath 卷将主机节点的文件系统中的文件或目录挂载到集群中，pod 删除的时候，卷不会被删除。
root@k8s-master1:/opt/k8s-data/yaml/linux39/case4# pwd
/opt/k8s-data/yaml/linux39/case4
root@k8s-master1:/opt/k8s-data/yaml/linux39/case4# cat deploy_hostPath.yml
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment-2
spec:
replicas: 1
selector:
matchLabels:
app: ng-deploy-80
template:
metadata:
labels:
app: ng-deploy-80
spec:
containers:
- name: ng-deploy-80
image: nginx
ports:
- containerPort: 80
volumeMounts:
- mountPath: /data/mysql
name: data-volume #和下面volumes的挂载名称要一致
volumes:
- name: data-volume #定义挂载的名称
hostPath: #指定使用什么类型 什么样子的卷
path: /data/mysqlroot@k8s-master1:/opt/k8s-data/yaml/linux39/case4# kubectl apply -f deploy_hostPath.yml
deployment.apps/nginx-deployment-2 created
root@k8s-master1:/opt/k8s-data/yaml/linux39/case4# kubectl get pod
NAME READY STATUS RESTARTS AGE
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d10h
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d10h
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d9h
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d9h
nginx-deployment-2-7944748bc4-5gg2m 1/1 Running 0 14s
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 2d3h
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 5d
root@k8s-master1:/opt/k8s-data/yaml/linux39/case4# kubectl get pod -o wide ##查看在哪一个 node 节点中，为 node-1
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d10h 10.10.5.18 node-3 <none> <none>
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d10h 10.10.4.14 node-2 <none> <none>
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d9h 10.10.6.16 node-1 <none> <none>
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d9h 10.10.4.15 node-2 <none> <none>
nginx-deployment-2-7944748bc4-5gg2m 1/1 Running 0 37s 10.10.6.54 node-1 <none> <none>
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 2d3h 10.10.4.16 node-2 <none> <none>
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 5d 10.10.5.17 node-3 <none> <none>- 可以去node-1主机中查看是否自动创建了一个/data/mysql的目录
root@k8s-master1:/opt/k8s-data/yaml/linux39/case4# kubectl exec -it nginx-deployment-2-7944748bc4-5gg2m bash
root@nginx-deployment-2-7944748bc4-5gg2m:# cd /data/mysql
root@nginx-deployment-2-7944748bc4-5gg2m:/data/mysql# mkdir logs
root@nginx-deployment-2-7944748bc4-5gg2m:/data/mysql# cd logs
root@nginx-deployment-2-7944748bc4-5gg2m:/data/mysql# echo bokebi > logs/nginx.logs- 在 node-1 中验证数据的存储目录
root@node-1:~# cd /data/mysql/logs/
root@node-1:/data/mysql/logs# ls
nginx.logs
root@node-1:/data/mysql/logs# cat nginx.logs #这份文件宿主机和pod容器就一份，同修改同删除
bokebi- 验证：在pod被删除时候，之前的数据是否被删除
root@k8s-master1:/opt/k8s-data/yaml/linux39/case4# kubectl delete -f deploy_hostPath.yml
deployment.apps "nginx-deployment-2" deletedroot@node-1:~# cat /data/mysql/logs/nginx.logs
bokebi- 总结：hostPath卷将主机节点的文件系统中的文件或目录挂载到集群中，pod删除的时候，卷不会被删除。
CASE5:nfs Volume
- nfs：
- nfs 卷允许将现有的 NFS（网络文件系统）共享挂载到容器中。不像 emptyDir，当删除 Pod 时，nfs 卷的内容被保留，卷仅仅是被卸载。这意味着NFS 卷可以预填充数据，并且可以在pod 之间“切换”数据。NFS 可以被多个写入者同时挂载。
警告:在您使用 NFS 卷之前，必须运行自己的 NFS 服务器并将目标 share 导出备用。
- 这里实验复用 HA-service 的机器：192.168.26.134
root@haproxy-server:~# apt install nfs-server -y
root@haproxy-server:~# mkdir /data/k8sdata -p
root@haproxy-server:~# vim /etc/exports
...省略...
/data/k8sdata *(rw,no_root_squash)
root@haproxy-server:~# systemctl restart nfs-server.service
root@haproxy-server:~# systemctl enable nfs-server.service - 在 node 节点上面查看共享出来的挂载卷。如果看不到说明配置有问题，后面实验也挂不上。
root@node-1:~# apt install nfs-common -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
nfs-common is already the newest version (1:1.3.4-2.1ubuntu5.2).
0 upgraded, 0 newly installed, 0 to remove and 168 not upgraded.
root@node-1:~# showmount -e 192.168.26.134
Export list for 192.168.26.134:
/data/k8sdata *- 在node节点的宿主机挂载
root@node-1:~# mount -t nfs 192.168.26.134:/data/k8sdata /mnt- 实验
root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# pwd
/opt/k8s-data/yaml/linux39/case5
root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# ll
total 16
drwxr-xr-x 2 root root 4096 Apr 4 03:58 ./
drwxr-xr-x 7 root root 4096 Apr 4 02:49 ../
-rw-r--r-- 1 root root 804 Mar 30 18:33 deploy_nfs2.yml
-rw-r--r-- 1 root root 800 Apr 4 03:58 deploy_nfs.yml
root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# cat deploy_nfs.yml
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: ng-deploy-80
template:
metadata:
labels:
app: ng-deploy-80
spec:
containers:
- name: ng-deploy-80
image: nginx
ports:
- containerPort: 80
volumeMounts:
- mountPath: /usr/share/nginx/html/mysite #挂载到 nfs 服务器的 pod 的指定目录
name: my-nfs-volume
volumes:
- name: my-nfs-volume
nfs:
server: 192.168.26.134 #nfs 服务器地址
path: /data/k8sdata #nfs 共享服务器共享的目录
---
apiVersion: v1
kind: Service
metadata:
name: ng-deploy-80
spec:
ports:
- name: http
port: 81
targetPort: 80
nodePort: 30011
protocol: TCP
type: NodePort
selector:
app: ng-deploy-80ot@k8s-master1:/opt/k8s-data/yaml/linux39/case5# kubectl apply -f deploy_nfs.yml
deployment.apps/nginx-deployment-3 created
service/ng-deploy-80 unchanged
root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# kubectl get pod
NAME READY STATUS RESTARTS AGE
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d10h
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d10h
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d10h
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d10h
nginx-deployment-3-688784467c-nb9st 1/1 Running 0 3m27s
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 2d4h
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 5d1h
root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# kubectl exec -it nginx-deployment-3-688784467c-nb9st bash
root@nginx-deployment-3-688784467c-nb9st:/# df -Th
Filesystem Type Size Used Avail Use% Mounted on
overlay overlay 98G 5.9G 88G 7% /
tmpfs tmpfs 64M 0 64M 0% /dev
tmpfs tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/sda1 ext4 98G 5.9G 88G 7% /etc/hosts
shm tmpfs 64M 0 64M 0% /dev/shm
192.168.26.134:/data/k8sdata nfs4 98G 4.7G 89G 5% /usr/share/nginx/html/mysite
tmpfs tmpfs 2.0G 12K 2.0G 1% /run/secrets/kubernetes.io/serviceaccount
tmpfs tmpfs 2.0G 0 2.0G 0% /proc/acpi
tmpfs tmpfs 2.0G 0 2.0G 0% /proc/scsi
tmpfs tmpfs 2.0G 0 2.0G 0% /sys/firmware
- 在共享数据中创建数据，看挂载node节点是否能够访问？
root@haproxy-server:/data# cd /data/k8sdata/
root@haproxy-server:/data/k8sdata# echo "linux39 test page" > linux39.html
- 新建一个 pod 看一个存储是否能够被多个 pod 所挂载
root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# cat deploy_nfs2.yml
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment-site2
spec:
replicas: 1
selector:
matchLabels:
app: ng-deploy-82
template:
metadata:
labels:
app: ng-deploy-82
spec:
containers:
- name: ng-deploy-82
image: nginx
ports:
- containerPort: 80
volumeMounts:
- mountPath: /usr/share/nginx/html/mysite #挂载到 nfs 服务器的 pod 的指定目录
name: my-nfs-volume
volumes:
- name: my-nfs-volume
nfs:
server: 192.168.26.134 #nfs 服务器地址
path: /data/k8sdata #nfs 共享服务器共享的目录
---
apiVersion: v1
kind: Service
metadata:
name: ng-deploy-82
spec:
ports:
- name: http
port: 80
targetPort: 80
nodePort: 30032 #对外暴露访问端口
protocol: TCP
type: NodePort
selector:
app: ng-deploy-82root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# kubectl apply -f deploy_nfs2.yml
deployment.apps/nginx-deployment-site2 unchanged
service/ng-deploy-82 created
root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# kubectl get pod
NAME READY STATUS RESTARTS AGE
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d11h
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d11h
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d10h
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d10h
nginx-deployment-3-688784467c-nb9st 1/1 Running 0 20m
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 2d4h
nginx-deployment-site2-66756d5865-hx6rx 1/1 Running 0 23s
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 5d1h
root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# kubectl exec -it nginx-deployment-site2-66756d5865-hx6rx bash
root@nginx-deployment-site2-66756d5865-hx6rx:/# df -Th
Filesystem Type Size Used Avail Use% Mounted on
overlay overlay 98G 6.6G 87G 8% /
tmpfs tmpfs 64M 0 64M 0% /dev
tmpfs tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/sda1 ext4 98G 6.6G 87G 8% /etc/hosts
shm tmpfs 64M 0 64M 0% /dev/shm
192.168.26.134:/data/k8sdata nfs4 98G 4.7G 89G 5% /usr/share/nginx/html/mysite
tmpfs tmpfs 2.0G 12K 2.0G 1% /run/secrets/kubernetes.io/serviceaccount
tmpfs tmpfs 2.0G 0 2.0G 0% /proc/acpi
tmpfs tmpfs 2.0G 0 2.0G 0% /proc/scsi
tmpfs tmpfs 2.0G 0 2.0G 0% /sys/firmware
- 实现了，一个 nfs 能被多个 pod 同时挂载，从而数据共享。
- 在一个 pod 里面同时挂载多个 NFS 服务，实现 pod 里面有多个数据源，类似服务的多个站点数据区分。
root@haproxy-server:~# mkdir /data/linux39
root@haproxy-server:~# vim /etc/exports
...省略...
/data/k8sdata *(rw,no_root_squash)
/data/linux39 *(rw,no_root_squash)
root@haproxy-server:~# systemctl restart nfs-server.service- node节点上查看是否共享成功搜索到，192.168.26.134 的共享挂载点
root@node-1:~# showmount -e 192.168.26.134
Export list for 192.168.26.134:
/data/linux39 *
/data/k8sdata *root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# cat deploy_nfs.yml
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment-3
spec:
replicas: 1
selector:
matchLabels:
app: ng-deploy-80
template:
metadata:
labels:
app: ng-deploy-80
spec:
containers:
- name: ng-deploy-80
image: nginx
ports:
- containerPort: 80
volumeMounts:
- mountPath: /usr/share/nginx/html/mysite #对应/data/k8sdata目录挂载
name: my-nfs-volume #区分挂载哪个nfs
- mountPath: /usr/share/nginx/html #对应 /data/linux39目录挂载
name: linux39-nfs-volume #区分挂载哪个nfs
volumes:
- name: my-nfs-volume
nfs:
server: 192.168.26.134
path: /data/k8sdata
- name: linux39-nfs-volume
nfs:
server: 192.168.26.134
path: /data/linux39
---
apiVersion: v1
kind: Service
metadata:
name: ng-deploy-80
spec:
ports:
- name: http
port: 81
targetPort: 80
nodePort: 30016
protocol: TCP
type: NodePort
selector:
app: ng-deploy-80root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# kubectl get pod
NAME READY STATUS RESTARTS AGE
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d11h
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d11h
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d11h
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d11h
nginx-deployment-3-674cf8d848-ltg7q 1/1 Running 0 50s
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 2d5h
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 5d2h
root@k8s-master1:/opt/k8s-data/yaml/linux39/case5# kubectl exec -it nginx-deployment-3-674cf8d848-ltg7q bash
root@nginx-deployment-3-674cf8d848-ltg7q:/# df -Th
Filesystem Type Size Used Avail Use% Mounted on
overlay overlay 98G 5.9G 88G 7% /
tmpfs tmpfs 64M 0 64M 0% /dev
tmpfs tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/sda1 ext4 98G 5.9G 88G 7% /etc/hosts
shm tmpfs 64M 0 64M 0% /dev/shm
192.168.26.134:/data/linux39 nfs4 98G 4.7G 89G 5% /usr/share/nginx/html
tmpfs tmpfs 2.0G 12K 2.0G 1% /run/secrets/kubernetes.io/serviceaccount
192.168.26.134:/data/k8sdata nfs4 98G 4.7G 89G 5% /usr/share/nginx/html/mysite
tmpfs tmpfs 2.0G 0 2.0G 0% /proc/acpi
tmpfs tmpfs 2.0G 0 2.0G 0% /proc/scsi
tmpfs tmpfs 2.0G 0 2.0G 0% /sys/firmware
-
NFS 服务挂载的时候，不仅仅给 pod 节点 pod 网段共享目录的权限，也要给宿主机网段权限，不然挂载报错：授权范围得注意,而且不是用 mount 去挂载
-
如：
root@HA-server1:/data/k8sdata# vim /etc/exports
/data/k8sdata 10.10.0.0/16 192.168.26.0/24(rw,no_root_squash) #用空格隔开
/data/linux39 10.10.0.0/16 192.168.26.0/24(rw,no_root_squash)CASE6:configmap
- configmap: https://kubernetes.io/zh/docs/concepts/storage/volumes/#configmap
- configmap 资源提供了向 Pod 注入配置数据的方法。 ConfigMap 对象中存储的数据可以被 configMap 类型的卷引用，然后被应用到 Pod 中运行的容器化应用。
- 配置信息大部分放到镜像里面,如果同一些配置信息给多个 pod 复用 可以用 configmap
root@k8s-master1:/opt/k8s-data/yaml/linux39/case6# pwd
/opt/k8s-data/yaml/linux39/case6
root@k8s-master1:/opt/k8s-data/yaml/linux39/case6# cat deploy_configmap.yml
apiVersion: v1
kind: ConfigMap #类型是 configMap
metadata:
name: nginx-config #通过这个名称来调用
data:
default: | #default 一个 key 的名称 下面为定义的服务内容
server {
listen 80;
server_name www.mysite.com;
index index.html;
location / {
root /data/nginx/html;
if (!-e $request_filename) {
rewrite ^/(.*) /index.html last;
}
}
}
---
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: ng-deploy-80
template:
metadata:
labels:
app: ng-deploy-80
spec:
containers:
- name: ng-deploy-80
image: nginx
ports:
- containerPort: 80
volumeMounts:
- mountPath: /data/nginx/html
name: nginx-static-dir
- name: nginx-config #调用 nginx-config 这个的类型是 configMap
mountPath: /etc/nginx/conf.d
volumes:
- name: nginx-static-dir
hostPath:
path: /data/nginx/linux39
- name: nginx-config
configMap:
name: nginx-config
items:
- key: default #key 调用最前面的 default
path: mysite.conf #挂载容器的路径，结合起来就是 /etc/nginx/conf.d/mysite.conf
---
apiVersion: v1
kind: Service
metadata:
name: ng-deploy-80
spec:
ports:
- name: http
port: 81
targetPort: 80
nodePort: 30019 #对外暴露访问端口
protocol: TCP
type: NodePort
selector:
app: ng-deploy-80root@k8s-master1:/opt/k8s-data/yaml/linux39/case6# kubectl get pod
NAME READY STATUS RESTARTS AGE
net-test1-5fcc69db59-c2plf 1/1 Running 3 5d11h
net-test1-5fcc69db59-xzvrl 1/1 Running 3 5d11h
net-test2-8456fd74f7-ckpsr 1/1 Running 3 5d11h
net-test2-8456fd74f7-nbzx8 1/1 Running 3 5d11h
nginx-deployment-4-8c449b55f-kzcxg 1/1 Running 0 59s
nginx-deployment-5f4dc447b5-wcg2t 1/1 Running 0 2d5h
tomcat-deployment-5cd65b4d74-d4x7s 1/1 Running 2 5d2h
root@k8s-master1:/opt/k8s-data/yaml/linux39/case6# kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 192.168.0.1 <none> 443/TCP 5d14h
magedu-nginx-service NodePort 192.168.3.55 <none> 80:30004/TCP 2d9h
magedu-tomcat-service NodePort 192.168.1.200 <none> 80:30005/TCP 5d2h
ng-deploy-80 NodePort 192.168.1.114 <none> 81:30019/TCP 3m36s
root@k8s-master1:/opt/k8s-data/yaml/linux39/case6# kubectl exec -it nginx-deployment-4-8c449b55f-kzcxg bash
root@nginx-deployment-4-8c449b55f-kzcxg:/# cat /etc/nginx/conf.d/mysite.conf
server {
listen 80;
server_name www.mysite.com;
index index.html;
location / {
root /data/nginx/html;
if (!-e $request_filename) {
rewrite ^/(.*) /index.html last;
}
}
}
- 在 node-1 中创建首页文件，看是否能够调用 configmap
root@node-1:~# cd /data/nginx/linux39/
root@node-1:/data/nginx/linux39# ll
total 8
drwxr-xr-x 2 root root 4096 Apr 4 04:49 ./
drwxr-xr-x 3 root root 4096 Apr 4 04:49 ../
root@node-1:/data/nginx/linux39# vim index.html
root@node-1:/data/nginx/linux39# cat index.html
configMap test page- 浏览器访问： http://192.168.26.144:30019/
DaemonSet
-
DaemonSet： https://kubernetes.io/zh/docs/concepts/workloads/controllers/daemonset/
-
DaemonSet 确保全部（或者某些）节点上运行一个 Pod 的副本。当有节点加入集群时， 也会为他们新增一个 Pod 。当有节点从集群移除时，这些 Pod 也会被回收。删除 DaemonSet 将会删除它创建的所有 Pod。
-
DaemonSet 会在当前k8s集群中的所有 node 创建相同的 node，主要用于在所有 node 执行相同的操作的场景。
1.日志收集
2.Prometheus
3.flannel
root@k8s-master1:/opt/k8s-data/yaml/linux39# pwd
/opt/k8s-data/yaml/linux39
root@k8s-master1:/opt/k8s-data/yaml/linux39# vim daemonset.yml
root@k8s-master1:/opt/k8s-data/yaml/linux39# cat daemonset.yml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluentd-elasticsearch
namespace: kube-system
labels:
k8s-app: fluentd-logging
spec:
selector:
matchLabels:
name: fluentd-elasticsearch
template:
metadata:
labels:
name: fluentd-elasticsearch
spec:
tolerations:
# this toleration is to have the daemonset runnable on master nodes
# remove it if your masters can't run pods
- key: node-role.kubernetes.io/master
effect: NoSchedule
containers:
- name: fluentd-elasticsearch
image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 200Mi
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
terminationGracePeriodSeconds: 30
volumes:
- name: varlog
hostPath:
path: /var/log #把/var/log挂载到pod里面
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers #把/var/lib/docker/containers挂载到pod里面root@k8s-master1:/opt/k8s-data/yaml/linux39# kubectl apply -f daemonset.yml
daemonset.apps/fluentd-elasticsearch created
root@k8s-master1:/opt/k8s-data/yaml/linux39# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-7f9c544f75-cz7f8 1/1 Running 4 5d11h
coredns-7f9c544f75-h9s2f 1/1 Running 2 5d4h
etcd-k8s-master1 1/1 Running 4 5d14h
etcd-k8s-master2 1/1 Running 2 5d13h
etcd-k8s-master3 1/1 Running 2 5d13h
fluentd-elasticsearch-ftmdh 0/1 ContainerCreating 0 48s
fluentd-elasticsearch-lfwkv 0/1 ContainerCreating 0 48s
fluentd-elasticsearch-s25qm 0/1 ContainerCreating 0 48s
fluentd-elasticsearch-v8fj6 0/1 ContainerCreating 0 48s
fluentd-elasticsearch-wfdnf 0/1 ContainerCreating 0 48s
fluentd-elasticsearch-xl9f4 0/1 ContainerCreating 0 48s- 可以看到集群中所有的 master 和所有的 node 节点中都创建了相同的一个 pod 可以用于相同资源的获取调用。