6.服务发布入门 | Evergreen Notes

1. Label 和 Selector

1.1 定义 Label

应用案例：公司与 xx 银行有一条专属的高速光纤通道，此通道只能与 192.168.7.0 网段进行通信，因此只能将与 xx 银行通信的应用部署到 192.168.7.0 网段所在的节点上，此时可以对节点添加 Label

BASH

root@k8s-master01:~# kubectl label node node-02 region=subnet7
node/node-02 labeled

然后可以通过 Selector 对其筛选

BASH

最后在 Deployment 或其他控制器中指定将 Pod 部署到该节点

YAML

查看

BASH

也可以用同样的方式对 Service 添加 Label

YAML

创建了一个名为 my-service 的 Service，并使用 Label 选择与之关联的 Pod。spec.selector 字段中的 app: frontend 表示选择具有 Label app: frontend 的 Pod 作为该 Service 的后端。

对 Service 添加 Label

创建service

BASH

root@k8s-master01:~/hhh# kubectl create -f service.yaml
service/my-service created

查看

BASH

打上标签

BASH

以查看所有 version 为 v1 的 Service

BASH

1.2 Selector 选择器

首先使用--show-labels 查看指定资源目前已有的 Label

BASH

选择匹配 app 为 my-service2 或者my-service3 的 Service：

BASH

选择 app 为 my-service2 或 my-service3但不包括 version=v1 的 svc：

BASH

选择 label 的 key 名为 app 的 svc：

BASH

1.3 修改标签（Label）

比如将 version=v1 改为 version=v2

BASH

1.4 删除标签（Label）

删除 key 名为 version 的标签

BASH

2 Service

2.1 定义 Service

YAML

该示例为 my-service:80 即可访问到具有 app=myapp 标签的 Pod 的 80 端口上。需要注意的是，Service 能够将一个接收端口映射到任意的 targetPort，如果 targetPort 为空， targetPort 将被设置为与 Port 字段相同的值。targetPort 可以设置为一个字符串，引用 backend Pod 的一个端口的名称，这样的话即使更改了 Pod 的端口，也不会对 Service 的访问造成影响。 Kubernetes Service 能够支持 TCP、UDP、SCTP 等协议，默认为 TCP 协议。

BASH

2.2 Service 类型

Kubernetes Service Type（服务类型）主要包括以下几种

ClusterIP：在集群内部使用，默认值，只能从集群中访问
NodePort：在所有安装了 Kube-Proxy 的节点上打开一个端口，此端口可以代理至后端 Pod，可以通过 NodePort 从集群外部访问集群内的服务，格式为 NodeIP:NodePort
LoadBalancer：使用云提供商的负载均衡器公开服务，成本较高
ExternalName：通过返回定义的 CNAME 别名，没有设置任何类型的代理，需要 1.7 或更高版本 kube-dns 支持

2.3 NodePort 类型

如果将 Service 的 type 字段设置为 NodePort，则 Kubernetes 将从--service-node-port-range 参数指定的范围（默认为 30000-32767）中自动分配端口，也可以手动指定 NodePort，创建该 Service 后，集群每个节点都将暴露一个端口，通过某个宿主机的 IP+端口即可访问到后端的应用.

定义一个 NodePort 类型的 Service 格式如下

YAML

2.4 使用 Service 代理 K8s 外部服务

使用场景

希望在生产环境中使用某个固定的名称而非 IP 地址访问外部的中间件服务

2.希望 Service 指向另一个 Namespace 中或其他集群中的服务
正在将工作负载转移到 Kubernetes 集群，但是一部分服务仍运行在 Kubernetes 集群之外的 backend

举个例子

YAML

ip: 10.200.135.155 为pod的nginx地址

BASH

创建

BASH

测试一下

BASH

我们不难发现我们通过http://nginx-svc-external 访问到services nginx-svc-external 10.100.241.177 其实后端为10.200.135.155:80

注意：Endpoint IP 地址不能是 loopback（127.0.0.0/8）、link-local（169.254.0.0/16）或者 linklocal 多播地址（224.0.0.0/24）

访问没有 Selector 的 Service 与有 Selector 的 Service 的原理相同，通过 Service 名称即可访问，请求将被路由到用户定义的 Endpoint。

2.5 ExternalName Service

ExternalName Service 是 Service 的特例，它没有 Selector，也没有定义任何端口和 Endpoint，它通过返回该外部服务的别名来提供服务。

比如可以定义一个 Service，后端设置为一个外部域名，这样通过 Service 的名称即可访问到该域名。使用 nslookup 解析以下文件定义的 Service，集群的 DNS 服务将返回一个值为 my.database.example.com 的 CNAME 记录：

YAML

BASH

2.6 多端口 Service

例如将 Service 的 80 端口代理到后端的 9376,443 端口代理到后端的 9377

3 Ingress

3.1 Ingress Controller 安装

官方安装文档：https://kubernetes.github.io/ingress-nginx/deploy/#bare-metal-clusters

YAML

BASH

3.2 使用域名发布 K8s 的服务

创建一个 web 服务

BASH

# kubectl create deploy nginx --image=registry.cn-beijing.aliyuncs.com/dotbalo/nginx:1.15.12

暴露服务

BASH

# kubectl expose deploy nginx --port 80

创建 Ingress

YAML

BASH

pathType：路径的匹配方式，目前有 ImplementationSpecific、Exact 和 Prefix 方式
Exact：精确匹配，比如配置的 path 为/bar，那么/bar/将不能被路由；
Prefix：前缀匹配，基于以 / 分隔的 URL 路径。比如 path 为/abc，可以匹配到/abc/bbb 等，比较常用的配置；
ImplementationSpecific：这种类型的路由匹配根据 Ingress Controller 来实现，可以当做一个单独的类型，也可以当做 Prefix 和 Exact。ImplementationSpecific 是 1.18 版本引入 Prefix 和 Exact 的默认配置；

3.3 Ingress 特例：不配置域名发布服务

YAML

3.4 Ingress 接口变化解析

1.19 之前的 v1beta1

YAML

root@k8s-master01:~# kubectl get nodes -l region=subnet7
NAME      STATUS   ROLES   AGE    VERSION
node-02   Ready    node    7d5h   v1.28.2

root@k8s-master01:~/hhh# cat nginx.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: registry.cn-beijing.aliyuncs.com/dotbalo/nginx:1.15.12-alpine
        name: nginx
      nodeSelector:
        region: subnet7

root@k8s-master01:~/hhh# kubectl create -f nginx.yaml
 
root@k8s-master01:~/hhh# kubectl get pod -owide
NAME                     READY   STATUS    RESTARTS   AGE   IP              NODE      NOMINATED NODE   READINESS GATES
nginx-68649d5db5-7ckb9   1/1     Running   0          15s   10.200.58.236   node-02   <none>           <none>
nginx-68649d5db5-hjww8   1/1     Running   0          15s   10.200.58.237   node-02   <none>           <none>
nginx-68649d5db5-msvt4   1/1     Running   0          15s   10.200.58.238   node-02   <none>           <none>

root@k8s-master01:~/hhh# cat service.yaml 
apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: frontend
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

root@k8s-master01:~/hhh# kubectl get svc --show-labels 
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE    LABELS
kubernetes   ClusterIP   10.100.0.1     <none>        443/TCP   7d5h   component=apiserver,provider=kubernetes
my-service   ClusterIP   10.100.148.0   <none>        80/TCP    23s    <none>

root@k8s-master01:~/hhh# kubectl label svc my-service env=canary version=v1
service/my-service labeled
 
root@k8s-master01:~/hhh# kubectl get svc --show-labels 
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE     LABELS
kubernetes   ClusterIP   10.100.0.1     <none>        443/TCP   7d5h    component=apiserver,provider=kubernetes
my-service   ClusterIP   10.100.148.0   <none>        80/TCP    2m25s   env=canary,version=v1

root@k8s-master01:~/hhh# kubectl get svc -l version=v1
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
my-service   ClusterIP   10.100.148.0   <none>        80/TCP    2m50s

root@k8s-master01:~/hhh# kubectl get svc --show-labels 
NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE     LABELS
kubernetes    ClusterIP   10.100.0.1       <none>        443/TCP   7d5h    component=apiserver,provider=kubernetes
my-service    ClusterIP   10.100.148.0     <none>        80/TCP    21m     env=canary,version=v1
my-service2   ClusterIP   10.100.171.60    <none>        80/TCP    4m27s   app=my-service2
my-service3   ClusterIP   10.100.92.228    <none>        80/TCP    4m21s   app=my-service3
my-service4   ClusterIP   10.100.227.255   <none>        80/TCP    4s      app=my-service4

root@k8s-master01:~/hhh# kubectl get svc -l 'app in (my-service2, my-service3)' --show-labels
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE     LABELS
my-service2   ClusterIP   10.100.171.60   <none>        80/TCP    3m18s   app=my-service2
my-service3   ClusterIP   10.100.92.228   <none>        80/TCP    3m12s   app=my-service3

root@k8s-master01:~/hhh#  kubectl get svc -l version!=v1,'app in (my-service2, my-service3)' --show-labels
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE     LABELS
my-service2   ClusterIP   10.100.171.60   <none>        80/TCP    7m26s   app=my-service2
my-service3   ClusterIP   10.100.92.228   <none>        80/TCP    7m20s   app=my-service3

root@k8s-master01:~/hhh# kubectl get svc -l app --show-labels
NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE     LABELS
my-service2   ClusterIP   10.100.171.60    <none>        80/TCP    8m17s   app=my-service2
my-service3   ClusterIP   10.100.92.228    <none>        80/TCP    8m11s   app=my-service3
my-service4   ClusterIP   10.100.227.255   <none>        80/TCP    3m54s   app=my-service4

root@k8s-master01:~/hhh# kubectl get svc --show-labels 
NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE     LABELS
kubernetes    ClusterIP   10.100.0.1       <none>        443/TCP   7d5h    component=apiserver,provider=kubernetes
my-service    ClusterIP   10.100.148.0     <none>        80/TCP    21m     env=canary,version=v1
my-service2   ClusterIP   10.100.171.60    <none>        80/TCP    4m27s   app=my-service2
my-service3   ClusterIP   10.100.92.228    <none>        80/TCP    4m21s   app=my-service3
my-service4   ClusterIP   10.100.227.255   <none>        80/TCP    4s      app=my-service4
 
root@k8s-master01:~/hhh# kubectl label svc my-service version=v2 --overwrite 
service/my-service labeled
 
root@k8s-master01:~/hhh# kubectl get svc  --show-labels
NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE     LABELS
kubernetes    ClusterIP   10.100.0.1       <none>        443/TCP   7d5h    component=apiserver,provider=kubernetes
my-service    ClusterIP   10.100.148.0     <none>        80/TCP    28m     env=canary,version=v2
my-service2   ClusterIP   10.100.171.60    <none>        80/TCP    11m     app=my-service2
my-service3   ClusterIP   10.100.92.228    <none>        80/TCP    11m     app=my-service3
my-service4   ClusterIP   10.100.227.255   <none>        80/TCP    6m44s   app=my-service4

root@k8s-master01:~/hhh# kubectl label svc my-service version-
service/my-service unlabeled
root@k8s-master01:~/hhh# kubectl get svc  --show-labels
NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE     LABELS
kubernetes    ClusterIP   10.100.0.1       <none>        443/TCP   7d5h    component=apiserver,provider=kubernetes
my-service    ClusterIP   10.100.148.0     <none>        80/TCP    30m     env=canary
my-service2   ClusterIP   10.100.171.60    <none>        80/TCP    12m     app=my-service2
my-service3   ClusterIP   10.100.92.228    <none>        80/TCP    12m     app=my-service3
my-service4   ClusterIP   10.100.227.255   <none>        80/TCP    8m19s   app=my-service4

root@k8s-master01:~/hhh# cat service.yaml 
kind: Service
apiVersion: v1
metadata:
  name: my-service
spec:
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.15.12
        ports:
        - containerPort: 80

root@k8s-master01:~/hhh# kubectl describe svc my-service 
Name:              my-service
Namespace:         default
Labels:            <none>
Annotations:       <none>
Selector:          app=nginx
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.100.227.218
IPs:               10.100.227.218
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.200.135.155:80,10.200.58.239:80,10.200.85.232:80
Session Affinity:  None
Events:            <none>
 
root@k8s-master01:~/hhh# kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.100.0.1       <none>        443/TCP   7d6h
my-service   ClusterIP   10.100.227.218   <none>        80/TCP    26s
 
root@k8s-master01:~/hhh# kubectl get pod -owide
NAME                                READY   STATUS    RESTARTS   AGE    IP               NODE      NOMINATED NODE   READINESS GATES
nginx-deployment-5c8578bf64-hrfzb   1/1     Running   0          2m9s   10.200.135.155   node-03   <none>           <none>
nginx-deployment-5c8578bf64-tbjdv   1/1     Running   0          2m9s   10.200.58.239    node-02   <none>           <none>
nginx-deployment-5c8578bf64-wfxcq   1/1     Running   0          2m9s   10.200.85.232    node-01   <none>           <none>

root@k8s-master01:~/hhh# cat nodeport.yaml 
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-nodeport
  namespace: kube-system
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 32000
  selector:
    k8s-app: kubernetes-dashboard

cat nginx-svc-external.yml
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx-svc-external
  name: nginx-svc-external
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  sessionAffinity: None
  type: ClusterIP
 
root@k8s-master01:~/hhh# cat nginx-ep-external.yaml 
apiVersion: v1
kind: Endpoints
metadata:
  labels:
    app: nginx-svc-external
  name: nginx-svc-external
subsets:
  - addresses:
      - ip: 10.200.135.155 
    ports:
      - name: http
        port: 80
        protocol: TCP

root@k8s-master01:~/hhh# kubectl get pod -owide
NAME                                READY   STATUS    RESTARTS   AGE   IP               NODE      NOMINATED NODE   READINESS GATES
nginx-deployment-5c8578bf64-hrfzb   1/1     Running   0          70m   10.200.135.155   node-03   <none>           <none>
nginx-deployment-5c8578bf64-tbjdv   1/1     Running   0          70m   10.200.58.239    node-02   <none>           <none>
nginx-deployment-5c8578bf64-wfxcq   1/1     Running   0          70m   10.200.85.232    node-01   <none>           <none>

# kubectl apply -f nginx-svc-external.yaml
# kubectl apply -f nginx-ep-external.yaml
#查看
root@k8s-master01:~/hhh# kubectl get services nginx-svc-external
NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
nginx-svc-external   ClusterIP   10.100.241.177   <none>        80/TCP    29m
root@k8s-master01:~/hhh# kubectl get endpoints nginx-svc-external
NAME                 ENDPOINTS           AGE
nginx-svc-external   10.200.135.155:80   22m
root@k8s-master01:~/hhh#

root@k8s-master01:~/hhh# kubectl run net-test1 --image=busybox:1.28.3 sleep 360000
 
root@k8s-master01:~/hhh# kubectl exec -it net-test1 sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
 
/ # wget http://10.200.135.155:80
Connecting to 10.200.135.155:80 (10.200.135.155:80)
index.html           100% |****************************************************************************************************************************************************************************************************************************************|   612   0:00:00 ETA
/ # wget http://nginx-svc-external
Connecting to nginx-svc-external (10.100.241.177:80)
wget: can't open 'index.html': File exists
/ # ls
bin         dev         etc         home        index.html  proc        root        sys         tmp         usr         var
/ # rm -f index.html 
/ # wget http://nginx-svc-external
Connecting to nginx-svc-external (10.100.241.177:80)
index.html           100% |****************************************************************************************************************************************************************************************************************************************|   612   0:00:00 ETA
/ # exit

kind: Service
apiVersion: v1
metadata:
  name: my-service8
spec:
  type: ExternalName
  externalName: www.baidu.com

root@k8s-master01:~/hhh# kubectl exec -it net-test1 sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # wget my-service8
Connecting to my-service8 (180.101.50.188:80)
wget: server returned error: HTTP/1.1 403 Forbidden
已经解析到百度地址 403是因为百度不允许跨域访问

kind: Service
apiVersion: v1
metadata:
  name: my-service
spec:
  selector:
    app: myapp
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 9376
  - name: https
    protocol: TCP
    port: 443
    targetPort: 9377

apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.7.1
  name:

root@k8s-master01:~/hhh# kubectl create -f deploy-ingress.yaml 
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created

vim web-ingress.yaml 
apiVersion: networking.k8s.io/v1 # k8s >= 1.22 必须 v1
kind: Ingress
metadata:
  name: nginx-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: nginx.test.com
    http:
      paths:
      - backend:
          service:
            name: nginx
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific

root@k8s-master01:~/hhh# kubectl get svc -n ingress-nginx 
NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.100.86.48    <none>        80:32698/TCP,443:31764/TCP   25m
ingress-nginx-controller-admission   ClusterIP   10.100.245.59   <none>        443/TCP                      25m
 
root@k8s-master01:~/hhh# kubectl describe ingress nginx-ingress 
Name:             nginx-ingress
Labels:           <none>
Namespace:        default
Address:          192.168.20.239
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host            Path  Backends
  ----            ----  --------
  nginx.test.com  
                  /   nginx:80 (10.200.135.155:80,10.200.58.239:80,10.200.58.244:80 + 1 more...)
Annotations:      <none>
Events:
  Type    Reason  Age                    From                      Message
  ----    ------  ----                   ----                      -------
  Normal  Sync    4m44s (x2 over 5m29s)  nginx-ingress-controller  Scheduled for sync
 
root@k8s-master01:~/hhh# kubectl get  ingress
NAME            CLASS   HOSTS            ADDRESS          PORTS   AGE
nginx-ingress   nginx   nginx.test.com   192.168.20.239   80      2m31s

# vim ingress-no-host.yaml 
apiVersion: networking.k8s.io/v1 # k8s >= 1.22 必须 v1
kind: Ingress
metadata:
  name: nginx-ingress-no-host
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - backend:
          service:
            name: nginx
            port:
              number: 80
        path: /no-host
        pathType: ImplementationSpecific

apiVersion: networking.k8s.io/v1beta1 # 1.22 之前可以使用 v1beta1
kind: Ingress
metadata:
  name: simple-fanout-example
  annotations:
    kubernetes.io/ingress.class: "nginx" # 不同的 controller，ingress.class
可能不一致
spec:
  rules:
    - host: foo.bar.com
      http:
        paths:
        - path: /foo
          pathType: Prefix
          backend:
            serviceName: service1
            servicePort: 4200
        - path: /bar
          pathType: ImplementationSpecific
          backend:
            serviceName: service2
            servicePort: 8080